We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

Queue and LSF versions

pdexheimerpdexheimer Member ✭✭✭✭

This is not a question, per se - I suppose it's more of an observation.

We recently upgraded LSF on one of our clusters to v9.0.1, and quickly discovered that Queue can't submit jobs. The reaction was rather violent - the entire JVM crashed, and the stack trace showed it dying in lsb_submit(). We downgraded LSF to v8.3.0, and everything is working fine (so far).

I know Queue is compiled against the LSF v7.0.6 API, it would appear that it's not binary-compatible with LSF 9.x.

Hope this helps others in the future...



  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Ouch, that's good to know, thank you for reporting this.

  • Hi Geraldine. We just ran into the same problem. Is there a patch in the works or some way I can compile and package a deployable jar file from the sources on github and try to manage my own patch?
    Many thanks!

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Hi @pmclaren,

    Unfortunately we don't have a solution at this time, and it's unlikely we will have one to offer in the near or even medium term, sorry. We currently don't have the resources to investigate the problem. Of course of someone in the community were to contribute a patch we'd be delighted to look at it.

  • inkink Member

    Hello, everyone!
    Just want to add another voice for the need of an LSF patch. We are running LSF 9.1.

  • pdexheimerpdexheimer Member ✭✭✭✭

    I've looked into the source a little - it's not necessarily a difficult patch, but it is a lot of coding. The GATK uses Java Native Access to talk to the LSF library. The kicker is that someone had to translate all of the C header files into Java structures - org.broadinstitute.sting.jna.lsf.v7_0_6.LibBat is around 20,000 lines.

    I have no experience with JNA, and so don't know if that amount of interface is strictly necessary. But it's not something I'm likely to mess with until I have to - and I imagine (though certainly don't speak for them) that the GSA folks feel similarly. So your best bet is probably to hope that someone out there does have the motivation to work on it (which I think is what happened with the SGE interface)

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    I've discussed this with Khalid (@kshakir) who originally developed Queue and ported the header files. He confirmed it's a pain in the rear to deal with. There is indeed probably some amount of unused code in that 20k-line file, but no-one wants to be the chump assigned to sort through it. On our end we have zero motivation to deal with it as long as our LSF version stays put (and there is no reason for it to change anytime soon). So I'm afraid for now our official position remains: we're happy to look at a patch, assuming someone out there needs it bad enough that they sit down and work through it.

  • Johan_DahlbergJohan_Dahlberg Member ✭✭✭

    I have very little experience in this compartment, but I just want to throw it out there that it might be possible to do this through drmaa instead. According to wikipedia, there is a drmaa implementation for LSF available: http://en.wikipedia.org/wiki/DRMAA.

  • rfarrerrfarrer Cambridge MAMember

    Dear Geraldine,

    I think we are having similar problems with LSF9 and Queue. Do you know if there is a work-around for Queue?


  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    @rfarrer Sorry, we don't have a workaround for this and are unlikely to put effort into finding one as we have switched to GridEngine. Johan's suggestion to use drmaa may be your best bet.

Sign In or Register to comment.