If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra

Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

Queue and LSF versions

pdexheimerpdexheimer Member ✭✭✭✭

This is not a question, per se - I suppose it's more of an observation.

We recently upgraded LSF on one of our clusters to v9.0.1, and quickly discovered that Queue can't submit jobs. The reaction was rather violent - the entire JVM crashed, and the stack trace showed it dying in lsb_submit(). We downgraded LSF to v8.3.0, and everything is working fine (so far).

I know Queue is compiled against the LSF v7.0.6 API, it would appear that it's not binary-compatible with LSF 9.x.

Hope this helps others in the future...



  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Ouch, that's good to know, thank you for reporting this.

  • Hi Geraldine. We just ran into the same problem. Is there a patch in the works or some way I can compile and package a deployable jar file from the sources on github and try to manage my own patch?
    Many thanks!

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Hi @pmclaren,

    Unfortunately we don't have a solution at this time, and it's unlikely we will have one to offer in the near or even medium term, sorry. We currently don't have the resources to investigate the problem. Of course of someone in the community were to contribute a patch we'd be delighted to look at it.

  • inkink Member

    Hello, everyone!
    Just want to add another voice for the need of an LSF patch. We are running LSF 9.1.

  • pdexheimerpdexheimer Member ✭✭✭✭

    I've looked into the source a little - it's not necessarily a difficult patch, but it is a lot of coding. The GATK uses Java Native Access to talk to the LSF library. The kicker is that someone had to translate all of the C header files into Java structures - org.broadinstitute.sting.jna.lsf.v7_0_6.LibBat is around 20,000 lines.

    I have no experience with JNA, and so don't know if that amount of interface is strictly necessary. But it's not something I'm likely to mess with until I have to - and I imagine (though certainly don't speak for them) that the GSA folks feel similarly. So your best bet is probably to hope that someone out there does have the motivation to work on it (which I think is what happened with the SGE interface)

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    I've discussed this with Khalid (@kshakir) who originally developed Queue and ported the header files. He confirmed it's a pain in the rear to deal with. There is indeed probably some amount of unused code in that 20k-line file, but no-one wants to be the chump assigned to sort through it. On our end we have zero motivation to deal with it as long as our LSF version stays put (and there is no reason for it to change anytime soon). So I'm afraid for now our official position remains: we're happy to look at a patch, assuming someone out there needs it bad enough that they sit down and work through it.

  • Johan_DahlbergJohan_Dahlberg Member ✭✭✭

    I have very little experience in this compartment, but I just want to throw it out there that it might be possible to do this through drmaa instead. According to wikipedia, there is a drmaa implementation for LSF available:

  • rfarrerrfarrer Cambridge MAMember

    Dear Geraldine,

    I think we are having similar problems with LSF9 and Queue. Do you know if there is a work-around for Queue?


  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    @rfarrer Sorry, we don't have a workaround for this and are unlikely to put effort into finding one as we have switched to GridEngine. Johan's suggestion to use drmaa may be your best bet.

Sign In or Register to comment.