Bug Bulletin: we have identified a bug that affects indexing when producing gzipped VCFs. This will be fixed in the upcoming 3.2 release; in the meantime you need to reindex gzipped VCFs using Tabix.

Queue and LSF versions

pdexheimerpdexheimer Posts: 297Member, GSA Collaborator ✭✭✭

This is not a question, per se - I suppose it's more of an observation.

We recently upgraded LSF on one of our clusters to v9.0.1, and quickly discovered that Queue can't submit jobs. The reaction was rather violent - the entire JVM crashed, and the stack trace showed it dying in lsb_submit(). We downgraded LSF to v8.3.0, and everything is working fine (so far).

I know Queue is compiled against the LSF v7.0.6 API, it would appear that it's not binary-compatible with LSF 9.x.

Hope this helps others in the future...

Tagged:

Comments

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 5,235Administrator, GSA Member admin

    Ouch, that's good to know, thank you for reporting this.

    Geraldine Van der Auwera, PhD

  • pmclarenpmclaren Posts: 1Member

    Hi Geraldine. We just ran into the same problem. Is there a patch in the works or some way I can compile and package a deployable jar file from the sources on github and try to manage my own patch? Many thanks!

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 5,235Administrator, GSA Member admin

    Hi @pmclaren,

    Unfortunately we don't have a solution at this time, and it's unlikely we will have one to offer in the near or even medium term, sorry. We currently don't have the resources to investigate the problem. Of course of someone in the community were to contribute a patch we'd be delighted to look at it.

    Geraldine Van der Auwera, PhD

  • inkink Posts: 3Member

    Hello, everyone! Just want to add another voice for the need of an LSF patch. We are running LSF 9.1. Thanks

  • pdexheimerpdexheimer Posts: 297Member, GSA Collaborator ✭✭✭

    I've looked into the source a little - it's not necessarily a difficult patch, but it is a lot of coding. The GATK uses Java Native Access to talk to the LSF library. The kicker is that someone had to translate all of the C header files into Java structures - org.broadinstitute.sting.jna.lsf.v7_0_6.LibBat is around 20,000 lines.

    I have no experience with JNA, and so don't know if that amount of interface is strictly necessary. But it's not something I'm likely to mess with until I have to - and I imagine (though certainly don't speak for them) that the GSA folks feel similarly. So your best bet is probably to hope that someone out there does have the motivation to work on it (which I think is what happened with the SGE interface)

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 5,235Administrator, GSA Member admin

    I've discussed this with Khalid (@kshakir) who originally developed Queue and ported the header files. He confirmed it's a pain in the rear to deal with. There is indeed probably some amount of unused code in that 20k-line file, but no-one wants to be the chump assigned to sort through it. On our end we have zero motivation to deal with it as long as our LSF version stays put (and there is no reason for it to change anytime soon). So I'm afraid for now our official position remains: we're happy to look at a patch, assuming someone out there needs it bad enough that they sit down and work through it.

    Geraldine Van der Auwera, PhD

  • Johan_DahlbergJohan_Dahlberg Posts: 80Member ✭✭✭

    I have very little experience in this compartment, but I just want to throw it out there that it might be possible to do this through drmaa instead. According to wikipedia, there is a drmaa implementation for LSF available: http://en.wikipedia.org/wiki/DRMAA.

Sign In or Register to comment.