GenomeSTrip+Queue+SGE

Will_GilksWill_Gilks University of Sussex, UKMember ✭✭

Hi,

We're trying to use GenomeStrip 2.0 on our Sun Grid Engine. Genomestrip itself works but there are problems with Queue.

We've tried to configure the shellscripts discover.sh and genotyping.sh by adding -jobRunner Drmaa -gatkJobRunner Drmaa -jobQueue serial.q to the QCommandLine but SGE barfs because the necessary modules aren't loaded on the compute node.

We need to pass parameters for number of cores, parallel environment, environment variables etc to Drmaa.

A workaround would be to omit the -jobRunner arguments and instead submit the shell script to SGE with a wrapper.

Attempting to use -jobNative to specify extra options to SGE (e.g. pe, environmental variables) results in errors. Below is the input and error message. (By the way, LD_LIBRARY_PATH includes the path to the drmaa library, and all our nodes have internet access. I note the installtest/ directory has both discovery.sh and discovery.sh~ which I'm assuming is ok though it looks weird to me. We're currently trying to add a parallel environment using -jobNative -pe openmp NUM where NUM is the number of cores.)

I was wondering if anyone had any idea how to fix this error. (Apologies if messages are confusing: I'm relaying information from a busy system administrator).

Sincerely,

William Gilks

doing
java -cp /cm/shared/apps/svtoolkit/2.0.1602/lib/SVToolkit.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/GenomeAnalysisTK.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/Queue.jar -Xmx4g org.broadinstitute.gatk.queue.QCommandLine -S /cm/shared/apps/svtoolkit/2.0.1602/qscript/SVPreprocess.q -S /cm/shared/apps/svtoolkit/2.0.1602/qscript/SVQScript.q -gatk /cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/GenomeAnalysisTK.jar -jobRunner Drmaa -gatkJobRunner Drmaa -cp /cm/shared/apps/svtoolkit/2.0.1602/lib/SVToolkit.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/GenomeAnalysisTK.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/Queue.jar -configFile conf/genstrip_installtest_parameters.txt -tempDir /lustre/scratch/tmp -R data/human_b36_chr1.fasta -genomeMaskFile data/human_b36_chr1.svmask.fasta -ploidyMapFile data/human_b36_chr1.ploidymap.txt -genderMapFile data/installtest_gender.map -copyNumberMaskFile data/human_b36_chr1.gcmask.fasta -runDirectory test2 -md test2/metadata -L 1:61700000-61900000 -reduceInsertSizeDistributions false -computeGCProfiles true -computeReadCounts false -jobLogDir test2/logs -I data/installtest.bam -jobNative -V -run -help || exit 1

ERROR ------------------------------------------------------------------------------------------
ERROR stack trace

org.broadinstitute.gatk.utils.commandline.InvalidArgumentValueException:
Argument '--job_native_arg' requires a value but none was provided
at org.broadinstitute.gatk.utils.commandline.ParsingEngine.validate(ParsingEngine.java:334)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:198)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:155)
at org.broadinstitute.gatk.queue.QCommandLine$.main(QCommandLine.scala:62)
at org.broadinstitute.gatk.queue.QCommandLine.main(QCommandLine.scala)

ERROR ------------------------------------------------------------------------------------------
ERROR A GATK RUNTIME ERROR has occurred (version ):
ERROR
ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
ERROR If not, please post the error message, with stack trace, to the GATK forum.
ERROR Visit our website and forum for extensive documentation and answers to
ERROR commonly asked questions http://www.broadinstitute.org/gatk
ERROR
ERROR MESSAGE: Argument '--job_native_arg' requires a value but none was provided

Issue · Github
by Sheila

Issue Number
105
State
closed
Last Updated
Closed By
vdauwera

Best Answers

Answers

  • Will_GilksWill_Gilks University of Sussex, UKMember ✭✭

    Sorry, I don't think I should've tagged this one as "Answered". Any chance you could change this?

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin
  • Will_GilksWill_Gilks University of Sussex, UKMember ✭✭

    Thanks @thibault . I think that was our main problem. Successful input shown below:

    module load sge
    module load genomestrip/2.0
    java \
    -cp /cm/shared/apps/svtoolkit/2.0.1602/lib/SVToolkit.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/GenomeAnalysisTK.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/Queue.jar -Xmx4g org.broadinstitute.gatk.queue.QCommandLine \
    -S /cm/shared/apps/svtoolkit/2.0.1602/qscript/SVPreprocess.q \
    -S /cm/shared/apps/svtoolkit/2.0.1602/qscript/SVQScript.q \
    -gatk /cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/GenomeAnalysisTK.jar \
    -jobRunner Drmaa \
    -gatkJobRunner Drmaa \
    -cp /cm/shared/apps/svtoolkit/2.0.1602/lib/SVToolkit.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/GenomeAnalysisTK.jar:/cm/shared/apps/svtoolkit/2.0.1602/lib/gatk/Queue.jar \
    -configFile conf/genstrip_installtest_parameters.txt \
    -tempDir /lustre/scratch/tmp \
    -R data/human_b36_chr1.fasta \
    -genomeMaskFile data/human_b36_chr1.svmask.fasta \
    -ploidyMapFile data/human_b36_chr1.ploidymap.txt \
    -genderMapFile data/installtest_gender.map \
    -copyNumberMaskFile data/human_b36_chr1.gcmask.fasta \
    -runDirectory test2 \
    -md test2/metadata \
    -L 1:61700000-61900000 \
    -reduceInsertSizeDistributions false \
    -computeGCProfiles true \
    -computeReadCounts false \
    -jobLogDir test2/logs \
    -I data/installtest.bam \
    -jobNative '-V -pe openmp 12 -q bioinf.q' \
    -l DEBUG
    -run || exit 1

  • thibaultthibault Broad InstituteMember, Broadie, Moderator, Dev admin

    Great to hear it, @Will_Gilks !

    Also a high-five to @bhandsaker for coming up with the same solution.

Sign In or Register to comment.