Attention:
The frontline support team will be unavailable to answer questions on April 15th and 17th 2019. We will be back soon after. Thank you for your patience and we apologize for any inconvenience!

CNVDiscovery Errors at Stage 7 (at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.ref)

Hi Genome STRiP users,

I met errors during running CNVDiscovery (latest Genome STRiP, 2.00.1833) on an LSF 9.X system with loading Drmaa. I started an interactive session on the cluster with requesting 4 cores on one node and 4GB memory. Then I run the CNVDiscovery with the following script.

module purge
module load r/3.4.1
module load samtools/1.6
module load tabix/0.2.6
module load java/1.8.0_112
module load drmaa/1.1.1

classpath="${SV_DIR}/lib/SVToolkit.jar:${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar:${SV_DIR}/lib/gatk/Queue.jar"
svpreprocess_dir="/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess"

java -Xmx4g -cp ${classpath} \
    org.broadinstitute.gatk.queue.QCommandLine \
    -S ${SV_DIR}/qscript/discovery/cnv/CNVDiscoveryPipeline.q \
    -S ${SV_DIR}/qscript/SVQScript.q \
    -cp ${classpath} \
    -gatk ${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar \
    -configFile ${SV_DIR}/conf/genstrip_parameters.txt \
    -R /nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa \
    -I /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54.list \
    -genderMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map \
    -ploidyMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map \
    -md ${svpreprocess_dir}/md_tempdir \
    -jobLogDir logs \
    -intervalList /nas02/home/m/i/minzhi/genomestrip/gs_script/reference_chromosomes16_1-500000.list \
    -tilingWindowSize 1000 \
    -tilingWindowOverlap 500 \
    -maximumReferenceGapLength 1000 \
    -boundaryPrecision 100 \
    -minimumRefinedLength 500 \
    -jobRunner Drmaa \
    -gatkJobRunner Drmaa \
    -run \
    || exit 1

Then I met three problems in the stage 7. The first one is here.

ERROR 15:08:29,573 FunctionEdge - Error:  'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.gatk.queue.QCommandLine'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStage7.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStageBase.q' '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryGenotyper.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/SVQScript.q'  '-gatk' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar'  '-jobLogDir' 'cnv_stage7/seq_chr16/logs'  '-memLimit' '2.0'  '-jobRunner' 'Drmaa'  '-gatkJobRunner' 'Drmaa'  -run  '-sequenceName' 'chr16'  '-runDirectory' 'cnv_stage7/seq_chr16'  '-sentinelFile' 'cnv_sentinel_files/stage_7_seq_chr16.sent'  --disableJobReport  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-R' '/nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  -disableGATKTraversal  '-I' 'cnv_stage6/seq_chr16/seq_chr16.merged_headers.bam'  '-vcf' 'cnv_stage4/seq_chr16/seq_chr16.merged.genotypes.vcf.gz'  '-siteListFile' 'cnv_stage6/seq_chr16/eval/SelectedVariants.list'  '-boundaryPrecision' '100'  '-maximumReferenceGapLength' '1000'  '-minimumRefinedLength' '500'  '-brigSiteVcfFile' 'cnv_stage7/seq_chr16/seq_chr16.brig.sites.vcf.gz'  
ERROR 15:08:29,580 FunctionEdge - Contents of /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/logs/CNVDiscoveryPipeline-10.out:
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaryStep(BoundaryRefinementAlgorithm.java:548)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaries(BoundaryRefinementAlgorithm.java:536)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.processVariant(BoundaryRefinementAlgorithm.java:232)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.run(RefineCNVBoundaries.java:204)
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:54)
    ... 5 more 

The second one is just following the first above one, but is the "Exception in thread "main" java.lang.RuntimeException" error:

ERROR 15:08:23,086 FunctionEdge - Error:  'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.sv.genotyping.RefineCNVBoundaries'  '-I' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage6/seq_chr16/seq_chr16.merged_headers.bam'  '-O' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage7/seq_chr16/P0003/seq_chr16.merged.brig.vcf'  '-R' '/nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-P' 'depth.readCountCacheIgnoreGenomeMask:true'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-vcf' 'cnv_stage4/seq_chr16/seq_chr16.merged.genotypes.vcf.gz'  '-site' 'cnv_stage7/seq_chr16/P0003.sites.list'  '-boundaryPrecision' '100'  '-minimumRefinedLength' '500'  '-maximumReferenceGapLength' '1000'  
ERROR 15:08:23,088 FunctionEdge - Contents of /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage7/seq_chr16/logs/CNVDiscoveryStage7-4.out:
INFO  15:08:07,885 HelpFormatter - ------------------------------------------------------------------ 
INFO  15:08:07,888 HelpFormatter - Program Name: org.broadinstitute.sv.genotyping.RefineCNVBoundaries 
INFO  15:08:07,893 HelpFormatter - Program Args: -I /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage6/seq_chr16/seq_chr16.merged_headers.bam -O /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage7/seq_chr16/P0003/seq_chr16.merged.brig.vcf -R /nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa -md /nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir -configFile /nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt -P depth.readCountCacheIgnoreGenomeMask:true -genderMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map -ploidyMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map -vcf cnv_stage4/seq_chr16/seq_chr16.merged.genotypes.vcf.gz -site cnv_stage7/seq_chr16/P0003.sites.list -boundaryPrecision 100 -minimumRefinedLength 500 -maximumReferenceGapLength 1000 
INFO  15:08:08,030 HelpFormatter - Executing as [email protected] on Linux 2.6.18-238.12.1.el5 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_112-b15. 
INFO  15:08:08,031 HelpFormatter - Date/Time: 2018/06/28 15:08:07 
INFO  15:08:08,031 HelpFormatter - ------------------------------------------------------------------ 
INFO  15:08:08,031 HelpFormatter - ------------------------------------------------------------------ 
INFO  15:08:08,037 RefineCNVBoundaries - Opening reference sequence ... 
INFO  15:08:08,041 RefineCNVBoundaries - Opened reference sequence. 
INFO  15:08:08,174 MetaData - Opening metadata ...  
INFO  15:08:08,175 MetaData - Adding metadata location /nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir ... 
INFO  15:08:08,191 MetaData - Opened metadata. 
INFO  15:08:08,203 RefineCNVBoundaries - Initializing input data set ... 
INFO  15:08:08,625 RefineCNVBoundaries - Initialized data set: 1 file, 409 read groups, 52 samples. 
INFO  15:08:09,163 ReadCountCache - Initializing read count cache with 1 file. 
Exception in thread "main" java.lang.RuntimeException
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:65)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
    at org.broadinstitute.sv.commandline.CommandLineProgram.runAndReturnResult(CommandLineProgram.java:29)
    at org.broadinstitute.sv.commandline.CommandLineProgram.run(CommandLineProgram.java:25)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.main(RefineCNVBoundaries.java:133)
Caused by: java.lang.NullPointerException
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.computeNextCoordinate(BoundaryRefinementAlgorithm.java:886)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.getRefinedInterval(BoundaryRefinementAlgorithm.java:704)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineOneBoundary(BoundaryRefinementAlgorithm.java:612)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaryStep(BoundaryRefinementAlgorithm.java:548)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaries(BoundaryRefinementAlgorithm.java:536)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.processVariant(BoundaryRefinementAlgorithm.java:232)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.run(RefineCNVBoundaries.java:204)
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:54)
    ... 5 more 

And the last one is the same as the second one.

Is the first error the reason of the following errors? Is there anything wrong related to the -jobRunner and -gatkRunner flag? Are the supporting softwares such as samtools and so on are in the correct version? Or could you share with me your script? May I have your suggestions about it? Thank you in advance.

Best regards,
Wusheng

Comments

  • bhandsakerbhandsaker Member, Broadie, Moderator admin

    I suspect this is also related to not downloading and correctly using a human reference metadata bundle. See the question here:
    https://gatkforums.broadinstitute.org/gatk/discussion/12308/svpreprocess-succeeded-for-fewer-samples-but-failed-for-large-number-of-samples

  • zhangwushengzhangwusheng Member

    Hi Bob,

    Thank you very much! This works ... to some extent. With your standard reference data, I can pass the first 10 stages in CNVDiscovery. However, there is a "missing -outputDirectory argument" error at the end of the CNVDiscovery process. Details are shown below.

    Job <271505> is submitted to default queue .
    INFO  15:35:33,251 DrmaaJobRunner - Submitted job id: 271505 
    INFO  15:35:33,252 QGraph - 0 Pend, 1 Run, 0 Fail, 14 Done 
    ERROR 15:36:03,056 FunctionEdge - Error:  'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.gatk.queue.QCommandLine'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStage12.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStageBase.q' '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryGenotyper.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/SVQScript.q'  '-gatk' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar'  '-jobLogDir' 'cnv_stage12/logs'  '-memLimit' '2.0'  '-jobRunner' 'Drmaa'  '-gatkJobRunner' 'Drmaa'  -run  '-runDirectory' 'cnv_stage12'  '-sentinelFile' 'cnv_sentinel_files/stage_12.sent'  --disableJobReport  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-R' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta' '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.lcmask.fasta'  '-copyNumberMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gcmask.fasta'  '-readDepthMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.rdmask.bed'  '-genderMaskBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gendermask.bed'  '-vdjBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.vdjregions.bed'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  -disableGATKTraversal  '-filteredVcf' 'cnv_stage11/seq_chr16/seq_chr16.filtered.genotypes.vcf.gz'  '-mergedVcfFile' 'cnv_stage12/gs_cnv.genotypes.vcf.gz'    '-genderGenotypeFilterFile' 'gender_gt_filters/gender_gt_filter.txt'  '-filterDescriptionFile' 'gender_gt_filters/gender_gt_filter_descr.txt'  
    ERROR 15:36:03,060 FunctionEdge - Contents of /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/logs/CNVDiscoveryPipeline-15.out:
    INFO  15:35:41,568 QScriptManager - Compiling 4 QScripts 
    INFO  15:35:49,112 QScriptManager - Compilation complete 
    ##### ERROR --
    ##### ERROR stack trace 
    org.broadinstitute.gatk.utils.commandline.MissingArgumentException: 
    Argument with name '--outputdirectory' (-outputDirectory) is missing.
        at org.broadinstitute.gatk.utils.commandline.ParsingEngine.validate(ParsingEngine.java:300)
        at org.broadinstitute.gatk.utils.commandline.ParsingEngine.validate(ParsingEngine.java:280)
        at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:225)
        at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
        at org.broadinstitute.gatk.queue.QCommandLine$.main(QCommandLine.scala:61)
        at org.broadinstitute.gatk.queue.QCommandLine.main(QCommandLine.scala)
    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR A GATK RUNTIME ERROR has occurred (version 3.7.GS-r1748-0-g74bfe0b):
    ##### ERROR
    ##### ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
    ##### ERROR If not, please post the error message, with stack trace, to the GATK forum.
    ##### ERROR Visit our website and forum for extensive documentation and answers to 
    ##### ERROR commonly asked questions https://software.broadinstitute.org/gatk
    ##### ERROR
    ##### ERROR MESSAGE: Argument with name '--outputdirectory' (-outputDirectory) is missing.
    ##### ERROR ------------------------------------------------------------------------------------------
    INFO  15:35:49,272 QCommandLine - Shutting down jobs. Please wait...  
    INFO  15:36:03,060 QGraph - Writing incremental jobs reports... 
    INFO  15:36:03,060 QJobsReporter - Writing JobLogging GATKReport to file /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/CNVDiscoveryPipeline.jobreport.txt 
    INFO  15:36:03,067 QGraph - 0 Pend, 0 Run, 1 Fail, 14 Done 
    INFO  15:36:03,068 QCommandLine - Writing final jobs report... 
    INFO  15:36:03,069 QJobsReporter - Writing JobLogging GATKReport to file /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/CNVDiscoveryPipeline.jobreport.txt 
    INFO  15:36:03,074 QJobsReporter - Plotting JobLogging GATKReport to file /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/CNVDiscoveryPipeline.jobreport.pdf 
    INFO  15:36:10,058 QCommandLine - Done with errors 
    INFO  15:36:10,059 QGraph - ------- 
    INFO  15:36:10,060 QGraph - Failed:   'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.gatk.queue.QCommandLine'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStage12.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStageBase.q' '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryGenotyper.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/SVQScript.q'  '-gatk' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar'  '-jobLogDir' 'cnv_stage12/logs'  '-memLimit' '2.0'  '-jobRunner' 'Drmaa'  '-gatkJobRunner' 'Drmaa'  -run  '-runDirectory' 'cnv_stage12'  '-sentinelFile' 'cnv_sentinel_files/stage_12.sent'  --disableJobReport  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-R' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta' '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.lcmask.fasta'  '-copyNumberMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gcmask.fasta'  '-readDepthMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.rdmask.bed'  '-genderMaskBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gendermask.bed'  '-vdjBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.vdjregions.bed'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  -disableGATKTraversal  '-filteredVcf' 'cnv_stage11/seq_chr16/seq_chr16.filtered.genotypes.vcf.gz'  '-mergedVcfFile' 'cnv_stage12/gs_cnv.genotypes.vcf.gz'    '-genderGenotypeFilterFile' 'gender_gt_filters/gender_gt_filter.txt'  '-filterDescriptionFile' 'gender_gt_filters/gender_gt_filter_descr.txt'  
    INFO  15:36:10,060 QGraph - Log:     /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/logs/CNVDiscoveryPipeline-15.out 
    INFO  15:36:10,061 QCommandLine - Script failed: 0 Pend, 0 Run, 1 Fail, 14 Done 
    ------------------------------------------------------------------------------------------
    Done. There were no warn messages.
    ------------------------------------------------------------------------------------------
    

    However, there is even no such argument all over all pipelines of Genome STRiP. May I have your suggestion about this argument or about this error please?

    Thank you in advance.

    Best regards,
    Wusheng

    For your reference, I am running CNVDiscovery on an LSF 9.X system. I started and interactive session as below

    bsub -q day -Is -n 4 -W 5:50 -R "rusage[mem=4096]" /bin/bash
    

    And then running the script as below.

    module purge
    module load r/3.4.1
    module load samtools/1.6
    module load tabix/0.2.6
    module load java/1.8.0_112
    module load drmaa/1.1.1
    
    classpath="${SV_DIR}/lib/SVToolkit.jar:${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar:${SV_DIR}/lib/gatk/Queue.jar"
    svpreprocess_dir="/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess"
    
    java -Xmx4g -cp ${classpath} \
        org.broadinstitute.gatk.queue.QCommandLine \
        -S ${SV_DIR}/qscript/discovery/cnv/CNVDiscoveryPipeline.q \
        -S ${SV_DIR}/qscript/SVQScript.q \
        -cp ${classpath} \
        -gatk ${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar \
        -configFile ${SV_DIR}/conf/genstrip_parameters.txt \
        -R /nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta \
        -I /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54.list \
        -genderMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map \
        -ploidyMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map \
        -md ${svpreprocess_dir}/md_tempdir \
        -jobLogDir logs \
        -intervalList /nas02/home/m/i/minzhi/genomestrip/gs_script/reference_chromosomes16_1-500000.list \
        -tilingWindowSize 1000 \
        -tilingWindowOverlap 500 \
        -maximumReferenceGapLength 1000 \
        -boundaryPrecision 100 \
        -minimumRefinedLength 500 \
        -jobRunner Drmaa \
        -gatkJobRunner Drmaa \
        -run \
        || exit 1
    
  • bhandsakerbhandsaker Member, Broadie, Moderator admin

    I believe this is because you didn't specify -runDirectory <dir>. The documentation says this is a required parameter, although the code doesn't explicitly enforce this (but instead fails in stage 12 with this admittedly unhelpful error message). I believe if you use -runDirectory . you should be able to continue processing from where you left off.

  • zhangwushengzhangwusheng Member

    Hi Bob,

    That really works. Now to the 54 samples case, I can finish all the CNVDiscovery stages on an LSF 9.X cluster with "0 fail". And I will move on to the 3418 samples case. Thank you very much.

    Best regards,
    Wusheng

Sign In or Register to comment.