CNVDiscovery Errors at Stage 7 (at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.ref)

Hi Genome STRiP users,

I met errors during running CNVDiscovery (latest Genome STRiP, 2.00.1833) on an LSF 9.X system with loading Drmaa. I started an interactive session on the cluster with requesting 4 cores on one node and 4GB memory. Then I run the CNVDiscovery with the following script.

module purge
module load r/3.4.1
module load samtools/1.6
module load tabix/0.2.6
module load java/1.8.0_112
module load drmaa/1.1.1

classpath="${SV_DIR}/lib/SVToolkit.jar:${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar:${SV_DIR}/lib/gatk/Queue.jar"
svpreprocess_dir="/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess"

java -Xmx4g -cp ${classpath} \
    org.broadinstitute.gatk.queue.QCommandLine \
    -S ${SV_DIR}/qscript/discovery/cnv/CNVDiscoveryPipeline.q \
    -S ${SV_DIR}/qscript/SVQScript.q \
    -cp ${classpath} \
    -gatk ${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar \
    -configFile ${SV_DIR}/conf/genstrip_parameters.txt \
    -R /nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa \
    -I /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54.list \
    -genderMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map \
    -ploidyMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map \
    -md ${svpreprocess_dir}/md_tempdir \
    -jobLogDir logs \
    -intervalList /nas02/home/m/i/minzhi/genomestrip/gs_script/reference_chromosomes16_1-500000.list \
    -tilingWindowSize 1000 \
    -tilingWindowOverlap 500 \
    -maximumReferenceGapLength 1000 \
    -boundaryPrecision 100 \
    -minimumRefinedLength 500 \
    -jobRunner Drmaa \
    -gatkJobRunner Drmaa \
    -run \
    || exit 1

Then I met three problems in the stage 7. The first one is here.

ERROR 15:08:29,573 FunctionEdge - Error:  'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.gatk.queue.QCommandLine'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStage7.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStageBase.q' '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryGenotyper.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/SVQScript.q'  '-gatk' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar'  '-jobLogDir' 'cnv_stage7/seq_chr16/logs'  '-memLimit' '2.0'  '-jobRunner' 'Drmaa'  '-gatkJobRunner' 'Drmaa'  -run  '-sequenceName' 'chr16'  '-runDirectory' 'cnv_stage7/seq_chr16'  '-sentinelFile' 'cnv_sentinel_files/stage_7_seq_chr16.sent'  --disableJobReport  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-R' '/nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  -disableGATKTraversal  '-I' 'cnv_stage6/seq_chr16/seq_chr16.merged_headers.bam'  '-vcf' 'cnv_stage4/seq_chr16/seq_chr16.merged.genotypes.vcf.gz'  '-siteListFile' 'cnv_stage6/seq_chr16/eval/SelectedVariants.list'  '-boundaryPrecision' '100'  '-maximumReferenceGapLength' '1000'  '-minimumRefinedLength' '500'  '-brigSiteVcfFile' 'cnv_stage7/seq_chr16/seq_chr16.brig.sites.vcf.gz'  
ERROR 15:08:29,580 FunctionEdge - Contents of /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/logs/CNVDiscoveryPipeline-10.out:
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaryStep(BoundaryRefinementAlgorithm.java:548)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaries(BoundaryRefinementAlgorithm.java:536)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.processVariant(BoundaryRefinementAlgorithm.java:232)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.run(RefineCNVBoundaries.java:204)
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:54)
    ... 5 more 

The second one is just following the first above one, but is the "Exception in thread "main" java.lang.RuntimeException" error:

ERROR 15:08:23,086 FunctionEdge - Error:  'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.sv.genotyping.RefineCNVBoundaries'  '-I' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage6/seq_chr16/seq_chr16.merged_headers.bam'  '-O' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage7/seq_chr16/P0003/seq_chr16.merged.brig.vcf'  '-R' '/nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-P' 'depth.readCountCacheIgnoreGenomeMask:true'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-vcf' 'cnv_stage4/seq_chr16/seq_chr16.merged.genotypes.vcf.gz'  '-site' 'cnv_stage7/seq_chr16/P0003.sites.list'  '-boundaryPrecision' '100'  '-minimumRefinedLength' '500'  '-maximumReferenceGapLength' '1000'  
ERROR 15:08:23,088 FunctionEdge - Contents of /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage7/seq_chr16/logs/CNVDiscoveryStage7-4.out:
INFO  15:08:07,885 HelpFormatter - ------------------------------------------------------------------ 
INFO  15:08:07,888 HelpFormatter - Program Name: org.broadinstitute.sv.genotyping.RefineCNVBoundaries 
INFO  15:08:07,893 HelpFormatter - Program Args: -I /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage6/seq_chr16/seq_chr16.merged_headers.bam -O /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/cnv_stage7/seq_chr16/P0003/seq_chr16.merged.brig.vcf -R /nas02/home/m/i/minzhi/genomestrip/gs_script/hs38DH.fa -md /nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir -configFile /nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt -P depth.readCountCacheIgnoreGenomeMask:true -genderMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map -ploidyMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map -vcf cnv_stage4/seq_chr16/seq_chr16.merged.genotypes.vcf.gz -site cnv_stage7/seq_chr16/P0003.sites.list -boundaryPrecision 100 -minimumRefinedLength 500 -maximumReferenceGapLength 1000 
INFO  15:08:08,030 HelpFormatter - Executing as [email protected] on Linux 2.6.18-238.12.1.el5 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_112-b15. 
INFO  15:08:08,031 HelpFormatter - Date/Time: 2018/06/28 15:08:07 
INFO  15:08:08,031 HelpFormatter - ------------------------------------------------------------------ 
INFO  15:08:08,031 HelpFormatter - ------------------------------------------------------------------ 
INFO  15:08:08,037 RefineCNVBoundaries - Opening reference sequence ... 
INFO  15:08:08,041 RefineCNVBoundaries - Opened reference sequence. 
INFO  15:08:08,174 MetaData - Opening metadata ...  
INFO  15:08:08,175 MetaData - Adding metadata location /nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir ... 
INFO  15:08:08,191 MetaData - Opened metadata. 
INFO  15:08:08,203 RefineCNVBoundaries - Initializing input data set ... 
INFO  15:08:08,625 RefineCNVBoundaries - Initialized data set: 1 file, 409 read groups, 52 samples. 
INFO  15:08:09,163 ReadCountCache - Initializing read count cache with 1 file. 
Exception in thread "main" java.lang.RuntimeException
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:65)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
    at org.broadinstitute.sv.commandline.CommandLineProgram.runAndReturnResult(CommandLineProgram.java:29)
    at org.broadinstitute.sv.commandline.CommandLineProgram.run(CommandLineProgram.java:25)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.main(RefineCNVBoundaries.java:133)
Caused by: java.lang.NullPointerException
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.computeNextCoordinate(BoundaryRefinementAlgorithm.java:886)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.getRefinedInterval(BoundaryRefinementAlgorithm.java:704)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineOneBoundary(BoundaryRefinementAlgorithm.java:612)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaryStep(BoundaryRefinementAlgorithm.java:548)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.refineBoundaries(BoundaryRefinementAlgorithm.java:536)
    at org.broadinstitute.sv.genotyping.BoundaryRefinementAlgorithm.processVariant(BoundaryRefinementAlgorithm.java:232)
    at org.broadinstitute.sv.genotyping.RefineCNVBoundaries.run(RefineCNVBoundaries.java:204)
    at org.broadinstitute.sv.commandline.CommandLineProgram.execute(CommandLineProgram.java:54)
    ... 5 more 

And the last one is the same as the second one.

Is the first error the reason of the following errors? Is there anything wrong related to the -jobRunner and -gatkRunner flag? Are the supporting softwares such as samtools and so on are in the correct version? Or could you share with me your script? May I have your suggestions about it? Thank you in advance.

Best regards,
Wusheng

Comments

  • bhandsakerbhandsaker Member, Broadie, Moderator admin

    I suspect this is also related to not downloading and correctly using a human reference metadata bundle. See the question here:
    https://gatkforums.broadinstitute.org/gatk/discussion/12308/svpreprocess-succeeded-for-fewer-samples-but-failed-for-large-number-of-samples

  • zhangwushengzhangwusheng Member

    Hi Bob,

    Thank you very much! This works ... to some extent. With your standard reference data, I can pass the first 10 stages in CNVDiscovery. However, there is a "missing -outputDirectory argument" error at the end of the CNVDiscovery process. Details are shown below.

    Job <271505> is submitted to default queue .
    INFO  15:35:33,251 DrmaaJobRunner - Submitted job id: 271505 
    INFO  15:35:33,252 QGraph - 0 Pend, 1 Run, 0 Fail, 14 Done 
    ERROR 15:36:03,056 FunctionEdge - Error:  'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.gatk.queue.QCommandLine'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStage12.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStageBase.q' '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryGenotyper.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/SVQScript.q'  '-gatk' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar'  '-jobLogDir' 'cnv_stage12/logs'  '-memLimit' '2.0'  '-jobRunner' 'Drmaa'  '-gatkJobRunner' 'Drmaa'  -run  '-runDirectory' 'cnv_stage12'  '-sentinelFile' 'cnv_sentinel_files/stage_12.sent'  --disableJobReport  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-R' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta' '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.lcmask.fasta'  '-copyNumberMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gcmask.fasta'  '-readDepthMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.rdmask.bed'  '-genderMaskBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gendermask.bed'  '-vdjBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.vdjregions.bed'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  -disableGATKTraversal  '-filteredVcf' 'cnv_stage11/seq_chr16/seq_chr16.filtered.genotypes.vcf.gz'  '-mergedVcfFile' 'cnv_stage12/gs_cnv.genotypes.vcf.gz'    '-genderGenotypeFilterFile' 'gender_gt_filters/gender_gt_filter.txt'  '-filterDescriptionFile' 'gender_gt_filters/gender_gt_filter_descr.txt'  
    ERROR 15:36:03,060 FunctionEdge - Contents of /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/logs/CNVDiscoveryPipeline-15.out:
    INFO  15:35:41,568 QScriptManager - Compiling 4 QScripts 
    INFO  15:35:49,112 QScriptManager - Compilation complete 
    ##### ERROR --
    ##### ERROR stack trace 
    org.broadinstitute.gatk.utils.commandline.MissingArgumentException: 
    Argument with name '--outputdirectory' (-outputDirectory) is missing.
        at org.broadinstitute.gatk.utils.commandline.ParsingEngine.validate(ParsingEngine.java:300)
        at org.broadinstitute.gatk.utils.commandline.ParsingEngine.validate(ParsingEngine.java:280)
        at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:225)
        at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
        at org.broadinstitute.gatk.queue.QCommandLine$.main(QCommandLine.scala:61)
        at org.broadinstitute.gatk.queue.QCommandLine.main(QCommandLine.scala)
    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR A GATK RUNTIME ERROR has occurred (version 3.7.GS-r1748-0-g74bfe0b):
    ##### ERROR
    ##### ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
    ##### ERROR If not, please post the error message, with stack trace, to the GATK forum.
    ##### ERROR Visit our website and forum for extensive documentation and answers to 
    ##### ERROR commonly asked questions https://software.broadinstitute.org/gatk
    ##### ERROR
    ##### ERROR MESSAGE: Argument with name '--outputdirectory' (-outputDirectory) is missing.
    ##### ERROR ------------------------------------------------------------------------------------------
    INFO  15:35:49,272 QCommandLine - Shutting down jobs. Please wait...  
    INFO  15:36:03,060 QGraph - Writing incremental jobs reports... 
    INFO  15:36:03,060 QJobsReporter - Writing JobLogging GATKReport to file /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/CNVDiscoveryPipeline.jobreport.txt 
    INFO  15:36:03,067 QGraph - 0 Pend, 0 Run, 1 Fail, 14 Done 
    INFO  15:36:03,068 QCommandLine - Writing final jobs report... 
    INFO  15:36:03,069 QJobsReporter - Writing JobLogging GATKReport to file /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/CNVDiscoveryPipeline.jobreport.txt 
    INFO  15:36:03,074 QJobsReporter - Plotting JobLogging GATKReport to file /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/CNVDiscoveryPipeline.jobreport.pdf 
    INFO  15:36:10,058 QCommandLine - Done with errors 
    INFO  15:36:10,059 QGraph - ------- 
    INFO  15:36:10,060 QGraph - Failed:   'java'  '-Xmx2048m'  '-XX:+UseParallelOldGC'  '-XX:ParallelGCThreads=4'  '-XX:GCTimeLimit=50'  '-XX:GCHeapFreeLimit=10'  '-Djava.io.tmpdir=/nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/.queue/tmp'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  'org.broadinstitute.gatk.queue.QCommandLine'  '-cp' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/SVToolkit.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar:/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/Queue.jar'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStage12.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryStageBase.q' '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/discovery/cnv/CNVDiscoveryGenotyper.q'  '-S' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/qscript/SVQScript.q'  '-gatk' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/lib/gatk/GenomeAnalysisTK.jar'  '-jobLogDir' 'cnv_stage12/logs'  '-memLimit' '2.0'  '-jobRunner' 'Drmaa'  '-gatkJobRunner' 'Drmaa'  -run  '-runDirectory' 'cnv_stage12'  '-sentinelFile' 'cnv_sentinel_files/stage_12.sent'  --disableJobReport  '-configFile' '/nas02/home/m/i/minzhi/genomestrip/svtoolkit/conf/genstrip_parameters.txt'  '-R' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta'  '-ploidyMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map'  '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.svmask.fasta' '-genomeMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.lcmask.fasta'  '-copyNumberMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gcmask.fasta'  '-readDepthMaskFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.rdmask.bed'  '-genderMaskBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.gendermask.bed'  '-vdjBedFile' '/nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.vdjregions.bed'  '-genderMapFile' '/nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map'  '-md' '/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess/md_tempdir'  -disableGATKTraversal  '-filteredVcf' 'cnv_stage11/seq_chr16/seq_chr16.filtered.genotypes.vcf.gz'  '-mergedVcfFile' 'cnv_stage12/gs_cnv.genotypes.vcf.gz'    '-genderGenotypeFilterFile' 'gender_gt_filters/gender_gt_filter.txt'  '-filterDescriptionFile' 'gender_gt_filters/gender_gt_filter_descr.txt'  
    INFO  15:36:10,060 QGraph - Log:     /nas02/home/m/i/minzhi/genomestrip/gs_test_svcnvdiscovery/logs/CNVDiscoveryPipeline-15.out 
    INFO  15:36:10,061 QCommandLine - Script failed: 0 Pend, 0 Run, 1 Fail, 14 Done 
    ------------------------------------------------------------------------------------------
    Done. There were no warn messages.
    ------------------------------------------------------------------------------------------
    

    However, there is even no such argument all over all pipelines of Genome STRiP. May I have your suggestion about this argument or about this error please?

    Thank you in advance.

    Best regards,
    Wusheng

    For your reference, I am running CNVDiscovery on an LSF 9.X system. I started and interactive session as below

    bsub -q day -Is -n 4 -W 5:50 -R "rusage[mem=4096]" /bin/bash
    

    And then running the script as below.

    module purge
    module load r/3.4.1
    module load samtools/1.6
    module load tabix/0.2.6
    module load java/1.8.0_112
    module load drmaa/1.1.1
    
    classpath="${SV_DIR}/lib/SVToolkit.jar:${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar:${SV_DIR}/lib/gatk/Queue.jar"
    svpreprocess_dir="/nas02/home/m/i/minzhi/genomestrip/gs_test_svpreprocess"
    
    java -Xmx4g -cp ${classpath} \
        org.broadinstitute.gatk.queue.QCommandLine \
        -S ${SV_DIR}/qscript/discovery/cnv/CNVDiscoveryPipeline.q \
        -S ${SV_DIR}/qscript/SVQScript.q \
        -cp ${classpath} \
        -gatk ${SV_DIR}/lib/gatk/GenomeAnalysisTK.jar \
        -configFile ${SV_DIR}/conf/genstrip_parameters.txt \
        -R /nas02/home/m/i/minzhi/genomestrip/Homo_sapiens_assembly38/Homo_sapiens_assembly38.fasta \
        -I /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54.list \
        -genderMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/JHS_54_all_male_gender.map \
        -ploidyMapFile /nas02/home/m/i/minzhi/genomestrip/gs_script/standard_ploidy.map \
        -md ${svpreprocess_dir}/md_tempdir \
        -jobLogDir logs \
        -intervalList /nas02/home/m/i/minzhi/genomestrip/gs_script/reference_chromosomes16_1-500000.list \
        -tilingWindowSize 1000 \
        -tilingWindowOverlap 500 \
        -maximumReferenceGapLength 1000 \
        -boundaryPrecision 100 \
        -minimumRefinedLength 500 \
        -jobRunner Drmaa \
        -gatkJobRunner Drmaa \
        -run \
        || exit 1
    
  • bhandsakerbhandsaker Member, Broadie, Moderator admin

    I believe this is because you didn't specify -runDirectory <dir>. The documentation says this is a required parameter, although the code doesn't explicitly enforce this (but instead fails in stage 12 with this admittedly unhelpful error message). I believe if you use -runDirectory . you should be able to continue processing from where you left off.

  • zhangwushengzhangwusheng Member

    Hi Bob,

    That really works. Now to the 54 samples case, I can finish all the CNVDiscovery stages on an LSF 9.X cluster with "0 fail". And I will move on to the 3418 samples case. Thank you very much.

    Best regards,
    Wusheng

Sign In or Register to comment.