Holiday Notice:
The Frontline Support team will be slow to respond December 17-18 due to an institute-wide retreat and offline December 22- January 1, while the institute is closed. Thank you for your patience during these next few weeks. Happy Holidays!

GenomicsDBImport

SystemSystem Administrator admin
This discussion was created from comments split from: New to the forum? Ask your questions here!.

Comments

  • jparker4jparker4 Member

    Hi, I'm having a problem with GenomicsDBImport that I have no idea how to solve. The error in question is: A USER ERROR has occurred: Failed to create reader from file:///mnt/fastdata/mbp15jdp/GTEx/SRA-private/SCAPT_gVCF/Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz

    And the following is the statment I am running:
    gatk --java-options -Xmx10G GenomicsDBImport -V Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4565-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4996-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4005-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4435-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4643-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4839-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4434-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4913-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4274-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-5025-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4262-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-5037-NPJ8.g.vcf.gz -V --genomicsdb-workspace-path VCFdatabase.dir --intervals chr19

    Any help would be much appreciated.

  • bhanuGandhambhanuGandham Member, Administrator, Broadie, Moderator admin
    edited December 3

    HI @jparker4

    Which version of gatk are you using? Are you running this on the firecloud/NFS system? Also Would you please post the entire error log?

    Regards
    Bhanu

    Post edited by bhanuGandham on
  • jparker4jparker4 Member
    My GATK version is: v4.0.11.0, I'm not running it on the firecloud/NFS system, but as part of a custom build variant calling pipeline.

    The entire error log is (I have pared down the full list of files submitted in the interest of readability):

    'exceptions.OSError(---------------------------------------
    Child was terminated by signal 2:
    The stderr was:
    Using GATK jar /shared/sudlab1/General/apps/bio/gatk-4.0.11.0/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar
    Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx10G -jar /shared/sudlab1/General/apps/bio/gatk-4.0.11.0/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar GenomicsDBImport -V Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4565-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4996-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4005-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4435-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4643-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4839-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4434-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4913-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4274-NPJ8.g.vcf.gz --genomicsdb-workspace-path VCFdatabase.dir --intervals chr19
    19:06:05.236 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/shared/sudlab1/General/apps/bio/gatk-4.0.11.0/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    19:06:07.513 INFO GenomicsDBImport - ------------------------------------------------------------
    19:06:07.513 INFO GenomicsDBImport - The Genome Analysis Toolkit (GATK) v4.0.11.0
    19:06:07.514 INFO GenomicsDBImport - Executing as [email protected] on Linux v3.10.0-862.14.4.el7.x86_64 amd64
    19:06:07.514 INFO GenomicsDBImport - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_102-b14
    19:06:07.514 INFO GenomicsDBImport - Start Date/Time: 29 November 2018 19:06:05 GMT
    19:06:07.515 INFO GenomicsDBImport - ------------------------------------------------------------
    19:06:07.515 INFO GenomicsDBImport - ------------------------------------------------------------
    19:06:07.515 INFO GenomicsDBImport - HTSJDK Version: 2.16.1
    19:06:07.515 INFO GenomicsDBImport - Picard Version: 2.18.13
    19:06:07.515 INFO GenomicsDBImport - HTSJDK Defaults.COMPRESSION_LEVEL : 2
    19:06:07.516 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    19:06:07.516 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    19:06:07.516 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    19:06:07.516 INFO GenomicsDBImport - Deflater: IntelDeflater
    19:06:07.516 INFO GenomicsDBImport - Inflater: IntelInflater
    19:06:07.516 INFO GenomicsDBImport - GCS max retries/reopens: 20
    19:06:07.516 INFO GenomicsDBImport - Requester pays: disabled
    19:06:07.516 INFO GenomicsDBImport - Initializing engine
    19:06:07.613 INFO GenomicsDBImport - Shutting down engine
    [29 November 2018 19:06:07 GMT] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 0.04 minutes.
    Runtime.totalMemory()=1171783680
    ***********************************************************************

    A USER ERROR has occurred: Failed to create reader from file:///mnt/fastdata/mbp15jdp/GTEx/SRA-private/SCAPT_gVCF/Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz

    ***********************************************************************
    Set the system property GATK_STACKTRACE_ON_USER_EXCEPTION (--java-options '-DGATK_STACKTRACE_ON_USER_EXCEPTION=true') to print the stack trace.

    gatk --java-options -Xmx10G GenomicsDBImport -V Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4565-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4996-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4005-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4435-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4643-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4839-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4434-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4913-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4274-NPJ8.g.vcf.gz --genomicsdb-workspace-path VCFdatabase.dir --intervals chr19
    -----------------------------------------)' raised in ...
    Task = def genomicdbimport(...):
    Job = [[Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz, Variantcalls.dir/Frontal_Cortex-4565-NPJ8.g.vcf.gz, Variantcalls.dir/Frontal_Cortex-4996-NPJ8.g.vcf.gz, Variantcalls.dir/Frontal_Cortex-4005-NPJ8.g.vcf.gz, Variantcalls.dir/Frontal_Cortex-4435-NPJ8.g.vcf.gz, Variantcalls.dir/Cerebellar_Hemisphere-4643-NPJ8.g.vcf.gz, Variantcalls.dir/Cerebellar_Hemisphere-4839-NPJ8.g.vcf.gz, Variantcalls.dir/Frontal_Cortex-4434-NPJ8.g.vcf.gz, Variantcalls.dir/Frontal_Cortex-4913-NPJ8.g.vcf.gz, Variantcalls.dir/Cerebellar_Hemisphere-4274-NPJ8.g.vcf.gz] -> chr19, NPJ8_chr19.database]

    Traceback (most recent call last):
    File "/home/mbp15jdp/.conda/envs/sharc/lib/python2.7/site-packages/ruffus/task.py", line 751, in run_pooled_job_without_exceptions
    register_cleanup, touch_files_only)
    File "/home/mbp15jdp/.conda/envs/sharc/lib/python2.7/site-packages/ruffus/task.py", line 567, in job_wrapper_io_files
    ret_val = user_defined_work_func(*params)
    File "/home/mbp15jdp/devel/pipeline_gVCF/pipeline_gVCF.py", line 331, in genomicdbimport
    P.run()
    File "/home/mbp15jdp/devel/CGATPipelines/CGATPipelines/Pipeline/Execution.py", line 578, in run
    ignore_errors=ignore_errors)
    File "/home/mbp15jdp/devel/CGATPipelines/CGATPipelines/Pipeline/Cluster.py", line 350, in collectSingleJobFromCluster
    "".join(stderr), statement))
    OSError: ---------------------------------------
    Child was terminated by signal 2:
    The stderr was:
    Using GATK jar /shared/sudlab1/General/apps/bio/gatk-4.0.11.0/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar
    Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx10G -jar /shared/sudlab1/General/apps/bio/gatk-4.0.11.0/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar GenomicsDBImport -V Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4565-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4996-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4005-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4435-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4643-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4839-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4434-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4913-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4274-NPJ8.g.vcf.gz --genomicsdb-workspace-path VCFdatabase.dir --intervals chr19

    19:06:05.236 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/shared/sudlab1/General/apps/bio/gatk-4.0.11.0/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    19:06:07.513 INFO GenomicsDBImport - ------------------------------------------------------------
    19:06:07.513 INFO GenomicsDBImport - The Genome Analysis Toolkit (GATK) v4.0.11.0
    19:06:07.514 INFO GenomicsDBImport - Executing as [email protected] on Linux v3.10.0-862.14.4.el7.x86_64 amd64
    19:06:07.514 INFO GenomicsDBImport - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_102-b14
    19:06:07.514 INFO GenomicsDBImport - Start Date/Time: 29 November 2018 19:06:05 GMT
    19:06:07.515 INFO GenomicsDBImport - ------------------------------------------------------------
    19:06:07.515 INFO GenomicsDBImport - ------------------------------------------------------------
    19:06:07.515 INFO GenomicsDBImport - HTSJDK Version: 2.16.1
    19:06:07.515 INFO GenomicsDBImport - Picard Version: 2.18.13
    19:06:07.515 INFO GenomicsDBImport - HTSJDK Defaults.COMPRESSION_LEVEL : 2
    19:06:07.516 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    19:06:07.516 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    19:06:07.516 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    19:06:07.516 INFO GenomicsDBImport - Deflater: IntelDeflater
    19:06:07.516 INFO GenomicsDBImport - Inflater: IntelInflater
    19:06:07.516 INFO GenomicsDBImport - GCS max retries/reopens: 20
    19:06:07.516 INFO GenomicsDBImport - Requester pays: disabled
    19:06:07.516 INFO GenomicsDBImport - Initializing engine
    19:06:07.613 INFO GenomicsDBImport - Shutting down engine
    [29 November 2018 19:06:07 GMT] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 0.04 minutes.
    Runtime.totalMemory()=1171783680
    ***********************************************************************

    A USER ERROR has occurred: Failed to create reader from file:///mnt/fastdata/mbp15jdp/GTEx/SRA-private/SCAPT_gVCF/Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz

    ***********************************************************************
    Set the system property GATK_STACKTRACE_ON_USER_EXCEPTION (--java-options '-DGATK_STACKTRACE_ON_USER_EXCEPTION=true') to print the stack trace.

    gatk --java-options -Xmx10G GenomicsDBImport -V Variantcalls.dir/Frontal_Cortex-5074-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4565-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4996-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4005-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4435-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4643-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4839-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4434-NPJ8.g.vcf.gz -V Variantcalls.dir/Frontal_Cortex-4913-NPJ8.g.vcf.gz -V Variantcalls.dir/Cerebellar_Hemisphere-4274-NPJ8.g.vcf.gz --genomicsdb-workspace-path VCFdatabase.dir --intervals chr19
    -----------------------------------------
  • bhanuGandhambhanuGandham Member, Administrator, Broadie, Moderator admin

    HI @jparker4

    Here are a couple of things you can do.
    1) As mentioned in the error log run --java-options '-DGATK_STACKTRACE_ON_USER_EXCEPTION=true' and send us the stack trace to debug this issue.

    2) Try to run SelectVariants with the input vcf to see if this issue is with the vcf or genomicsdbimport.

    Please send us your findings once you have done this.

    Regards
    Bhanu

  • bhanuGandhambhanuGandham Member, Administrator, Broadie, Moderator admin

    Hi @jparker4

    As we have not heard back from you in more than 2 business days, we are closing this issue for now. Please get back to us if you have more questions.

Sign In or Register to comment.