Heads up:
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!
Test-drive the GATK tools and Best Practices pipelines on Terra
Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
RealignerTargetCreator hangs
alexpe
GermanyMember ✭
Hi GATK team!
we have an issue with running the RealignerTargetCreator unfortunately. Commandline looks like this:
gatk -T RealignerTargetCreator -R ref.fasta -I /testsample.sorted.bam -nt 32 -o /testsample.intervals INFO 13:00:59,111 HelpFormatter - --------------------------------------------------------------------------------------------- INFO 13:00:59,141 HelpFormatter - The Genome Analysis Toolkit (GATK) vnightly-2017-07-11-g1f763d5, Compiled 2017/07/11 00:01:14 INFO 13:00:59,141 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute INFO 13:00:59,142 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk INFO 13:00:59,142 HelpFormatter - [Thu Jul 20 13:00:58 UTC 2017] Executing on Linux 3.10.0-327.3.1.el7.x86_64 amd64 INFO 13:00:59,142 HelpFormatter - Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 INFO 13:00:59,170 HelpFormatter - Program Args: -T RealignerTargetCreator -R ref.fasta -I /testsample.sorted.bam -nt 32 -o /testsample.intervals INFO 13:00:59,226 HelpFormatter - Executing as user on Linux 3.10.0-327.3.1.el7.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11. INFO 13:00:59,227 HelpFormatter - Date/Time: 2017/07/20 13:00:59 INFO 13:00:59,227 HelpFormatter - --------------------------------------------------------------------------------------------- INFO 13:00:59,228 HelpFormatter - --------------------------------------------------------------------------------------------- ERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/opt/gatk/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console…
After this, the application unfortunately hangs. Running this with GATK v3.7 stable is also not working, we had issues with the bug in HaplotypeCallers VectorHMM library. Any ideas what we can do?
Post edited by shlee on
Best Answer
-
Geraldine_VdAuwera
Cambridge, MA admin
Sorry for the delay everyone -- we had some technical issues. The fix is now in, and should be available in the nightly build starting tomorrow. Thanks for your patience!
Answers
For one, @alexpe, you can try removing the
-ntand run the command with a single thread.I did, same error (well, not an error - but the application freezes).
Validated the BAM file with Picard and it seems ok - no errors on the file front here. Mapping with the reference genome worked as well, so things should be fine here, too.
Hi @alexpe,
Given the error message:
It appears something is wonky with your build. The Log4j2 is an Apache package. The error message implies there is an alternative logging function,
SimpleLogger, that should enable the run to continue but that's where you say the run hangs.I just tested my v3.7 jar on a small BAM and RealignerTargetCreator and it iterates over the contigs. I suggest you test out a fresh prebuilt jar or try building your jar from source again. Another thing is that although this tool shouldn't require all that much memory, depending on the depth of coverage in your BAM, you may want to specify memory usage, e.g.
-Xmx16g.Hi!
This problem is also on the most recent V3.8. I can see a discussion on solving this in the github repo 10 days ago (https://github.com/broadinstitute/picard/issues/883), but don't know how to apply it to my case. Would you kindly assist?
I'm also not able to find links to jar files of previous versions (I know this wasn't an issue with V3.6)
Thank you,
Azza
Issue · Github
by Sheila
@alexpe @aeahmed
Hi,
I think there is a fix in progress for this issue. I am checking with the team now, and will get back to you.
-Sheila
Tried with GATK v3.8 and it still doesn't work - hangs in RealignerTargetCreator....
@alexpe,
If you try this on a small subset of your BAM using the
-Loption, does it still hang?@alexpe Are you running on CentOS by any chance? If so we do indeed have a fix for that which will be merged tonight if all goes well.
I tried out Singularity (containers, the container is actually Archlinux) but have to check what is used on the cluster here. Could as well be CentOS - will check back once I know.
Its definitely running on CentOS 7, maybe thats a possible fix then? Is there a bugreport for that case to figure out what went wrong?
Using CentOS 7.2.1511, We are having the same issue that RealignerTargetCreator hangs. Any solutions yet?
We have a fix for the hanging issue but we're seeing unexpected test failures, which we're investigating now. Stay tuned...
Sorry for the delay everyone -- we had some technical issues. The fix is now in, and should be available in the nightly build starting tomorrow. Thanks for your patience!
Hi, I still having the same issue, Can you tell me where can i get the nightly build file which fixed the issues.Thanks!
I'm still having this issue even using the nightly build from 25/08/2017. The exact error is:
StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/ionng/Software/NGS_Software/GATK_v3_8_250817/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties
The issue that was fixed was the program hanging on CentOS. The logger issue will be fixed separately. We also have a fix for that, it's in review.
@wxhyih the nightly builds are available from the Downloads section.
Dear @Geraldine_VdAuwera
I was directed to this thread as I am also having the problem with StatusLogger in HaplotypeCaller and GenotypeGVCF (GATK 3.8).
I wonder if this issue is related to the large memory requirements I am having for my runs (posted here.
If so, I wonder if there is an scheduled plan to fix it in the next days. In case the fix may take weeks, is there any way to go around it? I am in need of getting results ASAP...
Thanks for any help!
@apfuentes There was a problem with the proposed fix but I think there is a new fix that may be merged tomorrow, at which point it would be available from the nightly builds. Sorry for the inconvenience!
Hi everyone, I am having the same problem when I try to use PrintReads function and i've already updated to the lastest nightly build, but nothing has been fixed
INFO 08:43:33,025 HelpFormatter - ---------------------------------------------------------------------------------------------
ERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/home/user/software/gatknightly/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
INFO 08:43:33,088 GenomeAnalysisEngine - Deflater: IntelDeflater
INFO 08:43:33,089 GenomeAnalysisEngine - Inflater: IntelInflater
INFO 08:43:33,090 GenomeAnalysisEngine - Strictness is SILENT
ERROR ------------------------------------------------------------------------------------------
ERROR A USER ERROR has occurred (version nightly-2017-09-02-g5ee349d):
ERROR
ERROR This means that one or more arguments or inputs in your command are incorrect.
ERROR The error message below tells you what is the problem.
ERROR
ERROR If the problem is an invalid argument, please check the online documentation guide
ERROR (or rerun your command with --help) to view allowable command-line arguments for this tool.
ERROR
ERROR Visit our website and forum for extensive documentation and answers to
ERROR commonly asked questions https://software.broadinstitute.org/gatk
ERROR
ERROR Please do NOT post this error to the GATK forum unless you have really tried to fix it yourself.
ERROR
ERROR MESSAGE: Bad input: The GATK report has no version specified in the header
ERROR ------------------------------------------------------------------------------------------
Can anyone tell me what should I do to fix this? Any suggestions and details in perfomring?
Thanks everyone
Issue · Github
by Sheila
I have a similar issue: I see the ERROR, but SimpleLogger works for me and the process continues..
But I'd like not to get this error. Is it possible?
[2017-09-04 11:36:53] [18/25] Creating table with indel candidates for realignment [Indel Realignment 1/3] ...
Command: 'java -jar /soft/GenomeAnalysisTK-3.8-0-ge9d806836/GenomeAnalysisTK.jar -T RealignerTargetCreator -R /outputs/ref/GRCh38.d1.vd1/GRCh38.d1.vd1.fa -I /outputs/TCGA-DA-A3F2-06A-11D-A20D-08.rg.aligned-bwa.sorted-samtools.merged.rh
.dedupped.bam --known /outputs/ref/GRCh38.d1.vd1/dbSNP/All_20170710.vcf.gz -o /outputs/TCGA-DA-A3F2-06A-11D-A20D-08.intervals --num_threads 8'.
PID=24562 (last job)
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/tmp
INFO 11:36:56,252 HelpFormatter - ----------------------------------------------------------------------------------
INFO 11:36:56,256 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.8-0-ge9d806836, Compiled 2017/07/28 21:26:50
INFO 11:36:56,257 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute
INFO 11:36:56,257 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk
INFO 11:36:56,257 HelpFormatter - [Mon Sep 04 11:36:56 UTC 2017] Executing on Linux 4.4.0-87-generic amd64
INFO 11:36:56,257 HelpFormatter - Java HotSpot(TM) 64-Bit Server VM 1.8.0_144-b01
INFO 11:36:56,261 HelpFormatter - Program Args: -T RealignerTargetCreator -R /outputs/ref/GRCh38.d1.vd1/GRCh38.d1.vd1.fa -I /outputs/TCGA-DA-A3F2-06A-11D-A20D-08.rg.aligned-bwa.sorted-samtools.merged.rh.dedupped.bam --known /outputs/ref
/GRCh38.d1.vd1/dbSNP/All_20170710.vcf.gz -o /outputs/TCGA-DA-A3F2-06A-11D-A20D-08.intervals --num_threads 8
INFO 11:36:56,268 HelpFormatter - Executing as [email protected] on Linux 4.4.0-87-generic amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_144-b01.
INFO 11:36:56,268 HelpFormatter - Date/Time: 2017/09/04 11:36:56
INFO 11:36:56,268 HelpFormatter - ----------------------------------------------------------------------------------
INFO 11:36:56,268 HelpFormatter - ----------------------------------------------------------------------------------
ERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/soft/GenomeAnalysisTK-3.8-0-ge9d806836/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
INFO 11:36:56,455 GenomeAnalysisEngine - Deflater: JdkDeflater
INFO 11:36:56,455 GenomeAnalysisEngine - Inflater: JdkInflater
...
Has this issue been resolved in the latest nightly build?
@mizetrav @serge2016 @gloriawu
Hi everyone,
I have not seen any new developments on this, but I will check with the team and get back to you.
Thanks for your patience.
-Sheila
@gloriawu
Hi again,
Your error message may not be related to this issue being fixed in the latest nightly. Can you post the exact command you ran?
If you search for your final error message, you should find threads in the forum that may help.
-Sheila
@Sheila
below is the command I use
java -Xmx16g -jar /home/user/software/gatk3.7/GenomeAnalysisTK.jar -T PrintReads -R /home/user/reference/ucsc.hg19.fasta -I /home/wd2t/TWbiobank/NGS2_20150101A/NGS2_20150101A_S99_L999_08302017_bwamem.marked.realigned.bam -BQSR /home/wd2t/TWbiobank/NGS2_20150101A/NGS2_20150101A_S99_L999_08302017_bwamem.marked.realigned.bam.recal_data.grp -o /home/wd2t/TWbiobank/NGS2_20150101A/NGS2_20150101A_S99_L999_09012017_bwamem.marked.realigned.recal.bam
and i've not found other threads that might be useful. I've been trying with many builds of GATK
If it's not related to this issue, what else should I try? I've already reinstall java and running out of options to try next.
What is the final error message? I've posted everything that came out besides the script, or where can I obtain final error message?
Thank you again for replying, really appreciated!
@Sheila
below is the command I use
java -Xmx16g -jar /home/user/software/gatk3.7/GenomeAnalysisTK.jar -T PrintReads -R /home/user/reference/ucsc.hg19.fasta -I /home/wd2t/TWbiobank/NGS2_20150101A/NGS2_20150101A_S99_L999_08302017_bwamem.marked.realigned.bam -BQSR /home/wd2t/TWbiobank/NGS2_20150101A/NGS2_20150101A_S99_L999_08302017_bwamem.marked.realigned.bam.recal_data.grp -o /home/wd2t/TWbiobank/NGS2_20150101A/NGS2_20150101A_S99_L999_09012017_bwamem.marked.realigned.recal.bam
and i've not found other threads that might be useful. I've been trying with many builds of GATK
If it's not related to this issue, what else should I try? I've already reinstall java and running out of options to try next.
What is the final error message? I've posted everything that came out besides the script, or where can I obtain final error message?
Thank you again for replying, really appreciated!
Dear @Sheila
Does the latest nightly build include the fix for the logger issue?
Thanks
@Sheila
Would it be alright to use 3.7 for this phase and then use 3.8 for the remaining steps? The differences between versions seem minimal.
Thanks
@Sheila
gloriawu is c3h7oh, I end up with two accounts when I signed up with just one email address ><
I've already tried 3.7, and it is not working.
Some people told me that I need to have oracle java instead of openjdk, is that the case?
Cause I am running with ubuntu and openjdk is the default when I install java.
Or should I tried an even earlier version of GATK?
Thanks for the reply.
Hi all, the latest nightly build has the fix for the logger and the hanging issue. If you encounter any other issues beyond that, please open separate threads as they are separate problems that we will not address here.
@c3h7oh @gloriawu
Hi,
Alright. Does the latest nightly build fix your issue? If not, you may need to re-run the Base Recalibration step over again with the latest nightly build and see if that fixes the issue.
-Sheila
@mizetrav
Hi,
Is it possible to use 3.8 and 3.8 nightly build? I am assuming you wanted to use 3.7 to get around this issue, but now that the issue is fixed in the nightly, you should be good to go ?
-Sheila
I am running it now using the latest nightly build, thank you!
@Sheila
"A fatal error has been detected by the Java Runtime Environment"
The newest nightly build is crashing at the IndelRealigner phase. Should I open a separate thread with more detailed information? Maybe I should I talk with the server administrator and have them update Java beforehand? I haven't had any issues until a few weeks ago when I encountered the logger issue for the first time.
@mizetrav Please open a new discussion thread and post the error log details. Hopefully we can help you diagnose the issue, whether it's another bug in the GATK or something screwy in your environment.
I had the same problem with GenotypeGVCFs 3.8. I downloaded
GenomeAnalysisTK-nightly-2017-10-06-g1994025.jar, but it still fails. I have opened issue 10472 about it.@tommycarstensen
Hi Tommy,
Okay, we will try to help you in the other thread.
Sheila
I can confirm I still have the same issue with 3.8 as below:
INFO 09:34:52,843 HelpFormatter - ----------------------------------------------------------------------------------
INFO 09:34:52,854 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.8-0-ge9d806836, Compiled 2017/07/28 21:26:50
INFO 09:34:52,854 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute
INFO 09:34:52,854 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk
INFO 09:34:52,855 HelpFormatter - [Thu Oct 26 09:34:52 SGT 2017] Executing on Linux 3.16.0-77-generic amd64
INFO 09:34:52,855 HelpFormatter - OpenJDK 64-Bit Server VM 1.8.0_144-b01
INFO 09:34:52,860 HelpFormatter - Program Args: -T MuTect2 -R /speed/csioan/library/hg19.fa -I:tumor /var/www/html/csiwebportal/results/1524/DNA-Seq/FCHCY3CBBXX-HUMsqnEAAAHAAA-311_L2_tumor/FCHCY3CBBXX-HUMsqnEAAAHAAA-311_L2_tumor.picarded.rmdup.bam --cosmic /speed/csioan/library/hg19_cosmic_v54_120711.vcf --dbsnp /speed/csioan/library/dbsnp_132_b37.leftAligned.vcf --artifact_detection_mode --out /var/www/html/csiwebportal/results/1524/DNA-Seq/FCHCY3CBBXX-HUMsqnEAAAHAAA-311_L2_tumor/FCHCY3CBBXX-HUMsqnEAAAHAAA-311_L2_tumor-unpaired.mutect.vcf.temp
INFO 09:34:52,889 HelpFormatter - Executing as [email protected] on Linux 3.16.0-77-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_144-b01.
INFO 09:34:52,890 HelpFormatter - Date/Time: 2017/10/26 09:34:52
INFO 09:34:52,890 HelpFormatter - ----------------------------------------------------------------------------------
INFO 09:34:52,891 HelpFormatter - ----------------------------------------------------------------------------------
ERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/speed/csioan/software/anaconda2/opt/gatk-3.8/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
Any suggestion how to fix?
@arkanion
Hi,
Have you tried the latest nightly build?
Thanks,
Sheila
Dear Sheila,
unfortunately I am encountering the same problem. Latest version of GATK 3 installed, but I get the same error message as others describe it, and then the software hangs.
cheers,
Reto
I got the same problem with HaplotypeCaller.
My usual pipeline using GATK3.2.2, stopped working. I have now tried GATK 3.2.2, 3.6 and 3.8 (downloaded 20171102) , using different versions of java (I think the problems started when java was updated on our system last week?).
I'm currently using JRE version: Java(TM) SE Runtime Environment (8.0_151-b12) (build 1.8.0_151-b12). I have tried without multiple threads too without luck.
My command: java -XX:ParallelGCThreads=8 -jar GenomeAnalysisTK-3.8/GenomeAnalysisTK.jar -I 74118.recalibrated.bam -R WholeGenomeFASTA/WholeGenomeFASTA.fa -T HaplotypeCaller -pcrModel NONE --dbsnp Human_variations_NCBI/00-All_chrAdded.vcf -o 74118.NoPCR_GATKv3-8.vcf
The (latest) error:
ERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/mnt/NGS01/biss/opt/GenomeAnalysisTK-3.8-0-ge9d806836/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
And then the software hangs....
We're using the latest version of CentOS 7.
Cheers,
Carina
hi alll, I am experiencing the same issues with VariantsToTable (GATK v3.8.0)
ERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/software/GATK/GenomeAnalysisTK-3.8-0/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
Hi @burri, @carina and @Maki,
Sorry to hear your runs are hanging.
For one, the
ERROR StatusLoggermessage should be ignored for now if the run uses the SimpleLogger instead and completes successfully. If the run errors, e.g. hangs indefinitely, then the source of the error is likely to be something unrelated to the logger and we will need the command you ran and the error message that is at the very end of the stdout (if any).The user in this thread shows the issue is solved for them by using the latest nightly, which are mini-patched releases inbetween minor releases. You can find nightlies at https://software.broadinstitute.org/gatk/download/nightly.
Hi Shlee and Sheila
I finally got HaplotypeCaller to run with last nights version: GenomeAnalysisTK-nightly-2017-11-09-g45c474f. Same command and java version as previously mentioned (but it also seems to work with openjdk-1.8.0).
Thank you for your help
Carina
Great to hear @carina.
I was having a "ERROR StatusLogger Log4j2 could not find a logging implementation" error and I tried to rectify it with the nighly build but with no avail
The version of nightly build is : GenomeAnalysisTK-nightly-2017-12-11-1
The version of java is: 1.8.0_131
Now, instead of getting the aforementioned error, the command just hangs without any output.
The command used by me is:
nohup java -jar GenomeAnalysisTK-nightly-2017-12-11-1/GenomeAnalysisTK.jar -T RealignerTargetCreator -R ref.fasta -I Merge.bam -o INDEL.intervals &
https://gatkforums.broadinstitute.org/gatk/discussion/10131/gatk-3-8-logger-error (the thread here says that the nightly version has fixed the hanging/freezing issue)
Please help.
Issue · Github
by Sheila
Hi everyone
When I run GATK command
$ java -jar ~/biosoft/gatk/GenomeAnalysisTK-3.8/GenomeAnalysisTK.jar \ -R ref.fa \ -T RealignerTargetCreator \ -I sample.rmdup.bam \ -o sample.realign.intervalsI get a error message
So is there anyone could help me how to fix it?Thanks !
The command worked well though I got such a problem.Maybe this issue didn't affect the final result.
@shubhra @taoyan
Hi,
I suspect this is being fixed and will be released as a patch to version 3. I will get back to you with more details.
-Sheila
@taoyan Have you tried using the latest nightly build? See the Download page for details.
There is a patch version coming out soon. However you may be better off upgrading to GATK4, which will be released in a few weeks.
Hi，@Geraldine_VdAuwera，Thanks, I will try it!
Hi,
I just found the same error using RealignerTargetCreater in gatk 3.8.
ERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:
I cannot use gatk4 because HaplotypeCaller is not working for me (and for other users as seen in other threads) and they suggest using UnifiedGenotyper (which is not available for gatk4).
Despite this error, the command continues to go on and the final message is "Done. There were no warn messages."
My question is, is there anything I should worry about and solve? Or should I ignore the error?
Thanks!
Patch release 3.8-1 is finally out (in the archive downloads), sorry for the delay everyone. I blame legacy infrastructure used by the 3.x system (good riddance to that).
@Rebs No, you don't need to worry about logging error messages, they will have no effect on your results.
Alright, thanks @Geraldine_VdAuwera !
@Geraldine_VdAuwera
@Sheila
I also get this error but it runs through fine afterwards. Does it affect the performance in this case? Above is stated that it does not affect the results, so I suppose I can just ignore them?
Thanks!
@Juls
Hi,
You can ignore it as long as the tool runs to completion.
-Sheila