This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!
too many open files
I've seen this problem mentioned many times here, but I wonder if I may have some new contribution here. Our users are running GATK 3.8, and see this error:
##### ERROR MESSAGE: Unable to parse header with error: /tmp/org.broadinstitute.gatk.engine.io.stubs.VariantContextWriterStub3088197262002724909.tmp (Too many open files)
I've monitored some runs and noticed that while the run progresses, there is more and more leftover open file descriptors:
lsof | grep VariantContextWriterStub java 67922 frodeli 1563r REG 8,5 10740643 968 /tmp/org.broadinstitute.gatk.engine.io.stubs.VariantContextWriterStub498651467373697139.tmp (deleted) java 67922 frodeli 1564w REG 8,5 6282467 992 /tmp/org.broadinstitute.gatk.engine.io.stubs.VariantContextWriterStub2281266380489957735.tmp
VariantContextWriterStub files have a
(deleted) flag, and only some of them don't. After 24h of runtime 1432 out of 1467 entries are reported as deleted. This makes me wonder: maybe there is a resource leak in GATK, i.e., the temporary file is deleted without being closed before.