To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

Cromwell call-caching Failed to hash error

I get the following error when I enable call-caching in cromwell 30.1

2018-01-12 12:49:02,002 cromwell-system-akka.dispatchers.engine-dispatcher-57 ERROR - Failed to hash
java.io.IOException: Is a directory
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.FileDispatcherImpl.read(FileDispatcherImpl.java:46)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:159)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at org.apache.commons.codec.digest.DigestUtils.updateDigest(DigestUtils.java:794)
at org.apache.commons.codec.digest.DigestUtils.digest(DigestUtils.java:50)
at org.apache.commons.codec.digest.DigestUtils.md5(DigestUtils.java:274)
at org.apache.commons.codec.digest.DigestUtils.md5Hex(DigestUtils.java:310)
at cromwell.backend.impl.sfs.config.HashFileStrategy.$anonfun$hash$3(ConfigHashingStrategy.scala:82)
at cromwell.util.TryWithResource$.$anonfun$tryWithResource$1(TryWithResource.scala:16)
at scala.util.Try$.apply(Try.scala:209)
at cromwell.util.TryWithResource$.tryWithResource(TryWithResource.scala:10)
at cromwell.backend.impl.sfs.config.HashFileStrategy.hash(ConfigHashingStrategy.scala:82)
at cromwell.backend.impl.sfs.config.ConfigHashingStrategy.usingStandardInitData$1(ConfigHashingStrategy.scala:52)
at cromwell.backend.impl.sfs.config.ConfigHashingStrategy.getHash(ConfigHashingStrategy.scala:57)
at cromwell.backend.impl.sfs.config.ConfigBackendFileHashingActor.customHashStrategy(ConfigBackendFileHashingActor.scala:24)
at cromwell.backend.standard.callcaching.StandardFileHashingActor$$anonfun$fileHashingReceive$1.applyOrElse(StandardFileHashingActor.scala:70)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:514)
at akka.actor.Actor.aroundReceive$(Actor.scala:512)
at cromwell.backend.standard.callcaching.StandardFileHashingActor.aroundReceive(StandardFileHashingActor.scala:57)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
at akka.actor.ActorCell.invoke(ActorCell.scala:496)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2018-01-12 12:49:02,013 cromwell-system-akka.dispatchers.engine-dispatcher-72 ERROR - bec63d8d:complete.bwaaln:2:1: Hash error, disabling call caching for this job.
java.io.IOException: Is a directory
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.FileDispatcherImpl.read(FileDispatcherImpl.java:46)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:159)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at org.apache.commons.codec.digest.DigestUtils.updateDigest(DigestUtils.java:794)
at org.apache.commons.codec.digest.DigestUtils.digest(DigestUtils.java:50)
at org.apache.commons.codec.digest.DigestUtils.md5(DigestUtils.java:274)
at org.apache.commons.codec.digest.DigestUtils.md5Hex(DigestUtils.java:310)
at cromwell.backend.impl.sfs.config.HashFileStrategy.$anonfun$hash$3(ConfigHashingStrategy.scala:82)
at cromwell.util.TryWithResource$.$anonfun$tryWithResource$1(TryWithResource.scala:16)
at scala.util.Try$.apply(Try.scala:209)
at cromwell.util.TryWithResource$.tryWithResource(TryWithResource.scala:10)
at cromwell.backend.impl.sfs.config.HashFileStrategy.hash(ConfigHashingStrategy.scala:82)
at cromwell.backend.impl.sfs.config.ConfigHashingStrategy.usingStandardInitData$1(ConfigHashingStrategy.scala:52)
at cromwell.backend.impl.sfs.config.ConfigHashingStrategy.getHash(ConfigHashingStrategy.scala:57)
at cromwell.backend.impl.sfs.config.ConfigBackendFileHashingActor.customHashStrategy(ConfigBackendFileHashingActor.scala:24)
at cromwell.backend.standard.callcaching.StandardFileHashingActor$$anonfun$fileHashingReceive$1.applyOrElse(StandardFileHashingActor.scala:70)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:514)
at akka.actor.Actor.aroundReceive$(Actor.scala:512)
at cromwell.backend.standard.callcaching.StandardFileHashingActor.aroundReceive(StandardFileHashingActor.scala:57)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
at akka.actor.ActorCell.invoke(ActorCell.scala:496)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

It only happens in bwa align operations and only when I use hg19 reference (hg38 works fine).....
Using java version "1.8.0_151", cromwell 30.1.
Following is the cromwell config

include required(classpath("application"))
call-caching {
enabled = true
}
backend {
default = "Local"
providers {
Local {
actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
config {
run-in-background = true
submit = "/bin/bash ${script}"
concurrent-job-limit = 6
}
}
}
}
database {
profile = "slick.jdbc.MySQLProfile$"
db {
driver = "com.mysql.jdbc.Driver"
url = "jdbc:mysql://localhost/cromwell?useSSL=false&rewriteBatchedStatements=true"
user = "cg"
password = "*****"
connectionTimeout = 5000
}
}

workflow options

{
"workflow-failure-mode": "ContinueWhilePossible",
"write_to_cache": true,
"read_from_cache": true
}

relevant portion of inputs.json

"complete.reffasta": "/home/cg/resources/gatk/hg19/hg19ref.fa",
"complete.refindex": "/home/cg/resources/gatk/hg19/hg19ref.fa.fai",
"complete.refdict": "/home/cg/resources/gatk/hg19/hg19ref.dict",
"complete.bwaaln.refAmb": "/home/cg/resources/gatk/hg19/hg19ref.fa.amb",
"complete.bwaaln.refAnn": "/home/cg/resources/gatk/hg19/hg19ref.fa.ann",
"complete.bwaaln.refBwt": "/home/cg/resources/gatk/hg19/hg19ref.fa.bwt",
"complete.bwaaln.refPac": "/home/cg/resources/gatk/hg19/hg19ref.fa.pac",
"complete.bwaaln.refSa": "/home/cg/resources/gatk/hg19/hg19ref.fa.sa",
"complete.bwaaln.refAlt": "",

and relevant portion of wdl script

workflow complete {
File fastqPairsFile
Array[Array[String]] pairs = read_tsv(fastqPairsFile)
.....
scatter (pair in pairs) {
call bwaaln {
input:
name=name,
assembly=assembly,
bwa=bwa,
reffasta=reffasta,
refindex=refindex,
refdict=refdict,
pairID=pair[0],
id=pair[1],
sm=pair[2],
pl=pair[3],
lb=pair[4],
pu=pair[5],
reads1=pair[6],
reads2=pair[7]
}
....
task bwaaln {
File bwa
File reffasta
File refindex
File refdict
File? refAlt
File refAmb
File refAnn
File refBwt
File refPac
File refSa
String reads1
String reads2
String name
String id
String sm
String pl
String lb
String pu
String pairID
String assembly
command {
${bwa} mem -t 2 -M -R '@RG\tID:${id}\tSM:${sm}\tPL:${pl}\tLB:${lb}\tPU:${pu}' ${reffasta} ${reads1} ${reads2} > ${name}.${pairID}.${assembly}.bwa.aln.sam
}
output {
File outputsam = "${name}.${pairID}.${assembly}.bwa.aln.sam"
}
}

Thank you in advance for your time and effort.

Tagged:

Answers

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @cgiatzakis
    Hi,

    I am checking with the team and will get back to you soon

    -Sheila

  • Thank you Sheila!
    I also downloaded the hg19 reference files from gatk resource bundle and get the same error.

    inputs json relevant portion:
    "complete.reffasta": "/home/cg/resources/gatk/hg19/ucsc.hg19.fasta",
    "complete.refindex": "/home/cg/resources/gatk/hg19/ucsc.hg19.fasta.fai",
    "complete.refdict": "/home/cg/resources/gatk/hg19/ucsc.hg19.dict",
    "complete.bwaaln.refAmb": "/home/cg/resources/gatk/hg19/ucsc.hg19.fasta.amb",
    "complete.bwaaln.refAnn": "/home/cg/resources/gatk/hg19/ucsc.hg19.fasta.ann",
    "complete.bwaaln.refBwt": "/home/cg/resources/gatk/hg19/ucsc.hg19.fasta.bwt",
    "complete.bwaaln.refPac": "/home/cg/resources/gatk/hg19/ucsc.hg19.fasta.pac",
    "complete.bwaaln.refSa": "/home/cg/resources/gatk/hg19/ucsc.hg19.fasta.sa",
    "complete.bwaaln.refAlt": "",

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @cgiatzakis
    Hi,

    I just moved this to the WDL forum where @KateN can help you.

    -Sheila

  • jsotojsoto Broad InstituteMember, Broadie

    At a glance it looks like it
    "complete.bwaaln.refAlt": "",
    might be your issue. If there no alt file then there should not be a line in your inputs json. I believe hg19 doesn't have an alt file (so it shouldnt be in the inputs json) and hg38 does have an alt file (so it should be in the inputs json).

    Its ok to leave it out of the inputs json because it is an optional File input denoted by

    File? refAlt

    in the bwaaln task definition

Sign In or Register to comment.