To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

Picard ViewSam Performance

ayangayang TorontoMember
edited June 2016 in Ask the GATK team

Hi,

I have been playing around with the latest Picard Tools build (2.4.1) with the intention of eventually using HTSJDK directly to manipulate SAM/BAM/CRAM files. I've found that when I run the ViewSam Picard Tool on a CRAM, it takes a very long time to run. In this example, my "interval.txt" contains a single range 7:169000-170000 that has been converted from a Bed file into an Interval file with a SequenceDictionary I constructed from a FASTA file.

Running the same query in SamTools 1.3.1 View is extremely quick - a matter of seconds. The ViewSam Picard Tool takes 33 minutes.

java -jar /picard-tools-2.4.1/picard.jar ViewSam INPUT=sample.cram INTERVAL_LIST=interval.txt REFERENCE_SEQUENCE=genome.fa

[Wed Jun 22 20:41:31 UTC 2016] picard.sam.ViewSam INPUT=sample.cram INTERVAL_LIST=interval.txt REFERENCE_SEQUENCE=genome.fa ALIGNMENT_STATUS=All PF_STATUS=All HEADER_ONLY=false RECORDS_ONLY=false VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json

Running this on the BAM version of the file also takes a long time. The sample CRAM file I'm testing with is approximately 88 GB. Both my reference file and my CRAM file are indexed (I have genome.fa.fai and sample.cram.crai in the same directory as genome.fa and sample.cram)

What factors might be contributing to the long processing time? Are there additional parameters I should be specifying? Or is this expected due to how ViewSam is implemented. I haven't gone through the source code yet - I guess I'm hoping for a quick sanity check before committing time to chasing this down.

Thanks,
Andy

Tagged:

Answers

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie

    Hmm, that's a surprising observation, at least to me. Maybe try relaxing the validation stringency, in case it's due to the validation checks? That's the most obvious difference I can think of relative to samtools view.

    If that doesn't explain what you're seeing, and since it's more a developer question, I would recommend reposting it as an issue in the Picard repository on GitHub.

  • ayangayang TorontoMember

    Thanks for the suggestion. I set VALIDATION_ to SILENT but it still took the same time. I've reposted the question as a Picard issue in Github

Sign In or Register to comment.