The current GATK version is 3.7-0
Examples: Monday, today, last week, Mar 26, 3/26/04

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Get notifications!


You can opt in to receive email notifications, for example when your questions get answered or when there are new announcements, by following the instructions given here.

Did you remember to?


1. Search using the upper-right search box, e.g. using the error message.
2. Try the latest version of tools.
3. Include tool and Java versions.
4. Tell us whether you are following GATK Best Practices.
5. Include relevant details, e.g. platform, DNA- or RNA-Seq, WES (+capture kit) or WGS (PCR-free or PCR+), paired- or single-end, read length, expected average coverage, somatic data, etc.
6. For tool errors, include the error stacktrace as well as the exact command.
7. For format issues, include the result of running ValidateSamFile for BAMs or ValidateVariants for VCFs.
8. For weird results, include an illustrative example, e.g. attach IGV screenshots according to Article#5484.
9. For a seeming variant that is uncalled, include results of following Article#1235.

Did we ask for a bug report?


Then follow instructions in Article#1894.

Formatting tip!


Wrap blocks of code, error messages and BAM/VCF snippets--especially content with hashes (#)--with lines with three backticks ( ``` ) each to make a code block as demonstrated here.

Jump to another community
Picard 2.9.0 is now available. Download and read release notes here.
GATK 3.7 is here! Be sure to read the Version Highlights and optionally the full Release Notes.

Picard MergeBamAlignment issue

Hi,

I am attempting to merge the output of tophat in order to run some RNASeq
QC metrics. This is a single read 50 bp on a Hiseq. In order to get past
the fact that tophat gives a MAPQ of 255 to unmapped reads (not 0 as
expected by Picard) I used the following tool (
https://github.com/cbrueffer/tophat-recondition) to change it.

Once completed, I added read groups using picard and then sorted the
accepted_hits.bam by coordinate and sorted the unmapped reads by queryname.

tophat-recondition.py /home/ryan/NGS_Data/No_Dox

java -Xmx2g -jar /home/ryan/jar/picard-tools-1.129/picard.jar \
AddOrReplaceReadGroups \
I=/home/ryan/NGS_Data/unmapped_fixup.bam \
O=/home/ryan/NGS_Data/unmapped_fixup-RG.bam \
RGID=No_Dox RGLB=No_Dox RGPL=illumina RGPU=GCCAAT RGSM=No_Dox

java -Xmx2g -jar /home/ryan/jar/picard-tools-1.129/picard.jar \
AddOrReplaceReadGroups \
I=/home/ryan/NGS_Data/accepted_hits.bam \
O=/home/ryan/NGS_Data/accepted_hits-RG.bam \
SORT_ORDER=coordinate \
RGID=No_Dox RGLB=No_Dox RGPL=illumina RGPU=GCCAAT RGSM=No_Dox \
CREATE_INDEX=true

java -Xmx2g -jar /home/ryan/jar/picard-tools-1.129/picard.jar \
SortSam \
I=/home/ryan/NGS_Data/unmapped_fixup-RG.bam \
O=/home/ryan/NGS_Data/unmapped_fixup-RG-sorted.bam \
SORT_ORDER=queryname

java -Xmx2g -jar /home/ryan/jar/picard-tools-1.129/picard.jar \
SortSam \
I=/home/ryan/NGS_Data/accepted_hits-RG.bam \
O=/home/ryan/NGS_Data/accepted_hits-RG-sorted.bam \
SORT_ORDER=coordinate \
CREATE_INDEX=true

java -Xmx2g -jar /home/ryan/jar/picard-tools-1.129/picard.jar \
MergeBamAlignment \
UNMAPPED_BAM=/home/ryan/NGS_Data/unmapped_fixup-RG-sorted.bam \
ALIGNED_BAM=/home/ryan/NGS_Data/accepted_hits-RG-sorted.bam \
O=/home/ryan/NGS_Data/merge_unmapped_accepted_hits_No_Dox.bam \
SORT_ORDER=coordinate \
REFERENCE_SEQUENCE=/home/ryan/Reference/human_spikein/hg19_spikein.fa \
PROGRAM_RECORD_ID=Tophat \
PROGRAM_GROUP_VERSION=0.1 \
PROGRAM_GROUP_COMMAND_LINE=tophat/dummy \
PAIRED_RUN=false \
VALIDATION_STRINGENCY=LENIENT \
CREATE_INDEX=true

I then get the following warning followed by the error:

WARNING 2015-03-17 09:44:22 SamAlignmentMerger Exception merging
bam alignment - attempting to sort aligned reads and try again: Aligned
record iterator (HS3:608:C6LNLACXX:7:2101:9946:63417) is behind the
unmapped reads (HS3:608:C6LNLACXX:7:2101:9947:11009)
INFO 2015-03-17 09:44:33 SamAlignmentMerger Read 1000000 records
from alignment SAM/BAM.
INFO 2015-03-17 09:44:43 SamAlignmentMerger Read 2000000 records
from alignment SAM/BAM.
....
INFO 2015-03-17 09:58:01 SamAlignmentMerger Read 96000000 records
from alignment SAM/BAM.
INFO 2015-03-17 09:58:09 SamAlignmentMerger Read 97000000 records
from alignment SAM/BAM.
INFO 2015-03-17 09:58:15 SamAlignmentMerger Finished reading
97571897 total records from alignment SAM/BAM.
[Tue Mar 17 09:58:16 PDT 2015] picard.sam.MergeBamAlignment done. Elapsed
time: 14.32 minutes.
Runtime.totalMemory()=1908932608
To get help, see
http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" java.lang.IllegalStateException: Aligned record
iterator (HS3:608:C6LNLACXX:7:1101:10000:11036) is behind the unmapped
reads (HS3:608:C6LNLACXX:7:1101:10000:48402)
at
picard.sam.AbstractAlignmentMerger.mergeAlignment(AbstractAlignmentMerger.java:383)
at
picard.sam.SamAlignmentMerger.mergeAlignment(SamAlignmentMerger.java:153)
at picard.sam.MergeBamAlignment.doWork(MergeBamAlignment.java:248)
at
picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:187)
at
picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:95)
at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:105)

I have searched and have been unsuccessful at resolving this problem. Any
ideas?

I am running picard 1.129 using java version "1.7.0_60"

ValidateSamFile on both inputs into MergeBamAlignment returns no errors in
those files. I am at a lost and seeking for help!

Thanks,

Ryan

Tagged:

Issue · Github
by Geraldine_VdAuwera

Issue Number
186
State
closed
Last Updated
Assignee
Array
Closed By
chandrans

Answers

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAPosts: 11,736 admin

    Hi @Ryan ,

    I think the problem is that of the two sams you're merging, one is sorted by queryname and the other by coordinate. I expect if you change that to have both sorted the same way the error will go away.

    Geraldine Van der Auwera, PhD

  • RyanRyan Posts: 7

    Hi Geraldine,

    Thanks for thehelp. Unfortunately sorting both inputs by queryname does not resolve the issue and the same error pops up:

    WARNING 2015-03-18 09:51:14 SamAlignmentMerger Exception merging bam alignment - attempting to sort aligned reads and try again: Aligned record iterator (HS3:608:C6LNLACXX:7:1101:10000:11036) is behind the unmapped reads (HS3:608:C6LNLACXX:7:1101:10000:48402)
    INFO 2015-03-18 09:51:20 SamAlignmentMerger Read 1000000 records from alignment SAM/BAM.
    INFO 2015-03-18 09:51:26 SamAlignmentMerger Read 2000000 records from alignment SAM/BAM.
    ....
    INFO 2015-03-18 10:02:52 SamAlignmentMerger Read 97000000 records from alignment SAM/BAM.
    INFO 2015-03-18 10:02:59 SamAlignmentMerger Finished reading 97571897 total records from alignment SAM/BAM.
    [Wed Mar 18 10:02:59 PDT 2015] picard.sam.MergeBamAlignment done. Elapsed time: 11.75 minutes.
    Runtime.totalMemory()=1908932608
    To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
    Exception in thread "main" java.lang.IllegalStateException: Aligned record iterator (HS3:608:C6LNLACXX:7:1101:10000:11036) is behind the unmapped reads (HS3:608:C6LNLACXX:7:1101:10000:48402)
    at picard.sam.AbstractAlignmentMerger.mergeAlignment(AbstractAlignmentMerger.java:383)
    at picard.sam.SamAlignmentMerger.mergeAlignment(SamAlignmentMerger.java:153)
    at picard.sam.MergeBamAlignment.doWork(MergeBamAlignment.java:248)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:187)
    at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:95)
    at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:105)

    I am not sure what is causing this.

    Ryan

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAPosts: 11,736 admin

    Hi @Ryan,

    Would you be able to share some snippets of data that reproduce the problem so we can debug this locally? Instructions are here: https://www.broadinstitute.org/gatk/guide/article?id=1894

    Geraldine Van der Auwera, PhD

  • RyanRyan Posts: 7

    Hi Geraldine,

    Thanks for the help.
    I have uploaded snippets of data to your FTP server. It is named PICARD_MergeBam_error.tar.gz
    I really appreciate you looking into this. Let me know if you need additional information.

    Ryan

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAPosts: 11,736 admin

    Thanks @Ryan, we'll try to get to these asap.

    Geraldine Van der Auwera, PhD

  • 5581681555816815 TNPosts: 28
    edited May 2016

    @Ryan @Geraldine_VdAuwera
    Is this resolved? I got the same error for qn sorted bams.
    Thanks,

  • shleeshlee CambridgePosts: 597 admin

    Can you check to see if the read records that give the exception are present in both your uBAM and your aligned BAM? All reads in the aligned BAM must be in the uBAM.

  • 5581681555816815 TNPosts: 28
    edited May 2016

    Thanks @shlee
    Actually I resolved the issue by sorting my ubam file using picard...previously i used samtools.

    The queryname sorted bam are quite different detween picard and samtools.
    samtools seems to be numeric sorting on tile number
    picard seems to be text based like E00438:46:HMTV7CCXX:8:1101:10632:1661 is before E00438:46:HMTV7CCXX:8:1101:1073:10802

  • shleeshlee CambridgePosts: 597 admin

    Ah yes. There are minor inconsistencies between the tools so to avoid problems with GATK we recommend folks use Picard for data transformations. Good to hear you've sorted it out.

Sign In or Register to comment.