Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on November 11th and 13th 2019, due to the U.S. holiday(Veteran's day) and due to a team event(Nov 13th). We will return to monitoring the GATK forum on November 12th and 14th respectively. Thank you for your patience.

MarkDuplicates error Value was put into PairInfoMap more than once

henahena FinlandMember
edited April 2016 in Ask the GATK team

I get this error from a MarkDuplicates and all the questions I've found online comment on merging fastq file and ending up with read name from more than one original fastq files. I get this error without having merged fastq files and only having one pair of reads in bam file with same name with flags 129 and 81 (so read 2 and read 1). Is there any other reason why this message could come up?

I'm using an older version of the Picard (as a part of pipeline) though so if there has been bugs in this area then I'm likely hitting them.

Thanks,

Issue · Github
by Sheila

Issue Number
805
State
closed
Last Updated
Assignee
Array
Milestone
Array
Closed By
vdauwera

Answers

  • henahena FinlandMember

    I also got the same message with picard version 1.141 which I think is not that old version.

  • SheilaSheila Broad InstituteMember, Broadie admin

    @hena
    Hi,

    I will ask our Picard expert to get back to you. However, in the meantime, you can try using the latest version of Picard (2.2.1).

    -Sheila

  • SheilaSheila Broad InstituteMember, Broadie admin

    @hena
    Hi,

    Can you also try validating your bam file with ValidateSamFile?

    Thanks,
    Sheila

  • henahena FinlandMember
    edited April 2016

    2.2.1 seems to create same error same as running ValidateSamFile in summary mode. Here is the stacktrace from the ValidateSamFile

    Exception in thread "main" htsjdk.samtools.SAMException: Value was put into PairInfoMap more than once. 1: HWI-xxxxxxx:111:C8LG3ANXX:8:2209:18364:77568
    at htsjdk.samtools.CoordinateSortedPairInfoMap.ensureSequenceLoaded(CoordinateSortedPairInfoMap.java:133)
    at htsjdk.samtools.CoordinateSortedPairInfoMap.remove(CoordinateSortedPairInfoMap.java:86)
    at htsjdk.samtools.SamFileValidator$CoordinateSortedPairEndInfoMap.remove(SamFileValidator.java:749)
    at htsjdk.samtools.SamFileValidator.validateMateFields(SamFileValidator.java:484)
    at htsjdk.samtools.SamFileValidator.validateSamRecordsAndQualityFormat(SamFileValidator.java:282)
    at htsjdk.samtools.SamFileValidator.validateSamFile(SamFileValidator.java:200)
    at htsjdk.samtools.SamFileValidator.validateSamFileSummary(SamFileValidator.java:128)
    at picard.sam.ValidateSamFile.doWork(ValidateSamFile.java:187)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:209)
    at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:95)
    at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:105)

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    We've seen this error pop up for several different reasons; the main two are the fastq merging problem, and secondary hits not marked appropriately due to not using -M with BWA MEM. Not sure what's happening in your case, but if I remember the SAM flag rules correctly, the reads in a mate pair would have 129 and 65 respectively, or 161 and 81 (if on reverse strand). I think the program may not be recognizing that your reads are part of a mate pair because of that, and is instead considering them as part of different pairs. This could be caused by a mistake made by the aligner.

    If you only have one affected read pair out of your entire bam file I would advise just throwing out that one read pair and proceeding with your analysis. I believe you should be able to eliminate improperly paired reads by using samtools view -f 0x2 -- but don't take my word for it, read this blog post about sam flags and be sure to check the samtools docs too.

  • henahena FinlandMember

    The data is aligned with bwa aln + sampe, so there should not be secondary alignments. There is however single reads included but that has been working before only in this one read pair this problem has appeared. After removing that pair, it failed again with pair with flags 97 and 145 and after removing them as well I have pair again with 81 and 129. I also noticed that the mate information seems to be wrong on these reads and that they seem match other reads for some reason. Could this have an effect? I'm going to do name sort -> samtools fixmate -> position sort in order to see if that would help.

  • deklingdekling Broad InstituteMember admin

    Forgive my naiveté, but did you have a look at the following link? https://www.biostars.org/p/60263/ and the embedded links within?

  • SumuduSumudu Sri LankaMember

    Hi,

    I am doing variant calling with a human whole exome sample. I downloaded 30 exome samples from 1000Genomes project for joint v.calling with my sample.

    The 1000Genomes files are in .cram format. I used samtools view to covert them to .bam as below.

    samtools view -b HG04039.alt_bwamem_GRCh38DH.20150826.STU.exome.cram -T Homo_sapiens_assembly38.fasta -o HG04039.bam (samtools version1.9)

    Then I ran
    java -jar picard.jar ValidateSamFile I=HG04039.bam MODE=SUMMARY (picard version 2.20)

    I got an error: ERROR 2019-05-13 13:45:11 ValidateSamFile Value was put into PairInfoMap more than once. 3072: SRR794757.9422092

    I tried to fix this by using command
    samtools view -f 0x2 HG04039.bam -o HG04039_new.bam

    but again got errors after runing ValidateSamFile:
    ERROR:MISMATCH_FLAG_MATE_NEG_STRAND 1
    ERROR:MISMATCH_MATE_ALIGNMENT_START 2

    I also tried ValidateSamFile after runing MarkDuplicates with picard, still the same.

    Is it ok to proceed with these BAM files for v.calling in exome analysis? Appreciate any recommendations on this issue.

    Thank you in advance.
    Best,
    Sumudu

  • bshifawbshifaw Member, Broadie, Moderator admin

    @Sumudu

    Can you try running the following pipe to convert your cram files to bam then check it with ValidateSamFile

    samtools view -h -T ${RefFasta} ${InputCram} |
    samtools view -b -o ${SampleName}.bam -
    samtools index -b ${SampleName}.bam
    
  • SumuduSumudu Sri LankaMember

    Hi,

    Thank you for the reply. This is what I got after running the command. I am wondering the reference file has a incompatibility.

    [email protected]:~/1000G_BAMs$ samtools view -h -T /home/wes/resources/Homo_sapiens_assembly38.fasta HG03886.alt_bwamem_GRCh38DH.20150826.STU.exome.cram | samtools view -b -o new.bam - samtools index -b new.bam
    [bam_header_read] EOF marker is absent. The input is probably truncated.
    [sam_header_read2] 3366 sequences loaded.
    [bam_header_read] invalid BAM binary header (this is not a BAM file).
    [main_samview] fail to read the header from "-".

    When I ran the command : samtools view -h -T /home/wes/resources/Homo_sapiens_assembly38.fasta HG03886.alt_bwamem_GRCh38DH.20150826.STU.exome.cram

    This is the final part of the output:
    @SQ SN:HLA-DRB115:01:01:04 LN:11056
    @SQ SN:HLA-DRB1
    15:02:01 LN:10313
    @SQ SN:HLA-DRB115:03:01:01 LN:11567
    @SQ SN:HLA-DRB1
    15:03:01:02 LN:11569
    @SQ SN:HLA-DRB116:02:01 LN:11005
    [sam_read1] reference 'VN:1.5' is recognized as '
    '.
    Parse error at line 1: invalid CIGAR character
    Aborted (core dumped)

    Appreciate any suggestions to solve this problem.

    Best
    Sumudu

  • SumuduSumudu Sri LankaMember

    Hi,

    I solved the issue. The BAM was sorted by coordinate and ValidateSam passed.

    Best
    Sumudu

  • bshifawbshifaw Member, Broadie, Moderator admin

    Happy you solved the problem, and thanks for sharing!

Sign In or Register to comment.