Test-drive the GATK tools and Best Practices pipelines on Terra

Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

MarkDuplicates error Value was put into PairInfoMap more than once

henahena FinlandMember
edited April 2016 in Ask the GATK team

I get this error from a MarkDuplicates and all the questions I've found online comment on merging fastq file and ending up with read name from more than one original fastq files. I get this error without having merged fastq files and only having one pair of reads in bam file with same name with flags 129 and 81 (so read 2 and read 1). Is there any other reason why this message could come up?

I'm using an older version of the Picard (as a part of pipeline) though so if there has been bugs in this area then I'm likely hitting them.


Issue · Github
by Sheila

Issue Number
Last Updated
Closed By


  • henahena FinlandMember

    I also got the same message with picard version 1.141 which I think is not that old version.

  • SheilaSheila Broad InstituteMember, Broadie, Moderator admin


    I will ask our Picard expert to get back to you. However, in the meantime, you can try using the latest version of Picard (2.2.1).


  • SheilaSheila Broad InstituteMember, Broadie, Moderator admin


    Can you also try validating your bam file with ValidateSamFile?


  • henahena FinlandMember
    edited April 2016

    2.2.1 seems to create same error same as running ValidateSamFile in summary mode. Here is the stacktrace from the ValidateSamFile

    Exception in thread "main" htsjdk.samtools.SAMException: Value was put into PairInfoMap more than once. 1: HWI-xxxxxxx:111:C8LG3ANXX:8:2209:18364:77568
    at htsjdk.samtools.CoordinateSortedPairInfoMap.ensureSequenceLoaded(CoordinateSortedPairInfoMap.java:133)
    at htsjdk.samtools.CoordinateSortedPairInfoMap.remove(CoordinateSortedPairInfoMap.java:86)
    at htsjdk.samtools.SamFileValidator$CoordinateSortedPairEndInfoMap.remove(SamFileValidator.java:749)
    at htsjdk.samtools.SamFileValidator.validateMateFields(SamFileValidator.java:484)
    at htsjdk.samtools.SamFileValidator.validateSamRecordsAndQualityFormat(SamFileValidator.java:282)
    at htsjdk.samtools.SamFileValidator.validateSamFile(SamFileValidator.java:200)
    at htsjdk.samtools.SamFileValidator.validateSamFileSummary(SamFileValidator.java:128)
    at picard.sam.ValidateSamFile.doWork(ValidateSamFile.java:187)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:209)
    at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:95)
    at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:105)

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    We've seen this error pop up for several different reasons; the main two are the fastq merging problem, and secondary hits not marked appropriately due to not using -M with BWA MEM. Not sure what's happening in your case, but if I remember the SAM flag rules correctly, the reads in a mate pair would have 129 and 65 respectively, or 161 and 81 (if on reverse strand). I think the program may not be recognizing that your reads are part of a mate pair because of that, and is instead considering them as part of different pairs. This could be caused by a mistake made by the aligner.

    If you only have one affected read pair out of your entire bam file I would advise just throwing out that one read pair and proceeding with your analysis. I believe you should be able to eliminate improperly paired reads by using samtools view -f 0x2 -- but don't take my word for it, read this blog post about sam flags and be sure to check the samtools docs too.

  • henahena FinlandMember

    The data is aligned with bwa aln + sampe, so there should not be secondary alignments. There is however single reads included but that has been working before only in this one read pair this problem has appeared. After removing that pair, it failed again with pair with flags 97 and 145 and after removing them as well I have pair again with 81 and 129. I also noticed that the mate information seems to be wrong on these reads and that they seem match other reads for some reason. Could this have an effect? I'm going to do name sort -> samtools fixmate -> position sort in order to see if that would help.

  • deklingdekling Broad InstituteMember admin

    Forgive my naiveté, but did you have a look at the following link? https://www.biostars.org/p/60263/ and the embedded links within?

  • SumuduSumudu Sri LankaMember


    I am doing variant calling with a human whole exome sample. I downloaded 30 exome samples from 1000Genomes project for joint v.calling with my sample.

    The 1000Genomes files are in .cram format. I used samtools view to covert them to .bam as below.

    samtools view -b HG04039.alt_bwamem_GRCh38DH.20150826.STU.exome.cram -T Homo_sapiens_assembly38.fasta -o HG04039.bam (samtools version1.9)

    Then I ran
    java -jar picard.jar ValidateSamFile I=HG04039.bam MODE=SUMMARY (picard version 2.20)

    I got an error: ERROR 2019-05-13 13:45:11 ValidateSamFile Value was put into PairInfoMap more than once. 3072: SRR794757.9422092

    I tried to fix this by using command
    samtools view -f 0x2 HG04039.bam -o HG04039_new.bam

    but again got errors after runing ValidateSamFile:

    I also tried ValidateSamFile after runing MarkDuplicates with picard, still the same.

    Is it ok to proceed with these BAM files for v.calling in exome analysis? Appreciate any recommendations on this issue.

    Thank you in advance.

  • bshifawbshifaw Member, Broadie, Moderator admin


    Can you try running the following pipe to convert your cram files to bam then check it with ValidateSamFile

    samtools view -h -T ${RefFasta} ${InputCram} |
    samtools view -b -o ${SampleName}.bam -
    samtools index -b ${SampleName}.bam
  • SumuduSumudu Sri LankaMember


    Thank you for the reply. This is what I got after running the command. I am wondering the reference file has a incompatibility.

    [email protected]:~/1000G_BAMs$ samtools view -h -T /home/wes/resources/Homo_sapiens_assembly38.fasta HG03886.alt_bwamem_GRCh38DH.20150826.STU.exome.cram | samtools view -b -o new.bam - samtools index -b new.bam
    [bam_header_read] EOF marker is absent. The input is probably truncated.
    [sam_header_read2] 3366 sequences loaded.
    [bam_header_read] invalid BAM binary header (this is not a BAM file).
    [main_samview] fail to read the header from "-".

    When I ran the command : samtools view -h -T /home/wes/resources/Homo_sapiens_assembly38.fasta HG03886.alt_bwamem_GRCh38DH.20150826.STU.exome.cram

    This is the final part of the output:
    @SQ SN:HLA-DRB115:01:01:04 LN:11056
    15:02:01 LN:10313
    @SQ SN:HLA-DRB115:03:01:01 LN:11567
    15:03:01:02 LN:11569
    @SQ SN:HLA-DRB116:02:01 LN:11005
    [sam_read1] reference 'VN:1.5' is recognized as '
    Parse error at line 1: invalid CIGAR character
    Aborted (core dumped)

    Appreciate any suggestions to solve this problem.


  • SumuduSumudu Sri LankaMember


    I solved the issue. The BAM was sorted by coordinate and ValidateSam passed.


  • bshifawbshifaw Member, Broadie, Moderator admin

    Happy you solved the problem, and thanks for sharing!

Sign In or Register to comment.