This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!
(howto) Map and mark duplicates
See Tutorial#6747 for a comparison of MarkDuplicates and MarkDuplicatesWithMateCigar, downloadable example data to follow along, and additional commentary.
Map the read data to the reference and mark duplicates.
- This tutorial assumes adapter sequences have been removed.
- Identify read group information
- Generate a SAM file containing aligned reads
- Convert to BAM, sort and mark duplicates
1. Identify read group information
The read group information is key for downstream GATK functionality. The GATK will not work without a read group tag. Make sure to enter as much metadata as you know about your data in the read group fields provided. For more information about all the possible fields in the @RG tag, take a look at the SAM specification.
Compose the read group identifier in the following format:
\t stands for the tab character.
2. Generate a SAM file containing aligned reads
Run the following BWA command:
In this command, replace read group info by the read group identifier composed in the previous step.
bwa mem -M -R ’<read group info>’ -p reference.fa raw_reads.fq > aligned_reads.sam
<read group info> bit with the read group identifier you composed at the previous step.
-M flag causes BWA to mark shorter split hits as secondary (essential for Picard compatibility).
This creates a file called
aligned_reads.sam containing the aligned reads from all input files, combined, annotated and aligned to the same reference.
Note that here we are using a command that is specific for pair end data in an interleaved (read pairs together in the same file, with the forward read followed directly by its paired reverse read) fastq file, which is what we are providing to you as a tutorial file. To map other types of datasets (e.g. single-ended or pair-ended in forward/reverse read files) you will need to adapt the command accordingly. Please see the BWA documentation for exact usage and more options for these commands.
3. Convert to BAM, sort and mark duplicates
These initial pre-processing operations format the data to suit the requirements of the GATK tools.
Run the following Picard command to sort the SAM file and convert it to BAM:
java -jar picard.jar SortSam \ INPUT=aligned_reads.sam \ OUTPUT=sorted_reads.bam \ SORT_ORDER=coordinate
This creates a file called
sorted_reads.bam containing the aligned reads sorted by coordinate.
Run the following Picard command to mark duplicates:
java -jar picard.jar MarkDuplicates \ INPUT=sorted_reads.bam \ OUTPUT=dedup_reads.bam \ METRICS_FILE=metrics.txt
This creates a sorted BAM file called
dedup_reads.bam with the same content as the input file, except that any duplicate reads are marked as such. It also produces a metrics file called
metrics.txt containing (can you guess?) metrics.
Run the following Picard command to index the BAM file:
java -jar picard.jar BuildBamIndex \ INPUT=dedup_reads.bam
This creates an index file for the BAM file called