If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra

Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

suggestion for multi-samples RNAseq experiment

Hi everybody!
I've recently started working with GATK and after reading documents, tutorial and forum discussions, I set a pipeline for my experiment. I'm dealing with multi-sample RNAseq for which GATK tools are less improved and verified than DNAseq, thus I 'd like to have your suggestions.
Briefly, this is the experiment: 2 phenotypes of Sparus aurata, 8 libary per phenotype, each library consists on a pool (not barcoded) of 3 animals. Thus I have a total of** 16 samples**. My goal is to find the total number of variant sites and compare the allele frequencies between the two phenotypes. **I lack genome and SNPs database. **

Step by step:
1) I used STAR (not 2-pass) in order to map reads against my de novo assembly.
STAR --runThreadN 16 --genomeDir ./GenomeIndex --readFilesIn XXX.fastq --alignIntronMax 19 --outSAMtype BAM SortedByCoordinate --outSAMmapqUnique 60 --outFilterMultimapNmax 5 --outFilterMismatchNmax 4
2) I used the picard-tools to Mark duplicates
3) I used the picard-tools to AddOrReplaceReadGroups
4) I used the picard-tools to BuildBamIndex
5) I called the haplotype for the 16 samples with the following command:
GenomeAnalysisTK.jar -T HaplotypeCaller -R reference.fasta -I sample.bam -dontUseSoftClippedBases -ploidy 6 -ERC GVCF -o output.g.vcf
6) I used the GenotypeGVCFs to merge the samples from the same population in an unique vcf file as follow
GenomeAnalysisTK.jar -T GenotypeGVCFs -R reference.fasta -stand_call_conf 20 -stand_emit_conf 20 -ploidy 6 --variant sample1.g.vcf --variant sample2.g.vcf --variant sample3.g.vcf (8 samples) -o output_HC.vcf

Finally I'm going to filter the results with the Variant Filtration: GenomeAnalysisTK.jar -T VariantFiltration -R reference.fasta -V output.vcf -window 35 -cluster 3 -filterName FS -filter "FS > 30.0" -filterName QD -filter "QD < 2.0" -o utputFiltered.vcf

What do you think?

Now I'd like to compare the two populations, but how??? manually in excel files? vcf tools seem not to handle ploidy higher than 2.
Does anyone deal with these issues and can kindly give some tips?



  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Hi there,

    I'm afraid your preprocessing does not match our recommendations for RNAseq. Have a look at the methods documentation in the Best Practices section.

  • MariannnnaMariannnna ItalyMember

    Hi Geraldine,
    thank you for your feedback.
    I read the best practices (several times :neutral: ) but does it makes sense to perform any process related to splicing and indels if I don't have an annotated genome and a 50SE RNA seq experiment??


  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Not sure what you mean by "annotated genome"...

  • MariannnnaMariannnna ItalyMember

    Hi Geraldine,
    there isn't a reference genome. I'm working with a non-model species and intron-exon structure is not known. I'm dealing with a scaffold transcriptome assembled by myself.
    Hope to be clear...


  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Oh I see, thanks for clarifying. Then your workflow looks ok, but keep in mind you may still need to use the RNAseq-specific arguments (to allow N cigars if STAR generates any, and to assign mapping qualities).

    To compare the two populations, it depends what you want to compare, exactly. You may be able to use VariantEval according to this article:

  • MariannnnaMariannnna ItalyMember

    Ok, thank you.
    I will check the VariantEval tool!

    I'd appreciate your opinion about the pipeline I'm trying.
    I have two populations, 8 libraries/population. Each library is a pool of 3 individuals.
    The goal is to obtain all the variable positions and the allele of each sample.

    I called the haplotype of each sample (16) as following:

    GenomeAnalysisTK.jar -T HaplotypeCaller -R reference.fasta -I input.bam -dontUseSoftClippedBases -stand_call_conf 20 -stand_emit_conf 20 -ploidy 6 -ERC BP_RESOLUTION -o output.g.vcf

    Then I merged all the 16 g.vcf output as following:

    GenomeAnalysisTK.jar -T GenotypeGVCFs -R reference.fasta -ploidy 6 --variant xxx -o Genotypes.vcf

    Finally I filtered the snps by quality

    GenomeAnalysisTK.jar -T VariantFiltration -R reference.fasta -V Genotypes.vcf -window 35 -cluster 3 -filterName FS -filter "FS > 30.0" -filterName QD -filter "QD < 2.0" -o outputFiltered.vcf

    I'd really appreciate your opinion.

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Hi @Mariannna, that looks fine in principle; just keep in mind that we haven't validated using the GVCF workflow on RNAseq, so watch your results carefully. Note also that you don't need to give GenotypeGVCFs the ploidy argument, and the call/emit confidence thresholds should be given to GenotypeGVCFs, not HaplotypeCaller.

  • MariannnnaMariannnna ItalyMember

    Really thank you for your usefull suggestions!
    An additional clarification:
    -I've tried several times to run th GenotypeGVCF but I always encounter a problem with java memory. I'm usng the argument -Xmx62G (out of 64G RAM) -nt 10 (out of 16 threads). I'm working with 16 samples.
    Should I move to a more powerfull server?
    Or should I run the HC with -ERC GVCF mode? I read that the computational demand is lower. But: by using the GVCF mode, can I still distinguish samples having the reference allele from those completely lacking reads to call the variants?

    Thank you!!

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Try without multithreading and see if that reduces your memory usage.

Sign In or Register to comment.