Recommendation on performance using scatter/gather
I am currently running an analysis using the HaplotypeCaller on 300 large BAM files on our cluster and decided to chunk the the genome in 3MB bins in order for them to be processed in a decent time. I'm however experiencing very long runtimes as more and more jobs get scheduled to run in parallel on the same files. Looking at the GATK options, I saw these 2 that I thought could be of help and was wondering what were the recommendation for using them:
More precisely, does the num_bam_file_handles increase processing time by a lot? and what is the default value for --read_buffer_size ?
Thanks a lot,