Bug Bulletin: we have identified a bug that affects indexing when producing gzipped VCFs. This will be fixed in the upcoming 3.2 release; in the meantime you need to reindex gzipped VCFs using Tabix.

warnings and errors during score calibration

gilgigilgi Posts: 19Member
edited October 2012 in Ask the team

Dear GATK team,

The command I used:

 java -Xmx4g -jar /usr/local/src/gatk/GenomeAnalysisTK-2.0-0-g4c0ffd4/GenomeAnalysisTK.jar   -T BaseRecalibrator   -I my_merged_lane1.bam -R reference_genome.fasta  -knownSites my_snps.bed  -o recal_data_lane1.grp

It seemed it worked, but I got the following :

WARN  23:15:29,538 RestStorageService - Error Response: PUT '/GATK_Run_Reports/7wP8pzXrHgtLBjkDcJtGqpqLljupw7aN.report.xml.gz' -- ResponseCode: 403, ResponseStatus: Forbidden, Request Headers: [Content-Length: 343, Content-MD5: gmCP98zrgZqoNpLiyKC2+w==, Content-Type: application/octet-stream, x-amz-meta-md5-hash: 82608ff7cceb819aa83692e2c8a0b6fb, Date: Thu, 25 Oct 2012 21:15:28 GMT, Authorization: AWS AKIAJXU7VIHBPDW4TDSQ:rX//6IdcVn7cmu+vh3BM1OdRuG0=, User-Agent: JetS3t/0.8.1 (Linux/2.6.32-71.29.1.el6.x86_64; amd64; en; JVM 1.6.0_17), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers: [x-amz-request-id: 44927B6871494046, x-amz-id-2: 6PtxgxrhMcCLXlVmKviqkFWT+jHmrg/hOvEqtJ1Z160m9O7aoxTYVnNq/OSGMkg9, Content-Type: application/xml, Transfer-Encoding: chunked, Date: Thu, 25 Oct 2012 20:43:55 GMT, Connection: close, Server: AmazonS3] 
WARN  23:15:30,558 RestStorageService - Adjusted time offset in response to RequestTimeTooSkewed error. Local machine and S3 server disagree on the time by approximately -1894 seconds. Retrying connection. 
INFO  23:15:31,373 GATKRunReport - Uploaded run statistics report to AWS S3 

And for the -T PrintReads I got:

WARN  01:32:59,987 RestStorageService - Error Response: PUT '/GATK_Run_Reports/rDrSv8aayCQwBzzwAC3R4P1NNhU8eNtF.report.xml.gz' -- 
ResponseCode: 403, ResponseStatus: Forbidden, Request Headers:
[Content-Length: 347, Content-MD5: /HRSKSCV6FIXe/03JWlMwQ==, Content-Type:
application/octet-stream, x-amz-meta-md5-hash:
fc7452292095e852177bfd3725694cc1, Date: Thu, 25 Oct 2012 23:32:58 GMT,
Authorization: AWS AKIAJXU7VIHBPDW4TDSQ:ye/pDLSwNPHgH2kNUHKWCAt3YQ4=,
User-Agent: JetS3t/0.8.1 (Linux/2.6.32-71.29.1.el6.x86_64; amd64; en; JVM
1.6.0_17), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers:
[x-amz-request-id: 11DE8B74B2E028F3, x-amz-id-2:
/mV+Znu5Rq1j8tub42Y4CNz2lD5npQrtgFDM2OL5Tap3Whtt4rL4KOJLFtqhgNbA,
Content-Type: application/xml, Transfer-Encoding: chunked, Date: Thu, 25 Oct
2012 23:01:26 GMT, Connection: close, Server: AmazonS3] 
WARN  01:33:00,975 RestStorageService - Adjusted time offset in response to
RequestTimeTooSkewed error. Local machine and S3 server disagree on the time
by approximately -1893 seconds. Retrying connection. 

I will be grateful if you could let me know if I'm doing something wrong, and if should be worried about this.

Thanks a lot.

Post edited by Geraldine_VdAuwera on
Tagged:

Answers

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 5,230Administrator, GSA Member admin

    Hi there,

    It looks like the phone home feature is causing problems (likely to happen if you were running without an internet connection). See this article for an explanation and solution:

    http://www.broadinstitute.org/gatk/guide/article?id=1250

    Geraldine Van der Auwera, PhD

  • gilgigilgi Posts: 19Member

    Thanks a lot for the quick reply. I do have internet connection, so will try understand why it happens.

    I can use the output that were generated - right?

    Thanks.

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 5,230Administrator, GSA Member admin

    Yes, your output should be fine -- it's just the report upload that didn't work. It may just have been a transient issue on the amazon cloud side. If it doesn't happen again, don't worry about it.

    Geraldine Van der Auwera, PhD

  • didiercroesdidiercroes Posts: 4Member

    Hi Geraldine,

    Can you tell me why gatk try to upload a report to the amazon cloud. Is it for debugging purpose?

    Kind regards

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 5,230Administrator, GSA Member admin

    Yes, and to collect general usage statistics. It helps us understand what tools people use the most and what are the most common error modes. It can be turned off if it causes trouble, but we do prefer to have it on because the data is useful and helps us improve the software.

    Geraldine Van der Auwera, PhD

  • shingwanshingwan Posts: 16Member

    To be honest, I don't mind the phone home feature as I do believe it will help to improve the tools, however, I am experiencing problem with it (I guess) where more than half of my runs fail with the following error:

    WARN  12:24:45,542 RestStorageService - Error Response: PUT '/GATK_Run_Reports/hGXmuykExcJeliySbqP0nxFdoFtbIC9d.report.xml.gz' -- ResponseCode: 403, ResponseStatus: Forbidden, Request Headers: [Content-Length: 402, Content-MD5: 40vJhBYx0ynIeLx3O/8BqQ==, Content-Type: application/octet-stream, x-amz-meta-md5-hash: e34bc9841631d329c878bc773bff01a9, Date: Mon, 20 May 2013 04:24:43 GMT, Authorization: AWS AKIAIMHBU7X642TCHQ2A:xrZWTOIxf8j5dHmJtT61cUFMYvA=, User-Agent: JetS3t/0.8.1 (Linux/3.2.0-26-generic; amd64; en; JVM 1.6.0_24), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers: [x-amz-request-id: A5080602D6BE61D6, x-amz-id-2: 0a31I06+3RWGOjEjMhojsioYHbyjK8u5jxykQqSrtjg+SmggliXl50HkZqKTr5Zh, Content-Type: application/xml, Transfer-Encoding: chunked, Date: Mon, 20 May 2013 04:03:34 GMT, Connection: close, Server: AmazonS3]
    WARN  12:24:46,516 RestStorageService - Adjusted time offset in response to RequestTimeTooSkewed error. Local machine and S3 server disagree on the time by approximately -1271 seconds. Retrying connection.
    

    As we are using a server cluster and I am not the administrator, I really have no idea how to solve this problem locally, I guess that only leave me with the option to switch off the phone home feature?

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 5,230Administrator, GSA Member admin

    Hi @shingwan,

    Normally the failure of the phone home connection shouldn't be making the run itself fail. What you see there is just a warning about the connection failure, but if your run failed there should be more information in the console output about what is the actual problem that caused the failure.

    Geraldine Van der Auwera, PhD

  • shingwanshingwan Posts: 16Member

    @Geraldine_VdAuwera, thank you for your information. However, whenever my job fail, that is the only warning that I saw. Most of the time, a re-run will solve the problem (as in, just use the same parameter and execute the queue) . That makes me confuse. I am trying to reproduce the error and maybe able to send the error log out.

Sign In or Register to comment.