warnings and errors during score calibration

gilgi

Dear GATK team,

The command I used:

 java -Xmx4g -jar /usr/local/src/gatk/GenomeAnalysisTK-2.0-0-g4c0ffd4/GenomeAnalysisTK.jar   -T BaseRecalibrator   -I my_merged_lane1.bam -R reference_genome.fasta  -knownSites my_snps.bed  -o recal_data_lane1.grp

It seemed it worked, but I got the following :

WARN  23:15:29,538 RestStorageService - Error Response: PUT '/GATK_Run_Reports/7wP8pzXrHgtLBjkDcJtGqpqLljupw7aN.report.xml.gz' -- ResponseCode: 403, ResponseStatus: Forbidden, Request Headers: [Content-Length: 343, Content-MD5: gmCP98zrgZqoNpLiyKC2+w==, Content-Type: application/octet-stream, x-amz-meta-md5-hash: 82608ff7cceb819aa83692e2c8a0b6fb, Date: Thu, 25 Oct 2012 21:15:28 GMT, Authorization: AWS AKIAJXU7VIHBPDW4TDSQ:rX//6IdcVn7cmu+vh3BM1OdRuG0=, User-Agent: JetS3t/0.8.1 (Linux/2.6.32-71.29.1.el6.x86_64; amd64; en; JVM 1.6.0_17), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers: [x-amz-request-id: 44927B6871494046, x-amz-id-2: 6PtxgxrhMcCLXlVmKviqkFWT+jHmrg/hOvEqtJ1Z160m9O7aoxTYVnNq/OSGMkg9, Content-Type: application/xml, Transfer-Encoding: chunked, Date: Thu, 25 Oct 2012 20:43:55 GMT, Connection: close, Server: AmazonS3] 
WARN  23:15:30,558 RestStorageService - Adjusted time offset in response to RequestTimeTooSkewed error. Local machine and S3 server disagree on the time by approximately -1894 seconds. Retrying connection. 
INFO  23:15:31,373 GATKRunReport - Uploaded run statistics report to AWS S3 

And for the -T PrintReads I got:

WARN  01:32:59,987 RestStorageService - Error Response: PUT '/GATK_Run_Reports/rDrSv8aayCQwBzzwAC3R4P1NNhU8eNtF.report.xml.gz' -- 
ResponseCode: 403, ResponseStatus: Forbidden, Request Headers:
[Content-Length: 347, Content-MD5: /HRSKSCV6FIXe/03JWlMwQ==, Content-Type:
application/octet-stream, x-amz-meta-md5-hash:
fc7452292095e852177bfd3725694cc1, Date: Thu, 25 Oct 2012 23:32:58 GMT,
Authorization: AWS AKIAJXU7VIHBPDW4TDSQ:ye/pDLSwNPHgH2kNUHKWCAt3YQ4=,
User-Agent: JetS3t/0.8.1 (Linux/2.6.32-71.29.1.el6.x86_64; amd64; en; JVM
1.6.0_17), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers:
[x-amz-request-id: 11DE8B74B2E028F3, x-amz-id-2:
/mV+Znu5Rq1j8tub42Y4CNz2lD5npQrtgFDM2OL5Tap3Whtt4rL4KOJLFtqhgNbA,
Content-Type: application/xml, Transfer-Encoding: chunked, Date: Thu, 25 Oct
2012 23:01:26 GMT, Connection: close, Server: AmazonS3] 
WARN  01:33:00,975 RestStorageService - Adjusted time offset in response to
RequestTimeTooSkewed error. Local machine and S3 server disagree on the time
by approximately -1893 seconds. Retrying connection. 

I will be grateful if you could let me know if I'm doing something wrong, and if should be worried about this.

Thanks a lot.

Geraldine_VdAuwera

Hi there,

It looks like the phone home feature is causing problems (likely to happen if you were running without an internet connection). See this article for an explanation and solution:

http://www.broadinstitute.org/gatk/guide/article?id=1250

gilgi

Thanks a lot for the quick reply. I do have internet connection, so will try understand why it happens.

I can use the output that were generated - right?

Thanks.

Geraldine_VdAuwera

Yes, your output should be fine -- it's just the report upload that didn't work. It may just have been a transient issue on the amazon cloud side. If it doesn't happen again, don't worry about it.

gilgi

Thanks!

didiercroes

Hi Geraldine,

Can you tell me why gatk try to upload a report to the amazon cloud. Is it for debugging purpose?

Kind regards

Geraldine_VdAuwera

Yes, and to collect general usage statistics. It helps us understand what tools people use the most and what are the most common error modes. It can be turned off if it causes trouble, but we do prefer to have it on because the data is useful and helps us improve the software.

shingwan

To be honest, I don't mind the phone home feature as I do believe it will help to improve the tools, however, I am experiencing problem with it (I guess) where more than half of my runs fail with the following error:

WARN  12:24:45,542 RestStorageService - Error Response: PUT '/GATK_Run_Reports/hGXmuykExcJeliySbqP0nxFdoFtbIC9d.report.xml.gz' -- ResponseCode: 403, ResponseStatus: Forbidden, Request Headers: [Content-Length: 402, Content-MD5: 40vJhBYx0ynIeLx3O/8BqQ==, Content-Type: application/octet-stream, x-amz-meta-md5-hash: e34bc9841631d329c878bc773bff01a9, Date: Mon, 20 May 2013 04:24:43 GMT, Authorization: AWS AKIAIMHBU7X642TCHQ2A:xrZWTOIxf8j5dHmJtT61cUFMYvA=, User-Agent: JetS3t/0.8.1 (Linux/3.2.0-26-generic; amd64; en; JVM 1.6.0_24), Host: s3.amazonaws.com, Expect: 100-continue], Response Headers: [x-amz-request-id: A5080602D6BE61D6, x-amz-id-2: 0a31I06+3RWGOjEjMhojsioYHbyjK8u5jxykQqSrtjg+SmggliXl50HkZqKTr5Zh, Content-Type: application/xml, Transfer-Encoding: chunked, Date: Mon, 20 May 2013 04:03:34 GMT, Connection: close, Server: AmazonS3]
WARN  12:24:46,516 RestStorageService - Adjusted time offset in response to RequestTimeTooSkewed error. Local machine and S3 server disagree on the time by approximately -1271 seconds. Retrying connection.

As we are using a server cluster and I am not the administrator, I really have no idea how to solve this problem locally, I guess that only leave me with the option to switch off the phone home feature?

Geraldine_VdAuwera

Hi @shingwan,

Normally the failure of the phone home connection shouldn't be making the run itself fail. What you see there is just a warning about the connection failure, but if your run failed there should be more information in the console output about what is the actual problem that caused the failure.

shingwan

@Geraldine_VdAuwera, thank you for your information. However, whenever my job fail, that is the only warning that I saw. Most of the time, a re-run will solve the problem (as in, just use the same parameter and execute the queue) . That makes me confuse. I am trying to reproduce the error and maybe able to send the error log out.