To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

Picard IlluminaBaseCallsToSam - clocs file issue - more elements than expected

I am attempting to demultiplex a lane of Illumina HiSeq2500 data using Picard IlluminaBasecallsToSam (v2.5, Java(TM) SE Runtime Environment (build 1.8.0_20-b26)). This tool fails to complete and states that

picard.PicardException: Read the number of expected bins( 65600) but still had more elements in file( /Illumina/Basecalls/170808_SN218_0895_AHN3VWBCXY/Data/Intensities/L001/s_1_1107.clocs)

I've also run Picard CheckIlluminaDirectory and everything turns out to be fine. Also, I can successfully demultiplex lane 2 from this run that has the same read-structure. I suspected that a file was corrupted during network transfer so I re-ran RTA but Picard IlluminaBasecallsToSam still has the same result.

I'd appreciate any idea about what could be causing this error.

Complete error below:

[Tue Aug 15 11:01:59 EDT 2017] picard.illumina.IlluminaBasecallsToSam BASECALLS_DIR=/Illumina/Basecalls/170808_SN218_0895_AHN3VWBCXY/Data/Intensities/BaseCalls BARCODES_DIR=/Project/Capture LANE=1 RUN_BARCODE=HN3VWBCXY170810 READ_GROUP_ID=HN3VWBCXY170810 SEQUENCING_CENTER=ABC READ_STRUCTURE=98T8B6M8B98T LIBRARY_PARAMS=/Project/DemultCapture.txt NUM_PROCESSORS=6 IGNORE_UNEXPECTED_BARCODES=true TMP_DIR=[/Project/TMP]    PLATFORM=illumina ADAPTERS_TO_CHECK=[INDEXED, DUAL_INDEXED, NEXTERA_V2, FLUIDIGM] FORCE_GC=true APPLY_EAMSS_FILTER=true MAX_READS_IN_RAM_PER_TILE=1200000 MINIMUM_QUALITY=2 INCLUDE_NON_PF_READS=true MOLECULAR_INDEX_TAG=RX MOLECULAR_INDEX_BASE_QUALITY_TAG=QX VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
[Tue Aug 15 11:01:59 EDT 2017] Executing as rb@rcsgc22 on Linux 2.6.32-358.2.1.el6.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_20-b26; Picard version: 2.5.0(2c370988aefe41f579920c8a6a678a201c5261c1_1466708365)
INFO    2017-08-15 11:04:47 IlluminaBasecallsToSam  DONE_READING STRUCTURE IS 98T8B6M8B98T
INFO    2017-08-15 11:05:36 IlluminaBasecallsConverter  Read     1,000,000 records.  Elapsed time: 00:03:36s.  Time for last 1,000,000:   35s.  Last read position: */*
INFO    2017-08-15 11:06:14 IlluminaBasecallsConverter  Read     2,000,000 records.  Elapsed time: 00:04:14s.  Time for last 1,000,000:   37s.  Last read position: */*
INFO    2017-08-15 11:06:32 IlluminaBasecallsConverter  Read     3,000,000 records.  Elapsed time: 00:04:32s.  Time for last 1,000,000:   18s.  Last read position: */*
INFO    2017-08-15 11:07:00 IlluminaBasecallsConverter  Read     4,000,000 records.  Elapsed time: 00:04:59s.  Time for last 1,000,000:   27s.  Last read position: */*
INFO    2017-08-15 11:07:00 IlluminaBasecallsConverter  Before explicit GC, Runtime.totalMemory()=6693060608
INFO    2017-08-15 11:07:01 IlluminaBasecallsConverter  After explicit GC, Runtime.totalMemory()=6512705536
INFO    2017-08-15 11:07:22 IlluminaBasecallsConverter  Read     5,000,000 records.  Elapsed time: 00:05:22s.  Time for last 1,000,000:   22s.  Last read position: */*
INFO    2017-08-15 11:07:33 IlluminaBasecallsConverter  Read     6,000,000 records.  Elapsed time: 00:05:33s.  Time for last 1,000,000:   11s.  Last read position: */*
INFO    2017-08-15 11:07:41 IlluminaBasecallsConverter  Read     7,000,000 records.  Elapsed time: 00:05:41s.  Time for last 1,000,000:    8s.  Last read position: */*
INFO    2017-08-15 11:07:56 IlluminaBasecallsConverter  Read     8,000,000 records.  Elapsed time: 00:05:56s.  Time for last 1,000,000:   14s.  Last read position: */*
INFO    2017-08-15 11:08:09 IlluminaBasecallsConverter  Read     9,000,000 records.  Elapsed time: 00:06:09s.  Time for last 1,000,000:   12s.  Last read position: */*
INFO    2017-08-15 11:08:24 IlluminaBasecallsConverter  Read    10,000,000 records.  Elapsed time: 00:06:24s.  Time for last 1,000,000:   15s.  Last read position: */*
INFO    2017-08-15 11:08:35 IlluminaBasecallsConverter  Read    11,000,000 records.  Elapsed time: 00:06:35s.  Time for last 1,000,000:   10s.  Last read position: */*
INFO    2017-08-15 11:08:48 IlluminaBasecallsConverter  Read    12,000,000 records.  Elapsed time: 00:06:48s.  Time for last 1,000,000:   12s.  Last read position: */*
INFO    2017-08-15 11:08:58 IlluminaBasecallsConverter  Read    13,000,000 records.  Elapsed time: 00:06:58s.  Time for last 1,000,000:   10s.  Last read position: */*
INFO    2017-08-15 11:09:06 IlluminaBasecallsConverter  Read    14,000,000 records.  Elapsed time: 00:07:06s.  Time for last 1,000,000:    8s.  Last read position: */*
INFO    2017-08-15 11:09:16 IlluminaBasecallsConverter  Read    15,000,000 records.  Elapsed time: 00:07:16s.  Time for last 1,000,000:   10s.  Last read position: */*
INFO    2017-08-15 11:09:25 IlluminaBasecallsConverter  Read    16,000,000 records.  Elapsed time: 00:07:24s.  Time for last 1,000,000:    8s.  Last read position: */*
INFO    2017-08-15 11:09:38 IlluminaBasecallsConverter  Read    17,000,000 records.  Elapsed time: 00:07:38s.  Time for last 1,000,000:   13s.  Last read position: */*
INFO    2017-08-15 11:09:57 IlluminaBasecallsConverter  Write     1,000,000 records.  Elapsed time: 00:07:57s.  Time for last 1,000,000:   13s.  Last read position: */*
INFO    2017-08-15 11:10:03 IlluminaBasecallsConverter  Read    18,000,000 records.  Elapsed time: 00:08:03s.  Time for last 1,000,000:   25s.  Last read position: */*
INFO    2017-08-15 11:10:08 IlluminaBasecallsConverter  Write     2,000,000 records.  Elapsed time: 00:08:07s.  Time for last 1,000,000:   10s.  Last read position: */*
INFO    2017-08-15 11:10:17 IlluminaBasecallsConverter  Write     3,000,000 records.  Elapsed time: 00:08:17s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:10:27 IlluminaBasecallsConverter  Read    19,000,000 records.  Elapsed time: 00:08:26s.  Time for last 1,000,000:   23s.  Last read position: */*
INFO    2017-08-15 11:10:29 IlluminaBasecallsConverter  Write     4,000,000 records.  Elapsed time: 00:08:29s.  Time for last 1,000,000:   12s.  Last read position: */*
INFO    2017-08-15 11:10:39 IlluminaBasecallsConverter  Write     5,000,000 records.  Elapsed time: 00:08:38s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:10:48 IlluminaBasecallsConverter  Write     6,000,000 records.  Elapsed time: 00:08:48s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:10:48 IlluminaBasecallsConverter  Read    20,000,000 records.  Elapsed time: 00:08:48s.  Time for last 1,000,000:   21s.  Last read position: */*
INFO    2017-08-15 11:10:59 IlluminaBasecallsConverter  Write     7,000,000 records.  Elapsed time: 00:08:59s.  Time for last 1,000,000:   10s.  Last read position: */*
INFO    2017-08-15 11:11:08 IlluminaBasecallsConverter  Write     8,000,000 records.  Elapsed time: 00:09:08s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:11:14 IlluminaBasecallsConverter  Read    21,000,000 records.  Elapsed time: 00:09:14s.  Time for last 1,000,000:   25s.  Last read position: */*
INFO    2017-08-15 11:11:18 IlluminaBasecallsConverter  Write     9,000,000 records.  Elapsed time: 00:09:17s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:11:27 IlluminaBasecallsConverter  Write    10,000,000 records.  Elapsed time: 00:09:27s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:11:36 IlluminaBasecallsConverter  Read    22,000,000 records.  Elapsed time: 00:09:36s.  Time for last 1,000,000:   22s.  Last read position: */*
INFO    2017-08-15 11:11:37 IlluminaBasecallsConverter  Write    11,000,000 records.  Elapsed time: 00:09:36s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:11:46 IlluminaBasecallsConverter  Write    12,000,000 records.  Elapsed time: 00:09:45s.  Time for last 1,000,000:    8s.  Last read position: */*
INFO    2017-08-15 11:11:55 IlluminaBasecallsConverter  Write    13,000,000 records.  Elapsed time: 00:09:55s.  Time for last 1,000,000:    9s.  Last read position: */*
INFO    2017-08-15 11:11:56 IlluminaBasecallsConverter  Read    23,000,000 records.  Elapsed time: 00:09:56s.  Time for last 1,000,000:   20s.  Last read position: */*
INFO    2017-08-15 11:12:00 IlluminaBasecallsConverter  Before explicit GC, Runtime.totalMemory()=7281311744
INFO    2017-08-15 11:12:00 IlluminaBasecallsConverter  After explicit GC, Runtime.totalMemory()=7281311744
Exception in thread "pool-1-thread-6" ERROR 2017-08-15 11:12:04 IlluminaBasecallsConverter  Failure encountered in worker thread; attempting to shut down remaining worker threads and terminate ...
java.lang.InterruptedException
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.awaitWorkComplete(IlluminaBasecallsConverter.java:709)
    at picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:318)
    at picard.illumina.IlluminaBasecallsToSam.doWork(IlluminaBasecallsToSam.java:230)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:208)
    at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:95)
    at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:105)
picard.PicardException: Read the number of expected bins( 65600) but still had more elements in file( /Illumina/Basecalls/170808_SN218_0895_AHN3VWBCXY/Data/Intensities/L001/s_1_1107.clocs) 
    at picard.illumina.parser.readers.ClocsFileReader.hasNext(ClocsFileReader.java:150)
    at picard.illumina.parser.PosParser$1.hasNext(PosParser.java:98)
    at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:120)
    at picard.illumina.parser.PerTileParser.maybeAdvance(PerTileParser.java:99)
    at picard.illumina.parser.PerTileParser.next(PerTileParser.java:109)
    at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:133)
    at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:555)
    at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:657)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Exception in thread "pool-1-thread-1" java.lang.ArrayIndexOutOfBoundsException
    at htsjdk.samtools.util.BlockCompressedOutputStream.deflateBlock(BlockCompressedOutputStream.java:357)
    at htsjdk.samtools.util.BlockCompressedOutputStream.write(BlockCompressedOutputStream.java:250)
    at java.io.DataOutputStream.write(DataOutputStream.java:107)
    at htsjdk.samtools.util.BinaryCodec.writeBytes(BinaryCodec.java:206)
    at htsjdk.samtools.util.BinaryCodec.writeByteBuffer(BinaryCodec.java:174)
    at htsjdk.samtools.util.BinaryCodec.writeInt(BinaryCodec.java:220)
    at htsjdk.samtools.BAMRecordCodec.encode(BAMRecordCodec.java:132)
    at htsjdk.samtools.BAMFileWriter.writeAlignment(BAMFileWriter.java:134)
    at htsjdk.samtools.SAMFileWriterImpl.addAlignment(SAMFileWriterImpl.java:190)
    at picard.illumina.IlluminaBasecallsToSam$SAMFileWriterWrapper.write(IlluminaBasecallsToSam.java:483)
    at picard.illumina.IlluminaBasecallsToSam$SAMFileWriterWrapper.write(IlluminaBasecallsToSam.java:472)
    at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$3.run(IlluminaBasecallsConverter.java:831)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Exception in thread "pool-1-thread-5" java.lang.ArrayIndexOutOfBoundsException
    at htsjdk.samtools.util.BlockCompressedOutputStream.deflateBlock(BlockCompressedOutputStream.java:357)
    at htsjdk.samtools.util.BlockCompressedOutputStream.write(BlockCompressedOutputStream.java:250)
    at java.io.DataOutputStream.write(DataOutputStream.java:107)
    at htsjdk.samtools.util.BinaryCodec.writeBytes(BinaryCodec.java:206)
    at htsjdk.samtools.util.BinaryCodec.writeByteBuffer(BinaryCodec.java:174)
    at htsjdk.samtools.util.BinaryCodec.writeInt(BinaryCodec.java:220)
    at htsjdk.samtools.BAMRecordCodec.encode(BAMRecordCodec.java:131)
    at htsjdk.samtools.BAMFileWriter.writeAlignment(BAMFileWriter.java:134)
    at htsjdk.samtools.SAMFileWriterImpl.addAlignment(SAMFileWriterImpl.java:190)
    at picard.illumina.IlluminaBasecallsToSam$SAMFileWriterWrapper.write(IlluminaBasecallsToSam.java:483)
    at picard.illumina.IlluminaBasecallsToSam$SAMFileWriterWrapper.write(IlluminaBasecallsToSam.java:472)
    at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$3.run(IlluminaBasecallsConverter.java:831)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
[Tue Aug 15 11:12:06 EDT 2017] picard.illumina.IlluminaBasecallsToSam done. Elapsed time: 10.12 minutes.
Runtime.totalMemory()=7301234688
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" picard.PicardException: Failure encountered in worker thread; see log for details.
    at picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:321)
    at picard.illumina.IlluminaBasecallsToSam.doWork(IlluminaBasecallsToSam.java:230)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:208)
    at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:95)
    at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:105)
Exception in thread "pool-1-thread-3" picard.PicardException: IOException opening cluster binary file /Illumina/Basecalls/170808_SN218_0895_AHN3VWBCXY/Data/Intensities/BaseCalls/L001/s_1_1109.filter
    at picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:119)
    at picard.illumina.parser.readers.MMapBackedIteratorFactory.getByteIterator(MMapBackedIteratorFactory.java:66)
    at picard.illumina.parser.readers.FilterFileReader.<init>(FilterFileReader.java:68)
    at picard.illumina.parser.FilterParser$1.<init>(FilterParser.java:55)
    at picard.illumina.parser.FilterParser.makeTileIterator(FilterParser.java:54)
    at picard.illumina.parser.PerTileParser.advanceTile(PerTileParser.java:80)
    at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:121)
    at picard.illumina.parser.IlluminaDataProvider.hasNext(IlluminaDataProvider.java:104)
    at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:554)
    at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:657)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedByInterruptException
    at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
    at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:314)
    at picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:113)
    ... 12 more
Exception in thread "pool-1-thread-2" picard.PicardException: IOException opening cluster binary file /Illumina/Basecalls/170808_SN218_0895_AHN3VWBCXY/Data/Intensities/L001/s_1_1110.clocs
    at picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:119)
    at picard.illumina.parser.readers.MMapBackedIteratorFactory.getByteIterator(MMapBackedIteratorFactory.java:66)
    at picard.illumina.parser.readers.ClocsFileReader.<init>(ClocsFileReader.java:85)
    at picard.illumina.parser.PosParser.makeTileIterator(PosParser.java:83)
    at picard.illumina.parser.PerTileParser.advanceTile(PerTileParser.java:80)
    at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:121)
    at picard.illumina.parser.PerTileParser.maybeAdvance(PerTileParser.java:99)
    at picard.illumina.parser.PerTileParser.next(PerTileParser.java:109)
    at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:133)
    at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:555)
    at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:657)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.ClosedByInterruptException
    at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
    at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:314)
    at picard.illumina.parser.readers.MMapBackedIteratorFactory.getBuffer(MMapBackedIteratorFactory.java:113)
    ... 13 more
Exception in thread "pool-1-thread-7" picard.PicardException: Error reading from file /Project/CorrectCapture/s_1_1111_barcode.txt
    at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:120)
    at picard.util.AbstractInputParser.advance(AbstractInputParser.java:85)
    at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44)
    at htsjdk.samtools.util.AbstractIterator.hasNext(AbstractIterator.java:44)
    at picard.illumina.parser.readers.BarcodeFileReader.hasNext(BarcodeFileReader.java:42)
    at picard.illumina.parser.BarcodeParser$BarcodeDataIterator.hasNext(BarcodeParser.java:69)
    at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:120)
    at picard.illumina.parser.IlluminaDataProvider.hasNext(IlluminaDataProvider.java:104)
    at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:554)
    at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:657)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: htsjdk.samtools.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
    at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:74)
    at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:103)
    ... 12 more
Caused by: java.nio.channels.ClosedByInterruptException
    at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
    at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:163)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.BufferedReader.fill(BufferedReader.java:161)
    at java.io.BufferedReader.readLine(BufferedReader.java:324)
    at java.io.BufferedReader.readLine(BufferedReader.java:389)
    at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:70)
    ... 13 more

Answers

  • rbrnsrbrns DFCIMember

    Also, Illumina's bcl2fastq software does not throw any errors while Picard does. I've successfully de-multiplexed may lanes with this same read structure and barcode set with Picard before; this is the first time I've seen this error.

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @rbrns
    Hi,

    It looks like a threading issue. What happens if you set NUM_PROCESSORS =0?

    -Sheila

  • rbrnsrbrns DFCIMember

    Thank you for getting back to me about this. I've tried this again setting NUM_PROCESSORS=0 and still get a very similar error output where it seems to abort after encountering the clocs file that is mentioned.

    [Tue Aug 22 11:51:21 EDT 2017] picard.illumina.IlluminaBasecallsToSam BASECALLS_DIR=/Illumina/Basecalls/170808_SN218_0895_AHN3VWBCXY/Data/Intensities/BaseCalls BARCODES_DIR=/Project/Capture LANE=1 RUN_BARCODE=HN3VWBCXY170810 READ_GROUP_ID=HN3VWBCXY170810 SEQUENCING_CENTER=ABC READ_STRUCTURE=98T8B6M8B98T LIBRARY_PARAMS=/Project/DemultCapture.txt NUM_PROCESSORS=0 IGNORE_UNEXPECTED_BARCODES=true TMP_DIR=[/Project/TMP]    PLATFORM=illumina ADAPTERS_TO_CHECK=[INDEXED, DUAL_INDEXED, NEXTERA_V2, FLUIDIGM] FORCE_GC=true APPLY_EAMSS_FILTER=true MAX_READS_IN_RAM_PER_TILE=1200000 MINIMUM_QUALITY=2 INCLUDE_NON_PF_READS=true MOLECULAR_INDEX_TAG=RX MOLECULAR_INDEX_BASE_QUALITY_TAG=QX VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
    [Tue Aug 22 11:51:21 EDT 2017] Executing as rb@rcsgx4 on Linux 2.6.32-504.el6.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_20-b26; Picard version: 2.10.3-SNAPSHOT
    INFO    2017-08-22 11:53:45 IlluminaBasecallsToSam  DONE_READING STRUCTURE IS 98T8B6M8B98T
    INFO    2017-08-22 11:54:07 BasecallsConverter  Read     1,000,000 records.  Elapsed time: 00:00:22s.  Time for last 1,000,000:    6s.  Last read position: */*
    INFO    2017-08-22 11:54:33 BasecallsConverter  Read     2,000,000 records.  Elapsed time: 00:00:48s.  Time for last 1,000,000:   26s.  Last read position: */*
    INFO    2017-08-22 11:54:45 BasecallsConverter  Read     3,000,000 records.  Elapsed time: 00:00:59s.  Time for last 1,000,000:   11s.  Last read position: */*
    INFO    2017-08-22 11:55:29 BasecallsConverter  Read     4,000,000 records.  Elapsed time: 00:01:44s.  Time for last 1,000,000:   44s.  Last read position: */*
    INFO    2017-08-22 11:55:37 BasecallsConverter  Read     5,000,000 records.  Elapsed time: 00:01:51s.  Time for last 1,000,000:    7s.  Last read position: */*
    INFO    2017-08-22 11:55:42 BasecallsConverter  Read     6,000,000 records.  Elapsed time: 00:01:56s.  Time for last 1,000,000:    5s.  Last read position: */*
    INFO    2017-08-22 11:55:51 BasecallsConverter  Read     7,000,000 records.  Elapsed time: 00:02:05s.  Time for last 1,000,000:    8s.  Last read position: */*
    INFO    2017-08-22 11:55:54 BasecallsConverter  Read     8,000,000 records.  Elapsed time: 00:02:08s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:56:05 BasecallsConverter  Read     9,000,000 records.  Elapsed time: 00:02:19s.  Time for last 1,000,000:   11s.  Last read position: */*
    INFO    2017-08-22 11:56:42 BasecallsConverter  Read    10,000,000 records.  Elapsed time: 00:02:56s.  Time for last 1,000,000:   36s.  Last read position: */*
    INFO    2017-08-22 11:56:51 BasecallsConverter  Read    11,000,000 records.  Elapsed time: 00:03:05s.  Time for last 1,000,000:    9s.  Last read position: */*
    INFO    2017-08-22 11:56:58 BasecallsConverter  Read    12,000,000 records.  Elapsed time: 00:03:12s.  Time for last 1,000,000:    6s.  Last read position: */*
    INFO    2017-08-22 11:57:01 BasecallsConverter  Read    13,000,000 records.  Elapsed time: 00:03:15s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:57:09 BasecallsConverter  Read    14,000,000 records.  Elapsed time: 00:03:23s.  Time for last 1,000,000:    7s.  Last read position: */*
    INFO    2017-08-22 11:57:11 BasecallsConverter  Read    15,000,000 records.  Elapsed time: 00:03:25s.  Time for last 1,000,000:    2s.  Last read position: */*
    INFO    2017-08-22 11:57:14 BasecallsConverter  Read    16,000,000 records.  Elapsed time: 00:03:28s.  Time for last 1,000,000:    2s.  Last read position: */*
    INFO    2017-08-22 11:57:19 BasecallsConverter  Read    17,000,000 records.  Elapsed time: 00:03:33s.  Time for last 1,000,000:    5s.  Last read position: */*
    INFO    2017-08-22 11:57:28 BasecallsConverter  Read    18,000,000 records.  Elapsed time: 00:03:42s.  Time for last 1,000,000:    8s.  Last read position: */*
    INFO    2017-08-22 11:57:31 BasecallsConverter  Read    19,000,000 records.  Elapsed time: 00:03:46s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:57:36 BasecallsConverter  Read    20,000,000 records.  Elapsed time: 00:03:51s.  Time for last 1,000,000:    5s.  Last read position: */*
    INFO    2017-08-22 11:57:43 BasecallsConverter  Read    21,000,000 records.  Elapsed time: 00:03:57s.  Time for last 1,000,000:    6s.  Last read position: */*
    INFO    2017-08-22 11:57:52 BasecallsConverter  Read    22,000,000 records.  Elapsed time: 00:04:06s.  Time for last 1,000,000:    8s.  Last read position: */*
    INFO    2017-08-22 11:57:59 BasecallsConverter  Read    23,000,000 records.  Elapsed time: 00:04:13s.  Time for last 1,000,000:    6s.  Last read position: */*
    INFO    2017-08-22 11:58:02 BasecallsConverter  Read    24,000,000 records.  Elapsed time: 00:04:16s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:58:05 BasecallsConverter  Read    25,000,000 records.  Elapsed time: 00:04:19s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:58:13 BasecallsConverter  Read    26,000,000 records.  Elapsed time: 00:04:27s.  Time for last 1,000,000:    7s.  Last read position: */*
    INFO    2017-08-22 11:58:16 BasecallsConverter  Read    27,000,000 records.  Elapsed time: 00:04:30s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:58:21 BasecallsConverter  Read    28,000,000 records.  Elapsed time: 00:04:35s.  Time for last 1,000,000:    4s.  Last read position: */*
    INFO    2017-08-22 11:58:23 BasecallsConverter  Read    29,000,000 records.  Elapsed time: 00:04:37s.  Time for last 1,000,000:    2s.  Last read position: */*
    INFO    2017-08-22 11:58:30 BasecallsConverter  Read    30,000,000 records.  Elapsed time: 00:04:44s.  Time for last 1,000,000:    6s.  Last read position: */*
    INFO    2017-08-22 11:58:33 BasecallsConverter  Read    31,000,000 records.  Elapsed time: 00:04:47s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:58:38 BasecallsConverter  Read    32,000,000 records.  Elapsed time: 00:04:52s.  Time for last 1,000,000:    4s.  Last read position: */*
    INFO    2017-08-22 11:58:42 BasecallsConverter  Read    33,000,000 records.  Elapsed time: 00:04:56s.  Time for last 1,000,000:    3s.  Last read position: */*
    INFO    2017-08-22 11:58:45 IlluminaBasecallsConverter  Before explicit GC, Runtime.totalMemory()=25168445440
    INFO    2017-08-22 11:58:45 IlluminaBasecallsConverter  After explicit GC, Runtime.totalMemory()=25168445440
    INFO    2017-08-22 11:58:54 BasecallsConverter  Read    34,000,000 records.  Elapsed time: 00:05:08s.  Time for last 1,000,000:   12s.  Last read position: */*
    INFO    2017-08-22 11:58:59 BasecallsConverter  Write     1,000,000 records.  Elapsed time: 00:05:13s.  Time for last 1,000,000:   22s.  Last read position: */*
    INFO    2017-08-22 11:58:59 BasecallsConverter  Read    35,000,000 records.  Elapsed time: 00:05:13s.  Time for last 1,000,000:    4s.  Last read position: */*
    Exception in thread "pool-1-thread-7" ERROR 2017-08-22 11:59:06 IlluminaBasecallsConverter  Failure encountered in worker thread; attempting to shut down remaining worker threads and terminate ...
    java.lang.InterruptedException
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.awaitWorkComplete(IlluminaBasecallsConverter.java:612)
        at picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:237)
        at picard.illumina.IlluminaBasecallsToSam.doWork(IlluminaBasecallsToSam.java:265)
        at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:205)
        at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:94)
        at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:104)
    picard.PicardException: Read the number of expected bins( 65600) but still had more elements in file( /Illumina/Basecalls/170808_SN218_0895_AHN3VWBCXY/Data/Intensities/L001/s_1_1107.clocs) 
        at picard.illumina.parser.readers.ClocsFileReader.hasNext(ClocsFileReader.java:150)
        at picard.illumina.parser.PosParser$1.hasNext(PosParser.java:98)
        at picard.illumina.parser.PerTileParser.hasNext(PerTileParser.java:120)
        at picard.illumina.parser.PerTileParser.maybeAdvance(PerTileParser.java:99)
        at picard.illumina.parser.PerTileParser.next(PerTileParser.java:109)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:123)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
        at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:470)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$1.run(IlluminaBasecallsConverter.java:563)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Exception in thread "pool-1-thread-10" picard.PicardException: Error reading from file /Project/Capture/s_1_1110_barcode.txt
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:120)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:85)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44)
        at htsjdk.samtools.util.AbstractIterator.next(AbstractIterator.java:57)
        at picard.illumina.parser.readers.BarcodeFileReader.next(BarcodeFileReader.java:29)
        at picard.illumina.parser.BarcodeParser$BarcodeDataIterator$1.getBarcode(BarcodeParser.java:75)
        at picard.illumina.parser.BaseIlluminaDataProvider.addData(BaseIlluminaDataProvider.java:49)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:135)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
        at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:470)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$1.run(IlluminaBasecallsConverter.java:563)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: htsjdk.samtools.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:75)
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:103)
        ... 13 more
    Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:163)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:71)
        ... 14 more
    Exception in thread "pool-1-thread-3" picard.PicardException: Error reading from file /Project/Capture/s_1_1113_barcode.txt
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:120)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:85)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44)
        at htsjdk.samtools.util.AbstractIterator.next(AbstractIterator.java:57)
        at picard.illumina.parser.readers.BarcodeFileReader.next(BarcodeFileReader.java:29)
        at picard.illumina.parser.BarcodeParser$BarcodeDataIterator$1.getBarcode(BarcodeParser.java:75)
        at picard.illumina.parser.BaseIlluminaDataProvider.addData(BaseIlluminaDataProvider.java:49)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:135)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
        at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:470)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$1.run(IlluminaBasecallsConverter.java:563)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: htsjdk.samtools.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:75)
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:103)
        ... 13 more
    Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:163)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:71)
        ... 14 more
    Exception in thread "pool-1-thread-11" picard.PicardException: Error reading from file /Project/Capture/s_1_1111_barcode.txt
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:120)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:85)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44)
        at htsjdk.samtools.util.AbstractIterator.next(AbstractIterator.java:57)
        at picard.illumina.parser.readers.BarcodeFileReader.next(BarcodeFileReader.java:29)
        at picard.illumina.parser.BarcodeParser$BarcodeDataIterator$1.getBarcode(BarcodeParser.java:75)
        at picard.illumina.parser.BaseIlluminaDataProvider.addData(BaseIlluminaDataProvider.java:49)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:135)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
        at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:470)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$1.run(IlluminaBasecallsConverter.java:563)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: htsjdk.samtools.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:75)
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:103)
        ... 13 more
    Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:163)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:71)
        ... 14 more
    Exception in thread "pool-1-thread-8" picard.PicardException: Error reading from file /Project/Capture/s_1_1108_barcode.txt
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:120)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:85)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44)
        at htsjdk.samtools.util.AbstractIterator.next(AbstractIterator.java:57)
        at picard.illumina.parser.readers.BarcodeFileReader.next(BarcodeFileReader.java:29)
        at picard.illumina.parser.BarcodeParser$BarcodeDataIterator$1.getBarcode(BarcodeParser.java:75)
        at picard.illumina.parser.BaseIlluminaDataProvider.addData(BaseIlluminaDataProvider.java:49)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:135)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
        at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:470)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$1.run(IlluminaBasecallsConverter.java:563)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: htsjdk.samtools.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:75)
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:103)
        ... 13 more
    Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:163)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:71)
        ... 14 more
    Exception in thread "pool-1-thread-9" picard.PicardException: Error reading from file /Project/Capture/s_1_1109_barcode.txt
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:120)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:85)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44)
        at htsjdk.samtools.util.AbstractIterator.next(AbstractIterator.java:57)
        at picard.illumina.parser.readers.BarcodeFileReader.next(BarcodeFileReader.java:29)
        at picard.illumina.parser.BarcodeParser$BarcodeDataIterator$1.getBarcode(BarcodeParser.java:75)
        at picard.illumina.parser.BaseIlluminaDataProvider.addData(BaseIlluminaDataProvider.java:49)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:135)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
        at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:470)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$1.run(IlluminaBasecallsConverter.java:563)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: htsjdk.samtools.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:75)
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:103)
        ... 13 more
    Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:163)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:71)
        ... 14 more
    [Tue Aug 22 11:59:06 EDT 2017] picard.illumina.IlluminaBasecallsToSam done. Elapsed time: 7.75 minutes.
    Runtime.totalMemory()=27118796800
    To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
    Exception in thread "main" htsjdk.samtools.util.RuntimeIOException
        at htsjdk.samtools.util.BinaryCodec.close(BinaryCodec.java:611)
        at htsjdk.samtools.BAMFileWriter.finish(BAMFileWriter.java:146)
        at htsjdk.samtools.SAMFileWriterImpl.close(SAMFileWriterImpl.java:231)
        at picard.illumina.IlluminaBasecallsToSam$SAMFileWriterWrapper.close(IlluminaBasecallsToSam.java:542)
        at picard.illumina.IlluminaBasecallsConverter.doTileProcessing(IlluminaBasecallsConverter.java:260)
        at picard.illumina.IlluminaBasecallsToSam.doWork(IlluminaBasecallsToSam.java:265)
        at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:205)
        at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:94)
        at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:104)
    Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:314)
        at htsjdk.samtools.util.BlockCompressedInputStream.checkTermination(BlockCompressedInputStream.java:622)
        at htsjdk.samtools.util.BlockCompressedInputStream.checkTermination(BlockCompressedInputStream.java:607)
        at htsjdk.samtools.util.BlockCompressedInputStream.checkTermination(BlockCompressedInputStream.java:596)
        at htsjdk.samtools.util.BlockCompressedOutputStream.close(BlockCompressedOutputStream.java:286)
        at java.io.FilterOutputStream.close(FilterOutputStream.java:159)
        at htsjdk.samtools.util.BinaryCodec.close(BinaryCodec.java:607)
        ... 8 more
    Exception in thread "pool-1-thread-12" picard.PicardException: Error reading from file /Project/Capture/s_1_1112_barcode.txt
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:120)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:85)
        at picard.util.AbstractInputParser.advance(AbstractInputParser.java:44)
        at htsjdk.samtools.util.AbstractIterator.next(AbstractIterator.java:57)
        at picard.illumina.parser.readers.BarcodeFileReader.next(BarcodeFileReader.java:29)
        at picard.illumina.parser.BarcodeParser$BarcodeDataIterator$1.getBarcode(BarcodeParser.java:75)
        at picard.illumina.parser.BaseIlluminaDataProvider.addData(BaseIlluminaDataProvider.java:49)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:135)
        at picard.illumina.parser.IlluminaDataProvider.next(IlluminaDataProvider.java:41)
        at picard.illumina.IlluminaBasecallsConverter$TileReader.process(IlluminaBasecallsConverter.java:470)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$1.run(IlluminaBasecallsConverter.java:563)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Caused by: htsjdk.samtools.util.RuntimeIOException: java.nio.channels.ClosedByInterruptException
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:75)
        at picard.util.BasicInputParser.readNextLine(BasicInputParser.java:103)
        ... 13 more
    Caused by: java.nio.channels.ClosedByInterruptException
        at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
        at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:163)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at htsjdk.samtools.util.BufferedLineReader.readLine(BufferedLineReader.java:71)
        ... 14 more
    Exception in thread "pool-1-thread-4" java.util.concurrent.RejectedExecutionException: Task picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2@31b98a79 rejected from java.util.concurrent.ThreadPoolExecutor@ac72ca2[Shutting down, pool size = 6, active threads = 6, queued tasks = 0, completed tasks = 33]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.findAndEnqueueWorkOrSignalCompletion(IlluminaBasecallsConverter.java:682)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.access$500(IlluminaBasecallsConverter.java:489)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:739)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Exception in thread "pool-1-thread-2" java.util.concurrent.RejectedExecutionException: Task picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2@4811f928 rejected from java.util.concurrent.ThreadPoolExecutor@ac72ca2[Shutting down, pool size = 5, active threads = 5, queued tasks = 0, completed tasks = 34]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.findAndEnqueueWorkOrSignalCompletion(IlluminaBasecallsConverter.java:682)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.access$500(IlluminaBasecallsConverter.java:489)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:739)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Exception in thread "pool-1-thread-1" java.util.concurrent.RejectedExecutionException: Task picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2@6eeebd36 rejected from java.util.concurrent.ThreadPoolExecutor@ac72ca2[Shutting down, pool size = 4, active threads = 4, queued tasks = 0, completed tasks = 35]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.findAndEnqueueWorkOrSignalCompletion(IlluminaBasecallsConverter.java:682)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.access$500(IlluminaBasecallsConverter.java:489)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:739)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Exception in thread "pool-1-thread-13" java.util.concurrent.RejectedExecutionException: Task picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2@4e18fb3d rejected from java.util.concurrent.ThreadPoolExecutor@ac72ca2[Shutting down, pool size = 3, active threads = 3, queued tasks = 0, completed tasks = 36]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.findAndEnqueueWorkOrSignalCompletion(IlluminaBasecallsConverter.java:682)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.access$500(IlluminaBasecallsConverter.java:489)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:739)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    Exception in thread "pool-1-thread-5" java.util.concurrent.RejectedExecutionException: Task picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2@60842553 rejected from java.util.concurrent.ThreadPoolExecutor@ac72ca2[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 38]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.findAndEnqueueWorkOrSignalCompletion(IlluminaBasecallsConverter.java:682)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator.access$500(IlluminaBasecallsConverter.java:489)
        at picard.illumina.IlluminaBasecallsConverter$TileReadAggregator$2.run(IlluminaBasecallsConverter.java:739)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
    
    

    Issue · Github
    by Sheila

    Issue Number
    2456
    State
    closed
    Last Updated
    Assignee
    Array
    Milestone
    Array
    Closed By
    chandrans
  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @rbrns
    Hi,

    Okay, I need to check with the team. We will get back to you soon.

    -Sheila

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @rbrns
    Hi,

    Sorry for the delay. Can you submit a bug report? Instructions are here.

    Thanks,
    Sheila

Sign In or Register to comment.