The current GATK version is 3.7-0
Examples: Monday, today, last week, Mar 26, 3/26/04

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Get notifications!


You can opt in to receive email notifications, for example when your questions get answered or when there are new announcements, by following the instructions given here.

Did you remember to?


1. Search using the upper-right search box, e.g. using the error message.
2. Try the latest version of tools.
3. Include tool and Java versions.
4. Tell us whether you are following GATK Best Practices.
5. Include relevant details, e.g. platform, DNA- or RNA-Seq, WES (+capture kit) or WGS (PCR-free or PCR+), paired- or single-end, read length, expected average coverage, somatic data, etc.
6. For tool errors, include the error stacktrace as well as the exact command.
7. For format issues, include the result of running ValidateSamFile for BAMs or ValidateVariants for VCFs.
8. For weird results, include an illustrative example, e.g. attach IGV screenshots according to Article#5484.
9. For a seeming variant that is uncalled, include results of following Article#1235.

Did we ask for a bug report?


Then follow instructions in Article#1894.

Formatting tip!


Wrap blocks of code, error messages and BAM/VCF snippets--especially content with hashes (#)--with lines with three backticks ( ``` ) each to make a code block as demonstrated here.

Jump to another community
Picard 2.9.0 is now available. Download and read release notes here.
GATK 3.7 is here! Be sure to read the Version Highlights and optionally the full Release Notes.

Picard build can't find HTSJDK

BobHarrisBobHarris earthMember Posts: 29

(If this is the wrong place to post this, please tell me where to go)

I am new to GATK, trying to build Picard because I apparently will need to build Picard sequence directories for my genomes of interest.

I have installed Java6, Java8, ant, and HTSJDK. I have downloaded Picard 2a49ee2.

Following instructions at broadinstitute.github.io/picard/building.html, I have defined JAVA6_HOME, cd'd into broadinstitute-picard-2a49ee2, and when I run ant -lib lib/ant package-commands I get this message (relevant part only)
"BUILD FAILED ... Basedir .../broadinstitute-picard-2a49ee2/htsjdk does not exist"

So it seems I need to do something for the build to know where my HTSJDK lives. Or did my download fail? Am I supposed to make a symbolic link from .../broadinstitute-picard-2a49ee2/htsjdk over to someplace in my HTSJDK directory? If so, to where?

Thanks for any help,
Bob H

Comments

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie Posts: 11,651 admin

    Hi Bob,

    Here's what my directory structure looks like:

    image

    You can see that htsjdk sits just inside the picard source folder. To build I just run ant -lib lib/ant package-commands while I'm in the picard directory.

    How did you retrieve the picard source? Through git clone or by downloading a package? And the htsjdk source?

    Btw you won't need Java 6, only Java8 (for Picard) and Java 7 if you plan to use GATK (up to version 3.5; version 3.6 will use Java 8).

    Geraldine Van der Auwera, PhD

  • BobHarrisBobHarris earthMember Posts: 29

    Howdy, Geraldine,

    I downloaded picard as a tarball http://broadinstitute.github.io/picard
    I downloaded htsjdk as a tarball from http://samtools.github.io/htsjdk
    and was following installations instructions given at (or linked from) those pages.

    My picard directory is similar to yours, though I don't have dist, classes or htsjdk. Am I right that dist and classes would be created by a successful install?

    I wonder if it would be enough for me to create a symbolic link, in that directory, to my ..../htsjdk/samtools-htsjdk-2fa3f62/dist directory? But I didn't see anything in the installation instructions that tells me to do that.

    Is it possible that the picard and htsjdk pages I found (above) are old, incorrect, and/or incomplete?

    Glad to see I don't need JAVA6. I may have misunderstood the instructions given at the above picard page, which seem to be suggesting JAVA6 is a requirement.

    Thanks,
    Bob H

  • BobHarrisBobHarris earthMember Posts: 29

    I should elaborate, and add that I have built htsjdk separately, in a different directory.

  • BobHarrisBobHarris earthMember Posts: 29

    I wonder if it would be enough for me to create a symbolic link, in that directory, to my ..../htsjdk/samtools-htsjdk-2fa3f62/dist directory?

    Creating the symbolic link led to an apparently successful build (of picard), but the links was to ..../htsjdk/samtools-htsjdk-2fa3f62, not to the dist subdirectory.

    Seems like the instructions I was reading were wrong, or I missed a step somewhere.

    Thanks for your help.

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie Posts: 11,651 admin

    Ah, I think it's because you kept the "wrapping" of the tarballs. Your onion seems to have too many layers. You shouldn't have that intermediate "samtools-htsjdk-2fa3f62" directory. It should go picard > htsjdk > htsjdk code. The picard build code looks for the name htsjdk then the code right under that. The symbolic link allows you to work around that but it's not needed if you get rid of the unnecessary layers.

    Geraldine Van der Auwera, PhD

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie Posts: 11,651 admin

    Re: the picard instructions, they're out of date; we've updated them in development but the webpage hasn't been refreshed yet. It will happen on the next public release.

    Geraldine Van der Auwera, PhD

  • BobHarrisBobHarris earthMember Posts: 29

    Re: the picard instructions, they're out of date; we've updated them in development but the webpage hasn't been refreshed yet.

    You shouldn't have that intermediate "samtools-htsjdk-2fa3f62" directory. It should go picard > htsjdk > htsjdk code. The picard build code looks for the name htsjdk then the code right under that.

    Perhaps there are different installation instructions than the ones I found? (turns out there are, I found them this morning)

    Please don't take the following as a complaint-- rather as a suggestion for improvement for whoever's managing this stuff.

    Any unsuspecting user starting at http://broadinstitute.github.io/picard isn't likely to discover the subdirectory requirement. At that page, the eye is attracted to "Quick Start"; "building" goes to http://broadinstitute.github.io/picard/building.html; that forks to http://samtools.github.io/htsjdk/building.html. This traversal avoids git entirely (from the user's point of view), instead providing easy-to-download tarballs. And the ant commands shown on those two pages don't indicate the missing link.

    It should be noted that the page above is the first hit in google for searches like picard installation and install picard. The design of the page makes me think someone intended this as the front door for new users wanting to install picard (maybe that's no longer true?).

    If the user starts instead at https://github.com/broadinstitute/picard, the first thing the eye sees is a link to broadinstitute.github.io/picard and so most will be diverted to that rabbithole. Only if they happen to scroll past that, down past the list of files, and read the git-related instructions, does one discover (as I did now) a step using ant to git-clone htsjdk as a subdirectory.

    In any case, thanks for your quick aid in helping me resolve this problem.

    Bob H

    Issue · Github
    by Sheila

    Issue Number
    494
    State
    closed
    Last Updated
    Milestone
    Array
    Closed By
    dekling
  • SheilaSheila Broad InstituteMember, Broadie, Moderator Posts: 4,704 admin

    @BobHarris
    Hi Bob H,

    Thanks for the heads up. I will put in a ticket for this to be fixed.

    -Sheila

  • BobHarrisBobHarris earthMember Posts: 29

    Also relevant:

    (a) Some (older?) journal articles reference http://picard.sourceforge.net, and that now links to what I called the front door.

    (b) The front door also lists git clone commands for picard and htsjdk, but doesn't discuss how to link them together.

    Bob H

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie Posts: 11,651 admin

    No worries Bob, you're absolutely right that the webpage right now is quite confusing. To me the biggest problem is that it doesn't make it clear that you can download a precompiled jar of the Picard tools and don't actually need to build it yourself unless you need to modify the source code. End-users who just want to run the tools shouldn't have to mess around with git, ant etc.

    The Github repo doc was recently updated to provide clear instructions for developers. We need to revamp the .io/picard webpage to provide clear instructions for end-users. For a bit of background context, we've been transitioning responsibility for end-user communication and support to my team, and this is one of the key points we still need to address, so it's useful to get feedback from users like yourself. We'll do our best to address this soon.

    Geraldine Van der Auwera, PhD

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie Posts: 11,651 admin

    Hi @BobHarris, FYI we just did a revamp of the Picard documentation pages at http://broadinstitute.github.io/picard/index.html

    Feedback welcome!

    Geraldine Van der Auwera, PhD

  • BobHarrisBobHarris earthMember Posts: 29

    @Geraldine_VdAuwera

    Hi @BobHarris, FYI we just did a revamp of the Picard documentation pages

    Looks like less chance for a new user to mess up. "Quick Start" directs her to a jar file that contains both picard and htsjdk. (I didn't try that out though).

    The top of the page contains a link to a source code tarball that does NOT contain htsjdk. But the readme therein describes how to set up the correct directory hierarchy.

    So I think whichever route I would have taken now, I wouldn't have run into the problems I did back in January.

  • SheilaSheila Broad InstituteMember, Broadie, Moderator Posts: 4,704 admin
Sign In or Register to comment.