Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

How to resolve log4j version clash on SPARK

Hi,
by default SPARK 2 has no support for log4j2. It uses Log4j 1.2.17. When I use GATK 4.0.0 build "as-is" (depends on Log4j2) all jobs on SPARK fail due to version clash. How this is supposed to be solved?

As a workaround I was able:

  • relocate Log4j to different package
  • make sure it is included in the {{-spark}} jar (it is excluded by default)

But I believe there must be better and cleaner way of running current GATK 4.0.0 on SPARK.

Thanks!

Issue · Github
by Sheila

Issue Number
2887
State
open
Last Updated

Answers

  • Hi,

    and thanks for the comment. Since I am interested in running GATK-4.0.0.0 with Spark too,
    could you please give us a few more details on the workaround you've done?

    I would like to run the GATK pipeline in combination with Spark-2.1.0.
    For the current version of GATK, do we need Log4j 1.2.17? what do you mean by relocating to different package?

    Thanks in advance for you answer!

    Best,
    Giuseppe

  • SheilaSheila Broad InstituteMember, Broadie, Moderator admin

    @zykajan @SergioBenvenuti
    Hi,

    I asked someone from the team to get back to you.

    -Sheila

Sign In or Register to comment.