We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

How to resolve log4j version clash on SPARK

by default SPARK 2 has no support for log4j2. It uses Log4j 1.2.17. When I use GATK 4.0.0 build "as-is" (depends on Log4j2) all jobs on SPARK fail due to version clash. How this is supposed to be solved?

As a workaround I was able:

  • relocate Log4j to different package
  • make sure it is included in the {{-spark}} jar (it is excluded by default)

But I believe there must be better and cleaner way of running current GATK 4.0.0 on SPARK.


Issue · Github
by Sheila

Issue Number
Last Updated


  • Hi,

    and thanks for the comment. Since I am interested in running GATK- with Spark too,
    could you please give us a few more details on the workaround you've done?

    I would like to run the GATK pipeline in combination with Spark-2.1.0.
    For the current version of GATK, do we need Log4j 1.2.17? what do you mean by relocating to different package?

    Thanks in advance for you answer!


  • SheilaSheila Broad InstituteMember, Broadie ✭✭✭✭✭

    @zykajan @SergioBenvenuti

    I asked someone from the team to get back to you.


Sign In or Register to comment.