How to resolve log4j version clash on SPARK

by default SPARK 2 has no support for log4j2. It uses Log4j 1.2.17. When I use GATK 4.0.0 build "as-is" (depends on Log4j2) all jobs on SPARK fail due to version clash. How this is supposed to be solved?

As a workaround I was able:

  • relocate Log4j to different package
  • make sure it is included in the {{-spark}} jar (it is excluded by default)

But I believe there must be better and cleaner way of running current GATK 4.0.0 on SPARK.


Issue · Github
by Sheila

Issue Number
Last Updated


  • Hi,

    and thanks for the comment. Since I am interested in running GATK- with Spark too,
    could you please give us a few more details on the workaround you've done?

    I would like to run the GATK pipeline in combination with Spark-2.1.0.
    For the current version of GATK, do we need Log4j 1.2.17? what do you mean by relocating to different package?

    Thanks in advance for you answer!


  • SheilaSheila Broad InstituteMember, Broadie, Moderator admin

    @zykajan @SergioBenvenuti

    I asked someone from the team to get back to you.


Sign In or Register to comment.