Cromwell configuration on SLURM

grace_Ggrace_G Member

I'm so confused about cromwell configuration, however, since I will run on SLURM, so now I focus on solving this part:
If I run on SLURM, could my configuration just as Cromwell Docs gives
https://cromwell.readthedocs.io/en/stable/backends/SLURM/

But why herehttps://cromwell.readthedocs.io/en/develop/tutorials/ConfigurationFiles/ mentioned:

Creating your first configuration file
To get started customizing Cromwell via a configuration file, create a new empty text file, say your.conf. Then add this include at the top:

include required(classpath("application"))
The default Cromwell configuration values are set via Cromwell's application.conf. To ensure that you always have the defaults from the application.conf,** you must include it at the top of your new configuration file**.

But the first SLURM configuration doesn't have this "ensure" content? And I didn't find application.conf file? Thanks in advance!!!

Tagged:

Best Answer

  • RuchiRuchi admin
    Accepted Answer

    Hey @grace_G,

    Apologies that the terminology is confusing.

    When you download Cromwell, right off the bat it comes with an internal config file, known as the "application.conf". This contains many of the default values that Cromwell applies. When a user wants to customize Cromwell settings and override existing defaults, we recommend they do so by building a new config file.

    To build a new config file, the first required step is to insert this line at the top of your config:

    include required(classpath("application"))
    

    This allows your new config file to inherit defaults from the application.conf.

    In the link you pointed out, the example configuration is a snippet of what's needed for the config, it's a truncated version of the config and its missing a lot of the top and bottom. To complete the backend stanza -- here is an example below:

    backend {
      default = "SLURM"
      providers {
        SLURM {
          actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
          config {
            runtime-attributes = """
            Int runtime_minutes = 600
            Int cpus = 2
            Int requested_memory_mb_per_core = 8000
            String queue = "short"
            """
    
            submit = """
                sbatch -J ${job_name} -D ${cwd} -o ${out} -e ${err} -t ${runtime_minutes} -p ${queue} \
                ${"-n " + cpus} \
                --mem-per-cpu=${requested_memory_mb_per_core} \
                --wrap "/bin/bash ${script}"
            """
            kill = "scancel ${job_id}"
            check-alive = "squeue -j ${job_id}"
            job-id-regex = "Submitted batch job (\\d+).*"
          }
        }
      }
    }
    

    Let me know if I missed anything.

    Thanks!

Answers

  • RuchiRuchi Member, Broadie, Moderator, Dev admin
    Accepted Answer

    Hey @grace_G,

    Apologies that the terminology is confusing.

    When you download Cromwell, right off the bat it comes with an internal config file, known as the "application.conf". This contains many of the default values that Cromwell applies. When a user wants to customize Cromwell settings and override existing defaults, we recommend they do so by building a new config file.

    To build a new config file, the first required step is to insert this line at the top of your config:

    include required(classpath("application"))
    

    This allows your new config file to inherit defaults from the application.conf.

    In the link you pointed out, the example configuration is a snippet of what's needed for the config, it's a truncated version of the config and its missing a lot of the top and bottom. To complete the backend stanza -- here is an example below:

    backend {
      default = "SLURM"
      providers {
        SLURM {
          actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
          config {
            runtime-attributes = """
            Int runtime_minutes = 600
            Int cpus = 2
            Int requested_memory_mb_per_core = 8000
            String queue = "short"
            """
    
            submit = """
                sbatch -J ${job_name} -D ${cwd} -o ${out} -e ${err} -t ${runtime_minutes} -p ${queue} \
                ${"-n " + cpus} \
                --mem-per-cpu=${requested_memory_mb_per_core} \
                --wrap "/bin/bash ${script}"
            """
            kill = "scancel ${job_id}"
            check-alive = "squeue -j ${job_id}"
            job-id-regex = "Submitted batch job (\\d+).*"
          }
        }
      }
    }
    

    Let me know if I missed anything.

    Thanks!

  • Very clear, thanks!

Sign In or Register to comment.