@Marvin, how should I run

subprocess

calls when using a

SlurmCLUSTER

DaskTaskRunner

? Should I invoke the subprocess with a

srun

explicitly?

When using the

DaskTaskRunner

with a

SlurmCluster

in Prefect, you typically don't need to explicitly use

srun

for subprocess calls within your tasks. The

DaskTaskRunner

is designed to handle task distribution across the cluster using Dask's capabilities, which should manage the execution environment for you. However, if your specific use case requires invoking subprocesses that must run under

srun

, you can do so within your task function. This might be necessary if the subprocess needs to be aware of the SLURM environment or if specific SLURM resources are required. Here's a general approach: 1. Within your task function, you can use Python's

subprocess

module to invoke commands with

srun

if needed. 2. Ensure that your task runner is configured properly to submit tasks to the SLURM-managed Dask cluster. For more detailed guidance on using

DaskTaskRunner

, you can refer to the Prefect Dask documentation. If you have specific requirements or configurations for SLURM, you might need to adjust your SLURM scripts or environment variables accordingly.