@Marvin, I am getting an error when I am trying to use the

dask_jobqueue

SLURMCluster

as a

DaskTaskRunner

AttributeError: 'SLURMCluster' object has no attribute 'duplicate'

so how can I use a slurm interface to defining task runners?

From the information gathered, it looks like you're dealing with configuring a SLURM-based Dask cluster for use with Prefect's DaskTaskRunner. Here's what I found that can help you: 1. DaskTaskRunner Usage: Prefect allows using a SLURM cluster by specifying the

cluster_class

in the DaskTaskRunner configuration. You can specify the class itself, a function for creating a custom cluster, or the full import path for the class. You should also use

cluster_kwargs

to pass any additional parameters needed for the configuration. 2. Prefect 3.x Docs: You can create a custom Dask cluster by providing the

cluster_class

and potentially use SLURMCluster if properly configured. See more details here: Prefect Dask Usage Guide. 3. Prefect 2.x Docs: Prefect also provides flexibility in using different Dask cluster interfaces. Look into how Prefect connects to an existing cluster or creates a temporary one: Prefect Dask Task Runners. It seems there are no specific issues or direct AttributeError errors reported in GitHub related to your query, so reviewing the relevant configuration setups in the documentation might help resolve the error you encountered. If your SLURM cluster doesn’t seem to fit into this configuration naturally, you might have to create additional setup script functions or modify some existing configurations to work collaboratively with Prefect. If you need more in-depth assistance, consider reaching out to the Prefect community.