I am running a flow in Prefect 1.0 with a

DaskExecutor

with

cluster_class="dask_jobqueue.SLURMCluster"

. Everything works fine, unless SLURM kills the resources (e.g. due to time limit). Then the computation is naturally terminated, but in the Prefect cloud I still see the flow and task running. I did an experiment where SLURM kills the resouces after 2min, but the flow is still shown as running for 1h 38min and counting. I thought that the heartbeat option should kick in at some point to flag the flow as

Failed

. Has anyone experience with a similar setup?

you could consider asking here https://dask.discourse.group/