Alexander van Eck

09/14/2021, 2:09 PM
Morning team! 👋 Q: When a prefect flow is using DaskExecutor and is cancelled is there a way to force the workers to not finish the task they’re working on? What I’m seeing is that dask-workers are gracefully stopped when a flow is cancelled. For long running tasks it can be quite a while before the resources are freed.

Kevin Kho

09/14/2021, 2:12 PM
Hey @Alexander van Eck, the answer here is not really and the user has to manage that themselves because it’s very very hard to stop a Python process running on different compute. See this StackOverflow post

Alexander van Eck

09/14/2021, 2:21 PM
Alright! Then I’ll have to look for that somewhere else - possibly we may be able to kill the container the worker is running in 🤔