Hi folks, I most probably miss something here, but here is an issue I encounter: while running a docker flow (similar to this one: https://docs.prefect.io/core/examples/functional_docker.html) on a Kubernetes distributed dask cluster, there is a chance that the

GetContainerLogs

and

WaitOnContainer

tasks do not execute on the same dask worker, and therefore on the same cluster node than the

StartContainer

task. In this case, they won't be able to access the container right? Am I missing something here? How do you handle this?

Those docker tasks require the communication of a Docker daemon. I believe that if each task is given the same Docker daemon to talk to then the tasks should be able to interact with the same containers