https://prefect.io logo
Title
a

Adam

07/27/2020, 2:50 PM
Hi everyone, a quick question. What is the relation between the Docker container and Dask? I'm struggling to make the link between the two. To my knowledge Dask is not aware of containers? (I've never used Dask but used k8s/docker extensively so just looking to find the link between how the container is run against Dask)
c

Chris White

07/27/2020, 3:19 PM
Hi @Adam - it is best practice to have the same python package versions + environment on your client + all Dask workers. Docker makes this very easy to do if all the workers share a common image.
a

Adam

07/27/2020, 3:26 PM
Thanks @Chris White, where would one specify that image? In the yaml for setting up the dask workers?
c

Chris White

07/27/2020, 3:27 PM
That depends on how you are creating your Dask cluster; if you are using a k8s manifest (which it sounds like you are based on your mention of yaml) then yea that would be a good place to configure the image