Hello all, I am new to using prefect. I wanted to know when we use workers like kubernetes, AWS, azure etc does it internally use docker container for code or environment set up to run our flow. I actually want to use podman as it's open source. Appreciate your reply
r
Rob Rati
01/16/2025, 1:21 PM
There are many different type of workpools/workers so it depends on what you setup. If you use the kubernetes worker/work pool than the worker spawns kubernetes jobs. The job definition is defined by the template on the work pool
a
AyushB
01/16/2025, 1:34 PM
Yes the workers spawn kubernetes job, but Kubernetes run a container, so I am asking whether docker is the default one that are spawned or can we configured it use podman. If so how.
r
Rob Rati
01/16/2025, 3:02 PM
It's going to use whatever your cluster is setup to use. The worker is just going to submit a job manifest to kubernetes. Kubernetes is responsible for pulling the container and running it.
a
AyushB
01/16/2025, 3:25 PM
Yes as you said kubernetes pulls containers and runs it. To prefect I am giving code and deploying to work pool, now will the worker only set up docker container for my code to give to kubernetes or can I use podman container here. If so, how.
r
Rob Rati
01/16/2025, 3:27 PM
Again, prefect isn't doing any docker related activity. That is all kubernetes. The kubernetes worker can be configured to pull containers using podman, docker, etc. Prefect does not care or influence. All prefect does is submit a job manifest, which details a pod (containers, labels, etc) and kubernetes does the container pull/run.
In short, this isn't a problem prefect solves. Prefect isn't involved in the actual process of running a container when using kubernetes.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.