Ajeel Ahmed
02/27/2025, 7:43 PMflow.serve
in cases where I want to run the flow-runs
as standalone containers?Marvin
02/27/2025, 7:44 PMflow.serve
to run flow-runs
as standalone containers. Prefect's .serve
method allows you to elevate a flow to a deployment that listens for scheduled work to execute as a local process. This "local" process can be in a Docker container, which can be on any machine with Docker installed.
To achieve this, you can package your flow code and dependencies into a Docker container using a Dockerfile. Once your flow is containerized, you can run it on any machine with Docker. Here's a link to the Prefect documentation that provides a detailed example on how to serve flows in a long-lived Docker container.
For reference, Prefect's top-level API includes serve
, which is part of the offerings for managing flow deployments.Nate
02/27/2025, 8:12 PMmakes it sound like you want each flow run in its own container, which would not be conducive toas standalone containers?flow-runs
.serve
because .serve
will start a long lived process that listens for and executes scheduled runs
so you can put .serve in a long-lived container if you want flow runs to be executed as subprocesses in that one container, but if you want a container per flow run (dynamic dispatch) then you'll want a docker work pool and a docker worker that submits each flow run as a containerAjeel Ahmed
02/28/2025, 9:17 AM2.7
version of Prefect
and now I’m migrating to 3.2
. Now the problem is in the previous version we were using Prefect.infrastructure.DockerContainer
for each flow run and this doesn’t seem to be available anymore which is why I raised this question.Nate
02/28/2025, 5:26 PMNate
02/28/2025, 5:26 PM