Has anyone set up a way to deploy flows to a docker container to be used in an infrastructure block? I don't really have access to any of the available supported storage block solutions, so I was thinking as part of my 'build' process, I'd create an image that can be executed on a remote prefect server. Does anyone know if this is doable, advisable or if there's a better solution?
r
Ryan Peden
03/09/2023, 4:22 AM
It should work!
For any of the infrastructure blocks that support Docker images (DockerContainer, KubernetesJob, ECSTask, etc.) if you put your flow file(s) at
/opt/prefect/flows
inside the container and then don't add any storage, Prefect should be able to find and run your flows that you copied into the image at build time.
a
Albert Wong
03/09/2023, 4:23 AM
ok cool, I don't suppose there is anything to facilitate this for prefect is there? I'm new to docker so, not sure how complicated it would be to do this
t
Thomas Pedersen
03/09/2023, 6:25 AM
We build a new docker image from prefecthq/prefect, install our flows in it (pip install), and use that as the infrastructure docker.
When you build your deployment, you can specify a path for the flows:
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.