https://prefect.io logo
Title
d

David Harrington

11/12/2020, 5:07 PM
Hi! We are interested in running multiple types of Flows via the Prefect Server using the Kubernetes Agent. We understand this requires using
Docker
storage for each job. We construct the images ourselves and may have many flows. We want to avoid constructing a unique image for each specific Flow, and we also want to avoid creating an image that contains every flow. Instead, we want to provide a base image that the Kubernetes Agent can run, and subsequently the launched Job can ask for the necessary Flow at runtime. Ultimately, we want to avoid serializing the Flow within the built image and still use the Kubernetes Agent. A thought we had was to build an image whose entrypoint would query for the Flow, place the serialized Flow in the appropriate file location and start Prefect. Looking for any thoughts on how to best use k8s agent with these constraints! Thanks
@Adithya Ramanathan @Payam Vaezi
You can register flows individually and user a standard docker container for each one
🎯 1
d

David Harrington

11/12/2020, 6:51 PM
Thanks - this is exactly what we’re looking for!