Hi there,
I'm new to prefect, may I ask are there any demo or docs to introduce how to config prefect to execute a flow over multi kubernetes? I hope to plan a model training task on kubernetes A and execute a model evaluation task on kubernetes B after model training. Where can I find some demo/docs/best practice method?
Best Regards,
Hanwen
b
Bianca Hoch
02/11/2025, 11:19 PM
Hey Hanwen! If you're just getting started, this doc will help you out. It covers everything you'll need to run your Prefect workflows on Kubernetes. If you're looking to run two separate processes on different clusters, you may consider creating two sets of Kubernetes work pools and workers, one for each cluster. Each worker should be configured to watch its corresponding work pool. When you create deployments for your flows, associate each deployment with the appropriate work pool to make sure that they run in the correct cluster.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.