To deploy your flow such that the scheduled runs trigger a new pod on a Kubernetes Prefect worker instead of the Prefect agent, follow these steps:
1.
Configure a Kubernetes Work Pool:
- Set up a Kubernetes work pool with the required specifications for your flow runs.
2.
Specify Work Pool in Deployment:
- When creating a deployment, you need to specify the work pool and work queue that the deployed flows should use. This ensures that the Prefect worker corresponding to the work pool will pick up the runs.
3.
Run Prefect Worker on Kubernetes:
- Ensure the Prefect worker is running on your Kubernetes cluster. The worker is responsible for polling the work queue and executing the tasks within the Kubernetes environment.
Here are some resources to help you along the way:
-
Deploying Flows using Prefect workers
-
Upgrading from Agents to Workers
To summarize in code:
1.
Define the Work Pool in your deployment file:
yaml
deployments:
- name: my_deployment
work_pool:
name: my_k8s_work_pool
schedules:
- cron: "0 0 * * *"
timezone: "America/Chicago"
active: true
2.
Run the Worker:
- Run the following command to start a Kubernetes worker that will handle your flow runs:
bash
prefect worker start my_k8s_work_pool
For more detailed steps and additional options, please visit the documentation provided above.
Isn't it just delightful how complicated life can get? If you need more assistance, don't hesitate to ask.