Jon Young
07/02/2025, 1:42 AMMarvin
07/02/2025, 1:42 AMMarvin
07/02/2025, 1:43 AMpython
from prefect.client import PrefectClient
from prefect.client.schemas import WorkPoolCreate
async with PrefectClient() as client:
work_pool = await client.create_work_pool(
work_pool=WorkPoolCreate(
name="my-work-pool",
type="process", # or "kubernetes", "docker", etc.
base_job_template={
# Your job template configuration here
"job_configuration": {
"command": "{{ command }}",
# Additional configuration specific to your work pool type
}
}
)
)
2. CLI
bash
prefect work-pool create my-work-pool \
--type process \
--template '{"job_configuration": {"command": "{{ command }}"}}'
3. Terraform Provider
If you're using Terraform, you can use the official Prefect provider to manage work pools. Here's an example:
hcl
resource "prefect_work_pool" "my_work_pool" {
name = "my-work-pool"
type = "process"
description = "My IaC managed work pool"
base_job_template = jsonencode({
job_configuration = {
command = "{{ command }}"
}
})
}
The job template structure will vary depending on your work pool type (process, docker, kubernetes, etc.). You can find detailed documentation about work pool management at: https://docs-3.prefect.io/v3/how-to-guides/deployment_infra/manage-work-pools
For specific work pool types and their template configurations, you can check:
- Process: https://docs-3.prefect.io/v3/guides/deployment/process-workers/
- Docker: https://docs-3.prefect.io/v3/guides/deployment/docker-workers/
- Kubernetes: https://docs-3.prefect.io/v3/guides/deployment/kubernetes-workers/
Each worker type has specific configuration options that can be defined in the base job template, allowing you to fully customize the execution environment for your flows.Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by