Akash N
07/27/2026, 5:06 AMMarvin
07/27/2026, 5:06 AMMarvin
07/27/2026, 5:08 AMfrom prefect import flow, deploy
@flow
def process_data(customer: str, queue_name: str, shard: int):
print(f"customer={customer}, queue={queue_name}, shard={shard}")
# your task logic here
if __name__ == "__main__":
d1 = process_data.to_deployment(
name="worker-1",
parameters={
"customer": "A",
"queue_name": "queue-1",
"shard": 1,
},
)
d2 = process_data.to_deployment(
name="worker-2",
parameters={
"customer": "B",
"queue_name": "queue-2",
"shard": 2,
},
)
d3 = process_data.to_deployment(
name="worker-3",
parameters={
"customer": "C",
"queue_name": "queue-3",
"shard": 3,
},
)
deploy(
d1,
d2,
d3,
work_pool_name="my-work-pool",
image="my-registry/my-image:latest",
push=True,
)
That gives you 3 separate deployments:
- process-data/worker-1
- process-data/worker-2
- process-data/worker-3
All use the same image, but each has different default parameters.
How it works
- The image contains the same flow code
- Each deployment stores its own parameters
- When that deployment runs, Prefect injects those parameters into the flow function
So if your flow is:
@flow
def process_data(customer: str, shard: int):
...
then each deployment can pass a different customer / shard.
If you want to run them manually
You can trigger each deployment separately:
prefect deployment run 'process-data/worker-1'
prefect deployment run 'process-data/worker-2'
prefect deployment run 'process-data/worker-3'
You can also override parameters at run time:
prefect deployment run 'process-data/worker-1' --param shard=10
Important distinction
If by “distribute tasks to 3 different deployments” you mean:
Option A:
You want 3 independent flow deployments with different inputs
→ use the pattern above.
Option B:
You want one parent flow that decides which deployment to send work to
→ you’d usually trigger those deployments as separate flow runs.
Option C:
You want tasks inside one flow run to execute on different infrastructure
→ that is a different pattern, and deployments alone are not the mechanism.
For your description, Option A sounds like the right fit.
Docs
- Deploy via Python
- Create deployments
- Work pools
If you want, I can also show:
1. how to do this with prefect.yaml, or
2. how to make a parent flow submit work to those 3 deployments dynamically.