<@ULVA73B9P> I have usecase where I want to distri...
# ask-marvin
a
@Marvin I have usecase where I want to distribute tasks to 3 different deployment , the image/ container will be the same . How can I pass a parameter to the flow function and achieve this ?
m
thought for 110.2 seconds
Yes — in Prefect 3.x, the usual way is: 1. Define one flow with parameters 2. Create 3 deployments from that same flow 3. Give each deployment different default parameters 4. Point all 3 deployments at the same image / work pool So the container image stays the same, but each deployment runs the flow with different inputs. Example
Copy code
from prefect import flow, deploy

@flow
def process_data(customer: str, queue_name: str, shard: int):
    print(f"customer={customer}, queue={queue_name}, shard={shard}")
    # your task logic here

if __name__ == "__main__":
    d1 = process_data.to_deployment(
        name="worker-1",
        parameters={
            "customer": "A",
            "queue_name": "queue-1",
            "shard": 1,
        },
    )

    d2 = process_data.to_deployment(
        name="worker-2",
        parameters={
            "customer": "B",
            "queue_name": "queue-2",
            "shard": 2,
        },
    )

    d3 = process_data.to_deployment(
        name="worker-3",
        parameters={
            "customer": "C",
            "queue_name": "queue-3",
            "shard": 3,
        },
    )

    deploy(
        d1,
        d2,
        d3,
        work_pool_name="my-work-pool",
        image="my-registry/my-image:latest",
        push=True,
    )
That gives you 3 separate deployments: -
process-data/worker-1
-
process-data/worker-2
-
process-data/worker-3
All use the same image, but each has different default parameters. How it works - The image contains the same flow code - Each deployment stores its own
parameters
- When that deployment runs, Prefect injects those parameters into the flow function So if your flow is:
Copy code
@flow
def process_data(customer: str, shard: int):
    ...
then each deployment can pass a different
customer
/
shard
. If you want to run them manually You can trigger each deployment separately:
Copy code
prefect deployment run 'process-data/worker-1'
prefect deployment run 'process-data/worker-2'
prefect deployment run 'process-data/worker-3'
You can also override parameters at run time:
Copy code
prefect deployment run 'process-data/worker-1' --param shard=10
Important distinction If by “distribute tasks to 3 different deployments” you mean: Option A: You want 3 independent flow deployments with different inputs → use the pattern above. Option B: You want one parent flow that decides which deployment to send work to → you’d usually trigger those deployments as separate flow runs. Option C: You want tasks inside one flow run to execute on different infrastructure → that is a different pattern, and deployments alone are not the mechanism. For your description, Option A sounds like the right fit. Docs - Deploy via Python - Create deployments - Work pools If you want, I can also show: 1. how to do this with
prefect.yaml
, or 2. how to make a parent flow submit work to those 3 deployments dynamically.