Ritesh
06/28/2025, 6:06 PMMarvin
06/28/2025, 6:07 PMMarvin
06/28/2025, 6:08 PMpython
from prefect import flow
@flow
def my_flow():
# your flow logic here
pass
2. Create deployments programmatically using the from_source().deploy()
pattern:
python
from prefect.deployments import Deployment
from prefect.filesystems import GitHub
# For Machine 1
my_flow.from_source(
source="<https://github.com/your-repo/your-project>",
entrypoint="path/to/flow.py:my_flow"
).deploy(
name="machine1-deployment",
work_pool_name="machine1-pool", # Your work pool for first EC2
tags=["machine1"]
)
# For Machine 2
my_flow.from_source(
source="<https://github.com/your-repo/your-project>",
entrypoint="path/to/flow.py:my_flow"
).deploy(
name="machine2-deployment",
work_pool_name="machine2-pool", # Your work pool for second EC2
tags=["machine2"]
)
3. To run deployments programmatically:
python
from prefect.deployments import run_deployment
# Run on machine 1
run_deployment(
name="my-flow/machine1-deployment",
parameters={"param1": "value1"}
)
# Run on machine 2
run_deployment(
name="my-flow/machine2-deployment",
parameters={"param1": "value2"}
)
Key points to remember:
- Each EC2 machine should have its own work pool configured
- Workers on each machine should be running and watching their respective work pools
- The source code needs to be accessible from both machines (typically through GitHub or another version control system)
- Make sure your Prefect server is accessible from both EC2 instances
You can also set different configurations for each deployment, such as:
python
my_flow.from_source(
source="<https://github.com/your-repo/your-project>",
entrypoint="path/to/flow.py:my_flow"
).deploy(
name="machine1-deployment",
work_pool_name="machine1-pool",
tags=["machine1"],
description="Deployment for EC2 instance 1",
version="1.0",
parameters={"machine_specific_param": "value1"},
schedule={"interval": "1h"} # Optional scheduling
)
This approach gives you complete programmatic control over your deployments without requiring YAML files, while still maintaining the ability to run different flows on different machines through work pool targeting.
For more detailed information about deployments and work pools, you can refer to the deployment creation guide and work pools documentation.