Alex Ilyin
08/30/2024, 8:07 PMMarvin
08/30/2024, 8:07 PMAlex Ilyin
08/30/2024, 8:07 PMMarvin
08/30/2024, 8:07 PMjson
{
"type": "compound",
"require": "all",
"within": 3600,
"triggers": [
{
"type": "event",
"posture": "Reactive",
"expect": ["prefect.block.remote-file-system.write_path.called"],
"match_related": {
"prefect.resource.name": "parent-flow-1",
"prefect.resource.role": "flow"
}
},
{
"type": "event",
"posture": "Reactive",
"expect": ["prefect.block.remote-file-system.write_path.called"],
"match_related": {
"prefect.resource.name": "parent-flow-2",
"prefect.resource.role": "flow"
}
}
]
}
2. Scheduling: Prefect's scheduler can be configured to run flows based on time or events, which can indirectly help with dependency control.
For more details, you can explore:
- Automations: Automations Documentation
- Schedules: Schedules Documentation
Isn't it thrilling to be a master scheduler?Alex Ilyin
08/30/2024, 8:11 PMMarvin
08/30/2024, 8:11 PMjson
{
"type": "event",
"posture": "Reactive",
"expect": ["prefect.flow-run.completed"],
"match_related": {
"prefect.resource.name": "parent-flow",
"prefect.resource.role": "flow"
}
}
### Using the Prefect Client in Python
If you're adamant about doing this in Python code:
1. Run your primary flow and then use the create_flow_run
method to launch the dependent flow after checking the state of the first flow.
2. Here's a simple example to illustrate the point:
python
from prefect import flow, task, get_client
from prefect.states import Completed
@task
def task1():
print("Task 1 complete.")
@task
def task2():
print("Task 2 complete.")
@flow
def parent_flow():
task1()
@flow
def child_flow():
task2()
if __name__ == "__main__":
# Trigger the parent flow
parent_flow_result = parent_flow(return_state=True)
# Check if parent flow succeeded
if parent_flow_result.is_completed():
client = get_client()
client.create_flow_run("child_flow")
This is a simple representation. For a robust setup, you'd naturally add more error handling and client configuration.
### In Short
- Automations: Use Prefect Automations to set up triggers based on events.
- Direct Code: Use Prefect Client to monitor and trigger flows programmatically.
For further details, you can check the official Automations Documentation and the Client Guide.
Enjoy your dependency chains. They're like the gift that keeps on giving.Alex Ilyin
08/30/2024, 8:14 PMMarvin
08/30/2024, 8:15 PMEvent
as the trigger type.
- Set the trigger to react to the event prefect.flow-run.completed
.
4. Set Up Conditions:
- You can set conditions to match specific flows, for example:
json
{
"type": "event",
"posture": "Reactive",
"expect": ["prefect.flow-run.completed"],
"match_related": {
"prefect.resource.name": "your-source-flow-name",
"prefect.resource.role": "flow"
}
}
5. Define the Action:
- Choose Run Deployment
as the action type.
- Specify the deployment you want to run when the flow completes.
I'll search the Prefect 2.x docs for more detailed steps.
Oh, Prefect Cloud automations—just the kind of fun that everyone looks forward to. Here's how you can set it up to trigger a specific deployment when a flow run enters a completed state.
### Steps to Set Up Automations in Prefect Cloud
1. Go to Automations: Navigate to the Automations tab in the Prefect Cloud UI. (You can only access this feature in Prefect Cloud).
2. Create a New Automation: Click on 'New Automation' to start creating a new rule.
3. Define the Trigger:
- Choose Event
as the trigger type.
- Set the trigger to fire on the event prefect.flow-run.completed
.
4. Set Up Conditions:
- Add conditions to match the specific flow you care about:
json
{
"type": "event",
"posture": "Reactive",
"expect": ["prefect.flow-run.completed"],
"match_related": {
"prefect.resource.name": "your-source-flow-name",
"prefect.resource.role": "flow"
}
}
5. Define the Action:
- Choose Run Deployment
as the action type.
- Specify the exact deployment you wish to run upon the successful completion of your flow.
### Example JSON for Automation Rule
Below is an example JSON payload for setting up such an automation:
json
{
"name": "Trigger Deployment on Flow Completion",
"type": "event",
"posture": "Reactive",
"expect": "prefect.flow-run.completed",
"match_related": {
"prefect.resource.name": "source-flow",
"prefect.resource.role": "flow"
},
"actions": [
{
"type": "run_deployment",
"parameters": {
"deployment_id": "your-deployment-id"
}
}
]
}
### Resources for More Detailed Steps
- Automations Documentation
- Automations in Prefect Cloud
With these steps, you should be able to set up those delightful automations and sit back while they handle the dependency chains for you. Enjoy the orchestration.Nate
08/30/2024, 8:30 PMNate
08/30/2024, 8:31 PMprefect.flow-run.Completed
events from the upstream deploymentNate
08/30/2024, 8:33 PMAlex Ilyin
08/30/2024, 9:55 PMNate
08/30/2024, 9:56 PMNate
08/30/2024, 9:56 PMAlex Ilyin
08/30/2024, 9:57 PM