Joish
04/15/2024, 6:56 AMMarvin
04/15/2024, 6:56 AMbuild_from_flow
in Prefect:
python
from prefect import flow
@flow(log_prints=True) # send `print` statements as INFO logs
def my_workflow():
print("I can call any python code here, including prefect tasks or flows")
if __name__ == "__main__":
my_workflow() # call it like a normal python function
# or serve it as a long-lived process
# my_workflow.serve("my-deployment-name")
Please note that in this code, the build_from_flow
function is not directly used, but the serve
method for a flow is a similar concept in Prefect 2.0, where you can build and serve a flow as a long-lived process.
You may use this code structure as a starting point and then tailor the workflow to your specific needs or check the accompanying link for more context and examples:
Prefect example code on GitHubJoish
04/15/2024, 6:58 AMMarvin
04/15/2024, 6:58 AMpython
from prefect import flow, get_run_context
from prefect.deployments import run_deployment
@flow
def flow_b():
print("This is flow B.")
@flow
def flow_a():
print("Flow A starting Flow B as a deployment...")
context = get_run_context()
flow_b_deployment_name = "flow_b-deployment" # Replace with your actual deployment name for Flow B
run_deployment(flow_b_deployment_name, parameters={}, context=context)
if __name__ == "__main__":
flow_a() # Only run Flow A; it will handle starting Flow B
In this example, flow_a
uses run_deployment
to start flow_b
as a deployment. Ensure you have created a deployment for flow_b
with the name flow_b-deployment
before running this script.
Before using this pattern, verify that you have correctly configured your deployments in Prefect. You might want to check the following Prefect documentation links for more information:
- Interactive Workflows
- Prefect 2.0 Concepts - Flows
- Migration Guide - Subflows
These excerpts suggest that there are ways to have interactive flows and subflows in Prefect 2.0, which is indicated by the use of run_deployment
in an interactive fashion. Please ensure that you create deployments and reference them correctly when initiating them from another flow.Joish
04/15/2024, 7:01 AMMarvin
04/15/2024, 7:01 AMpython
from prefect import flow
@flow
def flow_b():
print("This is Flow B, being called by Flow A.")
@flow
def flow_a():
print("This is Flow A, and now it will call Flow B.")
flow_b()
if __name__ == "__main__":
flow_a() # Running Flow A, which in turn will call Flow B
In this updated pattern, flow_a()
directly calls flow_b()
within its definition. This is a simple invocation and does not require a deployment since both flows are defined in the same file.
The Prefect documentation indicates that you can have 'subflows' and you can call these from a main flow. Here's the excerpt that supports this pattern:
"Flows can include calls to tasks as well as to other flows, which Prefect calls 'subflows' in this context."
For further reference, you might want to take a look at Prefect 2.0 Concepts - Flows where subflows are described.