William Jamir
11/28/2023, 2:41 PMMarvin
11/28/2023, 2:42 PMwork1
and work2
in the main flow, you need to understand that a subflow run is created when a flow function is called inside the execution of another flow. The primary flow is the "parent" flow. The flow created within the parent is the "child" flow or "subflow." Subflow runs behave like normal flow runs. There is a full representation of the flow run in the backend as if it had been called separately. When a subflow starts, it will create a new task runner for tasks within the subflow. When the subflow completes, the task runner is shut down. Subflows will block execution of the parent flow until completion. However, asynchronous subflows can be run in parallel by using AnyIO task groups or asyncio.gather. Subflows differ from normal flows in that they will resolve any passed task futures. More Info
Regarding the issue of logs from work1
not showing, it could be due to several reasons. One common issue is that the task-level logs are not being captured correctly. You can check the console where you ran the agent to see if the logs are being displayed there. If you see the logs in the console but not in the Prefect Cloud interface, it could indicate an issue with the logging configuration or the communication between your agent and Prefect Cloud. More InfoWilliam Jamir
11/28/2023, 2:44 PMKevin Grismore
11/28/2023, 2:47 PMWilliam Jamir
11/28/2023, 2:48 PMWilliam Jamir
11/28/2023, 2:49 PMWilliam Jamir
11/28/2023, 2:56 PMNate
11/28/2023, 7:44 PMFlow.add_task(..., upstream_tasks=[...]
however, you're free to dynamically wrap vanilla python functions with task
or flow
like
from my_pure_python import some_fn
@flow
def dynamic_flow():
some_task = task(log_prints=True)(some_fn) # just set log_prints to show how to set options
some_flow = flow(log_prints=True)(some_fn)
task_result = some_task()