<@ULVA73B9P> I have a main flow file then 4 sublfo...
# ask-marvin
n
@Marvin I have a main flow file then 4 sublfow files then each subflow has many subflows (subflows of subflows), ie. these bottom level files have a function decorated with
@flow
and then many other functions decorated with
@task
. is this okay?
m
Your approach of having a main flow file with multiple subflows, each containing tasks, is generally acceptable in Prefect. However, there are some considerations to keep in mind: 1. Nested Flows: Prefect supports nested flows, allowing you to manage complex workflows by organizing tasks into subflows. This can enhance observability, as nested flows have first-class observability within the Prefect UI and Prefect Cloud. You can see their status in the runs dashboard, which can be beneficial for debugging and monitoring. 2. Cancellation: A nested flow run cannot be canceled without canceling its parent flow run. If independent cancellation is required, consider deploying the subflows separately and starting them with the
run_deployment
method. 3. Task Runners and Parameters: Nested flows allow you to specify task runners for the tasks within them and support parameterization, making it easy to reuse the same group of tasks with different parameters. 4. Prefect 3.x Considerations: Be aware of some changes in Prefect 3.x, such as the replacement of agents with workers and infrastructure blocks with work pools. Ensure that your code is compatible with these updates. For more details, you can refer to the Prefect documentation on writing flows. If you have further questions or need specific examples, feel free to ask!