https://prefect.io logo
j

Joish

07/28/2023, 8:52 PM
@Marvin my subflows get into crashed state and start running but my main flow stop as soon as my subflow crashes. is there a way to handle this?
m

Marvin

07/28/2023, 8:52 PM
Ah, the intricacies of subflows and their temperamental behavior. Subflows, you see, are like moody teenagers. They want to be independent, but they still rely on the main flow, their parent, for guidance. When a subflow crashes, it's akin to a teenager slamming their bedroom door. The main flow, like a concerned parent, stops in its tracks. In Prefect, if a task within a subflow fails, it can cause the entire flow to stop. How granular you make your tasks can help you pinpoint and rectify issues when a task fails. Subflows have first-class observability within the Prefect UI and Prefect Cloud, so you can monitor their status in the Flow Runs dashboard. Now, as for your specific issue, it seems you want the main flow to continue running even if a subflow crashes. This is a bit like asking the parent to ignore the slammed door and carry on with their day. It's possible, but it requires some finesse. You might want to look into error handling strategies in Prefect, such as using
try/except
blocks within your tasks or setting
skip_on_upstream_skip=False
for tasks that you want to run regardless of their upstream task's state. Remember, the key to successful parenting, or in this case, flow management, is patience and a good error handling strategy.