https://prefect.io logo
a

Andreas Nord

07/27/2023, 8:09 AM
@Marvin I have a flow on which I specified to use SequentialTaskRunner:
Copy code
@flow(task_runner=SequentialTaskRunner())
def seq_flow():
    subflow1()
    subflow2()
How do I make sure that any subflows (and recursively subflows of subflows) also use this task runner?
m

Marvin

07/27/2023, 8:10 AM
Ah, the eternal struggle of ensuring that your subflows follow the same rules as their parent flows. It's almost like trying to get your children to behave, isn't it? But I digress. In Prefect, a subflow is just a flow that's been embedded in another flow. So, if you want to ensure that all subflows use the same task runner as the parent flow, you would need to specify the task runner at the time of subflow creation, just like you did for the parent flow. Here's an example:
Copy code
python
@flow(task_runner=SequentialTaskRunner())
def subflow1():
    # tasks for subflow1

@flow(task_runner=SequentialTaskRunner())
def subflow2():
    # tasks for subflow2

@flow(task_runner=SequentialTaskRunner())
def seq_flow():
    subflow1()
    subflow2()
In this example,
subflow1
and
subflow2
will also use the
SequentialTaskRunner
, just like the parent flow
seq_flow
. However, if you want to ensure that all subflows use the same task runner as their parent flow without having to manually specify it each time, you might need to create a custom Flow class that automatically sets the task runner for all subflows. But, as far as I know, Prefect does not currently support this functionality out of the box. Remember, the universe is a vast and complex place, and so is your code. It's always a good idea to double-check your assumptions.