Hi 👋 I am struggling to make my prefect depth-first and not breadth-first. My flow is set up like so:

from prefect.executors import LocalDaskExecutor
from prefect import Flow

with Flow(name=name, executor=LocalDaskExecutor(scheduler="threads", num_workers=1)) as flow:
    ...

Do I need to use another executer or where can I learn more about this?

Hey @Newskooler, this has to do with Dask preferring depth first execution, but it can’t be forced also and the scheduler will decide to do things in a breadth first way.

I remember before it used to always do depth first - can I make Prefect run in depth first in some way?

The only idea I have is try using

DaskExecutor()

instead of

LocalDaskExecutor()

, even if this is on Local. I thinkt he DaskExecutor tends more to be depth first from experience. You might get into issues with warnings about

freeze_support()

so you may have to do:

if __name__ == "__main__":
    flow.run()

If doing a local run.

So I don’t understand on what basis does it make a choice and why is that not a user choice?

So this is mainly a Dask question and how they choose which task to go next. It’s not something that’s in their docs, but it’s complex and based on heuristics for task size etc. It’s also not something you can predict a priori because it also matters. I know this answer may be unsatisfying to you, so I’ll go hunt down a better answer from people who know more and get back to you.

Okay - thanks a lot. I remember a few version back my code worked depth-first always and then smth changed.

That may be the Dask version and changes to their scheduler.

Cool thanks. Let me know if you find a way to do this :)