I have a problem with the single-node job I'm running; it's chewing up lots of memory because it does all mapping sequentially starting with top-most tasks. A depth-first traversal of the DAG would be much more memory efficient for us since the "vertical" at the "same index" of the mapping has a full set of data in memory. Is there any way to make Prefect do a depth-first search instead?

IIRC both the

DaskExecutor

and the

LocalDaskExecutor

perform Depth First Execution if you chain mapped tasks.

Using the DaskExecutor will prefer depth-first execution. But the LocalExecutor can’t be forced unless you rearchitect your flow somehow like using subflows or task looping

right, ok so back to https://prefect-community.slack.com/archives/CL09KU1K7/p1635497245106100 for me then

You could try using a

LocalDaskExecutor

, I have found it slightly easier to set up.