I’m struggling to get DFE to kick in for a mapped task, we are using the Dask executor and version 0.15. The flow is really simple List->A->B where A and B map.

Have you seen this blog post? https://medium.com/the-prefect-blog/map-faster-mapping-improvements-in-prefect-0-12-0-7cacc3f14e16 you can call taska.map(taskb.map(some_list))

Dask will strongly prefer DFE, but it can’t be forced either when it’s scheduler decides

Hey Anna, yes that’s what we are doing, for some reason it’s waiting for all runs of A to finish. That’s interesting Kevin, is there any way to predict when DFE will kick in? In this case we were trying to manage disk space in a flow that looks like this: Download file -> Upload file -> Delete file. Maybe relying on DFE in this context is an antipattern.

Can you share your Flow structure? I think there might be something wrong. I think DFE should really kick in there. How many items also?

Thanks. 20 total. I'm not at my computer right now, will share later. Thanks.

Checking after holidays: LMK if this is still an open issue @Jacob Blanco