Dask will strongly prefer DFE, but it can’t be forced either when it’s scheduler decides
05/24/2022, 11:50 PM
Hey Anna, yes that’s what we are doing, for some reason it’s waiting for all runs of A to finish. That’s interesting Kevin, is there any way to predict when DFE will kick in? In this case we were trying to manage disk space in a flow that looks like this: Download file -> Upload file -> Delete file. Maybe relying on DFE in this context is an antipattern.
05/25/2022, 12:46 AM
Can you share your Flow structure? I think there might be something wrong. I think DFE should really kick in there. How many items also?
05/25/2022, 1:34 AM
Thanks. 20 total. I'm not at my computer right now, will share later. Thanks.
05/30/2022, 9:58 AM
Checking after holidays: LMK if this is still an open issue @Jacob Blanco