Hi again 👋 I a questions regarding the flow order. Below is a snapshot of a flow I have where I use map and non-mapped parameters. My question is (in the context of my diagram): Why does the flow wait for

get_url_data

to finish, until

transform_url_data

starts? Same goes for the next step. The problem I have is that

get_url_data

will grow quite large and I guess since the information is kept, I can run out of RAM and the whole thing can crash. If the flow kept going iteratively, then the information will be disposed in the process and I have no problem. I must be missing something here I guess and probably there is a better way to design. If anyone can point me in the right direction, it’s much appreciated : )

Hi @Newskooler - in this case you're talking about depth-first execution. Depth-first is something that Prefect supports through dask, so to enable the execution pattern you've described you'll need to use a Dask executor. You can read more about DFE here: https://medium.com/the-prefect-blog/map-faster-mapping-improvements-in-prefect-0-12-0-7cacc3f14e16

Thanks; that’s what I was looking, @nicholas Solving my issue yet again : )

😄