If I am to load 50 tables from 1 db to another. Would you implement that as 1 flow per table or 1 flow mapped over all tables? I guess if I make a loop to register 1 flow per table it would be easier to rerun only one table. And maybe easier with debugging. But it would be more cluttered in the UI.

Keep in mind, that when you are mapping over all tables with e.g. Dask Executor, Prefect will try to parallalize this taks and potentially open 50 db-Connections. Aside from that I would personally prefer a single flow, but I would also like to hear the suggestion of the prefect maintainer 🙂

Thanks for replaying. Is there a way to limit the number of parallel tasks. On flow level or in the Dask executor maybe.

There is a Task Concurrency limit in Prefect Cloud if you have it enabled on your account.