Hello Prefect Family! I’m running a flow using mapped tasks, the LocalDaskExecutor and ECS/Fargate. There are 8 mapped tasks and when I run locally all 8 run in parallel. However, when I run in Prefect cloud, it runs 2 mapped tasks at a time until all 8 have completed. Since this works locally it’s not a restriction on ECS/Fargate. Any ideas on what this might be? (thinking out loud…) Is it possible that the “threads” default between my laptop (M1 Mac) and the Fargate nodes is different such that I need to default it in the flow?

Hey @Robert Bastian, did you try increasing the

cpu

for

ECSRun

?

no, but I will try!

I assume you don’t have any task concurrency limiting right/

You probably have to increase the number of workers (n_workers argument in LocalDaskExecutor) or the number of cores available, as it will default to one worker per core. so if you only have two cores in ECS / Fargate, you will have two workers and thus two parallel mapped tasks

@Kathryn Klarich setting num_workers worked! ty!

great! np