Hi a few days ago I scheduled a flow to run daily I use Loca

Question

Hi a few days ago I scheduled a flow to run daily I use LocalDaskExecutor with 8 workers and the processes scheduler I need to run tasks in parallel I am using Docker as storage and ECS fargate as run config The problem is that today the execution failed because some mapped tasks were always in Queued state with the following logs ``` 07 17 34 INFO CloudTaskRunner Task XXX 12 Starting task run 07 17 35 INFO CloudTaskRunner Task XXX 12 Finished task run for task with final state Queued 07 18 05 INFO CloudTaskRunner Task XXX 12 Starting task run 07 18 05 INFO CloudTaskRunner Task XXX 12 Finished task run for task with final state Queued ``` It seems the tasks started but finished in Queued state again What could be the problem here I am using task concurrency for those tasks if that helps Thank you

jcozar · Answer

I was using 256 and 512 for cpu and mem in my flow My tasks are lightweight as they are just using boto3 starting fargate tasks but using 8 workers for local dask executor might not be enough However I didn t see any log from dask crashing or somethig neither the fargate task running the prefect flow

Kevin Kho · Answer

Hey < jcozar> just making sure you re on Cloud and not Server

jcozar · Answer

yes I am on cloud

jcozar · Answer

And I confirm that using more cpu and mem solved the problem I suppose that 8 workers for just 0 25vpu and 512MB ram is not enought and something chashed but with no logs

Kevin Kho · Answer

Thanks for the info