Does anyone know why a mapped task might indicate Ready to p

Question

Does anyone know why a mapped task might indicate Ready to proceed with mapping but it never proceeds It works fine with LocalExecutor but not with DaskExecutor via FargateCluster I m really frustrated because I know I ve encountered and solved this exact issue before

Devin McCabe · Answer

There is nothing in Dask scheduler worker logs until the scheduler realizes it s been idle and shuts everything down All dependent tasks to my mapped task were successful according to prefect cloud

Mason Menges · Answer

Hey < Devin McCabe> Would you be able to provided a Simplified reproducible example of the flow that s running Nothing Immediately comes to mind as to what could be causing this it s possible the workers don t have enough memory to run the tasks but I d normally expect their to be some logs on the dask side to indicate that

Bianca Hoch · Answer

Hello Devin I m going to throw some resources you way as well

Devin McCabe · Answer

Thanks it turns out that the mapped task was very briefly running out of memory and the scheduler was killing it My experience with Dask has been generally that it doesn t always generate appropriate or any logs when the scheduler kills or respawns workers due to timeouts or capacity issues