one of the recurring issues i have with prefect is that things often seem to fail for no reason, and frequently with absolutely no context. i restarted this flow a minute later with no changes at all, and it ran successfully. are there any recommendations about handling these situations?

Hey @Martim Lobao this sounds like ECS is just failing to start. Do you have CloudWatch logs enabled for this?

unfortunately our cloudwatch logs don’t offer any more context either

What is your launch type and do you use Dask? Do you use spot instances?

yes, we’re using

LocalDaskExecutor

and fargate

LocalDask should not be a problem. Sometimes, spinning up a Dask cluster fails. I think the issue here is Fargate. Could you see if these debugging methods help or give any insight? I think you are looking at task logs, but maybe we can look for container logs

unfortunately the agent’s logs provide even less details

Do you have anything under Details for the Flow container?