Hello, for some reason we are not seeing any logs in the prefect cloud UI today, even though the flows we ran yesterday were logging normally. Is there any recommended way to debug this? We are running flows via an ECS agent and all the logs for the task running the agent look normal. Flows are still completing successfully, just no logs.

Seconding this issue, I am also seeing no logs for any of our flow runs (running on GCP Kubernetes Engine) - logging had been fine yesterday

We also saw a little weirdness with our flow runs today, one had a “late” status which I have never seen before

I came here to see if I was messing up my logging somehow - I am also not seeing any logs in the Cloud UI today

Are you all using LocalDaskExecutor with processes by chance?

Yep, mine are all using LocalDaskExecutor

Happening to us with both plain ECS and Dask+ECS execution. A scheduled run from this morning generated logs. The same pipeline does not right now. (No changes to any part of that stack on our end.)

It looks like we had a slight service disruption from 12:55pm Eastern, peaking and concluding around 2:15pm Eastern. I will check if this was potentially related to this.

Thanks! Flow run

db45aa33-3026-442f-b0f3-6e83f0d6298f

is an example if you need one. It's not just that the Prefect Cloud UI doesn't show any logs; the associated Fargate task itself doesn't even generate Cloudwatch logs, despite the fact that the other tasks for my Dask scheduler and workers are generating logs as normal.

We are also using the LocalDaskExecutor

Yes we have logs not being written. Being investigated

FWIW we have flows using both LocalExecutor and LocalDaskExecutor and both were having issues with logs. We are seeing logs for flow runs now though

Yeah so the investigation ended and confirmed we lost logs for that time frame. The service is now up. Part of the logging service went down

Cool, we are seeing logs now. Thanks for digging into this!