https://prefect.io logo
Title
j

John O'Farrell

04/03/2023, 4:28 PM
Hello, I have a prefect agent hosted in my own AWS environement that seems to be suddenly switching to an unhealthy state, where checking the agent logs seems to throw an errror saying that the ecs role I've assigned to the agent does not have permission to get the log events for
arn:logs:{region}:{id}:log-group:prefect:log-stream:alpha6-matar/prefect/{log id}
during ecs flow runs. I would understand adding this permission during the setup, but I'm confused as to why it be working fine last week and suddenly experience hiccups (including a brief moment of uptime) today.
z

Zanie

04/03/2023, 4:36 PM
Perhaps you’re suddenly running deployments with
stream_output
enabled and were not previously?
j

John O'Farrell

04/03/2023, 5:03 PM
I did make that switch so that could explain the error. Would that explain why the pool is in an unhealthy state even though the agent has been running?
z

Zanie

04/03/2023, 5:17 PM
I’m not sure about the specifics of the pool health reports here. We try not to crash the agent during streaming errors though.
@alex?
a

alex

04/03/2023, 5:51 PM
Work queues are marked as unhealthy if they have 1 or more late runs or if the queue has not been polled in the past 60 seconds. Are flow runs for this work queue completing successfully?