https://prefect.io logo
Join Slack
Powered by
I'm working on V2 for the first time and having an...
# prefect-community
j
I'm working on V2 for the first time and having an issue with the agent. I have the agent set up and running in an ECS service, but the work queue still says it's never been polled and it's an unhealthy status. Shouldn't it be showing as healthy status?
z
It’s possible that your agent does not have the API URL configured correctly and is just talking to a local API instead of the remote one.
gratitude thank you 1
j
i checked that, but it says that it has the proper api url and it does not say it's running in an ephemeral instance
z
Interesting! Do you have DEBUG level logs enabled for your agent?
j
I do, and nothing exciting happening there. I don't have any flow deployments, would that cause this behavior? I assumed that even without deployments it would show as healthy and that it's polling
z
It should indeed! It’s possible that going to an older version means your client doesn’t actually send the healthcheck
What version is your agent running on?
j
The agent is running 2.6.9
z
Hm and the agent displays “Checking for scheduled flow runs...“?
j
I sent you a DM with the logs. The last log message is "Agent started! Looking for work from queue.."
z
Yeah there’s a separate debug level log when it actually polls for runs
it’d come later.
Let me get a Cloud engineer over here to look at it on the backend cc @Jake Kaplan
j
let me take a look, do you mind dming me your workspace id/work queue info?
I can see the work queue but it hasn't been polled. which tracks with no 
Checking for scheduled flow runs
so I am not sure why the agent would stall there, I also don't see any requests for that work queue (which makes sense)
z
Hm the agent should display that it is checking for runs before running queries
j
so i can poll the work queue from my local machine
although i still can't get the "checking for flow runs" message to appear
z
Is the ECS container able to access the outside network?
j
i'm not actually sure, i'm new to ecs tasks. any advice on a simple way to check that? replace the prefect agent start command with a ping command maybe?
z
Yeah that’d be a good idea
j
prefect flow-run ls
is a pretty good proxy, as long as you have the API_URL set
j
my pipeline broke and took forever to fix. so if i run the prefect flow-run ls command i get error messages from httpx
which is the same thing i get from the 2.7.* version of the prefect agent from just running prefect agent start
so i'm guessing it doesn't have external access, i'll see if i can play with that next week
Turns out the logic I had to get the api URL and key from the AWS secret was adding a beginning and end double quote, which was causing httpx to freak out
somewhere along the line it was getting encoded so it was trying to call the API at %22https...
so that's on me, sorry for the confusion. Thank you both for the help!
j
np at all! I'm glad you were able to resolve!
j
FWIW, i only realized it because I tried to run a flow and that process showed an error message about not being able to connect to the API at example-url.com - it might be worthing adding the same kind of check to 
prefect agent start
z
It ought to! It swallows errors for a while so that if it can’t connect temporarily it will not die. It sounds like something’s not quite right though.
Open in Slack
PreviousNext
Powered by