https://prefect.io logo
Title
b

Blake Stefansen

09/21/2022, 3:54 PM
Hey y'all I have flow runs created from a deployment in the cloud. Our agent is running on kubernetes. I create a flow run from my deployment, and the agent seems to be picking it up (the flow run doesn't show in the work queue) However, the flow runs are stuck on "pending". I'm not exactly sure what would cause this. Some suggestions in this channel said to nuke the queue and recreate it. No luck there. Any ideas?
t

Taylor Babin

09/22/2022, 1:12 PM
The same thing is happening to me. I'm not using kubernetes but mine is stuck in pending. I haven't gotten an answer but feel free to check my thread for a response
b

Blake Stefansen

09/22/2022, 1:38 PM
Hi Taylor. That is very interesting. I tried seeing if my agent can't access the flow code, but I don't think that's the case? I can run a local deployment (the deployment uses s3 as a remote filesystem) and my local agent runs it just fine. It's my understanding that if the filesystem (where the flow code is stored) is defined in the deployment, then my k8 agent should have access to the flow code. If that's the case, I don't think it's an issue with flow code access
t

Taylor Babin

09/22/2022, 2:24 PM
I believe so in regards to your agent having access. I also tested locally with a local agent and was fine. I'm not able to see an error besides my flows being stuck on pending. I had assumed that if it was an access issue the flow would fail
b

Blake Stefansen

09/22/2022, 2:36 PM
right, same here. I don't see any logs on the flow runs that are stuck, so it's kind of hard to debug
n

Nate

09/22/2022, 4:04 PM
Hi @Blake Stefansen
It's my understanding that if the filesystem (where the flow code is stored) is defined in the deployment, then my k8 agent should have access to the flow code.
in terms of prefect yes, but your pods will need to have the appropriate IAM permissions to interact with S3, but either way, you would get an error if your agent couldn't pull the flow code. if your agent isn't picking up your deployment's flow run at all, then 2 possible issues could be: • you could have started the agent pointing at a different queue than the one you attached to your deployment • your agent has environment variables pointing at a different workspace's URL / API key • your agent is running a significantly different version of prefect