Hey I m working through

Question

Hey I m working through <https github com anna geller dataflow ops|this template> to get an agents up and running as ECS Service I ve got it running and the agent successfully created the work queue in our Prefect Cloud account However the agent logs say that the agent is checking for flow runs but the work queue in Prefect Cloud is Unhealthy and says that the queue has never been polled Could you help me understand how work queues send and receive updates from the agent Since the command to create the queue was able to get out I m guessing that the issue has to do with our security infrastructure and our rules for inbound traffic Understanding the communication would help us debug Thank you

Christopher Boyd · Answer

Hi Claire Agents are outbound connections only

Christopher Boyd · Answer

if you are running the agent in ECS on a vm or something you could log into the system and test a curl connection to the api to verify your outbound rules are allowing outbound calls

Christopher Boyd · Answer

``` $ curl vvvv <https api prefect cloud> Trying 34 111 36 90 443 Connected to api prefect cloud 34 111 36 90 port 443 0 ```

Claire Herdeman · Answer

Thanks that s helpful Will report back

Claire Herdeman · Answer

I ve been taking a look at this off an on this is running as a task so its a bit more difficult to get into the container directly I looked at the ENI associated with the task and saw that traffic was getting rejected made some tweaks and was able to resolve The agent still doesn t seem to be connecting though it doesn t create the queue on startup and doesn t poll Does anything look off to you about this startup message does the URL look good etc ```2022 11 10 17 05 50 Agent started Looking for work from queue s dev 2022 11 10 17 05 50 23 05 50 200 | DEBUG | prefect client Connecting to API at 2022 11 10 17 05 50 7B 22PREFECT API KEY 22 22 lt key gt 22 22PREFE 2022 11 10 17 05 50 CT API URL 22 22 lt acct id gt 2022 11 10 17 05 50 workspaces lt ws id gt 2022 11 10 17 05 50 Starting v2 6 7 agent connected to 2022 11 10 17 05 50 PREFECT API KEY lt key gt PREFECT API URL 2022 11 10 17 05 50 lt acct id gt wor 2022 11 10 17 05 50 kspaces lt ws id gt ```

Claire Herdeman · Answer

Welll now my suspicion is that its just how the secret is getting passed looks like it should just say the url there Though it still says that it connects which seems odd

Christopher Boyd · Answer

it looks like you have a carriage return in your deployment specification

Christopher Boyd · Answer

```2022 11 10 17 05 50 7B 22PREFECT API KEY 22 22 lt key gt 22 22PREFE 2022 11 10 17 05 50 CT API URL 22 22<https api prefect cloud api accounts > lt acct id gt ```

Christopher Boyd · Answer

let me double check on my end real quick with debug logs

Christopher Boyd · Answer

I think your deployment spect might have some awry wiht your API key it looks like this for me there is no reference to my API key in logs ```13 15 09 004 | DEBUG | prefect client Connecting to API at <https api prefect cloud api accounts e> lt acct id gt workspaces lt ws id ```

Christopher Boyd · Answer

for context my spec ``` env name PREFECT API URL value gt <https api prefect cloud api accounts > lt +++ gt workspaces lt +++ gt name PREFECT API KEY value lt +++ gt name PREFECT LOGGING LEVEL value DEBUG```

Claire Herdeman · Answer

Perfect yeah i think there s something wonky with how the secrets are getting pulled though I m confused why it would say that the agent has started though if those items aren t reading

Christopher Boyd · Answer

The agent starting as a service is successful just connecting to the API is not you could put <http www google com|www google com> as your API URL and the service will still say that it started successfully but not pull proper data this is due to teh fact that every consumers API URL would presumably be different so it has to check and return a result

Claire Herdeman · Answer

Gochya we re up and running now I do think this should have thrown an error though Basically I wasn t pulling the values out of the json blob of the secret so the full json object was going into the slot for both API url and the API key Both fields would have been totally malformed like even if <http google com|google com> would it seems like it should recognize if its NOT a url or if the url is in the incorrect domain

Christopher Boyd · Answer

That s good feedback and I can raise that with the team