I am facing this error on the prefect agent :face_holding_back_tears:

All connection attempts failed

Is the Prefect agent trying to connect to

PREFECT_API_URL

?

Hi @Bernardo Galvao, yes the agent is looking for the PREFECt_API_URL and you can provide use the --api flag on the agent to make sure that is set correctly. https://docs.prefect.io/concepts/work-queues/#agent-options

It should be picking it up from the

PREFECT_API_URL

setting added to your

agent

container. What version of Prefect are you running in your container? I can do a quick test with Docker Compose to see if I can help you sort this out

Version 2.7.* (I can tell you the patch version soon) Mind you, this is running docker swarm that ignores

depends

keyword of compose. It restarts containers until they stop erroring out (unless I am missing something) But I can say that I set the

PREFECT_API_URL

(with

prefect config set

) in my local machine to the remote api server, to test my flows. No flows were registered in the Prefect UI, which I think is indicative that the agent is not picking up my flows (or do I need to register them as deployments first? 😅)

You'll need to create a deployment for an agent to run your flows. You can add a schedule to your deployment, or you can runt he deployment manually via the UI It sounds like the agent is unable to connect to Orion anyway, though. There might be some extra steps to make the Docker overlay network work properly with Docker Swarm. I'll do some tests and report back. And then if needed I will update the recipe on GitHub 🙂

alright I'll do my best to solve it too and report back 🙂

Thanks! Much appreciated 🙂

@Ryan Peden I rolled out @Kalise Richmond's fix in the entrypoint of the prefect agent, and it seems to be working alright 🙂 The fix: (image attached) Before the fix:

$ docker logs prefect_agent.1.i4bwqmxk2zlgz3ihvj96y3smq 
Starting v2.7.6 agent connected to <http://orion:4200/api>...

  ___ ___ ___ ___ ___ ___ _____     _   ___ ___ _  _ _____
 | _ \ _ \ __| __| __/ __|_   _|   /_\ / __| __| \| |_   _|
 |  _/   / _|| _|| _| (__  | |    / _ \ (_ | _|| .` | | |
 |_| |_|_\___|_| |___\___| |_|   /_/ \_\___|___|_|\_| |_|


Agent started! Looking for work from queue(s): master-q...
16:16:10.196 | ERROR   | prefect.agent - Server error '500 Internal Server Error' for url '<http://orion:4200/api/work_queues/0c44a025-39aa-42d4-b427-3096a9e2e4e8/get_runs>'
For more information check: <https://httpstatuses.com/500>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prefect/agent.py", line 173, in get_and_submit_flow_runs
    queue_runs = await self.client.get_runs_in_work_queue(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/client/orion.py", line 774, in get_runs_in_work_queue
    response = await <http://self._client.post|self._client.post>(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1848, in post
    return await self.request(
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/client/base.py", line 251, in send
    response.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/httpx/_models.py", line 749, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url '<http://orion:4200/api/work_queues/0c44a025-39aa-42d4-b427-3096a9e2e4e8/get_runs>'
For more information check: <https://httpstatuses.com/500>

After the fix:

$ docker logs prefect_agent.1.u1y8r9okouus0sf9yzhw5ny7i 
Starting v2.7.6 agent connected to 172.16.8.65:4200/api...

  ___ ___ ___ ___ ___ ___ _____     _   ___ ___ _  _ _____
 | _ \ _ \ __| __| __/ __|_   _|   /_\ / __| __| \| |_   _|
 |  _/   / _|| _|| _| (__  | |    / _ \ (_ | _|| .` | | |
 |_| |_|_\___|_| |___\___| |_|   /_/ \_\___|___|_|\_| |_|


Agent started! Looking for work from queue(s): master-q...

(and nothing else logged) Just one of those docker-swarm things I suppose!

Ahh, got it. In this case if you set

PREFECT_API_URL

to http://172.16.8.65:4200/api I think it would have the same effect. But it should be possible to use the

prefect

overlay network with swarm, so I'll see if there are extra steps needed to make it work 🙂 Thanks for the extra info!