Bernardo Galvao
01/10/2023, 2:44 PMAll connection attempts failed
Is the Prefect agent trying to connect to PREFECT_API_URL
?Kalise Richmond
01/10/2023, 4:07 PMRyan Peden
01/10/2023, 5:04 PMPREFECT_API_URL
setting added to your agent
container.
What version of Prefect are you running in your container? I can do a quick test with Docker Compose to see if I can help you sort this outBernardo Galvao
01/10/2023, 5:13 PMdepends
keyword of compose. It restarts containers until they stop erroring out (unless I am missing something)
But I can say that I set the PREFECT_API_URL
(with prefect config set
) in my local machine to the remote api server, to test my flows. No flows were registered in the Prefect UI, which I think is indicative that the agent is not picking up my flows (or do I need to register them as deployments first? π
)Ryan Peden
01/10/2023, 5:24 PMBernardo Galvao
01/10/2023, 5:25 PMRyan Peden
01/10/2023, 5:27 PMBernardo Galvao
01/10/2023, 5:50 PM$ docker logs prefect_agent.1.i4bwqmxk2zlgz3ihvj96y3smq
Starting v2.7.6 agent connected to <http://orion:4200/api>...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue(s): master-q...
16:16:10.196 | ERROR | prefect.agent - Server error '500 Internal Server Error' for url '<http://orion:4200/api/work_queues/0c44a025-39aa-42d4-b427-3096a9e2e4e8/get_runs>'
For more information check: <https://httpstatuses.com/500>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/prefect/agent.py", line 173, in get_and_submit_flow_runs
queue_runs = await self.client.get_runs_in_work_queue(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/prefect/client/orion.py", line 774, in get_runs_in_work_queue
response = await <http://self._client.post|self._client.post>(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1848, in post
return await self.request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/httpx/_client.py", line 1533, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/prefect/client/base.py", line 251, in send
response.raise_for_status()
File "/usr/local/lib/python3.11/site-packages/httpx/_models.py", line 749, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Server error '500 Internal Server Error' for url '<http://orion:4200/api/work_queues/0c44a025-39aa-42d4-b427-3096a9e2e4e8/get_runs>'
For more information check: <https://httpstatuses.com/500>
After the fix:
$ docker logs prefect_agent.1.u1y8r9okouus0sf9yzhw5ny7i
Starting v2.7.6 agent connected to 172.16.8.65:4200/api...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue(s): master-q...
(and nothing else logged)
Just one of those docker-swarm things I suppose!Ryan Peden
01/10/2023, 5:56 PMPREFECT_API_URL
to http://172.16.8.65:4200/api I think it would have the same effect.
But it should be possible to use the prefect
overlay network with swarm, so I'll see if there are extra steps needed to make it work π
Thanks for the extra info!