https://prefect.io logo
Powered by
Title
b

Brad

03/18/2022, 3:07 AM
Hey team - I’m trying to have a play with some work queues on a remote host and I think something might be not right with the api endpoints
I can hit my remote orion server to see the deployments: 
[brad@remote ~]$ prefect deployment ls
 Deployments
┏━━━━━━┳━━━━┓
┃ Name ┃ ID ┃
┡━━━━━━╇━━━━┩
└──────┴────┘
But not the work queues (and subsequently, I cannot start an agent on this host as I would like) 
[brad@remote ~]$ prefect work-queue ls
Traceback (most recent call last):
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/prefect/cli/base.py", line 59, in wrapper
    return fn(*args, **kwargs)
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 120, in wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/prefect/utilities/asyncio.py", line 67, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 56, in run
    return asynclib.run(func, *args, **backend_options)
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 233, in run
    return native_run(wrapper(), debug=debug)
  File "/home/brad/miniconda3/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/brad/miniconda3/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 228, in wrapper
    return await func(*args)
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/prefect/cli/work_queue.py", line 167, in ls
    queues = await client.read_work_queues()
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/prefect/client.py", line 804, in read_work_queues
    response = await <http://self._client.post|self._client.post>(f"/work_queues/filter", json=body)
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 137, in post
    return await self.request(
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/prefect/utilities/httpx.py", line 80, in request
    response.raise_for_status()
  File "/home/brad/.local/pipx/venvs/prefect/lib/python3.9/site-packages/httpx/_models.py", line 1510, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '404 Not Found' for url '<http://orion:4201/api/work_queues/filter>'
For more information check: <https://httpstatuses.com/404>
An exception occurred.
This might even be a problem in cloud, just tried on beta and its just hanging on 
Agent started! Looking for work from queue '34263d3a-c107-46f8-b75e-00ab4907328f'...
a

Anna Geller

03/18/2022, 11:03 AM
Thanks for reporting that! 1. Did you start Orion 
prefect orion start
? I think you can't list deployments or work queues unless Orion is started and running in the background 2. Can you share my favorite command "prefect version"? I will try to reproduce in the same setup Will try to run it with Cloud in a bit
b

Brad

03/18/2022, 7:56 PM
Hey @Anna Geller yes I had a prefect Orion backend running on my local host (see deployments returned something rather than an error). On my local machine: 
❯ prefect version                                                                                                                                            06:55:23
Version:             2.0b2
API version:         0.3.0
Python version:      3.9.7
Git commit:          b2a048c2
Built:               Thu, Mar 17, 2022 2:24 PM
OS/Arch:             linux/x86_64
Profile:             default
Server type:         hosted
On the remote host I’m trying to run an agent on: 
[bmcelroy@remote ~]$ prefect version
Version:             2.0b2
API version:         0.3.0
Python version:      3.9.5
Git commit:          b2a048c2
Built:               Thu, Mar 17, 2022 2:24 PM
OS/Arch:             linux/x86_64
Profile:             default
Server type:         hosted
a

Anna Geller

03/18/2022, 9:28 PM
sorry still didn't have the time to check it, will tag you once I do
b

Brad

03/20/2022, 12:00 AM
Copy that @Anna Geller - cheers!
a

Anna Geller

03/20/2022, 12:56 PM
@Brad I briefly looked at the issue now and I think you may try either: 1. Run your Orion API on the same VM 2. Do it the other way around and run Orion on the VM, then before you start your agent on your local machine, set the API endpoint of your Orion API running on VM 
prefect config set PREFECT_API_URL=<http://YOUR_REMOTE_HOST_IP:4200/api>
If I understood your setup, you try that an agent on a remote VM (e.g. on AWS) reaches Orion API started on your laptop and this won't work - but maybe I misunderstood? If you want to have e.g. one remote agent on a VM and use it with Cloud 2.0, this should work well if you follow the instructions from this page: https://orion-docs.prefect.io/ui/cloud/ As a Club42 member, perhaps you were using the early-access setup from the private Discourse category, but now this changed a bit after the official release. LMK if following this doc for Cloud 2.0 doesn't work for you or if I misunderstood the setup that you try to achieve (i.e. where Orion API should run vs. from where you create your deployments vs where the agent should be deployed to poll the work queue).
b

Brad

03/23/2022, 5:10 AM
Okay @Anna Geller - I think this may have been due to me using an uppercase workspace name - after a nuke and using all lowercase everything, things are working now
a

Anna Geller

03/23/2022, 8:58 AM
Amazing, thank you so much for the update! Let me create an issue to investigate this uppercase issue: @Marvin open "Orion: creating workspaces with uppercase letters in Cloud 2.0 should not be allowed as it causes issues"
m

Marvin

03/23/2022, 8:59 AM
https://github.com/PrefectHQ/prefect/issues/5591
9 Views
#prefect-communityJoin Slack
Powered by