Sang Young Noh
06/07/2022, 11:05 AMClient error '401 Unauthorized' for url '<https://api-beta.prefect.io/api/accounts/5c1f7ebd-7031-4f69-bc00-6d21a9b17548/workspaces/9a321454-aa9e-4fb8-8f4c-17e327a77a01/blocks/get_default_storage_block>'
For reference, I was trying to run some deployments on prefect cloud, and when that didnt work, I logged out of prefect cloud and activated a local orion server instead.
The deployment command I am running is:
prefect orion database reset && prefect deployment create manager.py
Any help on this would be much appreciated!Anna Geller
06/07/2022, 11:14 AMSang Young Noh
06/07/2022, 11:14 AMAnna Geller
06/07/2022, 11:15 AMSang Young Noh
06/07/2022, 11:17 AMTraceback (most recent call last):
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/cli/_utilities.py", line 44, in wrapper
return fn(*args, **kwargs)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/utilities/asyncio.py", line 122, in wrapper
return run_async_in_new_loop(async_fn, *args, **kwargs)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/utilities/asyncio.py", line 69, in run_async_in_new_loop
return anyio.run(partial(__fn, *args, **kwargs))
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 292, in run
return native_run(wrapper(), debug=debug)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/asyncio/base_events.py", line 608, in run_until_complete
return future.result()
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/cli/deployment.py", line 231, in create
await spec.validate()
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/client.py", line 95, in with_injected_client
return await fn(*args, **kwargs)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/deployments.py", line 232, in validate
self.flow_storage or await client.get_default_storage_block()
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/client.py", line 1269, in get_default_storage_block
response = await <http://self._client.post|self._client.post>("/blocks/get_default_storage_block")
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/httpx/_client.py", line 1842, in post
return await self.request(
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/httpx/_client.py", line 1527, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/prefect/client.py", line 233, in send
response.raise_for_status()
File "/Users/sang.yn/miniforge3/envs/intel_env/lib/python3.8/site-packages/httpx/_models.py", line 736, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '401 Unauthorized' for url '<https://api-beta.prefect.io/api/accounts/5c1f7ebd-7031-4f69-bc00-6d21a9b17548/workspaces/9a321454-aa9e-4fb8-8f4c-17e327a77a01/blocks/get_default_storage_block>'
Anna Geller
06/07/2022, 11:26 AMprefect version
Sang Young Noh
06/07/2022, 11:28 AMVersion: 2.0b5
API version: 0.3.1
Python version: 3.8.0
Git commit: 7b27c7cf
Built: Tue, May 17, 2022 4:54 PM
OS/Arch: darwin/x86_64
Profile: default
Server type: hosted
Anna Geller
06/07/2022, 11:39 AMSang Young Noh
06/07/2022, 12:11 PMAnna Geller
06/07/2022, 12:14 PMprefect cloud login -k yourkey -w youraccount/yourworkspace
Sang Young Noh
06/07/2022, 12:16 PMAnna Geller
06/07/2022, 12:16 PMprefect profile create cloud
here configure cloud
and to use local Orion, you can use the default profile:
prefect profile use default
If I understood your intention, you want to develop locally on a local Orion server, but when you are ready to deploy it, you want to switch to the cloud, correct? this way, you can just switch the profile to cloud then:
prefect profile use cloud
Sang Young Noh
06/07/2022, 12:18 PMAnna Geller
06/07/2022, 12:29 PMSang Young Noh
06/07/2022, 12:34 PMprefect storage ls
and the error presists:
Client error '401 Unauthorized' for url '<https://api-beta.prefect.io/api/accounts/5c1f7ebd-7031-4f69-bc00-6d21a9b17548/workspaces/9a321454-aa9e-4fb8-8f4c-17e327a77a01/blocks/get_default_storage_block>'
Anna Geller
06/07/2022, 12:44 PMprefect version
Sang Young Noh
06/07/2022, 12:49 PMFlowSchedule = rrule(
freq = MINUTELY,
interval= 5,
dtstart = datetime.datetime(2020, 1, 1, 8, 31, 0),
byhour = range(8, 9),
byminute = range(30, 52),
)
return DeploymentSpec(
flow=ng_daily_polling,
name=f"test",
schedule=RRuleSchedule.from_rrule(FlowSchedule),
#tags=deploy_tags,
#parameters=parameters,
flow_runner=SubprocessFlowRunner(),
)
As for the storage, following the documentation, I created the temporary storage
https://orion-docs.prefect.io/concepts/storage/
Finally, for the version, I’m getting this:
Version: 2.0b5
API version: 0.3.1
Python version: 3.9.13
Git commit: 7b27c7cf
Built: Tue, May 17, 2022 4:54 PM
OS/Arch: darwin/x86_64
Profile: default
Server type: hosted
Anna Geller
06/07/2022, 12:55 PMServer type: hosted
Can you build a new environment from scratch and take it all step by step? your deployment spec is fine but your installation is not if you want to use Cloud 2.0Sang Young Noh
06/07/2022, 12:56 PMAnna Geller
06/07/2022, 12:57 PMprefect cloud login
on this cloud profile and this will make things much easierSang Young Noh
06/07/2022, 1:22 PMAnna Geller
06/07/2022, 1:26 PMSang Young Noh
06/07/2022, 1:26 PMDeploymentSpec(
name="test",
flow=my_flow,
flow_runner=SubprocessFlowRunner(env={"MY_VARIABLE": "FOO"}, condaenv="test"),
)
within the flow, is it possible to call upon the environmental variables as os.environ[‘MY_VARIABLE’]?
ThanksVersion: 2.0b6
API version: 0.5.0
Python version: 3.10.4
Git commit: 502ee008
Built: Tue, Jun 7, 2022 4:58 PM
OS/Arch: darwin/x86_64
Profile: cloud
Server type: hosted
The server type says hosted, which means it isnt pointing to the cloud server, but looking at the config
prefect -p 'cloudtest' config view
I seem to get
PREFECT_PROFILE='cloudtest'
PREFECT_API_KEY='***"
PREFECT_API_URL='<http://beta.prefect.io/api>' (from profile)
The api_url is as shown in the docs, the api key is that which I did create, and yet, I seem to be getting this error:
prefect.exceptions.PrefectHTTPStatusError: Redirect response '301 Moved Permanently' for url '<http://beta.prefect.io/api/block_documents/get_default_storage_block_document>'
Redirect location: '<https://beta.prefect.io/api/block_documents/get_default_storage_block_document>'
For more information check: <https://httpstatuses.com/301>
Anna Geller
06/08/2022, 11:22 AMSang Young Noh
06/08/2022, 11:26 AMprefect storage create
2. Login to cloud with prefect cloud login --key *
3. prefect deployment create **.py # for the creation of deployments
4. Profit!
(New Prefect cloud UI is lovely by the way)
The steps from 1 to 2, of the local prefect pointing to the cloud after that has been configured with the PREFECT_API_URL however, has been the bottleneck. Sometimes it seems to work, sometimes it doesnt.Anna Geller
06/08/2022, 11:43 AMSang Young Noh
06/08/2022, 1:08 PMAnna Geller
06/08/2022, 1:27 PMSang Young Noh
06/08/2022, 1:30 PM