Hello I m currently trying to delete some deployments and re

Question

Hello I m currently trying to delete some deployments and redeploy onto the prefect cloud but I m getting the following error ```Client error 401 Unauthorized for url <https api beta prefect io api accounts 5c1f7ebd 7031 4f69 bc00 6d21a9b17548 workspaces 9a321454 aa9e 4fb8 8f4c 17e327a77a01 blocks get default storage block> ``` For reference I was trying to run some deployments on prefect cloud and when that didnt work I logged out of prefect cloud and activated a local orion server instead The deployment command I am running is ```prefect orion database reset amp amp prefect deployment create manager py``` Any help on this would be much appreciated

Anna Geller · Answer

Looks like your API key expired I don t think you need to recreate a deployment or change to local Orion you would only need to create a new API key you can set it to never expire

Sang Young Noh · Answer

Ah I see ok

Anna Geller · Answer

I think what caused the issue for you is that by default when you create an API key it adds an expiration date for better security and to make it never expire you need to select one more checkbox LMK if this doesn t fix the issue for you

Sang Young Noh · Answer

Hi I ve made a new key and tried to login and it was fine But I still seem to be getting the same error as before

Sang Young Noh · Answer

Just to give a larger scope of the error ```Traceback most recent call last File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect cli utilities py line 44 in wrapper return fn args kwargs File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect utilities asyncio py line 122 in wrapper return run async in new loop async fn args kwargs File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect utilities asyncio py line 69 in run async in new loop return anyio run partial fn args kwargs File Users sang yn miniforge3 envs intel env lib python3 8 site packages anyio core eventloop py line 70 in run return asynclib run func args backend options File Users sang yn miniforge3 envs intel env lib python3 8 site packages anyio backends asyncio py line 292 in run return native run wrapper debug=debug File Users sang yn miniforge3 envs intel env lib python3 8 asyncio runners py line 43 in run return loop run until complete main File Users sang yn miniforge3 envs intel env lib python3 8 asyncio base events py line 608 in run until complete return future result File Users sang yn miniforge3 envs intel env lib python3 8 site packages anyio backends asyncio py line 287 in wrapper return await func args File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect cli deployment py line 231 in create await spec validate File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect client py line 95 in with injected client return await fn args kwargs File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect deployments py line 232 in validate self flow storage or await client get default storage block File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect client py line 1269 in get default storage block response = await blocks get default storage block File Users sang yn miniforge3 envs intel env lib python3 8 site packages httpx client py line 1842 in post return await self request File Users sang yn miniforge3 envs intel env lib python3 8 site packages httpx client py line 1527 in request return await self send request auth=auth follow redirects=follow redirects File Users sang yn miniforge3 envs intel env lib python3 8 site packages prefect client py line 233 in send response raise for status File Users sang yn miniforge3 envs intel env lib python3 8 site packages httpx models py line 736 in raise for status raise HTTPStatusError message request=request response=self httpx HTTPStatusError Client error 401 Unauthorized for url ```

Sang Young Noh · Answer

Just to show that the key has no expiration date

Anna Geller · Answer

this error indicates that your flow runner is not authenticated to retrieve flow code from storage can you share the following 1 your full DeploymentSpec of that flow 2 how did you configure the default storage 3 the output of `prefect version`

Sang Young Noh · Answer

The prefect version shows as follows ```Version 2 0b5 API version 0 3 1 Python version 3 8 0 Git commit 7b27c7cf Built Tue May 17 2022 4 54 PM OS Arch darwin x86 64 Profile default Server type hosted```

Sang Young Noh · Answer

the default storage I made was local I will get the deployment spec now

Sang Young Noh · Answer

What I find odd is that I am not trying to deploy onto the orion server instead of the prefect cloud and yet <https api beta prefect io api> part of the error indicates that prefect infers that I am trying to deploy onto prefect cloud

Anna Geller · Answer

I m so confused do you want to use Cloud 2 0 or self host

Anna Geller · Answer

API keys are for Cloud 2 0 self hosted doesn t have API keys

Sang Young Noh · Answer

No sorry I was being a bit ambiguous so I apologize I was testing the prefect cloud and because that error was turning up I switched to the local orion server

Sang Young Noh · Answer

Because the error itself was referring to the api beta I thought the error originated from cloud

Anna Geller · Answer

but what s your end goal are you OK with still using Cloud 2 0 if so you can create an API key which doesn t expire and configure it using ```prefect cloud login k yourkey w youraccount yourworkspace```

Sang Young Noh · Answer

The end goal is to use prefect cloud yes Let me try that and get back to you

Anna Geller · Answer

to make it all less confusing you can create multiple profiles ```prefect profile create cloud``` here configure cloud and to use local Orion you can use the default profile ```prefect profile use default``` If I understood your intention you want to develop locally on a local Orion server but when you are ready to deploy it you want to switch to the cloud correct this way you can just switch the profile to cloud then ```prefect profile use cloud```

Sang Young Noh · Answer

If I understood your intention you want to develop locally on a local Orion server but when you are ready to deploy it you want to switch to the cloud correct Yes I am also doing some benchmarking between prefect 1 and 2 to see what the pros cons are between the two platforms hence switching between them to compare and contrast Sorry I didnt give a context to what I was doing

Anna Geller · Answer

I meant switching between local Prefect 2 0 and Cloud 2 0 I didn t mean Prefect 1 0

Anna Geller · Answer

I would recommend starting directly with Prefect 2 0 if you are starting with Prefect now

Sang Young Noh · Answer

Yes Ok I feel like slack is making things lost in translation and me being a numpty writing is causing no end to the confusion and I am sorry So just to clarify I have been trying to run some deployments on the prefect cloud 2 and comparing contrasting with prefect 1 which meant running some deployments on cloud 2 and testing and logging the features As we have a large infrastructure already built in prefect 1 it was essential from my side to ensure that the people who put me in charge of this is ok with the changes i e does the advantages of prefect 2 justify the effort put in to refactor Hence coming to the present the error was because I was playing around with deployments with prefect 2

Sang Young Noh · Answer

I do still seem to be getting the error I ve tried to just view the prefect storage ```prefect storage ls``` and the error presists ```Client error 401 Unauthorized for url <https api beta prefect io api accounts 5c1f7ebd 7031 4f69 bc00 6d21a9b17548 workspaces 9a321454 aa9e 4fb8 8f4c 17e327a77a01 blocks get default storage block> ```

Anna Geller · Answer

this error indicates that your flow runner is not authenticated to retrieve flow code from storage can you share the following 1 your full DeploymentSpec of that flow 2 how did you configure the default storage 3 the output of `prefect version`

Anna Geller · Answer

totally understandable if you are happy with Prefect 1 0 no need to change but at some point you will have to migrate to Prefect 2 0 since this is the LTS version

Sang Young Noh · Answer

So to answer the deployment spec ``` FlowSchedule = rrule freq = MINUTELY interval= 5 dtstart = datetime datetime 2020 1 1 8 31 0 byhour = range 8 9 byminute = range 30 52 return DeploymentSpec flow=ng daily polling name=f test schedule=RRuleSchedule from rrule FlowSchedule tags=deploy tags parameters=parameters flow runner=SubprocessFlowRunner ``` As for the storage following the documentation I created the temporary storage <https orion docs prefect io concepts storage > Finally for the version I m getting this ```Version 2 0b5 API version 0 3 1 Python version 3 9 13 Git commit 7b27c7cf Built Tue May 17 2022 4 54 PM OS Arch darwin x86 64 Profile default Server type hosted```

Sang Young Noh · Answer

The error only became apparent quite recently Previously the create deployment for this same batch of code was completely fine

Anna Geller · Answer

the problem currently is that you want to use Cloud 2 0 but your profile doesn t point to Cloud it points to a local configuration ```Server type hosted``` Can you build a new environment from scratch and take it all step by step your deployment spec is fine but your installation is not if you want to use Cloud 2 0

Sang Young Noh · Answer

Ok I will get on that now

Anna Geller · Answer

I think the easiest is to create a cloud profile switch to it then do `prefect cloud login` on this cloud profile and this will make things much easier

Sang Young Noh · Answer

I think switching to the cloud profile fixed the bug Thank you By the way I dont think this part of switching profiles is documented Maybe I can assist in writing this I dont know whether I know enough to do that but you never know haha

Anna Geller · Answer

it s <https orion docs prefect io concepts settings |here> but I agree there s never enough docs + tutorials examples

Sang Young Noh · Answer

Ahhh ok its just me being slow then hahaha

Sang Young Noh · Answer

Hi again Just on the Deployment spec and the flows for example in this particular flow ```DeploymentSpec name= test flow=my flow flow runner=SubprocessFlowRunner env= MY VARIABLE FOO condaenv= test ``` within the flow is it possible to call upon the environmental variables as os environ MY VARIABLE Thanks

Sang Young Noh · Answer

Hello Sorry again I m struggling with the deployments on cloud again For example I created a profile called cloudtest and the prefect version says ```Version 2 0b6 API version 0 5 0 Python version 3 10 4 Git commit 502ee008 Built Tue Jun 7 2022 4 58 PM OS Arch darwin x86 64 Profile cloud Server type hosted``` The server type says hosted which means it isnt pointing to the cloud server but looking at the config ```prefect p cloudtest config view ``` I seem to get ```PREFECT PROFILE= cloudtest PREFECT API KEY= PREFECT API URL= from profile ``` The api url is as shown in the docs the api key is that which I did create and yet I seem to be getting this error ```prefect exceptions PrefectHTTPStatusError Redirect response 301 Moved Permanently for url Redirect location For more information check ```

Anna Geller · Answer

you can add env variables to the UniversalFlowRunner and all flow runners that inherit from it incl the SubprocessFlowRunner

Anna Geller · Answer

Can you try creating new default storage

Anna Geller · Answer

if nothing else works try recreating a deployment after you created new storage and assigned it as default

Anna Geller · Answer

your deployment spec is fine

Sang Young Noh · Answer

ok I ll give it a try

Sang Young Noh · Answer

Ok you seem to have the miracle touch It seems the steps is 1 Ensure the local storage or later a S3 storage for example is sorted is set with ```prefect storage create ``` 2 Login to cloud with prefect cloud login key ```3 prefect deployment create py for the creation of deployments``` 4 Profit New Prefect cloud UI is lovely by the way The steps from 1 to 2 of the local prefect pointing to the cloud after that has been configured with the PREFECT API URL however has been the bottleneck Sometimes it seems to work sometimes it doesnt

Anna Geller · Answer

I see nice work figuring this out keep us posted if the issue persists and thanks for the nice feedback

Sang Young Noh · Answer

Just another question sorry again not related to the above issue in prefect 1 the logging of states of a flow or task was handled with state handlers but in prefect 2 am I to assume that futures and states and calling them within the flow itself will be the equivalent operation

Anna Geller · Answer

what problem are you trying to solve this way there is no concept of state handlers but we will have a functionality to send e g alerts on failure

Anna Geller · Answer

example <https discourse prefect io t how to take action on a state change of a task run task level state handler 82>

Sang Young Noh · Answer

Oh nothing in particular at the moment I am using a combination of futures and client states to force a flow into failure depending on a set criteria It was just because I am looking at a prefect 1 version of the same code and prefect 2 version I made and it was just a curiosity