https://prefect.io logo
Title
a

Adam Everington

11/09/2021, 3:00 PM
Hiya all, i'm trying to run prefect server through the provided helm chart, changing the postgres values for a provisioned postgres instance in azure. My command is as below:
helm install prefectprod prefecthq/prefect-server -n prod --set agent.enabled=true --set postgresql.postgresqlUsername=admin%40my-server --set postgresql.postgresqlPassword=p%40ssword123 --set postgresql.externalHostname=<http://my-server.postgres.databases.azure.net|my-server.postgres.databases.azure.net> --set postgresql.useSubChart=false
when looking at my pods I'm getting the following
a

Anna Geller

11/09/2021, 3:28 PM
@Adam Everington can you check the logs on the pods? maybe it can give us some clues…
do you run it on the Azure Kubernetes cluster or in a local Kubernetes cluster on your machine? it could be that some security groups must be opened up to allow access
a

Adam Everington

11/09/2021, 3:30 PM
Yeah on AKS, not locally
a

Anna Geller

11/09/2021, 3:34 PM
if you don’t want to share your logs here, you can DM it to me
a

Adam Everington

11/09/2021, 3:37 PM
Anna, you'll have to excuse me i'm a K8's noob. I've had to find the logs through the azure ui. When going into the apollo log as an example the error I found was: Error: secret "prefectprod-postgresql" not found However i'm specifying username / password so a secret isn't required?
...yep same in hasura
a

Anna Geller

11/09/2021, 3:45 PM
no worries. It looks like helm is expecting the database password as Kubernetes Secrets named “prefectprod-postgresql”. Let me check how you can create it
can you try this?
kubectl create secret generic prefectprod-postgresql --from-literal='username=YOUR_POSTGRES_USERNAME' --from-literal='password=YOUR_POSTGRES_PWD'
👀 1
not sure if you have to redeploy the helm chart then
a

Adam Everington

11/09/2021, 3:48 PM
i'll just install from fresh
👍 1
will the username / pw have to be encoded like in the set operation with %40?
a

Anna Geller

11/09/2021, 3:50 PM
I would say no, but it’s just my gut feeling
btw, did you know that there is a video tutorial about deploying Server to Azure with Helm?

https://www.youtube.com/watch?v=EwsMecjSYEU&amp;t=1s

a

Adam Everington

11/09/2021, 3:50 PM
Yes, it's fab, that's what i'm working from 🙂
👍 1
error: "'username" is not valid key name for a Secret a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+')
a

Anna Geller

11/09/2021, 3:53 PM
yet another btw 😄 btw, did you know that Prefect Cloud has now 20,000 free task runs every month? overall, getting started with Prefect Cloud is much easier than Prefect Server, and you wouldn’t have to care about maintaining Server, Postgres database etc.
a

Adam Everington

11/09/2021, 3:53 PM
yeah i'm leaning towards it !!
:upvote: 1
a q I have on cloud though is your quote is per Task right? so doesn't this encourage you to create bad code by sticking as much into a single task as possible?
a

Anna Geller

11/09/2021, 3:59 PM
Why do you see it that way? Given that: • there are 20,000 free task runs each month (and trust me, that’s actually a lot, I’ve never crossed it so far) • there are automatic discounts with more tasks, • mapping allows visibility into child tasks running in parallel I feel that Prefect encourages smaller tasks. But overall, you are free to design your tasks as you wish, there are no restrictions here.
a

Adam Everington

11/09/2021, 4:01 PM
I mean we almost definitely wouldn't hit that thinking about it
I really appreciate your help, I think I will look into the cloud a bit more
👍 1
a

Anna Geller

11/09/2021, 4:02 PM
and really when looking at the costs, it’s worth looking at the TCO - what costs more: $0.005 per task run, or human hours maintaining all the infrastructure. I’m honestly quite happy paying for a service that takes this headache away for me, but I understand everyone can approach it differently
a

Adam Everington

11/09/2021, 4:03 PM
Yeah, I mean running it on Azure would essentially cost me ~£130 a month
z

Zanie

11/09/2021, 6:03 PM
To add to
doesn't this encourage you to create bad code by sticking as much into a single task as possible?
You can game the system and put as much into a single task as you'd like. This doesn't really change your developer experience though, you can still split code into functions that are called within that task. This doesn't have to be bad code. Rolling things into a single task just affects the granularity at which Prefect is orchestrating your code. This is why it's a good metric for cost—it directly corresponds to the value we are providing. If you don't need your orchestration / visibility and can get away with fewer tasks, great. If you need fine-grained orchestration and visibility, you may eventually have to pay for it, but hopefully there's a clear benefit for your $$. And as Anna mentioned, the free tier should get you far!
:upvote: 1
🙌 1
I also believe the password needs to be a key-value pair in a secret where the key is
postgresql-password
and the secret name can be anything but must be set in
existingSecret
. See https://github.com/PrefectHQ/server/blob/master/helm/prefect-server/values.yaml#L52-L57