https://prefect.io logo
Title
m

Matt Drago

05/23/2022, 11:15 AM
Hey Folks, Been a while since I've frequented these slack channels, but I've finally gotten some time at work to upgrade the Prefect installation that I set up a while ago (version 0.14.14). I was looking at using the Helm charts so I could dispose of the ones that I hand crafted. One of the problems that I am trying to work out though is how to connect to the external DB. My deployment environment is a K8S cluster in GCP. In my current installation, I have configured cloud_sql_proxy sidecars for the graphql and hasura deployments. I cannot find a way to do this with the new charts. Is the only way to connect to the CloudSQL Postgres DB to use the private/public IP of the DB instance?
a

Anna Geller

05/23/2022, 1:04 PM
this is a hard question, I can ask the team and see if someone knows
t

Tyler Wanner

05/23/2022, 2:35 PM
your best bet for adding cloud sql proxy is probably going to be forking the chart as I don't believe it supports extra containers. If you can add the sidecar container with the connection name, you should just need to set the externalHostname as 127.0.0.1:5432 or something similar and set useSubChart to false
:thank-you: 1
a

Anna Geller

05/23/2022, 4:15 PM
another colleague also recommended forking the helm chart as your best option - explanation: • there doesn't seem to be any way to configure sidecar containers at present… in the templates, there’s a
containers
section and your desired behavior is to add a second container to the pod, so that there is a main container + secondary container (sidecar). So we’d need to modify the template to accept a value that allows the configuration of additional arbitrary containers - this could be a feature request for OSS contributors • As a workaround, you can modify the Helm chart - fork it and add the sidecar container manually to the deployments to include the sidecar container with the connection name - https://github.com/PrefectHQ/server/blob/a49dac5674d6e63bde399171c20215619d22160c/helm/prefect-server/templates/hasura/deployment.yaml#L42
:upvote: 1
m

Matt Drago

05/23/2022, 10:34 PM
Thanks folks. Now why didn't I think of forking the repo!! Thanks for the suggestions. I might attempt the PrivateIP route to get what I am working on done before looking at forking the repo. I think there was also a place that an
init
section was connecting to the database, so I'd have to work out if the sidecar would work for that as well.
👍 1
t

Tyler Wanner

05/23/2022, 10:41 PM
I believe if you set the externalHostname, it skips the initiaization of the database. I’m not 100% sure but if you run into issues there let me know! I would like to see this pattern
m

Matt Drago

05/23/2022, 10:43 PM
Ah okay. I'll keep this thread posted with where I get to.
🙏 2
So close, yet so far. I cannot load sidecars for the initContainers (as used in graphql/deployment.yaml. I'm going to do something a bit... nqr to get this going (i.e. do the db upgrade in the graphql container command).
Folks, I have something that started and I was able to hit the UI (I'm testing locally on minikube talking to a clone of our production db that is hosted in Google CloudSQL). I've pushed the changes I've made to a clone which you can have a look at https://github.com/PrefectHQ/server/compare/master...PaperCutSoftware:master Let me know your thoughts on the changes and if it is something that I should submit a PR for.
🙌 1
a

Anna Geller

05/24/2022, 11:47 AM
I'd say you can submit a PR and this way anyone interested in this feature can comment and other engineers can leave their feedback on the PR directly.
thanks for contributing
m

Matt Drago

05/24/2022, 10:07 PM
I'll check over a few things and then I'll submit the PR.
a

Anna Geller

05/30/2022, 9:47 AM
thanks for contributing!
m

Matt Drago

05/30/2022, 9:48 AM
No problemo