https://prefect.io logo
j

Jeremy Phelps

01/11/2021, 11:10 PM
How do you get your Google credentials into Kubernetes? I have a Google credentials JSON file on the Agent machine, but when my flow tries to run in Kubernetes, this error happens:
Copy code
Failed to load and execute Flow's environment: DefaultCredentialsError('Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see <https://cloud.google.com/docs/authentication/getting-started>')
c

Chris White

01/11/2021, 11:15 PM
Hi Jeremy - please see https://docs.prefect.io/orchestration/recipes/third_party_auth.html#third-party-authentication along with Google’s documentation for how to securely authenticate with their APIs via Docker / K8s
j

Jeremy Phelps

01/11/2021, 11:24 PM
There doesn't seem to be a way to do it at all. Google's SDK authenticates via a JSON file that contains the credentials. I have no idea how to place this file within the Kubernetes pods that Prefect creates.
Now I'm getting:
Copy code
Failed to load and execute Flow's environment: ValueError('Service account info was not in the expected format, missing fields client_email, token_uri.')
Is there no way to use my developer credentials? I have to go create a service account?
c

Chris White

01/11/2021, 11:43 PM
This sounds like a great learning opportunity for you then - there are many ways of doing it, perhaps in your case you can use a volume mount. For your follow-up question, please spend some time with the Google documentation to understand how Google’s SDK expects you to authenticate; this is not a Prefect specific question. Based on your questions and your comments today, you appear to require a step-by-step walkthrough of getting things running in Kubernetes which is not what our slack channel is intended for and not a good use of the Prefect engineering team’s time, especially given the tone of your responses.  We assume positive intent from our users and hope that you would do the same of us - we are here to help and are not actively trying to make your work difficult, in fact quite the opposite, but you have to be open to receiving that help instead of making blanket statements like “There doesn’t seem to be a way to do it at all” when hundreds of Prefect users do it everyday. On a go-forward basis, please reconsider your tone when interacting with our community and avoid snarky comments and unsolicited negative commentary, as it is deeply contrary to the spirit of mutual collaboration that everyone here is expected to uphold.