Steven Trimboli
12/07/2022, 6:09 PMfrom prefect_gcp import GcpCredentials
gcp_credentials_block = GcpCredentials.load("bq-credentials")
bigquery_client = bigquery.Client(credentials=gcp_credentials_block)
I receive this error message: "ValueError: This library only supports credentials from google-auth-library-python. See https://google-auth.readthedocs.io/en/latest/ for help on authentication with this library."
How can I pass credentials to GCP successfully?Kevin Grismore
12/07/2022, 6:21 PMCredentials
object. The data stored in the GcpCredentials block is service_account_info
. If you install the google-auth
python package, you can do something like:
from google.oauth2.service_account import Credentials
from prefect_gcp import GcpCredentials
gcp_credentials_block = GcpCredentials.load("bq-credentials")
creds = Credentials.from_service_account_info(gcp_credentials_block.service_account_info)
bigquery_client = bigquery.Client(credentials=creds)
gcp_credentials_block
directly into any of the prefect_gcp.bigquery
tasks without needing to make your own bigquery client, but you'll be limited to the functionality provided by the task collectionAndrew Huang
12/07/2022, 6:45 PMfrom prefect_gcp import GcpCredentials
gcp_credentials_block = GcpCredentials.load("bq-credentials")
google_auth_credentials = gcp_credentials_block.get_credentials_from_service_account()
bigquery_client = bigquery.Client(credentials=google_auth_credentials)
Alternatively, you can simply get the bigquery client directly from gcp_credentials
gcp_credentials_block = GcpCredentials.load("bq-credentials").get_bigquery_client()
or best practice is simply to use one of the built-in tasks:
https://github.com/PrefectHQ/prefect-gcp/blob/main/prefect_gcp/bigquery.pyKevin Grismore
12/07/2022, 6:46 PM.get_credentials_from_service_account()
existed. This'll help me too. Thanks!Steven Trimboli
12/07/2022, 6:48 PMAndrew Huang
12/07/2022, 6:56 PMgcp_credentials_block = await GcpCredentials.load("bq-credentials")
or, you can simply put everything into a flow, and it should work without the awaitSteven Trimboli
12/07/2022, 6:57 PMAndrew Huang
12/07/2022, 6:58 PM