Nate
04/04/2024, 7:41 PMMarvin
04/04/2024, 7:41 PMNate
04/04/2024, 7:42 PMMarvin
04/04/2024, 7:43 PMprefect_gcp
package to save your service account information into a GcpCredentials
block. This can be done directly with service account info or a file path, but using info ensures accessibility across containers.
python
from prefect_gcp import GcpCredentials
service_account_info = { # your service account info as a dictionary }
GcpCredentials(service_account_info=service_account_info).save("BLOCK-NAME-PLACEHOLDER")
You can then load the saved block credentials in your Prefect flows:
python
gcp_credentials = GcpCredentials.load("BLOCK-NAME-PLACEHOLDER")
3. Deploy a Prefect worker on Google Cloud Run - Use the gcloud
CLI to deploy your worker as a service with the necessary environment variables:
shell
gcloud run deploy prefect-worker --image=prefecthq/prefect:2-latest \
--set-env-vars PREFECT_API_URL=$PREFECT_API_URL,PREFECT_API_KEY=$PREFECT_API_KEY
Make sure to replace placeholders with your actual Prefect Account ID, Workspace ID, API Key, Service Account Name, and Work Pool Name.
For more details, here are some helpful links:
- Setting up Prefect with Google Cloud Run
- Using Prefect with Google Cloud Platform
Remember to install the prefect-gcp
package to access these features and optionally with extra support for BigQuery if needed. Consider additional configurations if you plan on using Google Cloud Platform services like BigQuery or Vertex AI.Nate
04/04/2024, 7:44 PM