Loic M01/15/2021, 5:36 PM
Is there something I am missing there ? Should I include the google credentials when building my flow's docker image ?
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see <https://cloud.google.com/docs/authentication/getting-started|https://cloud.google.com/docs/authentication/getting->started
Josh01/15/2021, 5:39 PM
Loic M01/16/2021, 1:32 PM
My code looks like this
If I remove the
import prefect from prefect.engine.results import GCSResult from prefect.run_configs import DockerRun from prefect.storage import Docker @prefect.task(checkpoint=True, result =GCSResult(bucket="redacted")) def task1(): ... storage = DockerStorage(...) run_config = DockerRun(dockerfile="Dockerfile") with prefect.Flow("myflow") as flow: results = task1() ... # Other tasks downstream flow.register("test")
parameter, everything runs fine. If I decide to run this flow locally (changing storage, config and using
, I can save my results to GCS (using an environment variable for authentication) I see two solutions at the moment: • include the service account key when building my docker Image so I can reach it from inside the container the same way as locally •
task to save my results (though it won't be possible to use these results as checkpoint), which in this case the GCP_CREDENTIALS secret from prefect Cloud should work ?