Loic M

01/15/2021, 5:36 PM
Hello everyone ! Sorry If this has already been answered here but my search did not return anything. I have trouble understanding authentication through google cloud to store my results with the help of prefect cloud secrets: Following this part of the documentation, I have setup a secret GCP_CREDENTIALS containing the JSON service account key. From what I understood, this should be enough to automatically authenticate my flows with the right credentials: however, when I run my docker Agent, my task fails with the following traceback:
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see <|>started
Is there something I am missing there ? Should I include the google credentials when building my flow's docker image ?


01/15/2021, 5:39 PM
I’ve bundled my credentials into the docker image, but I’d love to know if there’s a better way.


01/15/2021, 6:10 PM
Hi @Loic M - can you give some more information about your setup? Are you using Prefect Cloud and trying to run tasks from the Task library?

Loic M

01/16/2021, 1:32 PM
Hi @nicholas, I am using a cutsom task but try to save its result (a dictionary) into a
My code looks like this
Copy code
import prefect
from prefect.engine.results import GCSResult
from prefect.run_configs import DockerRun
from import Docker

@prefect.task(checkpoint=True, result =GCSResult(bucket="redacted"))
def task1():

storage = DockerStorage(...)
run_config = DockerRun(dockerfile="Dockerfile")

with prefect.Flow("myflow") as flow:
    results = task1()
    ... # Other tasks downstream

If I remove the
parameter, everything runs fine. If I decide to run this flow locally (changing storage, config and using
instead of
, I can save my results to GCS (using an environment variable for authentication) I see two solutions at the moment: • include the service account key when building my docker Image so I can reach it from inside the container the same way as locally • Create a
task to save my results (though it won't be possible to use these results as checkpoint), which in this case the GCP_CREDENTIALS secret from prefect Cloud should work ?
(Just tested this and was met with the same error as previously)