Ryan Abernathey
09/09/2019, 7:42 PMChris White
09/09/2019, 7:57 PM~/.prefect/config.toml
, and for GOOGLE_APPLICATIONS_CREDENTIALS
specifically you’ll need to use json.dumps(credentials)
and strip out the escaped newlines. See this issue: https://github.com/PrefectHQ/prefect/issues/1471Ryan Abernathey
09/09/2019, 8:19 PM~/.prefect/config.toml
according to the instructions. But when I try
from prefect.client import Secret
Secret("GOOGLE_APPLICATION_CREDENTIALS").get()
I am getting ValueError: Local Secret "GOOGLE_APPLICATION_CREDENTIALS" was not found.
[context.secrets]
section)Chris White
09/09/2019, 8:20 PMRyan Abernathey
09/09/2019, 8:21 PMChris White
09/09/2019, 8:21 PMRyan Abernathey
09/09/2019, 8:21 PMChris White
09/09/2019, 8:22 PMcontext
which is explicitly shipped to the workersRyan Abernathey
09/09/2019, 8:22 PMCould not deserialize key data
error when I try to actually use this. I will study https://github.com/PrefectHQ/prefect/issues/1471 more closely.Chris White
09/09/2019, 8:25 PMRyan Abernathey
09/09/2019, 8:26 PMwith open('pangeo-181919-0c1f01383379.json') as fp:
token = json.load(fp)
prefect.context.secrets['GOOGLE_APPLICATION_CREDENTIALS'] = token
Chris White
09/09/2019, 8:27 PMRyan Abernathey
09/09/2019, 8:27 PMAttributeError: 'Context' object has no attribute 'secrets'
config.toml
, but if I remove it, I can’t access the secrets dict)Chris White
09/09/2019, 8:28 PMcontext
like a dictionary as well:
prefect.context['secrets']['GOOGLE_APPLICATION_CREDENTIALS'] = token
Ryan Abernathey
09/09/2019, 8:30 PMKeyError: 'secrets'
Chris White
09/09/2019, 8:33 PMprefect.context.setdefault('secrets', {})['GOOGLE_APPLICATION_CREDENTIALS'] = token
Ryan Abernathey
09/09/2019, 8:35 PMprefect.context.secrets['KEY'] = 'VALUE'
as part of the API
- Add something to the docs that explains how to set a secret this wayChris White
09/09/2019, 8:36 PMRyan Abernathey
09/09/2019, 8:37 PMChris White
09/09/2019, 8:38 PMAbraão Zaidan
11/08/2019, 8:16 PM# Authentication for Prefect Google Cloud Tasks
# <https://docs.prefect.io/api/unreleased/tasks/google.html>
json_file = open(os.environ['GOOGLE_APPLICATION_CREDENTIALS'], 'r')
context.setdefault('secrets', {})
context.secrets['GOOGLE_APPLICATION_CREDENTIALS'] = json_file.read()
json_file.close()