Hi. Is it any way to pass env variable in ECSRun(e...
# ask-communityv
Vadym Dytyniak
12/08/2021, 2:54 PMHi. Is it any way to pass env variable in ECSRun(env={...}) as Prefect Secret?
a
Anna Geller
12/08/2021, 3:00 PM
Can you explain what are you trying to do? Prefect Secrets can be used directly in your flow as tasks, so there is no need to set it additionally as env variables.
Anna Geller
12/08/2021, 3:01 PM
or are you trying to set new Prefect Secrets this way?
k
Kevin Kho
12/08/2021, 3:02 PM
Same question as Anna but you can try
Copy code
PREFECT__CONTEXT__SECRETS__MYSECRET="MY SECRET VALUE"upvote 1
v
Vadym Dytyniak
12/08/2021, 3:03 PMI need to pass PIP_EXTRA_INDEX_URL as ECSRun env var to be available for EXTRA_PIP_PACKAGES
Vadym Dytyniak
12/08/2021, 3:09 PMI don't want to set Secret I want to use Secret during registration
Vadym Dytyniak
12/08/2021, 3:09 PMand pass as part of ECSRun
k
Kevin Kho
12/08/2021, 3:11 PM
I think this will work yeah
v
Vadym Dytyniak
12/08/2021, 3:12 PMBut here I have to set the value
Vadym Dytyniak
12/08/2021, 3:12 PM Copy code
PREFECT__CONTEXT__SECRETS__MYSECRET="MY SECRET VALUE"Vadym Dytyniak
12/08/2021, 3:12 PMvalue in Prefect secrets🙃
Vadym Dytyniak
12/08/2021, 3:13 PMI want to write something like:
Copy code
ECSRun(
image=self.BASE_DOCKER_IMAGE,
env={
'PIP_EXTRA_INDEX_URL': Secret('PIP_EXTRA_INDEX_URL').get(),
'EXTRA_PIP_PACKAGES': ' '.join(self.dependencies),
},
)Vadym Dytyniak
12/08/2021, 3:13 PMI am running this code from jenkins
Vadym Dytyniak
12/08/2021, 3:13 PMso I need jenkins to have access to secrets
k
Kevin Kho
12/08/2021, 3:14 PM
Oh yeah you are right the Secret is not meant for RunConfigs
v
Vadym Dytyniak
12/08/2021, 3:14 PMIs it possible to access secrets having prefect token?
a
Anna Geller
12/08/2021, 3:16 PM
yes. I was e.g. using this in a CircleCI CI pipeline, I think Jenkins would be probably similar:
Copy code
prefect auth login --key $PREFECT_API_KEY && export PREFECT__CLOUD__USE_LOCAL_SECRETS=false && prefect register --project your_project -p flows/k
Kevin Kho
12/08/2021, 3:17 PM
Yeah you mean read or write? For read you can use the Secret.get() you mentioned. For write, the
Clientset_secretv
Vadym Dytyniak
12/08/2021, 3:18 PM Copy code
client = Client()
pip_extra_index_url = client.graphql('{ secretValue(name: "PIP_EXTRA_INDEX_URL") }').data.secretValueVadym Dytyniak
12/08/2021, 3:18 PMactually Secret.get is failing
k
Kevin Kho
12/08/2021, 3:19 PM
What is your error?
v
Vadym Dytyniak
12/08/2021, 3:20 PMValueError: Local Secret "PIP_EXTRA_INDEX_URL" was not found.
Vadym Dytyniak
12/08/2021, 3:20 PMand prefect.context.config.use_local_secrets = False didn't help
k
Kevin Kho
12/08/2021, 3:21 PM
How did you set that? In the config.toml?
v
Vadym Dytyniak
12/08/2021, 3:21 PMwhat do you mean? I just set it in python code
Vadym Dytyniak
12/08/2021, 3:21 PM Copy code
prefect.context.config.use_local_secrets = False
print(Secret('PIP_EXTRA_INDEX_URL').get())k
Kevin Kho
12/08/2021, 3:23 PM
I don’t think that will work because the context is not really mutable after it is set. You can either put this in the
config.toml.prefectexport PREFECT___CLOUD____USE__LOCAL_SECRETS=false Copy code
import os
os.environ["PREFECT__CLOUD__USE_LOCAL_SECRETS"]="false"upvote 1
v
Vadym Dytyniak
12/08/2021, 3:25 PMit works, just did one mistake
Vadym Dytyniak
12/08/2021, 3:25 PMmissed cloud
Vadym Dytyniak
12/08/2021, 3:25 PM Copy code
prefect.context.config.cloud.use_local_secrets = False
print(Secret('PIP_EXTRA_INDEX_URL').get())a
Anna Geller
12/08/2021, 3:25 PM
the easiest way would be to set this Secret in the Prefect Cloud UI and then in your Jenkins you can register your flow as described above
v
Vadym Dytyniak
12/08/2021, 3:25 PMthank you!😀
👍 1
4 Views