https://prefect.io
Join Slack
Hi. Is it any way to pass env variable in ECSRun(env={...}) as Prefect Secret?
# ask-community
v
Hi. Is it any way to pass env variable in ECSRun(env={...}) as Prefect Secret?
a
Can you explain what are you trying to do? Prefect Secrets can be used directly in your flow as tasks, so there is no need to set it additionally as env variables.
or are you trying to set new Prefect Secrets this way?
k
Same question as Anna but you can try
Copy code
PREFECT__CONTEXT__SECRETS__MYSECRET="MY SECRET VALUE"
upvote 1
v
I need to pass PIP_EXTRA_INDEX_URL as ECSRun env var to be available for EXTRA_PIP_PACKAGES
I don't want to set Secret I want to use Secret during registration
and pass as part of ECSRun
k
I think this will work yeah
v
But here I have to set the value
Copy code
PREFECT__CONTEXT__SECRETS__MYSECRET="MY SECRET VALUE"
value in Prefect secrets🙃
I want to write something like:
Copy code
ECSRun(
    image=self.BASE_DOCKER_IMAGE,
    env={
        'PIP_EXTRA_INDEX_URL': Secret('PIP_EXTRA_INDEX_URL').get(),
        'EXTRA_PIP_PACKAGES': ' '.join(self.dependencies),
    },
)
I am running this code from jenkins
so I need jenkins to have access to secrets
k
Oh yeah you are right the Secret is not meant for RunConfigs
v
Is it possible to access secrets having prefect token?
a
yes. I was e.g. using this in a CircleCI CI pipeline, I think Jenkins would be probably similar:
Copy code
prefect auth login --key $PREFECT_API_KEY && export PREFECT__CLOUD__USE_LOCAL_SECRETS=false && prefect register --project your_project -p flows/
k
Yeah you mean read or write? For read you can use the Secret.get() you mentioned. For write, the 
Client
has a 
set_secret
method
v
Copy code
client = Client()
pip_extra_index_url = client.graphql('{ secretValue(name: "PIP_EXTRA_INDEX_URL") }').data.secretValue
actually Secret.get is failing
k
What is your error?
v
ValueError: Local Secret "PIP_EXTRA_INDEX_URL" was not found.
and prefect.context.config.use_local_secrets = False didn't help
k
How did you set that? In the config.toml?
v
what do you mean? I just set it in python code
Copy code
prefect.context.config.use_local_secrets = False
print(Secret('PIP_EXTRA_INDEX_URL').get())
k
I don’t think that will work because the context is not really mutable after it is set. You can either put this in the 
config.toml
in the 
.prefect
folder. Or you can 
export PREFECT___CLOUD____USE__LOCAL_SECRETS=false
. You can also do:
Copy code
import os
os.environ["PREFECT__CLOUD__USE_LOCAL_SECRETS"]="false"
but it has to be before you import Prefect
upvote 1
v
it works, just did one mistake
missed cloud
Copy code
prefect.context.config.cloud.use_local_secrets = False
print(Secret('PIP_EXTRA_INDEX_URL').get())
a
the easiest way would be to set this Secret in the Prefect Cloud UI and then in your Jenkins you can register your flow as described above
v
thank you!😀
👍 1
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by