Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    Hi. Is it any way to pass env variable in ECSRun(env={...}) as Prefect Secret?
    Anna Geller

    Anna Geller

    9 months ago
    Can you explain what are you trying to do? Prefect Secrets can be used directly in your flow as tasks, so there is no need to set it additionally as env variables.
    or are you trying to set new Prefect Secrets this way?
    Kevin Kho

    Kevin Kho

    9 months ago
    Same question as Anna but you can try
    PREFECT__CONTEXT__SECRETS__MYSECRET="MY SECRET VALUE"
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    I need to pass PIP_EXTRA_INDEX_URL as ECSRun env var to be available for EXTRA_PIP_PACKAGES
    I don't want to set Secret I want to use Secret during registration
    and pass as part of ECSRun
    Kevin Kho

    Kevin Kho

    9 months ago
    I think this will work yeah
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    PREFECT__CONTEXT__SECRETS__MYSECRET="MY SECRET VALUE"
    But here I have to set the value
    value in Prefect secrets๐Ÿ™ƒ
    I want to write something like:
    ECSRun(
        image=self.BASE_DOCKER_IMAGE,
        env={
            'PIP_EXTRA_INDEX_URL': Secret('PIP_EXTRA_INDEX_URL').get(),
            'EXTRA_PIP_PACKAGES': ' '.join(self.dependencies),
        },
    )
    I am running this code from jenkins
    so I need jenkins to have access to secrets
    Kevin Kho

    Kevin Kho

    9 months ago
    Oh yeah you are right the Secret is not meant for RunConfigs
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    Is it possible to access secrets having prefect token?
    Anna Geller

    Anna Geller

    9 months ago
    yes. I was e.g. using this in a CircleCI CI pipeline, I think Jenkins would be probably similar:
    prefect auth login --key $PREFECT_API_KEY && export PREFECT__CLOUD__USE_LOCAL_SECRETS=false && prefect register --project your_project -p flows/
    Kevin Kho

    Kevin Kho

    9 months ago
    Yeah you mean read or write? For read you can use the Secret.get() you mentioned. For write, the
    Client
    has a
    set_secret
    method
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    client = Client()
    pip_extra_index_url = client.graphql('{ secretValue(name: "PIP_EXTRA_INDEX_URL") }').data.secretValue
    actually Secret.get is failing
    Kevin Kho

    Kevin Kho

    9 months ago
    What is your error?
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    ValueError: Local Secret "PIP_EXTRA_INDEX_URL" was not found.
    and prefect.context.config.use_local_secrets = False didn't help
    Kevin Kho

    Kevin Kho

    9 months ago
    How did you set that? In the config.toml?
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    what do you mean? I just set it in python code
    prefect.context.config.use_local_secrets = False
    print(Secret('PIP_EXTRA_INDEX_URL').get())
    Kevin Kho

    Kevin Kho

    9 months ago
    I donโ€™t think that will work because the context is not really mutable after it is set. You can either put this in the
    config.toml
    in the
    .prefect
    folder. Or you can
    export PREFECT___CLOUD____USE__LOCAL_SECRETS=false
    . You can also do:
    import os
    os.environ["PREFECT__CLOUD__USE_LOCAL_SECRETS"]="false"
    but it has to be before you import Prefect
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    it works, just did one mistake
    missed cloud
    prefect.context.config.cloud.use_local_secrets = False
    print(Secret('PIP_EXTRA_INDEX_URL').get())
    Anna Geller

    Anna Geller

    9 months ago
    the easiest way would be to set this Secret in the Prefect Cloud UI and then in your Jenkins you can register your flow as described above
    Vadym Dytyniak

    Vadym Dytyniak

    9 months ago
    thank you!๐Ÿ˜€