Don't mind, adding another question... 😣 Topic: Cloud Secrets Context: My flow is dependent on some Secrets I added to the cloud. I have a custom dockerfile that tries to set things up, with the following code:

COPY configuration /opt/strdata-prefect/configuration
ENV PREFECT__USER_CONFIG_PATH /opt/strdata-prefect/configuration/config.toml

Config.toml:

backend = "cloud"

[cloud]
use_local_secrets = false

    [cloud.agent]
    name = "strdata-agent"

    # Setting it to `DEBUG` for verbose logging
    level = "DEBUG"

[logging]
# The logging level: NOTSET, DEBUG, INFO, WARNING, ERROR, or CRITICAL
level = "DEBUG"

# Send logs to Prefect Cloud
log_to_cloud = true

# Extra loggers for Prefect log configuration
extra_loggers = "[]"

Error Logs:

ValueError: Local Secret "REDSHIFT_PASSWORD" was not found.

Thanks again, Prefect Team!

Hi @bruno.corucho - can you confirm you see that secret in the Cloud UI?

1sec, will get you that @nicholas

Oh! Do you have a

config.toml

at

~/.prefect

?

no, I dont. I'm trying to change the default path to /opt/strdata-prefect/configuration/config.toml by using PREFECT__USER_CONFIG_PATH env. name

Got it @bruno.corucho - so it's possible you have an environment variable for local secrets overriding what's in your config, or that the user config path is incorrect (or not present for whatever reason). If you don't have that environment variable set (

printenv

should confirm this), try moving your config to the normal location and see if that works

Hi @nicholas I have a followup on this issue

@Luis Muniz or @bruno.corucho can you share your flow code or a min reproducible example of what you're seeing? I think you may be doing something weird with the secrets cause those shouldn't be resolving on serialization

hi, working on it

Sure, that'll work

I will try to explain this as well as I can, as I am a python newb. We were initializing our secrets by assigning them to global variables like this,

REDSHIFT_PASSWORD = Secret("REDSHIFT_PASSWORD").get()

#now defining a tesk
@task
def dosomething():
    connect_to_redshift(REDSHIFT_PASSWORD)

@Luis Muniz since

Secrets

are tasks, you're essentially trying to execute the task outside of a Prefect context. My suggestion is to move your secrets into the Flow context like this:

@task
def i_need_redshift(redshift_password):
  password = redshift_password.get()
  # do something with password

with Flow("your_flow") as flow:
  redshift_password = Secret("REDSHIFT_PASSWORD")

  i_need_redshift(redshift_password)

or if you'd prefer not to build the task into your dependency tree that way, you can call it inside the tasks that need it instead:

@task
def i_need_redshift(redshift_password):
  redshift_password = Secret("REDSHIFT_PASSWORD").get()
  # do something with redshift_password

yes, that's what we found out while trying to create a reproducible example

Unfortunately, on the file where we're assigning our flow, we're importing all method and variables from the file where we have our tasks defined... rip emcapsulation

now that you mention that they are tasks it makes total sense

Yeah it sounds like it'll require a bit of reworking of the code, sorry about that @bruno.corucho - this is a more secure pattern though and prevents secrets from leaking into an unintended context

true, we did not bother about refactoring this code was we were more focused on the full flow executation than the code itself xD

No worries at all, happy to help: )

thanks guys you rock, stellar support