is there a way to hide parameters? Let's say a flo...
# prefect-server
d
is there a way to hide parameters? Let's say a flow requires a secret to work, and that secret is given as a param. Can I hide that from the logs and UI?
a
Don't pass secrets as parameters 😄 There are several mechanisms for storing secrets: • using Prefect secrets • using external secrets manager • leveraging environment variables All of the above are better than using Secrets in parameters. And to answer your question directly: there is no way of hiding that because there is no need for it - parameter values are just default configuration that can be optionally overridden at runtime. Here is a deep dive
d
the secrets are dynamic and changes based on the flow run 🤔
a
if you have some sensitive value but not a secret, perhaps the KV Store is an option worth considering?
are you dynamically changing say DB password? I can't think of a single use case when secrets would change dynamically with each flow run - can you explain your use case more? especially then you should consider a secrets manager to update such frequently rotating secrets
d
are you dynamically changing say DB password
lets say I did this. how would I hide the password from the logs?
a
for log specifically, you could add a log filter - here is an example I got from Kevin:
Copy code
from prefect import task, Flow, Parameter
import time
import logging
import prefect

# Creating a filter
class SecureFilter(logging.Filter):
    def filter(self, rec):
        if 'secs' in rec.msg:
            return 0
        return 1

@task
def abc(x):
    time.sleep(x)
    return x

def get_logger():
    logger = logging.getLogger("prefect.TaskRunner")
    logger.addFilter(SecureFilter()) 
    return logger

with Flow("timer flow") as flow:
    logger = get_logger()
    secs= Parameter("secs", 1)
    abc.map([secs]*5)

flow.run()
but again, I don't think that this is the right approach (just to be transparent)
d
ah, nice! Then it would be hidden from the UI as well?
a
log filters filter out logs so that those are not collected and not sent to Prefect Cloud backend
k
You can dynamically change the Secret name you are passing right?
Copy code
@task
def get_secret_name(env):
    return env+"password"

with Flow(...) as flow:
    env = Parameter("env", default="staging")
    secret_name = get_secret_name(env)
    pw = PrefectSecret()(secret_name)
You just need to pass the secret name to the run method instead of the init