https://prefect.io logo
Powered by
Title
n

Nora Myer

10/26/2022, 4:20 PM
Hey hi - prefect noob here - we’re using Prefect 2.0 w/ Prefect Cloud in a new system were building, and I have a couple Q’s around secrets. Details in the thread - thanks in advance!!
✅ 1
Right now we have a task that grabs secrets from Vault (lets call Task A), and a task that uses those secrets to make api calls to internal services (Task B). Since you cant call tasks within tasks, the higher level flow that calls Task A, then Task B. The issue is by passing these secrets into the task as a parameter, I think we’d be exposing them in logs and in the Prefect Cloud 2.0 UI So my q’s: 1. Does using a secret Block always require uploading the Block to Prefect Cloud 2.0 (if thats the route were going)? We dont want any secrets to ever leave our own infra, so wouldnt want to use Blocks if thats the case 2. Is there any other way to pass secrets around within a flow and not expose them? 3. Obviously another option is, only grab the secrets from within the task level that is actually using them. So make Task A, not be a task. If this is the best route, also totally fine mostly just curious
k

Kalise Richmond

10/26/2022, 4:30 PM
Hi Nora, in order to ensure that your secrets never leaves your infrastructure option 3 would be best. Blocks do require uploading to Prefect Cloud 2.0. You could definitely make a github request for feature enhancement to have a way to obscure parameters in the UI.
r

Ryan Peden

10/26/2022, 4:38 PM
If you decide to pass them as parameters, consider using 
pydantic.SecretStr
like many of our blocks do. For example: 
from prefect import flow, get_run_logger
from pydantic import SecretStr

@flow
def flow1():
    secret = SecretStr("password")
    flow2(secret)
    
@flow
def flow2(secret: SecretStr):
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"secret: {secret}")

if __name__ == "__main__":
    flow1()
As this example shows, the secret remains obscured in the flow run parameters UI (screenshot attached), and if you accidentally log a secret, it will be hidden there as well: 
12:30:30.654 | INFO    | prefect.engine - Created flow run 'quiet-boobook' for flow 'flow1'
12:30:30.826 | INFO    | Flow run 'quiet-boobook' - Created subflow run 'crystal-sponge' for flow 'flow2'
12:30:30.856 | INFO    | Flow run 'crystal-sponge' - secret: **********
12:30:30.877 | INFO    | Flow run 'crystal-sponge' - Finished in state Completed()
12:30:30.891 | INFO    | Flow run 'quiet-boobook' - Finished in state Completed('All states completed.')
🙌 4
n

Nora Myer

10/26/2022, 6:05 PM
amazing - that was going to be my next Q yeah, does prefect respect the SecretStr type even outside of Blocks - thank you!
4 Views
#prefect-communityJoin Slack
Powered by