Hi folks! I'm having trouble finding best practices around moving sensitive data between Prefect tasks. We have secrets stored in AWS SSM and are using boto3 to grab them within a task at run-time. Is it unsafe to return those secrets from one task, to be used later by another? I can't figure out if they are stored insecurely in Prefect's own DB

Hi Steve! Great question and one that we should document more prominently; if you toggle

persist_result=False

within the task decorator (e..g,

@task(persist_result=False)

), Prefect will not attempt to store the return value anywhere, making it safe to return sensitive information. That being said, you aren't strictly required to encapsulate all logic within a task, so if you'd prefer you can also downgrade that call to a raw function (and maybe add a flow run level log or two to replace the task logs you would have gotten otherwise)

Thanks so much Chris! So if we used the default

persist_result=True

instead of

=False

, that would be unsafe? Also - if we don't encapsulate in a task, there's no data persisted?

Yea, if you set

persist_result=True

Prefect will store the results of the task in the configured result store (which could be local, but still risky for sensitive data). If you don't encapsulate it, then Prefect won't persist it

Okay got it! Thanks again for your help, I really appreciate your quick response

anytime 👍