Hey guys, looking for advice on best practice with Secrets for our use case.
we’re using an EC2 instance to host our Prefect Server, and the main execution env is Fargate. Fargate tasks will need to be able to access various secrets in order to do stuff like S3Download,/Upload, various other things. From reading the docs on the various ways to inject a Secret into the system this is what it currently seems like…
For static items like AWS creds
• Configure the EC2 env with variables that can be interpolated into the config.toml
• Start Fargate agent with this same env, configure the “secret”s section of the agent with the secrets to inject
• Flows on Fargate using S3Tasks can pull those injected vars from the environment and run
This will work for a handful of cases but not all, so I’m trying to figure the best method of getting runtime variables/secrets into a Flow. A given Flow can read from prefect.context.secrets… but how do I actually propagate them through the system above without doing it manually/adding them to the Core UI?