What is the standard way to set up prefect code to handle promotion through various environments? (dev -> stage -> prod). I’m using Prefect Cloud and would like to use the Secrets manager.

In Prefect 1.0, you would usually do this through labels. You can have an agent with the label

"dev"

. Then, you can change the label on the run configuration to "stage" to move it to this another environment (agent). And the easiest way to do it is to reregister your flow with an updated run configuration. Does this answer your question? How would you want to use Secrets manager?

Clarifying… It seems like you’re saying the preferred Prefect setup would be to: • run all three environments on a single AWS account, • but use different agents for each environment • re-register the flow when moving from one environment to another Is this close? I need to manage different AWS access keys to run prefect in different AWS accounts for each environment. I’d like to setup the flows to grab the correct keys depending on which environment the flow is running in.

Correct, a separate agent for each environment, and you will be able to match flows with agents via labels. To then promote a flow from dev to prod, you could reregister this flow with label dev, or you could have a separate flow (e.g. in project "dev") say FLOW_NAME_dev and register it with this separate agent label. Re AWS: you can have different credentials per AWS profile and switch between those

Interesting. We were planning to keep AWS access keys in Prefect Cloud Secrets and dynamically switch secret names to get the appropriate keys. We can use the same mechanic to implement something like your

set_run_config()

function. We use terraform to setup the Prefect agent on ECS, so I can set an environment variable on the agent, but I’m not sure how to get that to the container running the flow. Any thoughts on how I could do this?

If you are using ECS agent, you shouldn't need to use Prefect Secrets or manually set AWS credentials, instead you need to use IAM roles, let me send you some resources and examples

I really appreciate your help.