Hello! I'm working with prefect to orchestrate some dbt tasks, but I'm struggling with ECS deployment. I have one flow that I want to run for multiple customers, so I need data access to be single-tenant, but it doesn't seem efficient to run an agent for each customer. I was hoping to: • Run one agent as a long-running ECS task that listens to one queue • When a flow run is requested, that agent spins up a separate ECS task (with client-specific permissions) to run the entire flow • The ECS task runs a prefect flow that includes dbt shell command prefect tasks from prefect-dbt Could somebody help me think through how this can be done? Most of the ECS docs describe v1 not v2, and I only found this discussion in Discourse which said ECS wasn't really supported.

Yes, I think that is essentially what you want to do here. You'd probably want to use a separate deployment for each customer so it can set the customer-specific permissions for each flow run.

That makes sense! Very helpful, thank you! For the ECS task Docker image, I have prefect installed and I have a

flow.py

file on it, but do I just give it the command

python flow.py

in the Dockerfile or should the agent supply a command? I'm not clear on how the agent and the task interact, do you have docs you recommend for that?