Does anyone know if the ECS serverless push work pool can be configured for self-managed Prefect? We've already got prefect as an ECS service, and a worker service with several replicas currently handles flow executions, but it would be nifty to be able to get prefect to spin up ECS tasks in its cluster already. We do this manually right now with the ECSTask from
prefect-aws
but it'd be far nicer to have an ECS task per flow instead of an ECS task per prefect task just due to the provisioning time
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.