I am attempting to use a flow to launch a number of parallel flows (up to 45) using RunDeployment against ECS Fargate (recursive flattening of json from a NOSQL db and expansion into child tables) . I note two big issues and wonder if anyone else has seen this.
1) ECS execution time is very slow, like clock speed is turned way down or something like that
2) Startup of new ECS containers takes ~2 minutes but you need to wait 30 seconds or so between container startup or you get throttled.
We've tested giving additional CPU resources/RAM to the containers with no measurable impact. Each of the flows runs for between 5 minutes and 30 minutes depending on data change rates.
I expect that there's something fundamentally wrong about the way we're deploying to Fargate using Prefect given this but can't see what it might be. Suggestions?