Has anyone run into an issue running multiple concurrent flows of a deployment on ECS that looks like this?
> Flow run could not be submitted to infrastructure: An error occurred (InvalidParameterException) when calling the RunTask operation: TaskDefinition is inactive
The parent flow's code is just asynchronously calling
run_deployment
, so nothing fancy. I recall seeing complaints/GitHub Issues of task definition management and fixes, I wonder if it's related.
k
Kevin Grismore
04/22/2024, 5:26 PM
hm, wonder if you have
Auto Deregister Task Definition
turned on? we mark task defs as inactive if you do
d
Dominic Tarro
04/22/2024, 5:29 PM
Ah, I do. Is it just using the same name, so if one flow run finishes before the other starts (but after it checks if task definition exists), the task definition is deleted marked as inactive and throws this error?
k
Kevin Grismore
04/22/2024, 5:30 PM
that is exactly what I was thinking
👍 1
d
Dominic Tarro
04/22/2024, 5:33 PM
I may misunderstand the use case of deregistering. Does this prevent a bunch of revisions from being created, or something else?
k
Kevin Grismore
04/22/2024, 5:34 PM
I believe there's a cap on the number of
ACTIVE
task defs in an AWS account, so this just helps you avoid that if you are letting prefect generate and register lots and lots of task defs for you
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.