I'm reading through the source code of the ECSAgent and see that in the agent's deploy_flow method, and see that if the task ARN isn't supplied as a part of the flow's runconfig, then the agent is going to call the boto

register_task_definition()

method. Does this mean that even if the contents of task definition hasn't actually changed, (e.g. the flow is never re-registered, or it's registered but with an identical task definition), the Agent is still going to register a new task to AWS on each and every flow run?

Hey @Sean Talia, I believe this is the case. The ECSAgent will register a new Task definition if the Task ARN isn’t supplied with the ECSRun. So if it is pre-registered, you can supply the name with the 

task_definition_arn

 kwarg, otherwise Prefect will define this task for you upon registration. FYI I’m trying to have this verified, so I’ll update you once I know more.

Okay that's what I thought; I don't have much experience with ECS, so I don't have any strong opinions here, but it does seem to me like this would unnecessarily run up the

revision

count for the task (unless Amazon is doing some kind of hashing on their end and sees that in fact, nothing changed – maybe that is the case?)...is there a specific reason why the implementation was done this way rather than, say, registering the task with AWS at the same time that the Flow is registered to Prefect (e.g. more or less what happens when S3 is used for storage), caching the task ARN as a part of the Flow's metadata, and then just using that task ARN at Flow run time?

AFAIK it’s to give the user as many options as possible and accommodate both use cases, however you can accomplish this with an ECSRun. The Agent assumes your Task doesn’t exist yet and will use defaults for all submitted flow runs unless configured otherwise, while the

run_config

allows for those idempotency concerns. In general, Prefect doesn’t make any assumptions about how you’ve configured your infrastructure and accommodates a wide range of setups, but also can’t reach into your AWS infra to see what fits best, so it delegates each flow run to defaults in order of precedence (the run_config will override the Agent). If you’d prefer all flow runs sent to that Agent to have that definition, you can use the

task_definition_path

on the Agent. Therefore, if you reregister your flow with an

ECSRun

and a

task_definition_arn

, then your Task will be replaced with its updated version. However, if you reregister your flow without this defined and submit it to your Agent without a definition path/arn anywhere, Prefect can’t assume it already exists and reverts to creating a new Task definition.