Short answer: even with match_latest_revision_in_family on, you can still see a burst of RegisterTaskDefinition calls when many runs start at once, or when your “family” or task definition config varies between runs. A few common culprits and fixes are below. Why this happens - First-run stampede: match_latest_revision_in_family only reuses an existing ACTIVE revision in the family. If 500 runs start at the same time and the family doesn’t exist yet, each worker will race to register a new definition. The first few seconds can generate hundreds of RegisterTaskDefinition calls before a single “latest” exists to match. - Family mismatch: if you don’t set a fixed

family

, Prefect generates one like

prefect_{work_pool_name}_{deployment_id}

(or falls back to

flow_id

). If your deployment id changes (e.g., you redeployed) or you trigger runs across different deployments/pools, you’ll get different families, hence new registrations. - Config mismatch: Prefect compares the desired task definition (image, cpu/memory, roles, containerDefinitions, logs config, etc.) to the latest in the family. If anything differs, it will register a new revision. Per-run values baked into the task definition (not passed as runtime overrides) will force a new registration. What match_latest_revision_in_family actually does - When enabled, the ECS worker will describe the family, fetch the latest ACTIVE revision, and reuse it if it’s equivalent to the desired definition. This works across different worker replicas (no shared in‑memory cache required). - It does not serialize registrations; if the family does not exist yet, concurrent runs will all attempt to register. How to stop the flood and avoid AWS rate limits 1) Set a fixed family and enable matching (most important) - Put

match_latest_revision_in_family: true

under job_configuration (not variables). - Set a stable

family

so every run uses the same family. Example base job template snippet (work pool)

{
  "job_configuration": {
    "match_latest_revision_in_family": true
  },
  "variables": {
    "family": "my-stable-family"
  }
}

You can update your pool with:

prefect work-pool get-default-base-job-template --type ecs -f ecs.json
# edit ecs.json to add the fields above
prefect work-pool update "your-ecs-pool" --base-job-template ecs.json

2) Avoid per-run changes in the task definition - Keep these stable to prevent new registrations: - image tag, cpu, memory - taskRoleArn, executionRoleArn - containerDefinitions (including logs config) - family name - If you need per-run values (env, command, labels), pass them as runtime overrides (not baked into containerDefinitions) so they don’t change the registered task definition. 3) Pre-warm the family or pin to an ARN - Pre-warm: trigger a single run first (or a tiny batch) to create the initial revision, then fan out. - Pin: if you absolutely want zero registrations, set a

task_definition_arn

in your job variables to reuse an existing task definition. 4) Throttle initial concurrency (as a safety valve) - Set a pool concurrency limit to avoid stampedes while you validate config:

prefect work-pool set-concurrency-limit "your-ecs-pool" 20

A quick sanity checklist - Where did you set

match_latest_revision_in_family

? It must be in

job_configuration

(pool base job template or deployment’s job configuration). If it’s under variables, it’s ignored. - Do all runs share the same

family

? If not, set it explicitly. - Are you changing image/cpu/memory/logging/roles between runs? That will force new registrations. - Are you launching 500+ runs when the family has no revision yet? Pre‑warm one run or throttle initial concurrency. If you can share: - your work pool’s base job template JSON - the deployment’s job variables/config I can spot exactly which field is causing the re-registrations and suggest the smallest change to fix it. Helpful commands ```

# Inspect the current work pool config (to confirm your job template) prefect work-pool inspect "your-ecs-pool" # Set a concurrency limit as a backstop prefect work-pool set-concurrency-limit "your-ecs-pool" 20 ``` Docs - ECS worker (prefect-aws) docs: ECS worker - Work pools: Work pools overview