<@ULVA73B9P> I would like to manage the concurrency of my workspace through config as code, what are...
j
@Marvin I would like to manage the concurrency of my workspace through config as code, what are the options I have in this area?
m
thought for 138.2 seconds
Yes — you have a few config-as-code options for concurrency in Prefect 3.x, but they apply at different layers. Main options 1. Global concurrency limits 2. Tag-based task concurrency limits 3. Deployment concurrency limits 4. Work pool / work queue concurrency limits 5. Terraform / API / Python client automation to manage the above Best option if you want “workspace concurrency as code” broadly: use global concurrency limits plus either Terraform or a small bootstrap script. --- ## 1) Global concurrency limits These are the most flexible concurrency controls in Prefect. They let you define named limits in the workspace, then reference them in flow/task code with
concurrency()
or
rate_limit()
. Example use cases: - limit DB connections - cap access to an external API - serialize use of a shared resource - implement rate limiting Config-as-code options - Terraform:
prefect_global_concurrency_limit
- Python client: create/update limits programmatically - REST API - CLI for scripting Example Terraform:
Copy code
hcl
resource "prefect_global_concurrency_limit" "database" {
  name                  = "database-connections"
  limit                 = 20
  slot_decay_per_second = 0
  active                = true
}
Example runtime usage:
Copy code
python
from prefect import task
from prefect.concurrency.sync import concurrency

@task
def query_db():
    with concurrency("database-connections", strict=True):
        ...
Docs: - Global concurrency limits - How to apply global concurrency limits - Terraform resource --- ## 2) Tag-based task concurrency limits These apply automatically to tasks that carry a given tag. Example:
Copy code
python
from prefect import task

@task(tags=["database"])
def query_db():
    ...
Then define a limit for the
database
tag in the workspace. This is nice when you want concurrency tied directly to task classes/patterns, without wrapping code in a concurrency context manager. Config-as-code options - Terraform:
prefect_task_run_concurrency_limit
- Python client - CLI - REST API Example Terraform:
Copy code
hcl
resource "prefect_task_run_concurrency_limit" "database" {
  tag               = "database"
  concurrency_limit = 10
}
Docs: - Tag-based concurrency limits - How to limit concurrent task runs with tags - Terraform resource Note: in newer Prefect versions, tag-based limits are implemented on top of global concurrency infrastructure, but they’re still a separate user-facing workflow. --- ## 3) Deployment concurrency limits These limit how many flow runs of a specific deployment may run at once. Good for: - “only 1 prod ETL at a time” - “max 5 concurrent runs of this deployment” This is typically defined with the deployment itself, so it fits config-as-code well if you define deployments in Python,
prefect.yaml
, or Terraform. Example:
Copy code
python
from prefect import flow

@flow
def my_flow():
    pass

my_flow.deploy(
    name="prod",
    work_pool_name="my-pool",
    concurrency_limit=5,
    concurrency_options={
        "collision_strategy": "ENQUEUE"
    },
)
Docs: - Deployments - Terraform deployment resource --- ## 4) Work pool concurrency limits These cap total concurrency across a work pool. Good for: - limiting infra pressure
- constraining total runs routed to a specific execution environment Terraform supports this on the work pool resource. Example:
Copy code
hcl
resource "prefect_work_pool" "pool" {
  name              = "k8s-pool"
  type              = "kubernetes"
  concurrency_limit = 20
}
Docs: - Terraform work pool resource There are also work queue concurrency controls, but they’re less commonly managed as first-class IaC compared with pools/deployments. --- ## 5) Automation mechanisms for “config as code” If your goal is workspace state is declared in source control, these are the practical ways to do it: ### Option A: Terraform Best if you want true declarative workspace management. Use it for: - global concurrency limits - tag-based task concurrency limits - work pools - deployments Docs: - Prefect Terraform provider ### Option B: Python bootstrap script Best if you want versioned setup but don’t want Terraform. Pattern: - keep a Python file in repo - call Prefect client in CI/CD - upsert limits on deploy Example shape:
Copy code
python
import asyncio
from prefect.client.orchestration import get_client

LIMITS = [
    {"name": "database-connections", "limit": 20, "slot_decay_per_second": 0},
    {"name": "external-api", "limit": 5, "slot_decay_per_second": 1.0},
]

async def main():
    async with get_client() as client:
        for item in LIMITS:
            await client.upsert_global_concurrency_limit_by_name(
                name=item["name"],
                limit=item["limit"],
                slot_decay_per_second=item["slot_decay_per_second"],
            )

asyncio.run(main())
### Option C: REST API / CLI scripting Useful for lightweight provisioning in CI, but less declarative than Terraform. --- ## What you can’t fully express as workspace-only config This is the main limitation to be aware of: Concurrency definitions can live in workspace config, but where they are applied is often partly in code. Examples: - A global concurrency limit exists in the workspace, but code must call
concurrency("name")
or
rate_limit("name")
- Tag limits exist in the workspace, but tasks must be tagged in code - Deployment concurrency is attached to the deployment definition, not some global workspace policy engine - There isn’t a single “workspace concurrency policy file” covering all Prefect concurrency behavior So in practice, concurrency-as-code in Prefect is usually a combination of: - workspace resources declared via Terraform/API - flow/task code that opts into those controls - deployment definitions that set deployment-level concurrency --- ## Recommended patterns ### If you want central governance Use Terraform for: - global limits - tag limits - pool limits - deployment limits ### If you want application-level resource protection Use global concurrency limits referenced in code. ### If you want task-class throttling Use tags + tag concurrency limits. ### If you want “only N runs of this deployment” Use deployment concurrency limits. ### If you want infra-wide throttling Use work pool concurrency limits. --- ## Practical summary If you want to manage workspace concurrency through config as code, your main choices are: - Terraform for declarative management of limits/resources - Python client / REST API for scripted provisioning - Deployment definitions for deployment-specific concurrency - Flow/task code for applying global limits or tags If you want, I can give you one of these next: 1. a Terraform example that manages all concurrency layers together 2. a Python bootstrap script for workspace concurrency setup 3. a decision matrix for when to use global vs tag vs deployment vs pool limits
j
@Marvin is there such a way that we only have concurrency limits for the registration of a task, and therefore only concurrency restrictions for task registration in ECS
m
thought for 169.3 seconds
Short answer: not exactly. In Prefect 3.x, concurrency limits are generally enforced around execution/state transitions, not at the moment a task or flow is merely registered/submitted. So if by “registration” you mean: - creating a flow run / task run record, or - submitting work into Prefect, then no, Prefect does not provide a concurrency limit that blocks only that registration step. ## What Prefect can do instead ### 1) Deployment concurrency limits for flow runs This is the closest fit if your goal is:
“Don’t let too many ECS-backed runs start provisioning/registering at once.”
Deployment concurrency is enforced before workers proceed with provisioning infrastructure for a run, so it can prevent too many ECS launches from happening concurrently. That makes it a good choice for controlling ECS task startup pressure. Docs: - Deployments ### 2) Work pool concurrency limits If multiple deployments share the same ECS execution environment, a work pool concurrency limit can cap total concurrent work across them. Terraform docs: - Work pool resource ### 3) Global concurrency limits If you have a specific code section that performs ECS API calls and you want to throttle that exact section, you can wrap it in a global concurrency limit. Example:
Copy code
python
from prefect.concurrency.sync import concurrency

def register_or_launch_ecs():
    with concurrency("ecs-registration", strict=True):
        # call ECS APIs here
        ...
Docs: - Global concurrency limits --- ## Important distinction If you want: ### “Limit task *submission*” Prefect does not really have a native “submission concurrency” concept. ### “Limit task *start/provisioning/execution*” Prefect does support this well. That means in practice you usually control: - how many runs can start - how many tasks can run - how many workers/pools can pick up work rather than how many task/run records can be created. --- ## For ECS specifically If your concern is: - too many ECS
run_task
calls at once - too many task definitions/registrations happening in parallel - ECS/API throttling during startup then the best patterns are: ### Best: deployment concurrency Use this if each flow run corresponds to ECS-backed execution and you want to limit concurrent launches. ### Also good: global concurrency around ECS API calls Use this if your own code is making ECS registration/start calls and you want only that critical section throttled. ### Broader cap: work pool concurrency Use this if you want a shared upper bound across many ECS-routed deployments. --- ## So the direct answer is No, Prefect does not currently offer a concurrency limit that applies only to “task registration” in the abstract. But if your real goal is “only allow N ECS task registrations/launches at a time”, then the closest workable options are: 1. deployment concurrency limits 2. work pool concurrency limits 3. global concurrency limits around the ECS registration/launch code path If you want, I can help you map your exact ECS architecture to the right one: - Prefect flow -> ECS task per flow run - Prefect task -> code that calls ECS APIs - custom worker / ECS work pool pattern