Not strictly. In Prefect 3.x, blocked concurrency-limit acquisitions are not guaranteed to receive slots in strict FIFO order. What is true: - On the client side, acquisition attempts are serialized through a queue for a given limit set, which behaves FIFO. - On the server side, slot acquisition is effectively determined by whichever request successfully commits first. - If slots aren’t available, Prefect returns a retry interval with jitter, so retries are intentionally staggered rather than resumed in exact arrival order. So the practical answer is: Blocked operations are coordinated to avoid a thundering herd, but Prefect does not provide a strict fairness/ordering guarantee for who gets the next slot. If you need deterministic ordering, you should not rely on concurrency limits alone for that. Relevant source-backed summary: - client-side acquisition uses a queued service - unavailable slots trigger retry with randomized backoff - server-side allocation is based on atomic DB updates rather than a waiter queue If you want, I can also dig into whether this differs between: - global concurrency limits - tag-based/task-run concurrency limits - work-pool / worker concurrency controls

No — as far as I can tell, it did not used to be strictly ordered either. I checked the historical behavior, and I couldn’t find any implementation that maintained a real waiter queue or guaranteed FIFO slot assignment for blocked operations. What seems to have been true historically: - older concurrency-limit behavior would reject/block acquisitions and tell clients to retry later - blocked clients then competed again on retry - there was no persistent “you were first in line” tracking on the server So the behavior was more like: - slots become available - waiting clients retry - whoever wins the next successful acquisition gets them What did change more recently is that Prefect added client-side serialization for some acquisition attempts to reduce thundering-herd behavior, but that is not the same thing as a distributed ordering guarantee. So the short answer is: No, I don’t see evidence that concurrency limit slots were ever guaranteed to be ordered/FIFO for blocked operations. If you want, I can also point you to the relevant source areas or summarize the difference between the older v1/v2 concurrency-limit implementations.

Best practice: don’t use Prefect concurrency limits as your ordering mechanism. In Prefect 3.x, if you need ordered concurrency, the most reliable pattern is: 1. Keep ordering in your flow logic 2. Use Prefect only to control parallelism / orchestration 3. If you need strict FIFO across distributed workers, use an external queue A few good patterns: --- 1. Ordered batches — usually the simplest choice Process items in batches sequentially, but process items within a batch concurrently.

from prefect import flow, task
from prefect.futures import wait

@task
def process_item(x):
    print(f"processing {x}")
    return x * 2

@flow
def ordered_batched(items, batch_size=5):
    results = []

    for i in range(0, len(items), batch_size):
        batch = items[i:i+batch_size]
        futures = [process_item.submit(item) for item in batch]
        wait(futures)
        results.extend([f.result() for f in futures])

    return results

Why this is good: - preserves order between batches - gives concurrency within batches - easy to reason about and retry This is usually the best answer if your input is already a list and you want “do these in order, but not one-at-a-time.” --- 2. Explicit dependency chains — for strict ordering If you need true ordered execution, chain tasks with

wait_for

.

from prefect import flow, task

@task
def process_item(x):
    print(f"processing {x}")
    return x * 2

@flow
def strictly_ordered(items):
    futures = []

    for item in items:
        future = process_item.submit(
            item,
            wait_for=[futures[-1]] if futures else None
        )
        futures.append(future)

    return [f.result() for f in futures]

This gives strict order, but effectively reduces concurrency to 1. So this is only right when you truly mean serial execution. --- 3. Ordered producer + bounded worker pool If you want “preserve submission order, but allow some parallel work,” submit in order and bound concurrency with the task runner.

from prefect import flow, task
from prefect.task_runners import ThreadPoolTaskRunner

@task
def process_item(x):
    return x * 2

@flow(task_runner=ThreadPoolTaskRunner(max_workers=3))
def ordered_submit(items):
    futures = [process_item.submit(item) for item in items]
    return [f.result() for f in futures]

Important caveat: - results can be collected in input order - execution start/finish order is not guaranteed So this is good if you only care that outputs align with input order, not that work happens FIFO. --- 4. External queue for true distributed FIFO If your requirement is: - multiple workers - strict FIFO - controlled concurrency - durable queue semantics then the best solution is usually an external queue like: - SQS FIFO - Kafka - RabbitMQ - a DB-backed work queue Pattern: - Prefect flow reads from the queue in order - workers process up to

N

messages concurrently - queue system owns the ordering guarantee This is the right answer when “ordered concurrency” is a business requirement, not just a convenience. --- What not to rely on I would not rely on: - global concurrency limits - tag-based concurrency limits - worker concurrency - task runner scheduling for FIFO semantics. Those help limit concurrency, but they do not guarantee ordered slot acquisition or ordered execution. --- My recommendation Choose based on what “ordered” means: - Need strict item-by-item order? → use

wait_for

chain - Need ordered groups with some parallelism? → use sequential batches - Need outputs returned in input order, but execution can vary? → submit normally and gather results in order - Need strict FIFO across distributed systems? → use an external queue If you want, I can give you a concrete Prefect 3 pattern for your exact case, for example: - API requests in order with max concurrency 5 - database jobs in order across workers - Kafka/SQS + Prefect consumer flow - ordered mapping over a large list

For ordered runs between different deployments on the same dataset, I’d recommend one of these two patterns: Best overall: use a single orchestrator flow that calls the deployments in order Best decoupled/event-driven: use deployment triggers / events plus a dataset lock The key point is: Prefect can orchestrate cross-deployment order at the flow/deployment level, but it is not the right tool to enforce strict task-by-task FIFO across independent deployments by itself. ### Recommended patterns #### 1) Single orchestrator flow — most reliable If these pipelines must run in a strict sequence, create one parent flow that calls each deployment and waits for completion before starting the next. Prefect provides

run_deployment(...)

for this.

from prefect import flow
from prefect.deployments import run_deployment

@flow
def dataset_pipeline(dataset_id: str):
    run_deployment(
        name="extract/extract-prod",
        parameters={"dataset_id": dataset_id},
        timeout=None,
    )

    run_deployment(
        name="transform/transform-prod",
        parameters={"dataset_id": dataset_id},
        timeout=None,
    )

    run_deployment(
        name="publish/publish-prod",
        parameters={"dataset_id": dataset_id},
        timeout=None,
    )

Why this is usually best: - strict order is explicit - easier to debug - easier to retry/restart with logic - no ambiguity about “who goes next” Verified API: -

prefect.deployments.run_deployment(name, ..., timeout=None, poll_interval=5, as_subflow=True, ...)

If your “different deployments” are logically one pipeline, this is what I’d do. --- #### 2) Event-driven chaining — best if deployments should stay independent If you want each deployment to remain separate, have deployment B trigger when deployment A completes, and deployment C trigger when B completes. This is a good fit when: - teams own separate deployments - you want loose coupling - you want runs to react automatically to upstream completion Prefect supports deployment event triggers via

prefect.events.DeploymentEventTrigger

. High-level pattern: - deployment A runs - on

prefect.flow-run.Completed

, deployment B starts - deployment C listens for B’s completion This handles flow/deployment ordering, not shared-resource locking by itself. --- #### 3) Add a dataset lock if the real requirement is “never touch the same dataset at the same time” If multiple deployments might contend for the same dataset, add a global concurrency limit or an external lock around the code section that reads/writes it. Important caveat: - a concurrency limit gives mutual exclusion - it does not guarantee ordering So if you need both: - order → orchestrator or event chain - exclusive access → lock/concurrency guard Use both together. --- ### What I would recommend for your case Based on your description:

pipelines operate on the same dataset and need to wait for each other to complete before moving on

I would choose: Option A: one orchestrator flow per dataset This is the cleanest and strongest guarantee. Structure: - parent flow receives

dataset_id

- parent calls deployment A - waits - calls deployment B - waits - calls deployment C That gives you per-dataset ordering naturally. If you may have multiple datasets concurrently, you can still run multiple parent flows in parallel — each parent serializes the steps for its own dataset. --- ### If you need ordering per dataset across independently-triggered deployments Then Prefect alone is usually not enough at task level. You likely want: - deployment/event chaining for sequencing - plus an external dataset-state mechanism, such as: - database row lock - Postgres advisory lock - Redis lock - queue table keyed by

dataset_id

That lets you express: - “dataset X is in stage transform” - “publish cannot start until transform marks complete” - “only one actor may advance dataset X at a time”

This is much safer than trying to infer order from concurrency slots. --- ### Practical decision guide Use orchestrator flow if: - these deployments are one logical pipeline - you want strongest ordering guarantee - you control all deployments Use event triggers if: - deployments should stay independent - completion of one should launch another - a small amount of orchestration latency is fine Use external locking/state if: - correctness depends on dataset-level exclusivity - deployments may start from many places - you need guarantees stronger than “flow A finished before flow B started” --- ### What I would not recommend I would not use only: - global concurrency limits - tag-based concurrency limits - worker limits to solve this. They help prevent overlap, but they do not give dependable ordered execution between deployments. --- ### Useful API references -

run_deployment

in

prefect.deployments

-

DeploymentEventTrigger

in

prefect.events

Docs: - Deployments - Events and automations - Global concurrency limits If you want, I can sketch the exact architecture for your case, for example: - 3 deployments chained per dataset_id - event-driven with exclusive dataset locking - orchestrator flow that fans out across datasets but serializes stages within each one