Hi @Marvin can i run two flow run of the same deployment in parallel ?

Yes, you can run multiple flow runs of the same deployment in parallel with Prefect. Prefect's architecture allows for concurrent or parallel execution of flow runs, which can be performed using different infrastructures based on the worker setup. 1. Documentation Excerpt: Several guides discuss deploying multiple deployments of Prefect flows, parallelizing flow runs, and rescheduling late flow runs to handle parallel execution effectively. 2. GitHub Issues: There have been suggestions and implementations (e.g., a

Flow.submit

interface) to handle running flows in the background and submitting them to different infrastructures, further confirming the support for parallel execution. For detailed instructions, you can refer to the following resources: - Deploying Flows: Deploy Multiple Deployments - Flow Runs and Scheduling: Rescheduling Flow Runs - Flow and Task Execution: GitHub issues, e.g., discussion on

Flow.submit

feature. Let me know if you need assistance with the implementation or further details on deployment commands.

@Marvin Is there a way to make sure the same flow run of the deployment always gets queued instead of being run parallely ?

To ensure that the same flow run of the deployment always gets queued instead of being run in parallel, you can use concurrency limiting for deployments. This can prevent multiple instances of a deployment from running simultaneously, queuing the flow runs instead. Two approaches you can use are: 1. Concurrency Limit in Deployment: Prefect allows you to set concurrency limits for deployments, ensuring only a certain number of flow runs can execute concurrently. If the limit is reached, additional flow runs are queued. 2. Global Concurrency Limits: Another approach is to use Prefect's global concurrency limits, which can be applied to flow runs to control the maximum number of runs that can execute concurrently across the entire Prefect environment. Here's an example of how you might implement the global concurrency limit:

python
from prefect import flow, concurrency

GLOBAL_CONCURRENCY_LIMIT_NAME = "single-flow"

@flow
async def my_flow():
    async with concurrency(GLOBAL_CONCURRENCY_LIMIT_NAME, occupy=1):
        # Your flow logic here
        pass

You can read more details and learn about these methods from Prefect's documentation and related discussions: - Concurrency Limits: Concurrency Limits Guide - Deployment Configuration: Deployment Concepts Additionally, there was a notable GitHub issue discussing concurrency limits for deployments: - Add Concurrency limiting for Deployments: Issue #12456 These methods allow you to control and limit parallel execution of your deployments, ensuring subsequent runs are queued appropriately.