Hi all ! We're currently fighting with mapping features, which overflow the backend when mapping over many inputs (which is a requirement for our flows - typically over 50) and cause the flow to crash. When running locally, the flow crashes with a message that the database is locked (on the example below) - when running on the server, we do get an error 500 back after many creations in a row (

Crash detected! Execution was cancelled by the runtime environment.

and then

Crash detected! Execution was interrupted by an unexpected exception: httpx.HTTPStatusError: Server error '500 Internal Server Error' for url 'http://[our instance url]/api/task_runs/'

For more information check: <https://httpstatuses.com/500>

) A minimum running example is (in Python 3.10 with prefect 2.7.9) :

import time
from prefect import unmapped, flow, task

@task(name="Sleeper {id}")
def sleeper(id, seconds_to_wait: int):
    print(f"Start Sleeper {id}")
    time.sleep(seconds_to_wait)
    print(f"End Sleeper {id}")

@flow(
    name="Prefect 2 concurrency checker",
)
def flow(n_runs: int = 128, seconds_to_wait: int = 2):
    res = sleeper.map(list(range(n_runs)), seconds_to_wait=unmapped(seconds_to_wait))
    return res


if __name__ == "__main__":
    res = flow()
    print("done")

In Prefect 1, we had a way to limit the number of task run creations, but that's apparently not available in Prefect 2 AFAIK (note: we've tried tag-limiting, but that's not helping as it's the creation of the many tasks that kills it, not their actual run ; we've also tried an async approach, which fails in the same way). My questions to you: 1- Have you ever experienced this too? 2- Did you find a walk-around? 3- If not, do you have an idea of how to manage this?