@Marvin Hey, how much are we limited via the system's hardware while running tasks concurrently?

Ah, the intricacies of hardware and software dancing together. It seems that the number of tasks you can run concurrently in Prefect is not strictly limited by the system's hardware, but rather by the task concurrency limits you set within Prefect itself. Prefect has a built-in functionality for setting task concurrency limits. These limits use task tags and you can specify an optional concurrency limit as the maximum number of concurrent task runs in a Running state for tasks with a given tag. If a task has multiple tags, it will run only if all tags have available concurrency. Tags without explicit limits are considered to have unlimited concurrency. If the concurrency limit is set to 0 for a tag, any attempt to run a task with that tag will be aborted instead of delayed[^1^]. For example, if you have tasks that interact with a database that only allows 10 connections, you can set a concurrency limit to ensure that no more than 10 tasks that connect to this database are running at any given time[^1^]. Also, Prefect's ConcurrentTaskRunner can run tasks concurrently, allowing tasks to switch when blocking on IO[^4^]. Here are the sources for your perusal: - Prefect Documentation on Tasks - Prefect Documentation on Task Runners - Prefect Release Notes on GitHub Remember, with great power (or in this case, concurrency) comes great responsibility. Don't go overboard and try to run all your tasks at once, or you might find your system as overwhelmed as a human trying to juggle while riding a unicycle. [^1^]: Prefect Documentation on Tasks [^4^]: Prefect Documentation on Task Runners