@Marvin @Marvin Can I run thousands of tasks concurrently in a single flow, when using prefect cloud?

Yes, it is possible to run thousands of tasks concurrently in a single flow when using Prefect Cloud, but there are several considerations and limitations: 1. Task Concurrency Limits: - Prefect has built-in functionality for setting task concurrency limits using task tags. You can specify an optional concurrency limit as the maximum number of concurrent task runs for tasks with a given tag. Prefect Docs - Task Concurrency Limits 2. API Rate Limits: - Prefect Cloud imposes API rate limits, such as limits on the rate of flow, task creation, and log services. For example: - Flow and task creation rate limits: 400 per minute for personal accounts, 2,000 per minute for Pro accounts. - Log service rate limits: 700 logs per minute for personal accounts, 10,000 logs per minute for Pro accounts. - These limits ensure the stability of the Prefect Cloud service. Prefect Docs - API Rate Limits 3. Work Pool and Queue Concurrency Limits: - Flow run concurrency limits can be set at a work pool and/or work queue level. If you still have questions or need further assistance, you might want to contact Prefect support directly at help@prefect.io. Would you like more detailed information about any of the above points, or anything else?