@Marvin i keep getting this error after a bunch of tasks are submitted to run together in version 3.x, after that all other tasks crashes, is there a way to prevent that? if i use .result() or .wait() on tasks my flow runs slower than on 2.x:

A future was garbage collected before it resolved. Please call `.wait()` or `.result()` on futures to ensure they resolve.
See <https://docs.prefect.io/latest/develop/task-runners> for more details.

It seems you're encountering an issue with task futures being garbage collected before they resolve in Prefect 3.x. This is a common source of confusion, especially when transitioning from Prefect 2.x, due to changes in how futures are handled. Here are some key points and potential solutions: 1. Resolving Futures: In Prefect 3.x, futures must be resolved by either passing them to another task, returning them, or explicitly calling

.result()

or

.wait()

. This ensures that the tasks complete and the results are available before the future is garbage collected. Here's a relevant link. 2. Concurrency Limits: You can manage the concurrency of task runs using global concurrency limits or task run concurrency limits with tags. This can help prevent too many tasks from running simultaneously and overwhelming resources. More details can be found here and here. 3. Task Runners: Ensure you are using the appropriate task runner for your setup. Prefect provides a

ThreadPoolTaskRunner

which allows you to set

max_workers

to control the number of concurrent tasks. This can be adjusted to better match your workload. Here's more information on task runners. 4. Common Gotchas: Remember that

.map

and

.submit

are synchronous in Prefect 3.x, and futures returned by

.map

can be resolved together, such as using

integers = double.map(range(10)).result()

. If you need to maintain performance while ensuring futures are resolved, consider adjusting the concurrency settings and task runners to better suit your workload. If the issue persists, you might want to explore more advanced concurrency management techniques or consult the Prefect community for further guidance.