@Marvin in which cases would one prefer to use async tasks versus using the ConcurrentTaskRunner to execute tasks?

Oh, the joy of explaining this. You see, the choice between using async tasks and the ConcurrentTaskRunner in Prefect depends on your needs for concurrency and parallelism. Concurrency refers to a system that can handle more than one task at the same time, but not necessarily at the exact same moment. It's more about non-blocking execution: within the constraints of available resources and data dependencies, one task doesn't block the execution of others in a flow. On the other hand, parallelism is about doing more than one thing at the exact same time. The ConcurrentTaskRunner allows tasks to run concurrently, switching when blocking on IO. It submits tasks to a thread pool maintained by anyio, allowing for non-blocking execution. In contrast, async tasks in Prefect would be used when you want to take advantage of a parallel or distributed execution library like Dask or Ray. These libraries allow for true parallel execution. So, if you need to handle IO-bound tasks efficiently, ConcurrentTaskRunner might be your choice. But if you need to perform heavy computations across multiple cores or machines, you might want to use async tasks with a parallel execution library. I hope that clears up your confusion, or at least adds a new layer of confusion that's a bit more enlightening.

If our tasks are not computation heavy, and we don't need to use a parallel execution library, then is there a difference?

hi @Sophia Ponte - if you’re not compute bound and dont otherwise need true parallelism, the concurrent task runner (the default) is a solid choice. just remember to call .submit() on your tasks (which returns a future you would call .result() on) in order to leverage the task runner