@Marvin Any reason prefect would wait infinitely once a task that uses parallel processing has finished? I found that it hangs on this File "/usr/local/lib/python3.12/site-packages/prefect/engine.py", line 869, in orchestrate_flow_run waited_for_task_runs = await wait_for_task_runs_and_report_crashes(

It seems that you are encountering a deadlock issue similar to what has been reported on GitHub when using native Python

multiprocessing

with Prefect tasks. To help you, I suggest checking out the related issue where someone experienced a deadlock at the

.join()

step of multiprocessing, specifically within a docker container environment. Here is the relevant issue on GitHub. This appears to be an open problem as of the time of this response, and it may require further investigation by the Prefect team. In the meantime, you can follow or contribute to that issue for any updates or possible workarounds.