Hi!
I've had some problems with flows getting stuck after all tasks are finished and did not receive support so I've tried to recreate a minimal reproduceable example see code below.
What is happening for me:
The tasks can run fine, but after the tasks completes the memory usage explodes (goes to 100%). Depending on the size of the data the flow will either:
• get stuck after last finished task
• finish succesfully
• either of above intermittently
I can run the same flow perfectly fine in Prefect 1, it also runs a lot faster even when increasing data a lot. I attach a log of a unsuccesful prefect 2 run in the thread: