Surfacing this here for visibility. Does prefect have a memory leak? I've been having some serious issues with long running scripts in production. Minimum viable example to reproduce below 👀 https://prefect-community.slack.com/archives/CL09KU1K7/p1710454264307259?thread_ts=1709100985.856459&cid=CL09KU1K7

Copying my reply from the other thread here. @Matthew Bell What version are you on? 2.16.3 fixed two memory leaks, 2.16.0 fixed others. (I'm trying to root them all out.)

@Andrew Brookins See the message I just posted in #CL09KU1K7. I am seeing this same problem but with the webrunners. I'm using 2.16.5 which definitely improved the issue (I get through more files processed before the server worker crashes), but still the same issue. Like @Matthew Bell above, I have some long running processes I need to sleep that can't work async (in particular calling some very old and poorly designed nasa apis.). (by the way, thanks for working on the memory leak issues. I am quite sure it's for longer running tasks - some of mine have to wait 75seconds or more for responses and sadly do not give callbacks so sleeps and retries.).

Hi Daryl! I’ve been on vacation and still catching up. I’ll check your post soon.

Hey Andrew any chance you've had a moment to look into this? Also happy to jump in and help or validate the leak further however I can!

Hey Matthew! I’m still closing out other work before I can get back to looking at memory. The specific leak reported in the issue you commented on is fixed, so I expect there is another leak (I’ve run into a few by now). I had trouble reproducing the leak with your example, but that may have been my setup. Can you open an issue with the code you shared here and steps to reproduce, including terminal output e.g. from your usage of the profiler?

Thank you! I'll assign myself.

Thank you. I'm here if you need any additional info