Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi! I would like to profile the memory consumption of my prefect flow, and I found <https://github.com/zzstoatzz/prefect-memory-profiling|this one> but it's for prefect 1, and we're using the latest version of prefect. Can anyone recommend a good tool to profile memory (RAM) consumption over time (I'd like to plot it, or at least report peak sizes), either specifically for prefect, or some other general tool? So far I've tried <https://github.com/pythonprofilers/memory_profiler|mprof> but it seems to be way off in reporting true memory consumption, even if I do use `--include-children` . My prefect flow calls a lot of Docker containers btw, which might be the reason that the reporting is way off.