https://prefect.io logo
Join Slack
Powered by
Hi team, I am trying to find which flow is eating ...
# ask-community
k
Hi team, I am trying to find which flow is eating up all my agent's resources so have a couple of questions: • Is each run forked, ie will resources be reclaimed when the run finishes? • Is there a way to get memory usage per flow run?
z
Hi Kostas, how are you running your flows (locally, Kubernetes, etc)?
k
Hello! We use the cloud scheduler and an agent running on Heroku. I guess you could call it a local agent
So I logged into the host where the agent is running and I see 21 instances of /app/.heroku/python/bin/python /app/.heroku/python/bin/prefect execute cloud-flow and 42 instances of /app/.heroku/python/bin/python /app/.heroku/python/bin/prefect heartbeat task-run -i <some id> May I suspect that cycles are running on top of each other?
z
Hmmm. So the item you're seeing 21 instances of is the actual running of the flow, while the 42 instances are us updating our heartbeats.
I'm not super familiar with Heroku, but you may be able to use 
top
to get a better idea of which processes are using memory.
k
So can I assume that something is keeping each running of the flows going for longer than it should? E.g. something blocking them from exiting
Is it one of those processes per flow?
(I don't think the fact it's heroku is important anyway)
z
Honestly, I'd expect the process for the flow to complete once the flow reached a terminal state. I imagine you don't actually have 21 flow runs currently running?
k
not intentionally 🙂
in fact 21 flows is all we have lol
so I don't think they are ever terminating
and I think I see why...
So let me try and remove a bad line of code and see if that fixes, thanks!
Good datapoint that its one child process per flow, that is useful to know anyway
z
Sure thing, let me know if I can be of any more help!
Open in Slack
PreviousNext
Powered by