Hello friends Is it possible to pause a prefect workflow hav

Question

Hello friends Is it possible to pause a prefect workflow have it not consume resources then turn it back on later

jawnsy · Answer

You can pause and resume flow runs yes Have you seen these docs <https docs prefect io 2 11 4 concepts flows pause a flow run>

Patrick Barker · Answer

Hey < jawnsy> I saw this I m curious how many resources are being consumed when its waiting do you know by chance

Patrick Barker · Answer

I ideally want to fully shut it off if thats possible

jawnsy · Answer

That depends on how you re running things If you re running a Kubernetes worker it will run your workloads as Kubernetes jobs and if the flow is paused then it will not create those jobs allowing your cluster to scale down It requires some setup to get that autoscaling to work correctly though particularly you will need to set resource requests limits and use the cluster autoscaler If you re running with Cloud Run or ECS it will run workloads as Cloud Run Jobs or ECS Tasks and if the flow is paused then it won t create those So you should only need to pay for running the worker which should be relatively inexpensive since it s pretty lightweight If you use push work pools then you don t need to run an agent at all so that allows full scale to zero while stuff is paused

Patrick Barker · Answer

awesome thanks just to double check if I use a push work pool I wouldn t pay anything while paused Are there any other limitations of that I may need to scale this to thousands of workflows does that seem feasible

Patrick Barker · Answer

I may have a lot of these that potentially never finish

Patrick Barker · Answer

also in the non workpool configuration is it a worker per flow or are workers shared among many flows

Patrick Barker · Answer

and also what happens in the case of pausing long running flows with a push work pool if I pause out of process

Will Raphaelson · Answer

Hey gang just want to hop in to clear up some concepts what i understand the request to be is to pause the execution of a flow without consuming compute resources while the above around work pools is correct its not directly related to flow pause behavior You can get exactly what you want here by pausing flow runs in code from the API with reschedule=True check out the docs here in particular the callout box Pausing flow runs is blocking by default <https docs prefect io 2 11 4 concepts flows pause a flow run> Patrick feel free to reach out via DM if you want to hop on a call and discuss the use case

Patrick Barker · Answer

Thanks < Will Raphaelson> I have a couple of followups I ll DM you with