Hello friends,
Is it possible to pause a prefect ...
# ask-communityp
Patrick Barker
08/21/2023, 4:25 PMHello friends,
Is it possible to pause a prefect workflow, have it not consume resources, then turn it back on later?
j
jawnsy
08/21/2023, 4:57 PMYou can pause and resume flow runs, yes. Have you seen these docs? https://docs.prefect.io/2.11.4/concepts/flows/#pause-a-flow-run
p
Patrick Barker
08/21/2023, 5:01 PMHey @jawnsy I saw this, I'm curious how many resources are being consumed when its waiting, do you know by chance?
Patrick Barker
08/21/2023, 5:02 PMI ideally want to fully shut it off if thats possible
j
jawnsy
08/21/2023, 7:06 PMThat depends on how you’re running things. If you’re running a Kubernetes worker, it will run your workloads as Kubernetes jobs, and if the flow is paused, then it will not create those jobs, allowing your cluster to scale down. It requires some setup to get that autoscaling to work correctly though, particularly you will need to set resource requests/limits and use the cluster autoscaler.
If you’re running with Cloud Run or ECS, it will run workloads as Cloud Run Jobs or ECS Tasks, and if the flow is paused, then it won’t create those. So you should only need to pay for running the worker, which should be relatively inexpensive, since it’s pretty lightweight
If you use push work pools, then you don’t need to run an agent at all, so that allows full scale-to-zero while stuff is paused
p
Patrick Barker
08/21/2023, 7:35 PMawesome thanks, just to double check, if I use a push work pool, I wouldn't pay anything while paused? Are there any other limitations of that? I may need to scale this to thousands of workflows, does that seem feasible?
Patrick Barker
08/21/2023, 7:35 PMI may have a lot of these that potentially never finish
Patrick Barker
08/21/2023, 7:41 PMalso in the non-workpool configuration, is it a worker per flow? or are workers shared among many flows?
Patrick Barker
08/21/2023, 7:51 PMand also what happens in the case of pausing long running flows with a push work pool, if I pause out of process
w
Will Raphaelson
08/22/2023, 2:23 PMHey gang, just want to hop in to clear up some concepts. what i understand the request to be is to pause the execution of a flow without consuming compute resources. while the above around work pools is correct, its not directly related to flow pause behavior.
You can get exactly what you want here by pausing flow runs in code/from the API with reschedule=True. check out the docs here, in particular the callout box “*Pausing flow runs is blocking by default”.*
https://docs.prefect.io/2.11.4/concepts/flows/#pause-a-flow-run
Patrick, feel free to reach out via DM if you want to hop on a call and discuss the use case.
🙏 2
p
Patrick Barker
08/22/2023, 3:31 PMThanks @Will Raphaelson! I have a couple of followups I'll DM you with
109 Views