Hello friends, Is it possible to pause a prefect workflow, have it not consume resources, then turn it back on later?

You can pause and resume flow runs, yes. Have you seen these docs? https://docs.prefect.io/2.11.4/concepts/flows/#pause-a-flow-run

Hey @jawnsy I saw this, I'm curious how many resources are being consumed when its waiting, do you know by chance?

That depends on how you’re running things. If you’re running a Kubernetes worker, it will run your workloads as Kubernetes jobs, and if the flow is paused, then it will not create those jobs, allowing your cluster to scale down. It requires some setup to get that autoscaling to work correctly though, particularly you will need to set resource requests/limits and use the cluster autoscaler. If you’re running with Cloud Run or ECS, it will run workloads as Cloud Run Jobs or ECS Tasks, and if the flow is paused, then it won’t create those. So you should only need to pay for running the worker, which should be relatively inexpensive, since it’s pretty lightweight If you use push work pools, then you don’t need to run an agent at all, so that allows full scale-to-zero while stuff is paused

awesome thanks, just to double check, if I use a push work pool, I wouldn't pay anything while paused? Are there any other limitations of that? I may need to scale this to thousands of workflows, does that seem feasible?

Hey gang, just want to hop in to clear up some concepts. what i understand the request to be is to pause the execution of a flow without consuming compute resources. while the above around work pools is correct, its not directly related to flow pause behavior. You can get exactly what you want here by pausing flow runs in code/from the API with reschedule=True. check out the docs here, in particular the callout box “*Pausing flow runs is blocking by default”.* https://docs.prefect.io/2.11.4/concepts/flows/#pause-a-flow-run Patrick, feel free to reach out via DM if you want to hop on a call and discuss the use case.

Thanks @Will Raphaelson! I have a couple of followups I'll DM you with