Hi! 👋 I admit my use-case isn't what Prefect was designed for, but here I am. I have to have manual trigger to let flow continue. That's because that are first runs of this flow in production, we want to manually check some state before going further. The problem is that after pause new Container is created. All state (both variables and temporary files stored in previous container) is lost this way and after pause it's missing. Is there some way to come around this obstacle?

I haven't tried to use manual triggers, but I have had similar problems when I have tried to restart a failed flow. I found that I was able to re-run a failed flow once I set up Results for my flow. But, Secret tasks will not have their values stored. To get around this I needed to manually mark Secret Tasks as Failed. On a re-run, the Secrets got re-evaluated and got the values that they needed and the other Tasks got their values from the Results. Note that I do not know what the behaviour of setting a Task to failed and leaving the subsequent Task as Success and doing a re-run. The environment that I did this was with Prefect deployed in kubernetes using KubernetesJobs.

Hi @Lech Głowiak, As Matt mentioned above, if you store the outputs of Task Runs and other pieces of state in Results, your Flow Run should work properly. Behind the scenes, Prefect allows the infrastructure in a paused state to spin down until you approve it. When an approval happens, the flow run is re-submitted for work and the agent spins up a new process/job for it.

👍 Thanks for the answers @Matt Drago and @Dylan. I'll try this approach. I understand that there is no other way to pause without spin down?

That is not configurable at this time. If you think it’s functionality that others would appreciate, feel free to open an enhancement issue on GitHub =]

Well, I started with notice, that perhaps I'm not using the best tool for the job. I guess there's very little need of such feature in data engineering tool.

We definitely have other users that have a manual approval for some workflows

I think I'll be ok with Results if I arrange flow a bit differently, so all dirty localfilesystem files manipulation will happen between pauses.