Hello, Noob question here, How do I execute an agent run (docker agent) on a server so that if I exit/logout of the server, the process/agent stays running/executing scheduled flows? Your help is much appreciated.

Hey @Stephen Mugisha --

screen

is one of the simplest ways to do this

Thanks very much @Zanie

another noob question: is this something we would need to do usually? I was thinking the container configs would start the agent along with everything else... but I haven't got that far yet.

If you're using

prefect server start

the agent is started separately; if you're using the Prefect Server helm chart, you can install the agent to your cluster at the same time; if you're using Prefect Cloud, you'll need to start an agent somewhere. The agent is in charge of spawning the containers to run flows.

So the agent can run on the k8s cluster, which also hosts the server and , UI and apollo endpoint. I guess Hasura, Graphql and towel could also be on the cluster, but we would want postgress to be persistent as RDS or at a minimum getting regular snapshots saved to s3.

Happy to hear from you 🙂

Hi! I don't know how those relate yet, I haven't actually started building yet (I'm working out some integration questions with our existing scheduler). I guess what i need to know is, when setting up prefect server what is needed to ensure that my data flow history metadata is preserved. For example there will be hundreds of weekly/daily jobs, if the cluster dies I need to launch a new one and not lose the history of my data store. Related question: For Prefect Cloud, is there a way to export/backup that job run history in the case where we someday migrated off of Cloud?

All the persistent data is stored in postgres

cool thats good insight, i'm doing probably too much thinking ahead.