Hi all! We are transitioning our python scripts and others from Heroku to Prefect. We have prefect cloud, but what is the best way to run a prefect agent with the needed dependencies for our python files. We were just using prefect docker agent start on the VM, and pointing our flows to a dockerfile with the correct dependencies. This was working but would always stop working after a few flow runs. Does anyone know of any source code for a good docker compose for prefect agents to run on a VM, or where we should look for this info? Thanks!!

Hi @Sam Garvis, this sounds a bit weird. The docker agent spins a new container in the VM anyway so the environment should be consistent. You’re not concerned about the docker compose for the agents, I think you’re more concerned about the new container spun by the Flow. Could you show me an example of the current

DockerRun

you use in the Flow definition and share more details about how it fails?

The DockerRun we are using is this. test:latest is a Prefect container with the custom requirements we need installed. And when it stops working, the flows just don't reach the agent. When we run the flow they just turn into late flows that never end.

This looks right to me. I think it’s a separate issue on them not reaching the agent. There is this Discourse page that might help you get started

The issue is that when we run prefect docker agent start on the VM. When we close out of the VM on GCP, it stops the agent.

There are a couple ways to do that. Maybe

tmux

or

pm2

? You can also use

supervisor

. What you are looking for in general is to run them in a detached mode

@Kevin Kho Thank you so much. I will look into this

Nice! yeah i can recommend nohup in the future