https://prefect.io logo
#prefect-ui
Title
# prefect-ui
s

Sam Garvis

04/06/2022, 6:30 PM
Hi all! We are transitioning our python scripts and others from Heroku to Prefect. We have prefect cloud, but what is the best way to run a prefect agent with the needed dependencies for our python files. We were just using prefect docker agent start on the VM, and pointing our flows to a dockerfile with the correct dependencies. This was working but would always stop working after a few flow runs. Does anyone know of any source code for a good docker compose for prefect agents to run on a VM, or where we should look for this info? Thanks!!
k

Kevin Kho

04/06/2022, 6:32 PM
Hi @Sam Garvis, this sounds a bit weird. The docker agent spins a new container in the VM anyway so the environment should be consistent. You’re not concerned about the docker compose for the agents, I think you’re more concerned about the new container spun by the Flow. Could you show me an example of the current
DockerRun
you use in the Flow definition and share more details about how it fails?
s

Sam Garvis

04/07/2022, 4:15 PM
The DockerRun we are using is this. test:latest is a Prefect container with the custom requirements we need installed. And when it stops working, the flows just don't reach the agent. When we run the flow they just turn into late flows that never end.
k

Kevin Kho

04/07/2022, 4:23 PM
This looks right to me. I think it’s a separate issue on them not reaching the agent. There is this Discourse page that might help you get started
s

Sam Garvis

04/07/2022, 6:45 PM
The issue is that when we run prefect docker agent start on the VM. When we close out of the VM on GCP, it stops the agent.
What is the best way to keep it running in the background?
k

Kevin Kho

04/07/2022, 7:21 PM
There are a couple ways to do that. Maybe
tmux
or
pm2
? You can also use
supervisor
. What you are looking for in general is to run them in a detached mode
👍 1
s

Sam Garvis

04/07/2022, 7:39 PM
@Kevin Kho Thank you so much. I will look into this
In case this helps you at all, an easy fix for me was to run 'nohup prefect agent docker start &'. This has been working for days now and is a great shorterm fix for what we need. Ik this is very similar to supervisor, so thanks for your help for leading me in the right direction!
k

Kevin Kho

04/11/2022, 4:19 PM
Nice! yeah i can recommend nohup in the future
7 Views