Hey there, I have a question about the "right way" to run prefect agents.
I have the following setup: local environment (where I quickly test flows and tasks), github repo with CI which automatically redeploys new code, remote machine (which clones code from github, runs prefect deployment build and then starts prefect agent). And finally I am using Prefect Cloud for all remaining things.
I am wondering that would happen If I redeploy my codebase (adding new flow and deployment for instance) and at the same time I would have some flows running. Would these flows be able to resume at the "right point"? And in general, what is the best practice for redeploying the codebase with prefect if I use only one machine?
Thanks a lot!
P.S. I used Airflow heavily before Prefect and now I am migrating some jobs to Prefect.
09/29/2022, 1:42 PM
Maybe I don’t quite understand the question. but you could pause the work-queue / agent from picking up more jobs prior to re-deploying your codebase
Then once finished re-deploying your codebase, re-enabling the work-queue
Running jobs will complete as they are already executing in infrastructure, new jobs will not while the queue is paused
09/29/2022, 2:08 PM
Thanks, this is helpfull!
But what about a use case, when I have a both the infrastructure and the codebase at the same place?
I mean when I re-deploy the codebase my infrastructure (local process) also shuts down (container is killed and then it is built again)
09/29/2022, 2:14 PM
I think for use cases like that we typically see more resilient infrastructure like kubernetes / docker / vms to host your agent to keep your infra up and running