https://prefect.io logo
t

Tense Fragile

08/25/2021, 12:33 PM
Hello! I have Linux server that runs the agent that schedules new prefect flows' runs. And I have a GitLab CD pipeline that connects to this server on push to master and pulls master on that server. On push to master I have the latest version of the code on the server but I want the agent to start using the latest code while it keeps scheduling jobs with old code. Is there any kinda autoreload feature for prefect agent? Maybe some of you faced with that problem so I'm curious about how you solved it?
k

Kevin Kho

08/25/2021, 2:16 PM
Hey, is the code being pushed your Flow code? or some package that you code is dependent on? If it’s something that your Flow code relies on like a custom module, using
pip install -e .
would pull the latest version. If it is the actual Flow code, it likely would have to be reregistered. So for a CI/CD approach, you can either use
Docker
storage and package everything up in a single container. When the container is loaded by the agent, everything will be up to date. Some users don’t like this because it means you have to rebuild the container every time. The second approach is to store it somewhere like Gitlab, or S3 where the agent will pull it down to execute. Again though, the Flow code should be re-registered if any of the metadata is changing (task graph or RunConfig/Storage/Caching/etc.)
3 Views