Hello from Chicago! I've tried searching this Sla...
# prefect-community
j
Hello from Chicago! I've tried searching this Slack and the Prefect documentation but haven't found a good answer for this.
How should I handle prefect version differences between the environment where a flow is created and registered (
flow.register()
) and the environment where the agent runs?
So, concretely, let's say that I have an agent running using the image
prefecthq/prefect:0.12.1-python3.7
. I have some flows already being run by that agent, and those flows were created with
prefect
0.12.1. Imagine that
prefect
0.13.0 has just been released on PyPi, and data scientists on my team are going to
pip install prefect
, create flows, and register them with
flow.register()
What should I do? 1. Add a label to each agent with the prefect major + minor version (e.g. v0.12, v0.13). Make sure flows are registered with such a version label, so 0.12 flows run on the 0.12 agent and 0.13 flows run on the 0.13 agent.
prefect
uses semantic versioning, which means that there can be breaking changes between minor releases in the
0.x
series. 2. Do nothing. a flow created with 0.13.x should be expected to work with an agent running 0.12.x 3. Upgrade all those 0.12 flows to 0.13. Prefect Cloud is only ever running a single version of
prefect
and if your flows + agent are not in sync with that version, bad things will happen. 4. something else Thanks for your time and consideration!
k
Hello @james.lamb, Survey says.... #2! We're gonna play it! New versions of Prefect will work with an Agent running a previous version, the only caveat being a LocalAgent will generally require environment parity. All other agents will submit Docker containers to run in, in which case you can fully control the prefect version running within the container at build time.
j
Thanks! Is that a guarantee that I should rely on, or just a "best thing to try first"? It seems to me that the agent code is importing things from other parts of the
prefect
library, so I imagine that if those things change it could break older flows: https://github.com/PrefectHQ/prefect/blob/ac473c3b09c4175ecbbd2c8758c300ab49f9b6fe/src/prefect/agent/agent.py#L18-L24
k
You should be able to rely on 'doing nothing.' All new versions of Prefect will account for your situation, and the devs will notify everyone of any version breaking issues (which I've not yet seen happen). If anything does comes up however, please feel free to let the team know via Github issue or here on Slack.
If you're feeling sheepish though, going with your #1 option will definitely help you stay thoroughly organized.
j
ok fair enough, thanks very much for your quick response