https://prefect.io logo
d

Dolor Oculus

01/14/2021, 2:21 PM
Is there a way to prevent the prefect docker flow storage from pip installing the latest and greatest prefect? My build setup/team is aghast. (We prefer to explicitly manage dependencies ourselves.)
s

Spencer

01/14/2021, 2:38 PM
According to the source, you'll need to pre-install prefect. https://github.com/PrefectHQ/prefect/blob/master/src/prefect/storage/docker.py#L204-L210
b

Billy McMonagle

01/14/2021, 3:07 PM
In my build environment, I have
pip install prefect==PREFECT_VERSION
, and set explicitly the base image that docker storage is built from. Together these seem to work well.
c

Chris White

01/14/2021, 3:58 PM
The Docker storage interface determines what version to install based on your local version, so if it’s installing
master
or the latest version, that must be what you’re running on locally. If you want to specify a different version, there is a
prefect_version
kwarg to the interface that you can use to pin to whatever version you’d like
d

Dolor Oculus

01/15/2021, 4:09 PM
@Chris White 🔥 thank you
we're using conda though, so pip is still an issue.
I'll share the above with my build team though and see if they have thoughts.
k

Kevin Weiler

01/15/2021, 4:23 PM
yeah - to be clear @Chris White - we specifically DON’T want prefect installing anything in our environment. There are 2 reasons for this: 1. the container we base off has a conda environment (that we build) that already has prefect installed 2. because the environment is conda, it’s possible for pip to install dependencies over top of ours.
a bit of further explanation: We don’t particularly like conda, but until very recently, it was the only game in town as far as actual package resolution is concerned. Pip (prior to v20.3) doesn’t actually handle transitive dependencies, which is a huge problem in a sufficiently complex environment. As an example, if package A depends on package Z==2 and Package B depends on package Z==1, pip installs the transitive dependencies in whatever order it installs the explicit dependencies. So if package A is listed first in your
requirements.txt
then it installs package Z==2 first and then Z==1 over top of it second.
😱 1
2 Views