Hey All - In the process of upgrading to prefect 2, so far really loving it.
Having a few issues though. I am trying to build a custom image to run our flows. Starting with
prefecthq/prefect:2.6.1-python3.10-conda
as a base image. Couple of questions:
• When installing a python environment in the container does it mater what it is called? I see the image has a
base
env and a
prefect
env. Should packages be installed in
prefect
env of can I create a new one?
• If I create a new env should the Dockerfile include a line to activate the correct env.?
• Is there a recipe for creating a custom image from a conda base image? I looked around and did find one, but maybe I missed it?
✅ 1
z
Zanie
10/19/2022, 7:00 PM
We include
RUN echo "conda activate prefect" >> ~/.bashrc
to have the image use the Prefect environment by default
Zanie
10/19/2022, 7:01 PM
We need to create a separate environment from
base
in order to have different Python versions.
Zanie
10/19/2022, 7:01 PM
I’d recommend continuing to use the
prefect
environment. I do not know of a recipe, I don’t think we have many users of the conda flavored images yet.
Zanie
10/19/2022, 7:02 PM
@Serina may be worth considering for a recipe.
🙌 1
👍 1
m
Matt Denno
10/19/2022, 7:02 PM
Thanks so much @Zanie . Just what I needed to know.
z
Zanie
10/19/2022, 7:02 PM
You’re welcome :0
Zanie
10/19/2022, 7:02 PM
🙂 *
m
Matt Denno
10/19/2022, 7:03 PM
I am not usually a conda user myself, but it seemed like the easiest path to get the correct GDAL binaries and few other things into the container.
z
Zanie
10/19/2022, 7:08 PM
If you’re going to extend the image it might be just as easy to apt-get install them from our normal base image? Whatever’s easiest though 🙂
m
Matt Denno
10/19/2022, 7:17 PM
That was actually what I tried first but GDAL is a tricky one because the version we are using (the version available for Ubuntu 22.04) is not available in the Debian repos. I'm sure I could add some other repos and get the correct version that way, but conda includes the binary for whatever version is installed. We shall see how it goes. :)
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.