I have a private Python package bundled up and sto...
# prefect-server
I have a private Python package bundled up and stored in Google Artifact Registry (as opposed to an open-source package on PyPI). I'd like to include it in my flow, which uses a Docker storage instance in production. I know that I can use the
kwarg to the class to include open-source packages. But how do I get a private package from Google Artifact Registry included as well? Is there an established pattern for this?
🙌 1
So this is happening because you are using the
as the interface to build the image. If it’s too limiting, you can supply your own Dockerfile or image and just handle it yourself.
But for this, I think there is a chance we can get it to work by pointing your
to the Artifact Registry. Each one of the dependencies are just pip installed here by adding commands to the container
So this will work if doiing
pip install some_library"
magically worked. I’ll look a bit
You can also add extra commands to the Docker build if ever we can’t pip install it
I think you need to add your own
pip install
command with the extra things like this
upvote 1
should do the trick - you can add it to your Dockerfile like so:
Copy code
RUN pip install --index-url <https://LOCATION-python.pkg.dev/PROJECT/REPOSITORY/simple/> PACKAGE
But when you register your flow and build the image, your terminal must be authenticated with Artifact Registry Reader permissions. In my last job we were also using private artifactory and you could also just add the index URL at the top of your
like so:
Copy code
--index-url <https://LOCATION-python.pkg.dev/PROJECT/REPOSITORY/simple>
--extra-index-url <https://pypi.org/simple>

# PyPi dependencies

# your custom private packages