oleander-whirlwind
09/30/2025, 11:43 AMMarvin
09/30/2025, 11:43 AMMarvin
09/30/2025, 11:47 AMFROM python:3.11-slim
# Optional: set pip to use your Artifactory globally
RUN mkdir -p /etc/pip && \
printf "[global]\nindex-url = <https://artifactory.myco.com/api/pypi/pypi/simple>\ntrusted-host = <http://artifactory.myco.com|artifactory.myco.com>\n" > /etc/pip/pip.conf
# Or set via env vars (works at build and runtime)
ENV PIP_INDEX_URL="<https://artifactory.myco.com/api/pypi/pypi/simple>"
ENV PIP_TRUSTED_HOST="<http://artifactory.myco.com|artifactory.myco.com>"
# Install your deps
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Install your code
WORKDIR /app
COPY . /app
Then reference the image in your deployment:
# prefect.yaml (excerpt)
deployments:
- name: my-deployment
entrypoint: flows/my_flow.py:flow
image: <http://registry.myco.com/prefect/my-flow:latest|registry.myco.com/prefect/my-flow:latest>
Tip: If your Artifactory requires auth, don’t hardcode creds. Use Docker BuildKit secrets or a private base image with pip already configured.
2) Kubernetes worker with runtime installs
If you’re using Prefect’s Kubernetes worker and installing packages at container startup, configure pip and pass the packages to install. The worker image supports installing extra packages via an env var; set pip to use Artifactory via env vars or pip.conf.
Kubernetes Pod template snippet:
env:
- name: PIP_INDEX_URL
value: <https://artifactory.myco.com/api/pypi/pypi/simple>
- name: PIP_TRUSTED_HOST
value: <http://artifactory.myco.com|artifactory.myco.com>
- name: EXTRA_PIP_PACKAGES
value: "pandas==2.2.2 requests"
Docs: Prefect 3: Kubernetes deployments
Note: Runtime installs are convenient but slower and can be brittle. For repeatable runs, prefer baking deps into your image (option 1).
3) Local/VM process worker
If you run flows on a machine directly (ProcessWorker), configure pip on that machine:
- Set env vars before installing: PIP_INDEX_URL, PIP_EXTRA_INDEX_URL (if you mirror), PIP_TRUSTED_HOST
- Or create `~/.pip/pip.conf`:
[global]
index-url = <https://artifactory.myco.com/api/pypi/pypi/simple>
trusted-host = <http://artifactory.myco.com|artifactory.myco.com>
Auth notes
- Use credentials via env vars or a pip.conf that references a token, or use `.netrc`/keyring.
- In Kubernetes, store creds in Secrets and mount as env vars.
- In Docker builds, use BuildKit secrets so creds aren’t baked into the image layers.
Would you share how you’re running your flows today?
- Docker image + worker
- Kubernetes worker
- Local/VM process worker
I can provide a tailored snippet for your setup (including handling Artifactory auth).