<@ULVA73B9P> Using a flow.from_source() type of de...
# ask-marvin
r
@Marvin Using a flow.from_source() type of deployment pulling code from Github, how can I run custom code at run time before Prefect code execution? For instance, to install custom dependancies using uv rather than pip through EXTRA_PIP_PACKAGES.
m
thought for 596.9 seconds
Great question. Assuming you’re on Prefect 3.x and using
flow.from_source(...).deploy(...)
with a GitHub repo: Short answer: - There isn’t a dedicated “pre-run hook,” but you have a few supported ways to run code before your flow is imported/executed. - If your goal is “install deps with uv,” you may not need anything extra: Prefect’s official images already install
EXTRA_PIP_PACKAGES
with uv at runtime. Options, from simplest to most flexible: 1) Use EXTRA_PIP_PACKAGES (uses uv under the hood) - If you’re using Prefect’s official images and you haven’t overridden the entrypoint, setting
EXTRA_PIP_PACKAGES
will install packages with uv before the flow runs. - Prefect’s entrypoint calls:
Copy code
if [ ! -z "$EXTRA_PIP_PACKAGES" ]; then
    echo "+uv pip install $EXTRA_PIP_PACKAGES"
    uv pip install --system $EXTRA_PIP_PACKAGES
  fi
- So you already get uv rather than pip. Add env to your work pool or deployment job variables:
Copy code
job_variables={"env": {"EXTRA_PIP_PACKAGES": "pandas boto3"}}
- References: - Entrypoint source: scripts/entrypoint.sh - Docker guide: https://docs.prefect.io/v3/how-to-guides/deployment_infra/docker/ - Kubernetes guide: https://docs.prefect.io/v3/how-to-guides/deployment_infra/kubernetes/ 2) Add a pull step that runs a shell script (runs before your flow import) - Pull steps execute each time a flow run starts, before Prefect imports your flow code. You can add a shell step to install with uv (or do any other setup). - In prefect.yaml:
Copy code
pull:
    # Your code will already be cloned automatically by flow.from_source() (git_clone),
    # so only add extra steps you need.
    - prefect.deployments.steps.utility.run_shell_script:
        id: install-deps
        script: |
          uv pip install --system -r requirements.txt
          uv pip install --system "some-extra==1.2.3"
        expand_env_vars: true
- Docs: - Deploy via Python / from_source: https://docs.prefect.io/v3/how-to-guides/deployments/deploy-via-python/ - prefect.yaml and pull steps: https://docs.prefect.io/v3/how-to-guides/deployments/prefect-yaml/ - Steps API (pull + utility): https://docs.prefect.io/v3/api-ref/python/prefect-deployments-steps-pull/ 3) Custom Docker entrypoint (full control, still preserves Prefect behavior) - Create a wrapper entrypoint that runs your pre-run logic, then execs Prefect’s original entrypoint. Keep tini and the final exec to ensure clean signal handling. Dockerfile:
Copy code
FROM prefecthq/prefect:3-latest
  COPY scripts/custom-entrypoint.sh /opt/prefect/custom-entrypoint.sh
  RUN chmod +x /opt/prefect/custom-entrypoint.sh
  ENTRYPOINT ["/usr/bin/tini", "-g", "--", "/opt/prefect/custom-entrypoint.sh"]
scripts/custom-entrypoint.sh:
Copy code
#!/usr/bin/env bash
  set -euo pipefail

  # Your pre-run actions
  uv pip install --system -r /opt/prefect/requirements-extra.txt || true

  # Hand off to Prefect's entrypoint
  exec /opt/prefect/entrypoint.sh "$@"
- Works for Docker-based workers and Kubernetes (if you use this image in your job template). 4) Kubernetes-only: initContainers in the work pool’s base job template - You can run setup in an initContainer before the main container starts:
Copy code
"initContainers": [
    {
      "name": "setup",
      "image": "{{ image }}",
      "command": ["/bin/bash", "-c", "uv pip install --system boto3"]
    }
  ]
- Docs: https://docs.prefect.io/v3/how-to-guides/deployment_infra/kubernetes/ Notes and gotchas - There’s no special “pre-run hook,” but pull steps are the supported pre-import mechanism and work with
flow.from_source
. -
flow.from_source(...)
automatically adds the required pull step (e.g.,
git_clone
) for your storage. Only add extra steps you need; don’t duplicate the clone step.
- If you’re using a Process worker (no containers), pull steps run on the host, so ensure
uv
is installed on that machine/virtualenv. - EXTRA_PIP_PACKAGES only installs at runtime if you’re using Prefect’s official image and haven’t changed the entrypoint. If you can share which worker type you’re using (Docker, Kubernetes, or Process) and whether you’re on the Prefect base image, I can give you an exact snippet tailored to your setup.