<@ULVA73B9P> I have my code on git I have to run...
# ask-marvin
s
@Marvin I have my code on git I have to run this code on docker worker with dependencies what options I have to make it happen
m
thought for 325.5 seconds
Great question. Assuming you’re on Prefect 3.x, you’ve got two solid paths to run code from Git on a Docker worker with the right dependencies: 1) Pull your code from Git at runtime + install deps on container startup - Best when you want the latest code on each run and don’t want to rebuild images often - Use
flow.from_source(...)
to point Prefect at your repo - Let the official Prefect image install extra packages at runtime via the
EXTRA_PIP_PACKAGES
env var (handled by the image’s entrypoint) Example:
Copy code
from prefect import flow

@flow(log_prints=True)
def my_flow():
    import pandas
    print("Pandas version:", pandas.__version__)

if __name__ == "__main__":
    flow.from_source(
        source="<https://github.com/your-org/your-repo.git>",  # or a GitRepository with credentials
        entrypoint="flows.py:my_flow",
    ).deploy(
        name="git-runtime-install",
        work_pool_name="my-docker-pool",
        job_variables={
            "env": {
                "EXTRA_PIP_PACKAGES": "pandas boto3"
            }
        },
    )
Notes: - For private repos, use
prefect.runner.storage.GitRepository
with credentials. See Store flow code guide -
EXTRA_PIP_PACKAGES
works when you use Prefect’s official images and don’t override the container entrypoint - Prefer listing packages explicitly; using
-r requirements.txt
can be brittle because package install may occur before code is pulled 2) Bake your code and dependencies into a Docker image - Best for reproducibility, faster startup, and complex/native deps - You can let Prefect generate a Dockerfile from your project or point to your own Dockerfile Minimal example using an auto-generated Dockerfile:
Copy code
from prefect import flow

@flow
def my_flow():
    import pandas
    print("works!")

if __name__ == "__main__":
    my_flow.deploy(
        name="image-baked",
        work_pool_name="my-docker-pool",
        image="my-registry/my-image:latest",
        build=True,   # build an image from your current directory
        push=True,    # push to your registry
    )
Custom Dockerfile example:
Copy code
# Dockerfile
FROM prefecthq/prefect:3-python3.11
COPY requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
COPY . /opt/prefect/flows/
WORKDIR /opt/prefect/flows/
Deploy with:
Copy code
from prefect import flow
from prefect.docker import DockerImage

@flow
def my_flow():
    ...

if __name__ == "__main__":
    my_flow.deploy(
        name="custom-dockerfile",
        work_pool_name="my-docker-pool",
        image=DockerImage(
            name="my-registry/my-flows",
            tag="v1",
            dockerfile="Dockerfile",
        ),
        push=True,
    )
How to get your Docker worker running - Create a Docker work pool:
Copy code
prefect work-pool create "my-docker-pool" --type docker
- Start a worker that polls that pool:
Copy code
prefect worker start --pool my-docker-pool
When to choose which option - Pull from Git at runtime + EXTRA_PIP_PACKAGES: - Frequent code changes, don’t want builds - Simpler dependency sets - Bake into image: - Reproducible runs, faster startup - Complex/native dependencies, private indexes, or air‑gapped Helpful docs - Docker worker/infrastructure: Run on Docker - Storing flow code (Git, S3, etc.): Store flow code A few clarifying questions to get you set up quickly: - Is your Git repo public or private? If private, do you prefer passing a token or using a credentials block? - Do you already have a requirements.txt/pyproject you want baked into an image, or would runtime installs via
EXTRA_PIP_PACKAGES
be fine? - Do you have a Docker registry to push to if we go with the image build route?