Arseniy Shishaev
12/11/2025, 11:27 AMMarvin
12/11/2025, 11:28 AMMarvin
12/11/2025, 11:31 AMfrom prefect import flow
from prefect.runner.storage import GitRepository
from prefect.blocks.system import Secret
# Optional: for private repos or sparse checkout
storage = GitRepository(
url="<https://github.com/org/my-repo.git>",
# directories limits clone to subdirs (good for monorepos)
directories=["services/data-pipeline"],
credentials={"access_token": Secret.load("github-token")},
branch="main",
)
flow.from_source(
source=storage, # or just "<https://github.com/org/my-repo.git>" for public repos
entrypoint="services/data-pipeline/flows/etl.py:etl_flow", # path is from repo root
).deploy(
name="data-pipeline-etl",
work_pool_name="my-work-pool",
)
Option B: Use a prefect.yaml with pull steps
- Use the git_clone step (supports sparse checkout via directories).
- Then set the working directory to the subfolder that contains your flows.
- Entrypoint becomes relative to that working directory.
Example prefect.yaml:
prefect-version: "3.x"
name: my-monorepo
pull:
- prefect.deployments.steps.git_clone:
id: clone
repository: <https://github.com/org/my-repo.git>
branch: main
# Optional (recommended for monorepos): clone only what you need
directories:
- services/data-pipeline
# For private repos:
# access_token: "{{ prefect.blocks.secret.github-token }}"
- prefect.deployments.steps.set_working_directory:
directory: "{{ clone.directory }}/services/data-pipeline"
deployments:
- name: data-pipeline-etl
entrypoint: flows/etl.py:etl_flow # now relative to the working dir above
work_pool:
name: my-work-pool
Then deploy:
prefect deploy --name data-pipeline-etl
# or
prefect deploy --all
Notes and tips
- Entrypoint format is file.py:function_name. With flow.from_source it’s relative to the repo root; with prefect.yaml after set_working_directory it’s relative to that directory.
- For private repos, use a Secret or credentials block; avoid hardcoding tokens.
- If your flow imports shared code from other monorepo packages, either:
- Build an image that installs those packages, or
- Set PYTHONPATH via your work pool’s job_variables env so imports resolve at runtime.
Docs to learn more
- Deployments overview: https://docs.prefect.io/latest/deploy/
- CLI deploy command (for prefect.yaml): https://docs.prefect.io/latest/deploy/#deploy-your-project
If you can share:
- The subdirectory where your flow lives
- Whether you prefer Python-based or prefect.yaml-based deploys
- Whether the repo is private
…I can tailor an exact config for your repo.Arseniy Shishaev
12/11/2025, 11:47 AMfrom google import genai
ModuleNotFoundError: No module named 'google'
although this module is in my uv projectArseniy Shishaev
12/11/2025, 11:47 AMfrom google import genai
ModuleNotFoundError: No module named 'google'
although this module is in my uv projectMarvin
12/11/2025, 11:47 AMMarvin
12/11/2025, 11:51 AMFROM prefecthq/prefect:3.1.0-python3.12
WORKDIR /app
COPY pyproject.toml uv.lock ./
# If you’re using uv for the project:
# Install your project deps (choose the approach you use locally)
# Example with a requirements file:
# COPY requirements.txt .
# RUN uv pip install -r requirements.txt
# Or install the package itself:
# COPY . .
# RUN uv pip install .
# If you prefer standard pip, use pip install instead of uv.
# Minimal example if using requirements.txt:
# COPY requirements.txt .
# RUN uv pip install -r requirements.txt
COPY . .
Then set the deployment to use that image (via .deploy(..., image=...) or in prefect.yaml with job_variables.image).
2) Install dependencies on the worker host (Process/Local work pools)
- If you’re running a Process worker on a VM/laptop, install into the same environment the worker uses:
uv pip install google-genai
# or
pip install google-genai
Then restart the worker and re-run.
3) Install at runtime with a pull step (easy for dev; slower per run)
- Add a pip_install_requirements step to your prefect.yaml so the worker installs deps before running the flow:
pull:
- prefect.deployments.steps.git_clone:
id: clone
repository: <https://github.com/your-org/your-repo.git>
- prefect.deployments.steps.set_working_directory:
directory: "{{ clone.directory }}/path/to/your/subdir" # if using a monorepo
- prefect.deployments.steps.pip_install_requirements:
directory: "{{ clone.directory }}/path/to/your/subdir"
requirements_file: requirements.txt # ensure it lists `google-genai`
Important notes
- The package name to install is likely google-genai (which provides google.genai). Make sure that’s what you’re installing; installing google alone won’t give you google.genai.
- If your repo has a top-level folder named google/, it can shadow the namespace and cause import issues. Rename it if present.
- If you’re relying on pyproject.toml/uv only (no requirements.txt), either:
- Use a custom Dockerfile that runs uv pip install . (or uv sync) to install your project, or
- Generate a requirements.txt (uv can export one) and use the runtime install step above.
If you can share:
- Your work pool type (Docker/Kubernetes/Process)
- How you’re deploying (Python .deploy, prefect.yaml, or flow.from_source)
- Whether you have a Docker image configured
- Where your requirements live (pyproject/requirements.txt and path)
…I can give an exact snippet tailored to your setup.
Refs:
- Deploy projects and manage environments: https://docs.prefect.io/latest/deploy/
- Steps (git_clone, pip_install_requirements, set_working_directory): https://docs.prefect.io/latest/deploy/#pull-stepsArseniy Shishaev
12/11/2025, 11:55 AMMarvin
12/11/2025, 11:55 AMMarvin
12/11/2025, 11:56 AMfrom google import genai, you need the google-genai package (not google).
- Make sure your pyproject has something like:
[project]
dependencies = [
"google-genai>=0.3.0",
"prefect>=3"
]
Pick one of these setups:
1) Docker/Kubernetes work pool (recommended)
Bake your uv project into the image used by the deployment so the worker already has deps.
Dockerfile:
FROM prefecthq/prefect:3.1.0-python3.12
WORKDIR /app
# Copy only lock files first for better caching
COPY pyproject.toml uv.lock ./
# Create a project venv with your locked deps
RUN uv sync --frozen --no-dev
ENV VIRTUAL_ENV=/app/.venv
ENV PATH="/app/.venv/bin:${PATH}"
# Copy the rest of your code
COPY . .
Then reference this image in your deployment (Python or prefect.yaml):
# prefect.yaml (example)
deployments:
- name: my-deployment
entrypoint: path/to/flow.py:my_flow
work_pool:
name: my-docker-pool
job_variables:
image: "your-registry/your-image:tag"
2) Process work pool (no containers)
Run the worker from your uv virtual environment so it uses that interpreter for all flow runs:
cd /path/to/your/project
uv sync # creates .venv with deps from uv.lock
source .venv/bin/activate
prefect worker start -p your-work-pool
Now the worker’s Python has google-genai, and your flow imports will succeed.
3) Install deps at runtime (good for dev; slower)
If you prefer pull steps, install into the current interpreter before the flow runs. With uv projects you can either:
- Maintain a requirements.txt and use pip_install_requirements, or
- Run uv directly via a shell step and install packages into the current interpreter.
Example using uv directly:
pull:
- prefect.deployments.steps.git_clone:
id: clone
repository: <https://github.com/your-org/your-repo.git>
- prefect.deployments.steps.set_working_directory:
directory: "{{ clone.directory }}/path/to/subdir"
- prefect.deployments.steps.run_shell_script:
script: |
# Install directly into the interpreter the worker uses
uv pip install google-genai
# add any other packages you need here
Note: uv sync creates a separate venv; unless you also switch the interpreter Prefect uses, the flow won’t see that venv. Prefer uv pip install ... if you go this route.
Helpful refs
- Deployments and environments: https://docs.prefect.io/latest/deploy/
- Pull steps (git_clone, run_shell_script, pip_install_requirements): https://docs.prefect.io/latest/deploy/#pull-steps
If you tell me:
- Your work pool type (Docker/Kubernetes/Process)
- How you’re deploying (Python .deploy vs prefect.yaml)
- Whether your project is a package (so uv pip install . is viable)
…I can give you an exact, copy-paste config for your setup.Arseniy Shishaev
12/11/2025, 11:57 AMMarvin
12/11/2025, 11:58 AMMarvin
12/11/2025, 12:00 PMflow.from_source(
source="<https://github.com/your-org/your-repo.git>",
entrypoint="path/to/flow.py:my_flow",
).deploy(
name="my-deployment",
work_pool_name="your-managed-pool",
job_variables={
"pip_packages": ["google-genai"] # provides `from google import genai`
},
)
prefect.yaml:
deployments:
- name: my-deployment
entrypoint: path/to/flow.py:my_flow
work_pool:
name: your-managed-pool
job_variables:
pip_packages:
- google-genai
2) Use pull steps and run uv in the managed runtime (good if you want to install your project)
- Prefer uv pip install ... (installs into the current interpreter). Avoid uv sync (it creates a separate venv the run won’t use).
Example (monorepo-friendly):
pull:
- prefect.deployments.steps.git_clone:
id: clone
repository: <https://github.com/your-org/your-repo.git>
# optional sparse checkout
# directories: ["services/my-service"]
- prefect.deployments.steps.set_working_directory:
directory: "{{ clone.directory }}/services/my-service" # adjust if needed
# Install exactly what you need:
- prefect.deployments.steps.run_shell_script:
script: |
# Install the package you’re missing
uv pip install google-genai
# If your subdir is a Python package with pyproject.toml:
# uv pip install .
Notes
- Double-check your uv deps include google-genai (not google).
- If your repo has a top-level google/ folder, it can shadow the namespace; rename it if present.
If you confirm:
- The pool is “Prefect Managed” (serverless)
- Where your pyproject lives (path in the monorepo)
…I can tailor the exact prefect.yaml or .deploy() snippet for your repo.
Refs:
- Deployments overview: https://docs.prefect.io/latest/deploy/
- Pull steps (git_clone, run_shell_script): https://docs.prefect.io/latest/deploy/#pull-stepsArseniy Shishaev
12/11/2025, 12:05 PMMarvin
12/11/2025, 12:05 PMMarvin
12/11/2025, 12:06 PMpip_packages or pull steps).
If you want to run your own image without hosting a worker, use a “push” work pool (still serverless, but on your cloud provider):
- AWS ECS (Fargate): ecs:push
- Azure Container Instances: azure-container-instance:push
- Google Cloud Run: cloud-run:push
- Modal / Coiled: push pools with their own packaging models
How you’d reference your image in a deployment
Python:
from prefect import flow
from prefect.docker import DockerImage
@flow
def my_flow():
...
my_flow.deploy(
name="genai",
work_pool_name="my-cloud-run-pool", # or ecs/aci/etc
image=DockerImage(
name="<http://gcr.io/my-proj/myimage:latest|gcr.io/my-proj/myimage:latest>",
# Cloud Run requires amd64; specify if you built on arm:
platform="linux/amd64",
),
)
prefect.yaml:
deployments:
- name: genai
entrypoint: path/to/flow.py:my_flow
work_pool:
name: my-ecs-pool
job_variables:
image: "<http://123456789012.dkr.ecr.us-east-1.amazonaws.com/myimage:latest|123456789012.dkr.ecr.us-east-1.amazonaws.com/myimage:latest>"
Building your image with uv
# Dockerfile
FROM prefecthq/prefect:3.1.0-python3.12
WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev \
&& . .venv/bin/activate \
&& python -c "import sys; print(sys.executable)"
ENV VIRTUAL_ENV=/app/.venv
ENV PATH="/app/.venv/bin:${PATH}"
COPY . .
Recap:
- Prefect Managed (Prefect-hosted): no custom images; add deps via pip_packages or pull steps like run_shell_script: uv pip install google-genai.
- Push pools (provider serverless): yes to custom images; ideal if you need uv-locked envs and system packages.
If you tell me which provider you prefer (ECS, ACI, Cloud Run), I’ll share a minimal, copy-paste setup for the work pool and deployment. Links:
- Work pools overview: https://docs.prefect.io/latest/concepts/work-pools/
- Deployments: https://docs.prefect.io/latest/deploy/