Robert Szomolanyi
10/21/2025, 3:44 PMMarvin
10/21/2025, 3:44 PMMarvin
10/21/2025, 3:51 PMfrom yourpkg.flows import etl_flow # your flow is installed and importable
# Creates a deployment that imports "yourpkg.flows:etl_flow" at run time
etl_flow.deploy(
name="etl-prod",
work_pool_name="process-pool",
tags=["prod"]
)
When to use: your workers run in environments where yourpkg is installed (e.g., Docker image, VM, or venv) and can import your flow.
Option B — Your flow lives in a local directory on disk
- Use Flow.from_source(...) to point to a directory and a file-based entrypoint like ./path/to/file.py:flow_function.
- This works only if the worker can see the same path (e.g., same machine or shared filesystem).
Example:
from prefect import Flow
Flow.from_source(
source=".", # project root on disk
entrypoint="flows/etl.py:etl_flow"
).deploy(
name="etl-local",
work_pool_name="process-pool"
)
Option C — Source is in Git (great when workers run remotely)
- Reference a Git repo and the same file-based entrypoint.
- For private repos, configure credentials via environment/blocks.
Example:
from prefect import Flow
Flow.from_source(
source="<https://github.com/org/repo>", # or git@github.com:org/repo.git
entrypoint="flows/etl.py:etl_flow"
).deploy(
name="etl-git",
work_pool_name="process-pool"
)
Starting a worker
- Make sure a worker is polling your work pool; otherwise scheduled runs won’t get picked up.
- Verified command:
prefect worker start -p process-pool
Notes and gotchas
- Installed-code approach (Option A): no extra storage is needed, but every worker that executes your flow must be able to import the module (same package/version).
- Local-path approach (Option B): only works if the worker has the same filesystem path available. If your worker runs elsewhere (e.g., k8s, VM), use Git (Option C) or package your code into the worker’s image/environment.
- For Docker/Kubernetes, you can set image= on deploy() and control build/push, but that’s optional if you’re not containerizing.
If you prefer the CLI
- From a project directory, you can do:
prefect deploy ./flows/etl.py:etl_flow -n etl-prod -p process-pool
This expects a file-based entrypoint in a Prefect project directory.
A couple of clarifying questions to tailor this to you:
- Will your worker run on the same machine as your local virtualenv?
- Is your flow code installed as a package (e.g., pip install -e .) in the worker environment, or do you need to pull from Git?Robert Szomolanyi
10/21/2025, 4:04 PMMarvin
10/21/2025, 4:05 PMMarvin
10/21/2025, 4:06 PMyour_flow.deploy(...) directly (Option A), Prefect uses a Python-module entrypoint (e.g., yourpkg.flows:etl_flow). For module-based entrypoints, Prefect requires a code distribution mechanism. That’s why you see: “ValueError: Either an image or remote storage location must be provided…”.
- Since you don’t want an image or remote storage, the fix is to deploy with a local source directory (on disk) so the worker reads the code directly from the filesystem.
Since your worker is on the same machine AND the code is installed in the same virtualenv, you can point Prefect at the installed package directory in site-packages.
Do this instead (Option B using your installed package’s folder as the source):
import importlib
from pathlib import Path
from prefect import Flow
# Replace 'yourpkg' and entrypoint below to match your package structure
pkg = importlib.import_module("yourpkg")
pkg_root = Path(pkg.__file__).parent
# e.g. .../venv/lib/python3.x/site-packages/yourpkg
Flow.from_source(
source=pkg_root,
entrypoint="flows/etl.py:etl_flow" # relative to pkg_root
).deploy(
name="etl-local",
work_pool_name="process-pool",
build=False, # ensure no image build attempt
push=False # ensure no push attempt
)
How to pick the entrypoint string
- It’s the file path (relative to pkg_root) plus :function_name of the flow.
- Examples:
- If your flow is defined in yourpkg/flows.py as etl_flow, use entrypoint="flows.py:etl_flow".
- If it’s in yourpkg/flows/etl.py as etl_flow, use entrypoint="flows/etl.py:etl_flow".
Start a worker for your pool
prefect worker start -p process-pool
Why this works
- Prefect now treats the installed package directory as a local source of truth. Since your worker is on the same machine/venv, it can read those files directly—no image and no remote storage needed.
If you want, share:
- The module path to your flow function (e.g., yourpkg.flows:etl_flow)
- The package layout (where the file actually lives)
I can give you the exact entrypoint string to use.Robert Szomolanyi
10/21/2025, 4:22 PMMarvin
10/21/2025, 4:22 PMMarvin
10/21/2025, 4:23 PMflow.deploy(...) with a module entrypoint, Prefect requires an image or remote storage. To replicate the CLI behavior in Python without image/remote storage, construct a RunnerDeployment with LocalStorage and then call the top-level deploy function.
Do this:
from pathlib import Path
from prefect.deployments import deploy
from prefect.deployments.runner import RunnerDeployment
from prefect.storage import LocalStorage
# Use the project root as your working directory, just like your prefect.yaml
project_root = Path(".").resolve()
deployment = RunnerDeployment(
name="LIF_0197",
entrypoint="lif_0197.main:lif_0197", # module:func, same as your prefect.yaml
work_pool_name="process-pool",
storage=LocalStorage(path=project_root), # local working dir; no image/remote storage
# optional: tags=[], parameters={}, description="...", etc.
)
# Create/update the deployment in Prefect
deploy(deployment, build=False, push=False)
Why this works
- LocalStorage(path=...) mirrors your prefect.yaml pull step (set_working_directory: "."), telling the worker to run from that directory.
- With a module entrypoint, Prefect accepts a local storage location (working directory) instead of an image or remote storage, as long as the worker is on the same machine and can import the module (your uv editable install handles that).
Start the worker
prefect worker start -p process-pool
Notes
- Keep build=False, push=False to avoid image building/pushing.
- You don’t need to reference the virtualenv’s site-packages path; pointing storage to your project root (".") is enough, matching the CLI behavior you already have.
- If your worker starts from a different CWD, prefer an absolute path for LocalStorage(path=...) as shown above.Robert Szomolanyi
10/21/2025, 4:29 PMMarvin
10/21/2025, 4:29 PMMarvin
10/21/2025, 4:31 PMprefect.storage the way I suggested.
Use this instead (mirrors your CLI prefect.yaml with set_working_directory="."):
from prefect import Flow
Flow.from_source(
source=".", # same as set_working_directory: "."
entrypoint="lif_0197.main:lif_0197" # module:function, same as your CLI
).deploy(
name="LIF_0197",
work_pool_name="process-pool",
build=False,
push=False
)
Notes:
- Run this from your project root so source="." points to the correct working directory.
- This avoids any image or remote storage; your worker is on the same machine and can import the editable module via uv, so it will work just like the CLI.
- Start your worker if it’s not already running:
prefect worker start -p process-pool
Docs for reference:
- Deployments overview
- prefect deploy CLIRobert Szomolanyi
10/21/2025, 4:33 PMMarvin
10/21/2025, 4:33 PMMarvin
10/21/2025, 4:33 PMfrom pathlib import Path
from prefect.deployments import deploy
from prefect.deployments.runner import RunnerDeployment
from prefect.runner.storage import LocalStorage # correct import
project_root = Path(".").resolve()
deployment = RunnerDeployment(
name="LIF_0197",
entrypoint="lif_0197.main:lif_0197", # module:function, same as your CLI
work_pool_name="process-pool",
storage=LocalStorage(path=str(project_root)), # mirrors set_working_directory: "."
)
# Register/update the deployment; no image and no remote storage
deploy(deployment, build=False, push=False)
Then start your worker:
prefect worker start -p process-pool
Why this fixes the error:
- A module-based entrypoint without storage requires either an image or a storage location.
- The CLI sets a working directory via the pull step; the above code does the same by providing LocalStorage, so Prefect does not require an image or remote storage.
References:
- Deployments overview
- prefect deploy CLI
If you still hit issues, please share your project root and the exact lif_0197 layout; I’ll tailor the entrypoint and LocalStorage path precisely.