Great question. In Prefect 3.x, the simplest way to run flows from code that already exists in a virtual environment on the worker machine is: - Start a Process worker from inside that virtual environment - Create a deployment that references the local source on disk (no code upload) Because the Process worker spawns flow runs with the same Python interpreter as the worker, starting the worker from your venv ensures your flows run in that venv. Here’s a concrete setup you can follow. 1) Create a Process work pool

prefect work-pool create process-local --type process

Optional: set a default working directory for runs so relative paths in your deployment resolve correctly.

prefect work-pool get-default-base-job-template --type process --file base-job-template.json
# edit base-job-template.json to set "working_dir" to your project root on the worker, e.g. "/opt/flows"
prefect work-pool create process-local --type process --base-job-template base-job-template.json --overwrite

2) Start the worker inside your virtual environment Activate your venv on the worker machine, then start the worker:

source /path/to/venv/bin/activate    # (Windows) .\venv\Scripts\activate
prefect worker start -p process-local

This guarantees the flow subprocess uses that venv’s interpreter and packages. 3) Create a deployment that uses local source (no upload) Option A – Python API:

from prefect import flow

@flow(log_prints=True)
def my_flow():
    print("Hello from my venv")

if __name__ == "__main__":
    my_flow.from_source(
        source="/path/on/worker/project",          # directory that exists on the worker machine
        entrypoint="flows/my_flow.py:my_flow",     # relative to that directory
    ).deploy(
        name="local-venv",
        work_pool_name="process-local",
    )

Option B – CLI from the project root on the worker machine:

prefect deploy ./flows/my_flow.py:my_flow -n local-venv -p process-local

Notes and tips - Path must exist on the worker: If you deploy from a different machine, use an absolute path in

source

that also exists on the worker, or just run

prefect deploy

on the worker itself from the project root. - Multiple venvs: If you need different venvs, start separate workers (each activated in its own venv) and point deployments to the appropriate work pool/queue. - Installed-as-package scenario: If your flows are installed into the venv as a Python package (not a local repo), add a small entrypoint file in a local project that imports the packaged flow, then set

source

to that local project directory and

entrypoint

to the wrapper file. Helpful links - Work pools and workers overview - Prefect 3 GA overview - Flow.from_source (source code reference) - Job template details (custom worker docs) Would you like me to tailor the commands for your exact setup? If so: - Are your flows on disk in a project directory, or installed as a package in the venv? - Do all workers share the same absolute path to the project? - Which OS are you running on?

Thanks — here’s a setup tailored to your environment. First, one correction: Prefect 3.x uses a project file named

prefect.yaml

(not

prefect.toml

) for deployments. You can generate one with

prefect init

, or create it yourself. If you specifically need TOML for something else, let me know, but deployments use YAML. Given: 1) You’re using a pyenv-managed virtualenv 2) Your flow is installed in that venv and importable as

lif_0197.main:run

3) Your worker is already running inside that same venv 4) You want the project file created Recommended approach - Create a tiny Prefect project on the same machine/venv as the worker. - Add a small entrypoint file that re-exports your installed flow function. - Use

prefect.yaml

to define the deployment and point at that entrypoint. - Deploy with

prefect deploy

. No code is uploaded; the worker imports the flow from the venv. 1) Create a minimal project directory on the worker machine

mkdir -p ~/prefect/lif_0197_project
cd ~/prefect/lif_0197_project

2) Create a tiny entrypoint file that exposes your installed flow This file just imports the flow function from your installed package so Prefect can reference it by file:path.

# file: entry.py
from lif_0197.main import run  # `run` must be a Prefect @flow

3) Create the Prefect project file (prefect.yaml)

# file: prefect.yaml
name: lif-0197-project
prefect-version: ">=3.0.0"

deployments:
  - name: lif-0197
    entrypoint: entry.py:run
    work_pool:
      name: process-local     # use your existing process work pool name here
      # work_queue_name: default  # optional
    parameters: {}            # add any default parameters if needed
    tags: []                  # optional
    schedules: []             # optional

Notes: - Replace

process-local

with the actual name of your Process work pool if different. - Because the worker is started inside your pyenv venv, the flow run will use that same interpreter and the installed

lif_0197

package. 4) Deploy Run this from the project directory:

prefect deploy

5) Run - From the UI or CLI, trigger the “lif-0197” deployment. The worker (already running in your pyenv) will import

lif_0197.main:run

and execute it in that venv. Optional: create the work pool (if you don’t already have one)

prefect work-pool create process-local --type process

Then start your worker from inside your pyenv (you mentioned it’s already running):

prefect worker start -p process-local

Why this works with installed packages - The worker is running inside your pyenv, so all imports resolve to that environment’s site-packages. - The entrypoint file gives Prefect a file:path to find the flow object (

entry.py:run

), but the flow itself comes from the installed package. Docs - Prefect 3 project file (prefect.yaml): prefect.yaml guide - CLI overview: prefect deploy (CLI) If you truly need a TOML file for a specific reason, tell me what config you’d like in it and I’ll translate it appropriately.

You don’t need a separate entry.py. In Prefect 3.x you can use an import-style entrypoint directly. Just point the deployment at

lif_0197.main:run

and make sure your worker is started inside the same pyenv virtualenv where

lif_0197

is installed. Here’s the minimal setup: 1) Ensure a Process work pool and worker (inside your pyenv)

prefect work-pool create process-local --type process
# Activate your pyenv virtualenv
prefect worker start -p process-local

2) Create a Prefect project file that uses your import entrypoint

# prefect.yaml
name: lif-0197-project
prefect-version: ">=3.0.0"

deployments:
  - name: lif-0197
    entrypoint: lif_0197.main:run
    work_pool:
      name: process-local
    parameters: {}

3) Deploy

prefect deploy

That’s it—the worker will import and run

lif_0197.main:run

directly from your pyenv virtualenv. No local source directory or extra entrypoint file needed. Notes - Ensure

run

is a Prefect flow (decorated with

@flow

); otherwise deployment will fail to locate a Flow. - If you prefer the CLI without a file, you can also do:

prefect deploy lif_0197.main:run -n lif-0197 -p process-local

(run from any small project directory; using a prefect.yaml is the most repeatable approach) Docs - prefect.yaml guide - prefect deploy (CLI)

Short answer: your worker likely isn’t running with the same Python interpreter/venv that has lif_0197 installed, or a different worker picked up the run. Why this happens - Process workers spawn flow runs using the same interpreter they were started with (roughly

sys.executable -m prefect.engine

). If the worker was started outside your pyenv venv (or from a different pyenv version), the flow run process won’t see your installed package and you’ll get ModuleNotFoundError. - If more than one worker is polling that work pool/queue, a different worker (without the package) may have executed the run. How to fix 1) Guarantee the worker uses your pyenv venv - Activate your pyenv venv, then start the worker using the venv’s Python directly:

/path/to/pyenv/versions/<your-venv>/bin/python -m prefect worker start -p process-local

This ensures the worker (and flow runs) use that interpreter. - Alternatively, if you run

prefect worker start

directly, double-check that the CLI is coming from the venv:

which prefect
which python
python -c "import sys; print(sys.executable)"
python -c "import lif_0197, sys; print(sys.executable, lif_0197.__file__)"

All of these should point into the same pyenv venv. 2) Make sure the correct worker is executing your deployment - Use a dedicated work pool for this venv and point your deployment at it. Ensure no other workers are attached to that pool/queue. - In the UI Flow Run details, confirm which worker handled the run. 3) Your deployment entrypoint is fine -

entrypoint: lif_0197.main:run

is valid. No local file is required. The import error is purely an environment/interpreter mismatch. Optional diagnostics inside the flow - Add a quick check to confirm the interpreter used at runtime:

from prefect import flow
import sys, lif_0197

@flow
def run():
    print("Interpreter:", sys.executable)
    print("lif_0197:", lif_0197.__file__)
    ...

If you’re running the worker as a service (systemd, launchd, etc.), ensure the service environment activates pyenv or uses the full venv python in ExecStart:

ExecStart=/path/to/pyenv/versions/<your-venv>/bin/python -m prefect worker start -p process-local

If you want, share: - The exact command you used to start the worker - Output of

which prefect

,

which python

, and the

python -c ...

checks above from that same shell - Your work pool/queue names I can spot-check the mismatch quickly.

Got it — if you’re truly on the exact same venv and still seeing ModuleNotFoundError for an import-style entrypoint, the usual culprits are: 1) A different worker actually executed the run - Even if you started a worker in your venv, another worker on the same pool/queue could have picked it up. - Check the Flow Run details in the UI and confirm the “Worker” name. Make sure only your venv-backed worker is attached to that pool/queue. 2) pyenv/shims or services masking the interpreter - Systemd/launchd or pyenv shims can make it look like you’re in the right venv when you start the worker, but the spawned flow process uses a different interpreter. - The most bulletproof fix is to pin the work pool’s command to your venv’s exact Python so Prefect uses that interpreter for the flow run:

prefect work-pool get-default-base-job-template --type process --file base-job-template.json
# Edit base-job-template.json and set:
# "command": "/absolute/path/to/pyenv/versions/<venv>/bin/python -m prefect.engine"
prefect work-pool create process-local --type process --base-job-template base-job-template.json --overwrite

Then restart the worker:

/absolute/path/to/pyenv/versions/<venv>/bin/python -m prefect worker start -p process-local

3) Editable install (pip -e) pointing to a path not accessible in the worker context - If

lif_0197

was installed in editable mode, the .pth file points to your source directory. If the worker runs under a different user/service/cwd where that path isn’t available, imports can fail even in the “same venv”. - Quick check with the exact interpreter your worker uses:

/absolute/path/to/pyenv/versions/<venv>/bin/python -c "import lif_0197, sys; print(sys.executable, lif_0197.__file__)"
/absolute/path/to/pyenv/versions/<venv>/bin/python -m pip show lif-0197

- If this fails or shows a path that’s not present for the worker, reinstall non-editable:

/absolute/path/to/pyenv/versions/<venv>/bin/pip install --upgrade --force-reinstall lif-0197

(or install from a wheel, not -e) 4) User-scoped install vs service user - If the worker runs under a different OS user (services often do), the package might be installed to the invoking user’s site-packages only. Use the absolute venv Python for both

pip install

and

prefect worker start

as shown above, or ensure the service user sees the same venv. 5) Sanity checks to pinpoint the mismatch - Verify the worker’s Python and import from the same shell where you start the worker:

which prefect
which python
python -c "import sys; print(sys.executable)"
python -c "import lif_0197, sys; print(sys.executable, lif_0197.__file__)"

- In the Flow Run details, confirm the “Worker” that executed it. - If needed, add a tiny diagnostic flow temporarily:

from prefect import flow
import sys
@flow
def debug_flow():
    import lif_0197
    print("Interpreter:", sys.executable)
    print("sys.path[0:5]:", sys.path[:5])
    print("lif_0197:", lif_0197.__file__)

Deploy with

entrypoint: lif_0197.main:run

or run

debug_flow

the same way to surface the runtime interpreter and path. Your

entrypoint: lif_0197.main:run

is valid. This error almost always traces back to which interpreter actually launched the flow run. Pinning the Process work pool’s “command” to your venv’s Python removes all ambiguity. If you share: - How you start the worker (exact command) - The work pool name and the worker shown on the failed run - Output of the two interpreter checks above using your venv’s absolute Python …I can pinpoint the cause quickly. Refs: - Work pools and workers overview: https://www.prefect.io/blog/work-pools-and-workers-deploy-python-securely - Custom/job template details (setting “command”): https://docs-3.prefect.io/v3/advanced/developing-a-custom-worker

Glad you found it — that explains the import error. To make this rock solid going forward, here’s a quick checklist: - Install into the exact venv the worker uses

/absolute/path/to/pyenv/versions/<venv>/bin/python -m pip install --upgrade --force-reinstall <your-package-or-wheel>
# or from your source tree:
# /.../bin/python -m pip install .
# (avoid `-e` unless you really need editable installs)

- Verify with the same interpreter your worker will use

/absolute/path/to/pyenv/versions/<venv>/bin/python -c "import lif_0197, sys; print(sys.executable, lif_0197.__file__)"

- Pin the worker and flow runs to your venv’s Python to avoid any shim/path surprises

prefect work-pool get-default-base-job-template --type process --file base-job-template.json
# Edit base-job-template.json:
# "command": "/absolute/path/to/pyenv/versions/<venv>/bin/python -m prefect.engine"
prefect work-pool create process-local --type process --base-job-template base-job-template.json --overwrite

# Start the worker with the same interpreter
/absolute/path/to/pyenv/versions/<venv>/bin/python -m prefect worker start -p process-local

- Keep your deployment entrypoint as-is

# prefect.yaml
name: lif-0197-project
prefect-version: ">=3.0.0"
deployments:
  - name: lif-0197
    entrypoint: lif_0197.main:run
    work_pool:
      name: process-local

Then:

prefect deploy

Tips - If you were using an editable install (-e), consider switching to a wheel install to avoid path issues (python -m build; pip install dist/*.whl). - When you update the package, you typically do not need to change the deployment; just reinstall the new version into the venv your worker uses. Refs - Work pools and workers overview - Job template details (setting command) If you want, paste the exact pip install command you’re using and how you start the worker — I’ll double-check for any edge cases.