Hi folks, quick question about the Prefect agent a...
# prefect-community
Hi folks, quick question about the Prefect agent and virtual environments. I have a machine with two registered flows, each flow runs inside its own virtual environment (I'm using
). If I have two separate agents running on the machine, each started from within their own venv, they can both execute their own flows successfully. As soon as I try to cut it down to one agent listening for both flows, the agent fails to execute any flows which live inside another venv (
Failed to load and execute Flow's environment: ModuleNotFoundError
). I'd like to keep the separated venvs if possible and just have one agent running, which would execute each of the flows within the context of their own venv. Is this possible, or would I need to bite the bullet and have one venv for both flows?
this seems somewhat tricky as it has to deal with the prefect serialization internals. prefect serialize the flows using
which contains references to the modules it imports. You can make the flows itself only depend on prefect imports and use local imports in the tasks. During task execution you may be able to have more flexibility to modify the runtime. Imagine something like this
Copy code
def do_foo():
    import foo

def do_bar():
    import bar
I have somewhat similar requirements but I use docker containers via subprocess as I mainly process files