Hi folks, quick question about the Prefect agent and virtual environments. I have a machine with two registered flows, each flow runs inside its own virtual environment (I'm using

venv

). If I have two separate agents running on the machine, each started from within their own venv, they can both execute their own flows successfully. As soon as I try to cut it down to one agent listening for both flows, the agent fails to execute any flows which live inside another venv (

Failed to load and execute Flow's environment: ModuleNotFoundError

). I'd like to keep the separated venvs if possible and just have one agent running, which would execute each of the flows within the context of their own venv. Is this possible, or would I need to bite the bullet and have one venv for both flows?

this seems somewhat tricky as it has to deal with the prefect serialization internals. prefect serialize the flows using

cloudpickle

which contains references to the modules it imports. You can make the flows itself only depend on prefect imports and use local imports in the tasks. During task execution you may be able to have more flexibility to modify the runtime. Imagine something like this

@task(env='foo')
def do_foo():
    import foo
    foo.foo()

@task(env='bar')
def do_bar():
    import bar
    bar.bar()