https://prefect.io
Join Slack
Hi! I'm looking for some details about specifying which virtual environment a flow/deployment should...
# best-practices
j
Hi! I'm looking for some details about specifying which virtual environment a flow/deployment should be executed in. I was following the virtual environment tutorial but got an error running it on my own, and then found these release notes saying that flow runners are deprecated. The infrastructure docs aren't super clear to me -- can/should a conda/venv be specified in a Process infrastructure block? Would just starting the agent after activating the correct venv work? The pattern I'm following is this: I have several different projects that each have their own venv, and I would like to orchestrate them all using prefect. Prefect is installed in each venv, and again in a "prefect venv" in which I launch Orion. When I launch deployments/queues/agents, I want to make sure that the flows within are executed in the proper venv. Is this the right way to think about/approach this?
k
Hi Jack, the tutorial you follow is appreciated. You can have a Prefect agent running different in different locations. However, the locations can’t be local. I’m not sure if you need to install all packages needed to run the flow in the environment where the agent is in though. Can you help me with this @Anna Geller @Zanie?
z
Right now, we’ve removed virtual environment support from the Process infrastructure block. Instead, you can either • Run your agent in the virtual env you want flows to execute in. Use a work queue per virtual env. • Modify the command that is used for the flow run entrypoint to use the Python in the virtual environment
🙏 3
There is likely to be a third option in the future, but we’re still designing that experience.
j
Got it. Running an agent in each virtual environment seems to be working well. Thanks for your help!
o
Hey Jack, just wondering if this is still working for you, I run the agent in the venv but it doesn't seem to be picking up the correct venv
a
Can you try the same using Conda?
z
This should work with venv too. Please file a bug report if it's not.
👍 1
(especially if it works with conda)
o
I will try it with conda and post the bug based on that
Hi Michael and Anna, it seems that the issue was when the agent was started in a venv that had spaces in its name. I created a new venv in a path that doesn't spaces and this worked. Nothing else changed
👍 1
🙏 1
z
Are you using Windows?
o
Yes I'm using Windows 11
z
Interesting, thanks! Would you mind opening a bug report?
23 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by