Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
pacc-clearcover-june-12-2023
pacc-may-31-2023
ppcc-may-16-2023
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
j
Jack Prominski
08/16/2022, 1:38 PMHi! I'm looking for some details about specifying which virtual environment a flow/deployment should be executed in.
I was following the virtual environment tutorial but got an error running it on my own, and then found these release notes saying that flow runners are deprecated. The infrastructure docs aren't super clear to me -- can/should a conda/venv be specified in a Process infrastructure block? Would just starting the agent after activating the correct venv work?
The pattern I'm following is this: I have several different projects that each have their own venv, and I would like to orchestrate them all using prefect. Prefect is installed in each venv, and again in a "prefect venv" in which I launch Orion. When I launch deployments/queues/agents, I want to make sure that the flows within are executed in the proper venv. Is this the right way to think about/approach this?
k
Khuyen Tran
08/16/2022, 3:14 PMHi Jack, the tutorial you follow is appreciated.
You can have a Prefect agent running different in different locations. However, the locations can’t be local. I’m not sure if you need to install all packages needed to run the flow in the environment where the agent is in though.
Can you help me with this @Anna Geller @Zanie?
z
Zanie
08/16/2022, 3:42 PMRight now, we’ve removed virtual environment support from the Process infrastructure block. Instead, you can either
• Run your agent in the virtual env you want flows to execute in. Use a work queue per virtual env.
• Modify the command that is used for the flow run entrypoint to use the Python in the virtual environment
:thank-you: 3
There is likely to be a third option in the future, but we’re still designing that experience.
j
Jack Prominski
08/17/2022, 7:00 PMGot it. Running an agent in each virtual environment seems to be working well. Thanks for your help!
o
Omar Sultan
10/16/2022, 4:34 PMHey Jack, just wondering if this is still working for you, I run the agent in the venv but it doesn't seem to be picking up the correct venv
a
Anna Geller
10/16/2022, 8:16 PMCan you try the same using Conda?
z
Zanie
10/16/2022, 8:30 PMThis should work with venv too. Please file a bug report if it's not.
👍 1
(especially if it works with conda)
o
Omar Sultan
10/16/2022, 10:42 PMI will try it with conda and post the bug based on that
Hi Michael and Anna, it seems that the issue was when the agent was started in a venv that had spaces in its name. I created a new venv in a path that doesn't spaces and this worked. Nothing else changed
👍 1
:thank-you: 1
z
Zanie
10/17/2022, 1:39 PMAre you using Windows?
o
Omar Sultan
10/18/2022, 4:21 AMYes I'm using Windows 11
z
Zanie
10/18/2022, 2:39 PMInteresting, thanks! Would you mind opening a bug report?