Any help would be greatly appreciated! Looking for...
# ask-community
r
Any help would be greatly appreciated! Looking for help on how to run my deployment/flow in a virtual environment that's deployed to the cloud.
j
What are you running your python in?
Anaconda?
r
no just through normal python -m venv
j
Best way to do this on Mac or PC is to use Anaconda Navigator, make a new virtualenv using a recent version of python, then open that virtualenv and do your pip install of prefect there
Which OS are you using here?
r
wouldn't that have just one virtual environment for all prefect flows?
windows
j
Oh yeah I'd use anaconda
r
I was hoping to have a virtual environment for each flow
j
That's a lot of virtual environments, potentially. Do you have conflicting python libraries needed for each flow?
That may not be a handy setup unless you have a known issue like that
j
If you want to do this you can update your command on your deployment to use the python executable from your env
1 sec i can get you an example
j
Ah a proper prefect expert is here. I'll step back now
😂 1
r
not at the moment, but as we build more flows I'd like to have the option for a newer flow to use the latest version of a module while not needing to test/upgrade a previous flow that's using an older version
✅ 1
thanks for helping @Jacob Bedard!
j
The default command on a deployment is
python -m prefect.engine
which is what will kick off your flow run, get the deployment, execute the pull steps, etc. you'll need to use the python from your existing virtualenv. For example you can change the command to
/Users/jakekaplan/opt/anaconda3/envs/py310/bin/python3 -m prefect.engine
r
thanks Jake, where do I enter this override command
is it in the deployment? Any way I can specify this override in the prefect.yaml?
is it here?
oh nvm I think it's on the work-pool
If I were to do that it will still only be one virtual environment per work-pool
but I'd like to have a virtual environment per deployment/flow
j
so it can be both. the work pool generates a template. if you want to hard code the
command
on all deployments that use that work pool, put it on the work pool. if you want to do it on the deployment, you can put in infra overrides
command: ...
r
oh cool, I'll try it on the deployment
j
by default
command
is a temptable value (this is a random work pool, just to illustrate)
r
do you have an example of what I should put in the deployment infra override in the UI?
j
sure it would look like
Copy code
{
  "command": "/Users/jakekaplan/opt/anaconda3/envs/py310/bin/python3 -m prefect.engine"
}
except replace it with the path to the python executable from the virutal env you would like to use
r
thank you!! I'll give it a try and let you and the community know how it goes
it works!! Thank you so much 🙂
🙌 1
One more question.. I promise. How would I do this in the prefect.yaml file?
this didn't work
sorry @Jake Kaplan, any luck?
I've tried all combinations and nothing is working. I'm just trying to set infra overrides from prefect.yaml but nothing is working