https://prefect.io logo
t

Todd Lindstrom

05/06/2021, 4:18 PM
Agent Issue. I'm just trying prefect out - using the backend server and running on an EC2 instance m5.large with 8080 and 4200 ports allowed external.  I have the server running, the UI attached, the graphql connected, and I started an agent on the same machine. For me I installed prefect using a conda activation - so to start the agent - i did the conda activation then prefect agent start command. HOWEVER, when I add a flow and run it - it fails on
unexpected error: NameError("name 'prefect' is not defined
This appears to me that it is not running in the condra activation and yes if thats true then the pip install will not be there. Please advise how I can set the python environment of my local agent.
I found this answer: https://stackoverflow.com/questions/63881231/prefect-modulenotfounderror-when-running-from-ui But it assumes that there is a path - and I think this is a different issue of getting the environment correct at agent launch.
k

Kevin Kho

05/06/2021, 4:27 PM
Hi @Todd Lindstrom, I think in general conda activation on the agent is quite tricky. Even if you package everything in a Docker container and use a
docker agent
, it’s hard to activate, but I think it can be done. We normally recommend just using pip inside the image since the container provides isolation already. Do you have another Python installation on the agent?
t

Todd Lindstrom

05/06/2021, 4:51 PM
well I was told in another thread that conda is the best way to ensure you have the right python version ----- I guess that was correct for my mac implementation but on linux I suppose just using the default (which on EC2 is just python 3.7.x which I think is fine)
I'll just do a pip install on the machine and it will likely work.
At some point we would probably want to run agents on separate EC2 instances with a load balancer or something like that - is there any documentation on how to do that outside of kubernetes?
otherwise - if not thank you for the link - that will solve my problem.
k

Kevin Kho

05/06/2021, 5:00 PM
Oh that is true though, conda really easily lets you pin the Python version. I think the agent itself is a lightweight process that I don’t expect to need multiple processes. Is the separate EC2 instances for failure handling?
Yes you may need Docker with the link I provided to use the environment set up
t

Todd Lindstrom

05/06/2021, 5:04 PM
multiple agents is a future thing - but I'm still facing the issue. I removed conda from my instance, and did pip install prefect.
i started up the server, the default tenant was already there, created a project,
added 2 flows and run - it still says
Copy code
State Message:
Unexpected error: NameError("name 'prefect' is not defined")
k

Kevin Kho

05/06/2021, 5:12 PM
What agent and runconfig are you using?
Do you think you have multiple Python installations on the agent?
t

Todd Lindstrom

05/06/2021, 5:14 PM
Kevin - thanks for your help - I am past that error and now its just python debug. Its finding the import prefect now.
k

Kevin Kho

05/06/2021, 5:14 PM
Oh nice job figuring it out!