Hi all! I am new developer and an new to prefect. ...
# prefect-community
t
Hi all! I am new developer and an new to prefect. Currently we are trying to implement prefect for our ETLS at my job. We use vmware and windows sever, we found out the hard way that using prefect with nested virtualization does not work well. So now are moving to prefect cloud, using local agents. Will this work? Is it necessary to run multiple agents for multiple flows or can one agent run multiple flows?
k
Yes that will work. One agent can run multiple flows for sure. The agent is a lightweight process that just kicks off other processes
t
Thank sthis worked! Now I am having the problem that when I try to execute another flow, it's looking for code to be in the sa/me directory as the agent. What am I doing wrong
k
I think this will explain that, even if your error may be different
t
Thank you I will review!
[22 March 2022 448pm] Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'config\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - prefect: (flow built with \'0.15.13\', currently running with \'1.1.0\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
I went through the article, and I am still kind of stuck. I am using a config file that it does not appear to like for some reason. I have been trying to figure this out for a few hours.
k
What is your RunConfig? I think it’s pulling the latest Prefect image
Whereas you registered with 0.15.13, which will cause issues with serialization
t
Sorry, how do I check my run config?
k
Ah what kind of agent are you using?
t
I am using a local agent.
k
You agent then seems to be Prefect 1.1 and you need to either upgrade your registration to 1.1 or downgrade the agent to 0.15.13 so you have matching versions on both registration and execution time
t
Ok let me try that. Thanks!
k
I would recommend registering with Prefect 1.1. Just
pip install -U prefect
t
It appears that the problem was in the way that I was registering the flow. I registered the flow in the same directory using only the folder path and project and everything is working.
k
Oh glad you got it figured out
t
Thank you for your help.