Channels
#prefect-community
Title
# prefect-community
t
Tabari Brannon
03/18/2022, 9:35 PMHi all! I am new developer and an new to prefect. Currently we are trying to implement prefect for our ETLS at my job. We use vmware and windows sever, we found out the hard way that using prefect with nested virtualization does not work well. So now are moving to prefect cloud, using local agents. Will this work? Is it necessary to run multiple agents for multiple flows or can one agent run multiple flows?
k
Kevin Kho
03/18/2022, 9:37 PM
Yes that will work. One agent can run multiple flows for sure. The agent is a lightweight process that just kicks off other processes
t
Tabari Brannon
03/21/2022, 4:06 PMThank sthis worked! Now I am having the problem that when I try to execute another flow, it's looking for code to be in the sa/me directory as the agent. What am I doing wrong
k
Kevin Kho
03/21/2022, 4:09 PM
I think this will explain that, even if your error may be different
t
Tabari Brannon
03/21/2022, 4:10 PMThank you I will review!
[22 March 2022 448pm] Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'config\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - prefect: (flow built with \'0.15.13\', currently running with \'1.1.0\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
I went through the article, and I am still kind of stuck. I am using a config file that it does not appear to like for some reason. I have been trying to figure this out for a few hours.
k
Kevin Kho
03/23/2022, 1:33 AM
What is your RunConfig? I think it’s pulling the latest Prefect image
Whereas you registered with 0.15.13, which will cause issues with serialization
t
Tabari Brannon
03/23/2022, 4:15 PMSorry, how do I check my run config?
k
Kevin Kho
03/23/2022, 4:16 PM
Ah what kind of agent are you using?
t
Tabari Brannon
03/23/2022, 4:17 PMI am using a local agent.
k
Kevin Kho
03/23/2022, 4:19 PM
You agent then seems to be Prefect 1.1 and you need to either upgrade your registration to 1.1 or downgrade the agent to 0.15.13 so you have matching versions on both registration and execution time
t
Tabari Brannon
03/23/2022, 4:20 PMOk let me try that. Thanks!
k
Kevin Kho
03/23/2022, 4:21 PM
I would recommend registering with Prefect 1.1. Just
pip install -U prefectt
Tabari Brannon
03/28/2022, 11:25 PMIt appears that the problem was in the way that I was registering the flow. I registered the flow in the same directory using only the folder path and project and everything is working.
k
Kevin Kho
03/28/2022, 11:38 PM
Oh glad you got it figured out
t
Tabari Brannon
03/28/2022, 11:39 PMThank you for your help.
8 Views