https://prefect.io logo
#prefect-community
Title
# prefect-community
t

Tabari Brannon

03/18/2022, 9:35 PM
Hi all! I am new developer and an new to prefect. Currently we are trying to implement prefect for our ETLS at my job. We use vmware and windows sever, we found out the hard way that using prefect with nested virtualization does not work well. So now are moving to prefect cloud, using local agents. Will this work? Is it necessary to run multiple agents for multiple flows or can one agent run multiple flows?
k

Kevin Kho

03/18/2022, 9:37 PM
Yes that will work. One agent can run multiple flows for sure. The agent is a lightweight process that just kicks off other processes
t

Tabari Brannon

03/21/2022, 4:06 PM
Thank sthis worked! Now I am having the problem that when I try to execute another flow, it's looking for code to be in the sa/me directory as the agent. What am I doing wrong
k

Kevin Kho

03/21/2022, 4:09 PM
I think this will explain that, even if your error may be different
t

Tabari Brannon

03/21/2022, 4:10 PM
Thank you I will review!
[22 March 2022 448pm] Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'config\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - prefect: (flow built with \'0.15.13\', currently running with \'1.1.0\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
I went through the article, and I am still kind of stuck. I am using a config file that it does not appear to like for some reason. I have been trying to figure this out for a few hours.
k

Kevin Kho

03/23/2022, 1:33 AM
What is your RunConfig? I think it’s pulling the latest Prefect image
Whereas you registered with 0.15.13, which will cause issues with serialization
t

Tabari Brannon

03/23/2022, 4:15 PM
Sorry, how do I check my run config?
k

Kevin Kho

03/23/2022, 4:16 PM
Ah what kind of agent are you using?
t

Tabari Brannon

03/23/2022, 4:17 PM
I am using a local agent.
k

Kevin Kho

03/23/2022, 4:19 PM
You agent then seems to be Prefect 1.1 and you need to either upgrade your registration to 1.1 or downgrade the agent to 0.15.13 so you have matching versions on both registration and execution time
t

Tabari Brannon

03/23/2022, 4:20 PM
Ok let me try that. Thanks!
k

Kevin Kho

03/23/2022, 4:21 PM
I would recommend registering with Prefect 1.1. Just
pip install -U prefect
t

Tabari Brannon

03/28/2022, 11:25 PM
It appears that the problem was in the way that I was registering the flow. I registered the flow in the same directory using only the folder path and project and everything is working.
k

Kevin Kho

03/28/2022, 11:38 PM
Oh glad you got it figured out
t

Tabari Brannon

03/28/2022, 11:39 PM
Thank you for your help.
8 Views