Channels
announcements
ask-marvin
best-practices-coordination-plane
data-ecosystem
data-tricks-and-tips
events
find-a-prefect-job
geo-bay-area
geo-berlin
geo-boston
geo-chicago
geo-colorado
geo-dc
geo-israel
geo-japan
geo-london
geo-nyc
geo-seattle
geo-texas
gratitude
introductions
marvin-in-the-wild
pacc-clearcover-june-12-2023
pacc-may-31-2023
ppcc-may-16-2023
prefect-ai
prefect-aws
prefect-azure
prefect-cloud
prefect-community
prefect-contributors
prefect-dbt
prefect-docker
prefect-gcp
prefect-getting-started
prefect-integrations
prefect-kubernetes
prefect-recipes
prefect-server
prefect-ui
random
show-us-what-you-got
Title
t
Tabari Brannon
03/18/2022, 9:35 PMHi all! I am new developer and an new to prefect. Currently we are trying to implement prefect for our ETLS at my job. We use vmware and windows sever, we found out the hard way that using prefect with nested virtualization does not work well. So now are moving to prefect cloud, using local agents. Will this work? Is it necessary to run multiple agents for multiple flows or can one agent run multiple flows?
k
Kevin Kho
03/18/2022, 9:37 PMYes that will work. One agent can run multiple flows for sure. The agent is a lightweight process that just kicks off other processes
t
Tabari Brannon
03/21/2022, 4:06 PMThank sthis worked! Now I am having the problem that when I try to execute another flow, it's looking for code to be in the sa/me directory as the agent. What am I doing wrong
k
Kevin Kho
03/21/2022, 4:09 PMI think this will explain that, even if your error may be different
t
Tabari Brannon
03/21/2022, 4:10 PMThank you I will review!
[22 March 2022 4:48pm]: Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'config\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - prefect: (flow built with \'0.15.13\', currently running with \'1.1.0\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
I went through the article, and I am still kind of stuck. I am using a config file that it does not appear to like for some reason. I have been trying to figure this out for a few hours.
k
Kevin Kho
03/23/2022, 1:33 AMWhat is your RunConfig? I think it’s pulling the latest Prefect image
Whereas you registered with 0.15.13, which will cause issues with serialization
t
Tabari Brannon
03/23/2022, 4:15 PMSorry, how do I check my run config?
k
Kevin Kho
03/23/2022, 4:16 PMAh what kind of agent are you using?
t
Tabari Brannon
03/23/2022, 4:17 PMI am using a local agent.
k
Kevin Kho
03/23/2022, 4:19 PMYou agent then seems to be Prefect 1.1 and you need to either upgrade your registration to 1.1 or downgrade the agent to 0.15.13 so you have matching versions on both registration and execution time
t
Tabari Brannon
03/23/2022, 4:20 PMOk let me try that. Thanks!
k
Kevin Kho
03/23/2022, 4:21 PMI would recommend registering with Prefect 1.1. Just
pip install -U prefectt
Tabari Brannon
03/28/2022, 11:25 PMIt appears that the problem was in the way that I was registering the flow. I registered the flow in the same directory using only the folder path and project and everything is working.
k
Kevin Kho
03/28/2022, 11:38 PMOh glad you got it figured out
t
Tabari Brannon
03/28/2022, 11:39 PMThank you for your help.