Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi all! I am new developer and an new to prefect. Currently we are trying to implement prefect for our ETLS at my job. We use vmware and windows sever, we found out the hard way that using prefect with nested virtualization does not work well. So now are moving to prefect cloud, using local agents. Will this work? Is it necessary to run multiple agents for multiple flows or can one agent run multiple flows?

Yes that will work. One agent can run multiple flows for sure. The agent is a lightweight process that just kicks off other processes

Thank sthis worked! Now I am having the problem that when I try to execute another flow, it's looking for code to be in the sa/me directory as the agent. What am I doing wrong

I think <https://discourse.prefect.io/t/when-i-run-my-flow-i-see-an-error-failed-to-load-and-execute-flows-environment-modulenotfounderror-no-module-named-users-username-what-is-happening/33|this> will explain that, even if your error may be different

[22 March 2022 4:48pm]: Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'config\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - prefect: (flow built with \'0.15.13\', currently running with \'1.1.0\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')

I went through the article, and I am still kind of stuck. I am using a config file that it does not appear to like for some reason. I have been trying to figure this out for a few hours.

What is your RunConfig? I think it’s pulling the latest Prefect image

Whereas you registered with 0.15.13, which will cause issues with serialization

You agent then seems to be Prefect 1.1 and you need to either upgrade your registration to 1.1 or downgrade the agent to 0.15.13 so you have matching versions on both registration and execution time

I would recommend registering with Prefect 1.1. Just `pip install -U prefect`

It appears that the problem was in the way that I was registering the flow. I registered the flow in the same directory using only the folder path and project and everything is working.