https://prefect.io
Join Slack
Anybody ever seen this before? I'm running a local agent in a conda env with python 3.8.13, and it's...
# ask-community
j
Anybody ever seen this before? I'm running a local agent in a conda env with python 3.8.13, and it's the same one I registered the flow on. The unpickling seems to be the issue, but I'm really confused about the other python version mentioned in this error msg.
Copy code
Failed to load and execute flow run: FlowStorageError('An error occurred while unpickling the flow:\n  ModuleNotFoundError("No module named \'snowflake\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n  - python: (flow built with \'3.8.13\', currently running with \'3.8.10\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
I created another flow that logged the environment, then ran that one via the UI too. It reported that it was running in the right env. Where is this 3.8.10 version coming from? And what's the solution for the issue of not being able to pull up snowflake? I didn't have this issue with my last prefect setup, so I'm just kinda scratching my head here
✅ 1
a
in fact I'm seeing this almost every week asked here in the community 😂 you're not alone Jacob
j
But isn't it strange that it's doing this with snowflake?
I installed that one with pip
Well, it's running at the moment. I kinda hacked it with an import of the snowflake connector in the task. Seems to have gotten beyond that initial issue
ouch. It failed at the 'import snowflake' stage. It just can't find snowflake, even though I installed it in the normal way
a
I recommend going through the Discourse thread, it includes many options you can try
building a package, modifying pythonpath, adding the path to the agent, ...
j
I think that's the part where I'm a bit confused
I read through the doc, and I see how it makes sense if you have your own package being used. The tricky part for me is that I installed via pip, so I thought it would be picked up by prefect. Do I have to specify the path in the conda env where the packages are installed?
Oh, I'm getting closer here:
Copy code
Exception: Version mismatch: this is the 'cffi' package version 1.15.0, located in '/home/ubuntu/anaconda3/envs/prefect-env/lib/python3.8/site-packages/cffi/api.py'.  When we import the top-level '_cffi_backend' extension module, we get version 1.14.0, located in '/usr/lib/python3/dist-packages/_cffi_backend.cpython-38-x86_64-linux-gnu.so'.  The two versions should be equal; check your installation.
Oh man that was brutal. I scoured the internet once I got the new error msg. Turns out I needed to uninstall and reinstall prefect
That fixed my issue. It was looking over to some other ".so" config file I was unfamiliar with (shown above). It wiped the slate clean once I reinstalled prefect. I also did the reinstall with conda instead of pip. Not sure if that had an impact, but I'm up and running now 🙂 Thanks for the help - I wouldn't have gotten to the next error message without it
🙌 1
a
thanks for the update and nice work finding the solution!
8 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by