Hello there! :slightly_smiling_face: I'd apprecia...
# ask-community
l
Hello there! 🙂 I'd appreciate any help, as I'm truly stuck. I've locally developed a flow on Jupyter Notebook (Python). Agent - Vertex Storage - GCP run_config - VertexRun Executer - LocalExecutor In VertexRun I've defined an image, which I've locally created on my CMD (Based on a simple Docker file, based on prefect:latest + run pip installs that are relevant for the specific flow). I've pushed this image to my repository in docker hub. Following what I've read here, I understood that on my script I should login to my docker account before activating the agent, and so I've done. One of the packages I've set to install on my image is "Pandas". Yet, when trying to run the flow through the UI I get this error: "Failed to load and execute Flow's environment: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'pandas\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - cloudpickle: (flow built with \'1.3.0\', currently running with \'2.0.0\')\n - prefect: (flow built with \'0.15.9\', currently running with \'0.15.10\')\n - python: (flow built with \'3.7.6\', currently running with \'3.7.12\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')" What do I miss? Thanks in advance! 🍪
a
@Liri Rozenthal this is the classic version mismatch error - probably one of the most common errors so you’re not alone 🙂 you registered (and therefore serialized) your flow using Prefect 0.15.9 and cloudpickle 1.3, but your agent is deployed using the latest version of Prefect 0.15.10 and cloudpickle 2.0. Same for Python version mismatch. The easiest way to solve is is to create a virtual environment identical with your agent environment and re-register your flow in this environment
thanks for the cookie! 😄
😁 1
l
Thanks!
Hi again, after modifing the cloudpickle and prefect packages identical to the agent's version, I'm having issues defining my python version. For Anaconda users, the last 3.7 version can be set is 3.7.11. Version 3.7.12 is just not downloadable. That is why I'm getting this error: Failed to load and execute Flow's environment: FlowStorageError('An error occurred while unpickling the flow:\n ModuleNotFoundError("No module named \'pandas\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - python: (flow built with \'3.7.11\', currently running with \'3.7.12\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.') BTW,
3.7.9
 was the last binary release for 
3.7
. Now 
3.7
 is only provided security fixes which are source-only releases. 
3.7.12
 is a security fix release so it doesn't contain any new feature, optimisation or improvement. How can I modify the agent's python version?
a
you could create a virtual environment with a specific version, e.g.
Copy code
conda create --name prefectAgent python=3.7
conda activate prefectAgent
pip install prefect==0.15.10 # or whichever version you need
alternatively, you could switch to Script storage rather than using pickle storage: https://docs.prefect.io/orchestration/flow_config/storage.html#pickle-vs-script-based-storage
l
Thanks Anna, However setting version 3.7 on conda is not enough, as you can see in the error log I've shared, I need specifically version 3.7.12, which can't be downloaded. Isn't there anyway to modify the agent's python version? maybe by changing the image I use?
a
Correct, I can confirm that - see image below 1. Why do you need specifically 3.7.12 on your agent? Could you instead deploy your agent e.g. in 3.8 and also build an equivalent virtual environment from which you register your flow that also has e.g. 3.8? This 3.8 is just an example, the point is: creating a new fresh virtual env for both agent and registration environment 2. Perhaps you can instead use Script storage? If I understood correctly, you use a local agent correct? If you register your flows using Prefect CLI, then script storage is the default, e.g.:
Copy code
prefect register --project yourproject -p yourflow.py
again, this doc explains it more: https://docs.prefect.io/orchestration/flow_config/storage.html#pickle-vs-script-based-storage I can definitely recommend script storage when possible because it makes your flows more compatible with various python and prefect versions as opposed to pickle