https://prefect.io logo
#prefect-community
Title
# prefect-community
p

Pim Claessens

08/22/2022, 2:06 PM
Hello, I have a question about prefect 1 (1.3). We are currently running a very outdated instance on our own server (0.15.9) and we are in progress of getting that to the cloud. We are now first trying with version 1.3. But when trying to run a flow I get this error: Failed to load and execute Flow’s environment: FlowStorageError(’An error occurred while unpickling the flow:\n AttributeError(“Can\’t get attribute \‘_make_function\’ on <module \‘cloudpickle.cloudpickle\’ from \‘/usr/local/lib/python3.8/site-packages/cloudpickle/cloudpickle.py\’>“)\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - cloudpickle: (flow built with \‘2.1.0\’, currently running with \‘2.0.0\’)\n - prefect: (flow built with \‘1.3.0\’, currently running with \‘0.15.9\’)\n - python: (flow built with \‘3.10.4\’, currently running with \‘3.8.12\‘)’) I have created it with prefect 1.3 on my mac, so that should work right? But it still seems to somehow mix the two old version. Any idea of where I’m going wrong?
1
k

Khuyen Tran

08/22/2022, 2:50 PM
It seems like you built your flow with both 2.0 and 1.0. Di you happen to run 2.0 before?
p

Pim Claessens

08/22/2022, 2:52 PM
No, I’m currently running 1.3 locally. The one I ran before was 0.15
j

Jeff Hale

08/22/2022, 3:10 PM
Looks like your flows were built in a in your updated environment, but you are trying to run them in the old environment.
Copy code
cloudpickle: (flow built with \'2.1.0\', currently running with \'2.0.0\')\n - 
prefect: (flow built with \'1.3.0\', currently running with \'0.15.9\')\n - 
python: (flow built with \'3.10.4\', currently running with \'3.8.12\')')
What happens if you try to run them in Prefect Cloud (v1)?
p

Pim Claessens

08/22/2022, 3:12 PM
This error is thrown in the prefect cloud
🤔 1
j

Jeff Hale

08/22/2022, 3:41 PM
Is your agent running in a virtual environment that has python 3.10.4, Prefect 1.3.0, and cloudpickle 2.1.0?
p

Pim Claessens

08/22/2022, 3:43 PM
I use: • python:3.10-slim-buster • RUN python3 -m pip install prefect==1.3.0 boto3 “prefect[aws]” inside the docker file of the agent
j

Jeff Hale

08/22/2022, 4:47 PM
It looks like your flows are still running on your server running .15. Looks like you might need to run
prefect backend cloud
to get them on the cloud.
p

Pim Claessens

08/23/2022, 7:11 AM
Well that’s the weird thing. I’m indeed still running the server ECS agent (as well as the new Cloud agent). And I can see in the logs that it is picked up by the cloud agent. We use S3 as the storage for both (and the same bucket, I just overwrite the file). Could that have anything to do with it?
If I don’t specify an image in the ECSRun config I get this error: Failed to load and execute flow run: FlowStorageError(“An error occurred while unpickling the flow:\n TypeError(‘code() takes at most 15 arguments (16 given)’)\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n - python: (flow built with ‘3.10.4’, currently running with ‘3.7.13’)“)
Ah, the error was in the outdated ECSRun image. Somehow the repository did not update when it said it did. Not sure why, but after creating a whole new image it is fixed. Thanks for the help!
👍 1