https://prefect.io logo
j

jack

01/04/2021, 10:24 PM
Hey all! I am still having issues running any newly registered flows from Prefect Cloud (all prefect versions - even with simple 'hello world' prefect flows that run on my local agent just fine) for the past weeks. I have checked if my kubernetes agents were still alive by running previously registered flows and they all worked fine. I am getting either of these two error messages:
{'type': ['Unsupported value: UniversalRun']}
{'_schema': 'Invalid data type: None'}
Not really able to run any newly built flows, so massive impact to my work. Did anyone have this issue and able to resolve it?
k

Kyle Moon-Wright

01/04/2021, 10:52 PM
Hey @jack, You’ll likely need to reinstall your Agents with the latest version of Prefect if your newly registered flows are using a later version. Agents will be able to pick up flows of lesser or equal versions of Prefect. If that doesn’t work though, let us know!
j

jack

01/05/2021, 1:26 AM
Hey @Kyle Moon-Wright, thanks for that! I reinstalled my agents with the latest prefect version and my old flows still work (thankfully). Then i registered my new flows and they gave me this error. Could you help me understand the issue here?
Copy code
14:22:37
ERROR
execute flow-run
Failed to load and execute Flow's environment: ValueError('unsupported pickle protocol: 5')

14:22:47
ERROR
k8s-infra
Pod prefect-job-c30cdca2-x9qzh failed.
	Container 'flow' state: terminated
		Exit Code:: 1
		Reason: Error
k

Kyle Moon-Wright

01/05/2021, 1:37 AM
Hmm, that one is tougher but is likely due to a Python mismatch of some sort - protocol 5 was introduced as the default in Python 3.8. This user encountered something similar as seen in this thread.
j

jack

01/05/2021, 3:53 AM
@Kyle Moon-Wright Now, the flow seems to run when I register the flow with prefect version up to
0.13.19
. The moment I register flow with Prefect version
0.14.0/0.14.1
, it throws me that
unsupported pickle protocol: 5
error. I am not quite sure if I understood that thread properly but the python version I run on my local machine is `3.8.6`How and where/what am I checking for the remote python version?
The docker image is push with the latest image
docker pull prefecthq/prefect:latest-python3.8
and it's a kubernetes agent
k

Kyle Moon-Wright

01/05/2021, 5:03 PM
Hmm, tough to say where the discrepancy is but in general I would recommend pulling latest with
docker pull prefecthq/prefect:latest
as long as you can ensure your newly registered flows and your Agent are running the most recent versions (0.14.1).
j

jack

01/06/2021, 2:37 AM
I have done so and re-pushed the image to the ECR repository. Then I reinstalled the EKS cluster agent using the 0.14.1 prefect version (which shows the prefect version on my agent tab in Prefect UI). Still not really able to run any flows at 0.14.0+ for the same unsupported pickle protocol issue. FYI - I used this medium blog to setup our agent https://medium.com/the-prefect-blog/seamless-move-from-local-to-aws-kubernetes-cluster-with-prefect-f263a4573c56
3 Views