https://prefect.io logo
Powered by
Title
r

Riley Hun

08/12/2020, 4:33 PM
Happy Wednesday - I'm encountering issues with several of the docker images. They seem to be related to the healthcheck.py file. I've tried 
prefecthq/prefect:0.12.1-python3.7
and 
prefecthq/prefect:0.12.5-python3.7
, which resulted in the following error: 
AttributeError: module 'types' has no attribute 'CellType'
I've also tried 
FROM prefecthq/prefect:0.11.5-python3.7
, which resulted in this error: 
ModuleNotFoundError: No module named 'prefect.core.parameter'
My full docker file looks something like this: 
FROM prefecthq/prefect:0.12.1-python3.7

ADD ./alternative_data_pipelines /root/lib/alternative_data_pipelines
ADD key.json /creds/key.json
ADD feature_lib_creds.json /creds/feature_lib_creds.json

RUN pip install /root/lib/alternative_data_pipelines

ENV PYTHONPATH ${PYTHONPATH}:/root/lib/

RUN apt-get update \
&& apt-get -qq -y install curl

# Downloading gcloud package
RUN curl <https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz> > /tmp/google-cloud-sdk.tar.gz

# Installing the package
RUN mkdir -p /usr/local/gcloud \
  && tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
  && /usr/local/gcloud/google-cloud-sdk/install.sh

# Adding the package path to local
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin
c

Chris White

08/12/2020, 4:35 PM
Hi Riley - the images themselves are fine, those errors are most likely arising because you are registering a Flow from a different version of prefect / python
👍 1
r

Riley Hun

08/12/2020, 4:36 PM
Ahh got it. Will check now - thanks!
j

james.lamb

08/12/2020, 9:24 PM
I just saw this same behavior, and I don't think it's related to Docker storage. I'm using: • storage: 
Webhook
• agent: 
KubernetesAgent
• environment: 
KubernetesJobEnvironment
• executor: 
DaskExecutor
• prefect version: 0.13.2 And I got this same error during the job created by 
KubernetesAgent
to run a flow. This issue (https://github.com/mlflow/mlflow/issues/2429) makes me think that the problem is incompatible versions of 
cloudpickle
between the environment where 
flow.register(build=True)
was called and the one where the flow runs. I have 
cloudpickle
1.5.0 in the environment where I registered the flow and 
cloudpickle
1.4.1 in the image where it's being run. This is extra tough because the most recent Dask 
distributed
has a floor of 1.5.0 
ERROR: distributed 2.22.0 has requirement cloudpickle>=1.5.0, but you'll have cloudpickle 1.4.1 which is incompatible.
I tried downgrading 
cloudpickle
to 1.4.1 and re-registering my flow (with 
build=True
), and then re-ran it. The new run was successful.
I kind of wrote that comment one-step at a time while experimenting. So let me summarize:
The error 
AttributeError: module 'types' has no attribute 'CellType'
is the result of different minor versions of 
cloudpickle
between the environment where a flow runs and the environment where it was registered.
c

Chris White

08/12/2020, 9:35 PM
hmmm that’s interesting and a great catch — we should definitely document this traceback message in github but I wonder if we can make this error more informative
:thank-you: 1
@Marvin archive “AttributeError: module ‘types’ has no attribute ‘CellType’”
m

Marvin

08/12/2020, 9:35 PM
https://github.com/PrefectHQ/prefect/issues/3148
#prefect-communityJoin Slack
Powered by