Hi Im am using docker storage and I am trying to m...
# ask-community
d
Hi Im am using docker storage and I am trying to make a docker image with my flow. I am getting an error. Any idea about how I can solve this?
Copy code
Step 15/15 : RUN python /opt/prefect/healthcheck.py '["flow-data_transformation.py"]' '(3, 10)'
 ---> Running in 2853f6886cd5
  File "/opt/prefect/healthcheck.py", line 17
    def system_check(python_version: str):
                                   ^
SyntaxError: invalid syntax
the image is using
node:16
as base and has
python3
installed
a
I have two options for you: a thorough one and a hack - choose wisely 😄 #1 The thorough one - there is a reason why the health check is failing and that's probably due to some Python version mismatch between your local flow registration environment and the Python version used in your image, or some things in your flow not being serializable. This docs page dives deeper into how you can debug this. #2 The hack: you can ignore the healthcheck
Copy code
flow.storage = Docker(ignore_healthchecks=True)
To give you some background on option #1 - by default, Prefect pickles your flow using cloudpickle during the flow registration (build time). The error indicates python version mismatch which may have the effect that your flow gets serialized incorrectly and cannot be deserialized at runtime. If you want to avoid pickling your flow completely, you can copy your flow file into your Docker image within your Dockerfile and use this approach with
stored_as_script=True
.
d
The health check is not really failing. It has a "syntax error" within itself. When calling
python
in the container it runs
Python 2.7.16
. I am guessing the health check needs python 3.5+ ? 🤔 Is there a way I can have prefect Docker builder call
python3
instead?
a
Can you share your Dockerfile? Two notes on that: 1. If you go to the setup.py in the Prefect repo for any release, you can see the minimal required Python version that is supported https://github.com/PrefectHQ/prefect/blob/master/setup.py#L144 2. You don't have to use the Prefect Docker build process even if you use Docker storage. The above link I shared with script storage requires you to build the image yourself and just provide it as a reference - I think especially in use cases such as yours, it can be beneficial to build the Docker image yourself so that you have full control over the process. The Docker builder is more for simple use cases where you may just need a handful of packages to be installed and you don't have any complex dependencies (and given the npm and python 2.7, you certainly do)
d
Ah oki I see. never mind
node.js
. I do not have a dependency on
python2
. My deps are
python3
and some
pip
packages.
Copy code
FROM node:16

COPY ./src/flows/data_transformation ./
COPY ./requirements.txt ./requirements.txt

RUN apt-get update
RUN apt-get install -y python3-pip

RUN pip3 install --no-cache --upgrade pip setuptools
RUN pip3 install -r requirements.txt
a
I see. I'd recommend building your own image then. The repo I shared provides an example Dockerfile. Btw, It would be beneficial to move your COPY commands below to speed up your builds. This way, the previous Docker layers are cached and when your file dependencies change, you can just copy them over without having to reinstall your code dependencies (because they are cached).
Copy code
FROM node:16

RUN apt-get update
RUN apt-get install -y python3-pip

RUN pip3 install --no-cache --upgrade pip setuptools
COPY ./requirements.txt ./requirements.txt
RUN pip3 install -r requirements.txt

COPY ./src/flows/data_transformation ./
d
ah nice! thanks for the advice:)
👍 1