jack
11/04/2021, 1:47 AM/bin/sh: prefect: command not found
The Dockerfile used to build the image based on amazonlinux:2 runs pip install prefect[aws]
, and when I run the image locally prefect
is in the path.Kevin Kho
11/04/2021, 1:49 AMjack
11/04/2021, 2:22 AMFROM amazonlinux:2.0.20211005.0
WORKDIR /project-name
# Add files required for provisioning
ADD requirements.txt /project-name/
ADD bin /project-name/bin/
# Install things
RUN yum update -y
RUN amazon-linux-extras enable python3.8
RUN yum install -y mlocate python38
RUN updatedb
RUN python3.8 -m pip install --upgrade pip
RUN pip3.8 install -r requirements.txt
# Add remaining required files
ADD dispatcher.py /project-name/
ADD models /project-name/models/
CMD ["python3.8", "dispatcher.py"]
command
is set to
["python3.8", "dispatcher.py"]
but the ECS task instead appears to run
["/bin/sh","-c","prefect execute flow-run"]
Kevin Kho
11/04/2021, 2:53 AMprefect execute flow-run
. So this command is used internally to run it on the image. This is working as intended.gcc
. How did you install it? This is what I’m trying:
FROM amazonlinux:2.0.20211005.0
# Install things
RUN yum update -y
RUN amazon-linux-extras enable python3.8
RUN yum install -y mlocate python38
RUN yum install gcc
RUN python3.8 -m pip install --upgrade pip
RUN pip3.8 install prefect
dispatcher.py
? You shouldn’t need it to run a flow on ECSjack
11/04/2021, 3:00 AMRUN yum install gcc
from the Dockerfile you pasted,it builds on my machine.Kevin Kho
11/04/2021, 3:03 AM#9 7.622 unable to execute 'gcc': No such file or directory
#9 7.622 C compiler or Python headers are not installed on this system. Try to run:
#9 7.622 sudo yum install gcc python3-devel
#9 7.622 error: command 'gcc' failed with exit status 1
And I tried installing python3-devel
and gcc
and it’s not working for me. What is your Docker version? (though I think this shouldn’t matter)jack
11/04/2021, 3:04 AMjd /tmp/td docker --version
Docker version 20.10.8, build 3967b7d
dispatcher.py
models/
model_1.py
model_2.py
The flow is defined in dispatcher.py.
The flow's tasks use models.model_1 and models.model_2.
I assumed that dispatcher.py and the models directory needed to live on the image that runs the flow. If that is not the case, then where would this code live?Kevin Kho
11/04/2021, 3:11 AMCMD ["python3.8", "dispatcher.py"]
as part of the Dockerfile because Prefect goes in and runs the flow code for you by adding that command to the container that you are seeing.jack
11/04/2021, 3:12 AMCMD
statement then.Kevin Kho
11/04/2021, 3:13 AMprefect
command is not found. But if you exec into the container, you can use it right?prefect
command explicitly to the PATH of your container like this ?jack
11/04/2021, 3:16 AM$ docker run -it --rm my-image bash
bash-4.2# prefect version
0.15.7
sbash-4.2# sh
sh-4.2# prefect version
0.15.7
Kevin Kho
11/04/2021, 3:21 AMjack
11/04/2021, 3:23 AM/usr/local/bin
is already in $PATH when I run the image locally. I'm not sure if $PATH is different when run via ECSKevin Kho
11/04/2021, 3:24 AMjack
11/04/2021, 3:25 AMAnna Geller
11/04/2021, 9:58 AMamazonlinux:2.0.20211005.0
? Prefect base images are optimized to work with Prefect flows, and they are all based on Python slim base image. This way, your image should be smaller in size and a bit faster to spin up on ECS than installing Python from Amazon Extras:
RUN amazon-linux-extras enable python3.8
jack
11/04/2021, 1:56 PMprefecthq/prefect
image and requires no external python modules.
ECS errors out with "*Stopped reason:*
*Essential container in task exited*"Anna Geller
11/04/2021, 2:05 PMjack
11/04/2021, 3:00 PM