https://prefect.io logo
Title
м

Михаил Бараковский

03/21/2023, 8:40 AM
Hello. When I run a shell in python in docker using prefect, I get an error /bin/sh: 1: yc: not found. If I try to run script in docker container it works correctly. For example:
>>> import subprocess
>>> subprocess.run('yc --help',  shell=True, capture_output=True, text=True)
j

jpuris

03/21/2023, 8:44 AM
I can help, but I'd need to reproduce this. Could you please share the dockerfile? Also how is the flow run i.e. fetched from the queue or some other way?
м

Михаил Бараковский

03/21/2023, 8:48 AM
FROM python:3.9

RUN apt-get update && apt-get install -y \
                    openssl \
                    libssl-dev

RUN update-ca-certificates

RUN apt-get install -y python3-opencv \
                       python3-dev \
                       default-mysql-client \
                       freetds-dev \
                       uuid \
                       curl

RUN curl <https://storage.yandexcloud.net/yandexcloud-yc/install.sh> | bash -s -- -a



ENV TINI_VERSION v0.19.0
ADD <https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini> /usr/local/bin/tini
RUN chmod +x /usr/local/bin/tini

COPY docker/prefect/requirements.txt /app/requirements.txt
RUN pip install 'prefect[git]==1.2.4' && \
    python3.9 -m pip install -r /app/requirements.txt

#COPY config.toml /root/.prefect/config.toml
COPY docker/prefect/freetds.conf /etc/freetds/freetds.conf

COPY replicator /tmp/replicator
ENV PYTHONPATH /tmp

ENTRYPOINT ["tini", "--"]
Flow run using quick run
j

jpuris

03/21/2023, 8:55 AM
Ahhh.. it is using prefect 1.2.4 and I only have Prefect cloud 2 to test it with. Pretty sure Prefect 1 agent will not be able to talk with Prefect Cloud 2 API. I will see if I can reproduce this without initiating the flow run via Cloud / Orion.
I was able to reproduce this without prefect. You need to provide the full path to the
yc
bin In your flow code, run
subprocess.run('/root/yandex-cloud/bin/yc --help',  shell=True, capture_output=True, text=True)
instead of
subprocess.run('yc --help',  shell=True, capture_output=True, text=True)
This happens because when you run this within the container by first initing the shell, the PATH containing
yc
gets exported. When this is run outside the shell (i.e. prefect agent process),
yc
is not in the PATH
м

Михаил Бараковский

03/21/2023, 11:17 AM
THX, It works. 🙏
How to add YC in PATH?
j

jpuris

03/28/2023, 3:27 PM
you could try something like this 🤷 I've not tested it
import os

os.environ['PATH'] += ':/root/yandex-cloud/bin/'

subprocess.run('yc --help',  shell=True, capture_output=True, text=True)
м

Михаил Бараковский

03/28/2023, 3:43 PM
Thx, it work 2