https://prefect.io
Join Slack
Hello. When I run a shell in python in docker using prefect, I get an error /bin/sh: 1: yc: not fou...
# prefect-cloud
м
Hello. When I run a shell in python in docker using prefect, I get an error /bin/sh: 1: yc: not found. If I try to run script in docker container it works correctly. For example:
Copy code
>>> import subprocess
>>> subprocess.run('yc --help',  shell=True, capture_output=True, text=True)
j
I can help, but I'd need to reproduce this. Could you please share the dockerfile? Also how is the flow run i.e. fetched from the queue or some other way?
м
Copy code
FROM python:3.9

RUN apt-get update && apt-get install -y \
                    openssl \
                    libssl-dev

RUN update-ca-certificates

RUN apt-get install -y python3-opencv \
                       python3-dev \
                       default-mysql-client \
                       freetds-dev \
                       uuid \
                       curl

RUN curl <https://storage.yandexcloud.net/yandexcloud-yc/install.sh> | bash -s -- -a



ENV TINI_VERSION v0.19.0
ADD <https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini> /usr/local/bin/tini
RUN chmod +x /usr/local/bin/tini

COPY docker/prefect/requirements.txt /app/requirements.txt
RUN pip install 'prefect[git]==1.2.4' && \
    python3.9 -m pip install -r /app/requirements.txt

#COPY config.toml /root/.prefect/config.toml
COPY docker/prefect/freetds.conf /etc/freetds/freetds.conf

COPY replicator /tmp/replicator
ENV PYTHONPATH /tmp

ENTRYPOINT ["tini", "--"]
Flow run using quick run
j
Ahhh.. it is using prefect 1.2.4 and I only have Prefect cloud 2 to test it with. Pretty sure Prefect 1 agent will not be able to talk with Prefect Cloud 2 API. I will see if I can reproduce this without initiating the flow run via Cloud / Orion.
I was able to reproduce this without prefect. You need to provide the full path to the 
yc
bin In your flow code, run
Copy code
subprocess.run('/root/yandex-cloud/bin/yc --help',  shell=True, capture_output=True, text=True)
instead of
Copy code
subprocess.run('yc --help',  shell=True, capture_output=True, text=True)
This happens because when you run this within the container by first initing the shell, the PATH containing 
yc
gets exported. When this is run outside the shell (i.e. prefect agent process), 
yc
is not in the PATH
м
THX, It works. 🙏
How to add YC in PATH?
j
you could try something like this 🤷 I've not tested it
Copy code
import os

os.environ['PATH'] += ':/root/yandex-cloud/bin/'

subprocess.run('yc --help',  shell=True, capture_output=True, text=True)
м
Thx, it work 2
18 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by