https://prefect.io
Join Slack
what should the entrypoint be for custom docker images in a `KubernetesRun`?
# ask-community
z
what should the entrypoint be for custom docker images in a 
KubernetesRun
?
stuck on the below error. I use poetry as my package manager, so thinking maybe I’m using the incorrect entrypoint
Copy code
Failed to load and execute Flow's environment: ImportError('Unable to import Github, please ensure you have installed the github extra')
Dockerfile below
Copy code
FROM python:3.7.4-slim

# libpq-dev is required to install psycopg2-binary
RUN apt-get update && \
    apt-get install libpq-dev curl -y && \
    apt-get clean

WORKDIR /app

RUN useradd -m python && \
    chown python:python -R /app
USER python

ENV POETRY_VERSION=1.1.4
ENV PATH=/home/python/.poetry/bin:/opt/bin:/home/python/.local/bin:$PATH
ENV PYTHONUNBUFFERED=1

RUN curl -sSL <https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py> | python

ADD pyproject.toml .
ADD poetry.lock .

RUN poetry config virtualenvs.create false && \
    poetry run pip install --user -U pip && \
    if [ "$install_dev" = "true" ] ; \
      then poetry install ; \
      else poetry install --no-dev ; \
    fi

CMD poetry run python
k
Hi @Zach Schumacher! The entrypoint should be fine because Prefect adds commands. It seems like you should install the Github extra so 
pip install prefect[github]
Or use poetry to install it
z
where can I find the command that prefect adds in the src code?
k
Not 100% sure but I think it’s here.
z
No matter what I do I’m still running into the same error.. even am explicitly pip installing pygithub. any other ideas?
Copy code
FROM prefecthq/prefect:0.14.16-python3.7

RUN apt-get update && \
    apt-get install libpq-dev gcc libunwind-dev curl -y && \
    apt-get clean

ENV POETRY_VERSION=1.1.4
ENV PATH=/root/.poetry/bin:$PATH

RUN curl -sSL <https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py> | python

ADD pyproject.toml .
ADD poetry.lock .

RUN pip install pygithub
RUN poetry config virtualenvs.create false && \
    poetry run pip install -U pip && \
    poetry install
Copy code
26 April 2021,11:50:20 	agent	INFO	Submitted for execution: Job prefect-job-4b671fee
26 April 2021,11:50:23 	execute flow-run	ERROR	Failed to load and execute Flow's environment: ImportError('Unable to import Github, please ensure you have installed the github extra')
26 April 2021,11:50:25 	execute flow-run	ERROR	Failed to load and execute Flow's environment: ImportError('Unable to import Github, please ensure you have installed the github extra')
26 April 2021,11:50:27 	k8s-infra	ERROR	Pod prefect-job-4b671fee-4m2cb failed.
	Container 'flow' state: terminated
		Exit Code:: 1
		Reason: Error
26 April 2021,11:50:27 	k8s-infra	ERROR	Pod prefect-job-4b671fee-cp9kj failed.
	Container 'flow' state: terminated
		Exit Code:: 1
		Reason: Error
k
How are you passing the Docker image to Kubernetes?
z
Image kwarg in the KubernetesRun object
Copy code
@task()
def print_hello(name: str):
    <http://logger.info|logger.info>(f"hello {name}!")


storage = GitHub(
    repo="Simplebet/sbprefect", path="sbprefect/flows/example.py", access_token_secret="GITHUB_ACCESS_TOKEN"
)
environment = os.environ.get("ENVIRONMENT", "dev")

with Flow("Example", storage=storage) as flow:
    example_task = print_hello("world")
    flow.run_config = KubernetesRun(labels=[f"k8s-{environment}"], image=f"simplebet/sbprefect:{environment}")
👀 1
k
Can you try moving the run_config outside of the 
with
block?
flow.run_config should be aligned with 
with
z
i think you just suggested two different things unless im misunderstanding
k
Sorry I wasn’t clear. Will type out the code
Copy code
with Flow("Example", storage=storage) as flow:
    example_task = print_hello("world")

flow.run_config = KubernetesRun(labels=[f"k8s-{environment}"], image=f"simplebet/sbprefect:{environment}")
z
No, same error unfortunately
k
I think the way to diagnose this would be to try using 
DockerRun
and seeing if you get the same error. If we get the same error, then at least we know it’s the container. If it works, then we know it’s on the k8s side.
If it’s on the k8s side, I’ll grab someone with more experience there.
You could also try using DockerStorage with KubernetesRun and seeing if that works.
Also, be aware that the default image pull policy on kubernetes is 
IfNotPresent
and you can change this to 
Always
to always pull the newest image. Some users find that their Docker container doesn’t update because Kubernetes already sees a version of the image.
z
It looks like the docker image was being cached, so it was not pulling the latest version of dev. Any idea how to overide that?
yeah i think that is exactly what is happening
k
This thread may help you with that
z
thanks @Kevin Kho, i was able to resolve. So basically there are two ways to fix it: • use an image that is tagged as 
:latest
• use a job template like the below
Copy code
apiVersion: batch/v1
kind: Job
spec:
  template:
    spec:
      containers:
        - name: flow
          imagePullPolicy: Always
🙏 1
k
Thanks for the information @Zach Schumacher! Glad you figured it out. 🙂
z
maybe a good addition to the docs? Save someone else from spinning on it like me 😅
happy to PR it if it makes sense
k
Hey @Zach Schumacher A PR would be welcome of course under this page maybe. You’re probably the third user or so who ran into this in the last 2 months so I can definitely see it being useful 🙂
z
k
Thanks for that! It will be reviewed by the core team
5 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by