Hi everyone, I’m having some difficulty with GitHub storage in a KubernetesJob.
I’m trying to use our existing app base docker images with Prefect installed on top so I have my dependencies preinstalled, then GitHub storage for the flow code.
But when I try and run the Jobs I see this error
/usr/local/bin/python: Error while finding module specification for 'prefect.engine' (ModuleNotFoundError: No module named 'prefect')
Our base image has a non-standard
WORKDIR
at
/home/app
but setting
PREFECT_HOME
on the container doesn’t seem to fix it.
Any help would be greatly appreciated.
r
Ryan Peden
03/30/2023, 1:12 PM
That's strange. Python is saying it doesn't think Prefect is installed at all.
Perhaps try open a shell in the image locally by running something like
docker run -it my-image:latest bash
. Once you're in, try running
python -m prefect.engine
and see if you get the same error message. If so, run
pip list
to see what Python actually thinks is installed.
a
Aiden Price
03/31/2023, 12:27 AM
Ah that’s helped, turns out the base image installs all the dependencies with Poetry, so the other Pods all start up with commands like
poetry run my-web-app
So what is the command/entrypoint that Prefect runs on the start up of a
KubernetesJob
? Is there some way to override it?
Aiden Price
03/31/2023, 3:44 AM
Ah got it, I’ve overridden the command for the job definition. Thanks!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.