https://prefect.io logo
Join Slack
Powered by
Hi guys! I'm trying to run a flow with a custom fl...
# ask-community
g
Hi guys! I'm trying to run a flow with a custom flow class that requires 
python >= 3.8
. When running it with the local agent, it works fine, but when I try to run it with the kubernetes agent, using as image one that extends 
prefecthq/prefect:latest-python3.8
, I get a python version mismatch error saying that it is trying to run the flow with a python 3.7.10. Is there a way I can change this python version?
k
Hey @Gustavo de Paula, what is the Python version of your agent?
g
Its 3.8.11
k
Can I see your error message? Was is registered with 3.7.10?
g
Copy code
Failed to load and execute Flow's environment: StorageError('An error occurred while unpickling the flow:\n  ModuleNotFoundError("No module named \'My_lib\'")\nThis may be due to one of the following version mismatches between the flow build and execution environments:\n  - python: (flow built with \'3.8.11\', currently running with \'3.7.10\')\nThis also may be due to a missing Python module in your current environment. Please ensure you have all required flow dependencies installed.')
k
Ok can you show me how you set-up KubernetesRun? and your Storage?
g
Sure
k
If your agent is 3.8.11, I think it’s the image causing the 3.7.10.
g
Copy code
with My_Flow("My flow2", storage=Azure(container="flows",connection_string = 'my_connection_string')) as flow:
    flow.run_config=KubernetesRun(labels=["kubeagent"])
k
Wait I’m a bit confused. How do you specify what image to use for this flow?
g
I specify it in the job_spec file
k
I think you need to specify it in the RunConfig instead? 
KubernetesRun(image="xxx", labels=["xxx"])
Actually can you try 
storage=Azure(container="flows",connection_string = 'my_connection_string', stored_as_script=True))
so that this flow will not get serialized?
g
Sure
Hi @Kevin Kho! The solution proposed solved the error, but now I'm getting this one when the Kubernetes job starts.
k
Hey @Gustavo de Paula, does the image you are using for KubernetesRun contain Prefect?
I guess you said you extended it. Can I see your Dockerfile and do you have another place you are specifying an image like the job template?
g
I was extending it but I was having some problems trying to pip install some packages. So now I'm trying a different approach.
Copy code
FROM python:3.8.11-slim
RUN apt update
RUN apt install -y python-pyodbc
RUN apt install tini
RUN pip install my-custom-package
RUN pip install prefect[Kubernetes] azure-storage-blob

COPY entrypoint.sh /usr/local/bin/entrypoint.sh
ENTRYPOINT ["tini", "-g", "--", "entrypoint.sh"]
k
You don’t need the ENTRYPOINT for Prefect Flows (though I think it’s fine). Prefect adds those commands in your container
What was your problem with pip installing?
g
Unfortunately I lost the log, but I was unable to install any package when building the image
I'll try removing the entrypoint
I tried removing the entrypoint and still got the same error
k
RUN pip install prefect[Kubernetes] azure-storage-blob
. Did you use big K or small k? I think this needs to be small
g
I used big K but it got the kubernetes agent correctly. The agent is up and running, the problem happens only at job level
k
ah ok if it’s something to do with the flow, maybe you can try a very simple flow to confirm that?
g
got the same error with a hello world flow
k
then it seems to be the container setup right? i guess try a container that just inherits prefect with no other libraries?
g
Hey @Kevin Kho, I changed the docker build server I was using and the extended image worked just fine. I guess there was something wrong with the docker installation, idk. Apparently everything is working for now. Thanks for all the help!
k
Gotcha! Good to hear!
5 Views
Open in Slack
PreviousNext
Powered by