https://prefect.io
Join Slack
If i am building a container run as a KubernetesJob, do I need anything outside of the standard pref...
# prefect-kubernetes
g
If i am building a container run as a KubernetesJob, do I need anything outside of the standard prefect package?
✅ 1
j
Hi! If you’re referring to the container that your flow will fun on - all you should need is a base prefect image! ofc if you have any dependencies those would also need to be installed (i.e. pandas)
g
@Jamie Zieziula OK i figured out my issue, I am running K8s in GKE and the Worker is passing the command 
prefect -m prefect.engine
to the container under the args fields, which is not running in GKE
I updated the advanced config to use command instead and it still passes through under args
@Nate have you seen this from any other customers?
n
i generally recommend avoiding overriding the 
command
prefect uses to start flow runs, unless you really need to why did you end up doing it here?
g
Ok so I had to dig into the prefect dockerfiles to figure this one out
I am manually build my job images with the code in them, and passing the args was fail in k8s. This same container ran fine in GCP Cloudrun
Because the K8s worker has the args field hardcode and even changing the job template doesn't change that
I ended up adding 
ENTRYPOINT ["/tini", "--"]
to my dockerfile to fix the issue
marvin 1
and let k8s accept the args attribute
j
@ Gregory Hunt, I'm new to GKE and have a question for you regarding GKE. I followed the steps https://docs.prefect.io/2.11.4/guides/deployment/helm-worker/ to deploy a Prefect worker on GKE. It is working, but how do you keep the worker alive? I set up a Google Cloud Shell session and started my prefect worker (prefect worker start --pool gke-kubernetes-cluster), but that worker will 'die' after I close the Google Cloud Shell session. Would you happen to have any tips on keeping the worker alive?
g
@John Horn I deployed the worker using this helm chart https://github.com/PrefectHQ/prefect-helm/tree/main/charts/prefect-worker
and it deploys it as a deployment, you set the values for which workpool you want to listen to
j
Got it, but don't you afterwards need to start the worker on the deployment?
Or does the helm chart take care of that?
👍 1
j
native k8s / the helm chart should take care of that
j
Oh wow, that is super neat
I thought I needed to run the command 'prefect worker start --pool gke'
in a google cloud session to start the worker
because I had to do that when developing locally with minikube
the helm chart deployment didn't work locally on minikube if I did not also start a worker
I re-opened the google cloud shell terminal and the prefect_worker window is still running. I'll kill it and see if the deployment creates a worker automatically
Wow, it worked!
Do you all know why in minikube we need to manually run a worker?
g
not sure but minikube can be funny
👍 1
j
It is amazing that the helm chart makes this worker creation so seamless
🙌 1
wow
Appreciate the help @Gregory Hunt and @Jamie Zieziula
🎉 1
👍 1
6 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by