https://prefect.io logo
g

Gregory Hunt

08/23/2023, 4:42 PM
If i am building a container run as a KubernetesJob, do I need anything outside of the standard prefect package?
βœ… 1
j

Jamie Zieziula

08/23/2023, 4:58 PM
Hi! If you’re referring to the container that your flow will fun on - all you should need is a base prefect image! ofc if you have any dependencies those would also need to be installed (i.e. pandas)
g

Gregory Hunt

08/23/2023, 5:59 PM
@Jamie Zieziula OK i figured out my issue, I am running K8s in GKE and the Worker is passing the command
prefect -m prefect.engine
to the container under the args fields, which is not running in GKE
I updated the advanced config to use command instead and it still passes through under args
@Nate have you seen this from any other customers?
n

Nate

08/23/2023, 8:09 PM
i generally recommend avoiding overriding the
command
prefect uses to start flow runs, unless you really need to why did you end up doing it here?
g

Gregory Hunt

08/23/2023, 8:11 PM
Ok so I had to dig into the prefect dockerfiles to figure this one out
I am manually build my job images with the code in them, and passing the args was fail in k8s. This same container ran fine in GCP Cloudrun
Because the K8s worker has the args field hardcode and even changing the job template doesn't change that
I ended up adding
ENTRYPOINT ["/tini", "--"]
to my dockerfile to fix the issue
marvin 1
and let k8s accept the args attribute
j

John Kang

08/24/2023, 6:35 PM
@ Gregory Hunt, I'm new to GKE and have a question for you regarding GKE. I followed the steps https://docs.prefect.io/2.11.4/guides/deployment/helm-worker/ to deploy a Prefect worker on GKE. It is working, but how do you keep the worker alive? I set up a Google Cloud Shell session and started my prefect worker (prefect worker start --pool gke-kubernetes-cluster), but that worker will 'die' after I close the Google Cloud Shell session. Would you happen to have any tips on keeping the worker alive?
g

Gregory Hunt

08/24/2023, 6:37 PM
@John Horn I deployed the worker using this helm chart https://github.com/PrefectHQ/prefect-helm/tree/main/charts/prefect-worker
and it deploys it as a deployment, you set the values for which workpool you want to listen to
j

John Kang

08/24/2023, 6:38 PM
Got it, but don't you afterwards need to start the worker on the deployment?
Or does the helm chart take care of that?
πŸ‘ 1
j

Jamie Zieziula

08/24/2023, 6:38 PM
native k8s / the helm chart should take care of that
j

John Kang

08/24/2023, 6:38 PM
Oh wow, that is super neat
I thought I needed to run the command 'prefect worker start --pool gke'
in a google cloud session to start the worker
because I had to do that when developing locally with minikube
the helm chart deployment didn't work locally on minikube if I did not also start a worker
I re-opened the google cloud shell terminal and the prefect_worker window is still running. I'll kill it and see if the deployment creates a worker automatically
Wow, it worked!
Do you all know why in minikube we need to manually run a worker?
g

Gregory Hunt

08/24/2023, 6:46 PM
not sure but minikube can be funny
πŸ‘ 1
j

John Kang

08/24/2023, 6:47 PM
It is amazing that the helm chart makes this worker creation so seamless
πŸ™Œ 1
wow
Appreciate the help @Gregory Hunt and @Jamie Zieziula
πŸŽ‰ 1
πŸ‘ 1
4 Views