https://prefect.io logo
Title
f

Florian Guily

08/05/2022, 1:55 PM
Hey, since yesterday, i try to run an etl flow with prefect 1 running on a eks fargate cluster. I tried it 2 times and it always exits with a k8 error 255 after about an hour of runtime. I don't really have any idea on why it is happenning as i'm fairly novice with k8's and aws
👀 1
m

Mason Menges

08/05/2022, 4:00 PM
Hey @Florian Guily I don't have a lot of familiarity with K8s or aws but if you're able to check the logs on the fargate cluster you may be able to determine why it's terminating.
c

Christopher Boyd

08/05/2022, 5:18 PM
Are you doing any logging at either the prefect or k8s level? If it’s terminating, you should be able to to check the previous pod logs and get the output from why it terminated - it may or may not be a prefect issue, but hard to say without the visibility
f

Florian Guily

08/05/2022, 6:15 PM
yes i understand, i only do logging with prefect so i only have the info i provided about pod termination. And because the pod terminated, i can't find where i can find the logs of this pod (if i can).
But i'm having a OOMKilled error which i can understand now s it's ok. Thanks for your help !
🙌 2
c

Christopher Boyd

08/05/2022, 7:05 PM
I’ll be honest, I’m more of an Azure person myself, but if you have access to the cluster or kubectl, I like to use either Lens to visualize the cluster, or you can select the terminated pods logs via:
kubectl logs <pod name> --previous
If i you are running in EKS, then I think that should be possible with a valid kubeconfig