https://prefect.io logo
m

Mikaël Ferreira de Almeida

08/28/2023, 3:24 PM
Hello everyone ! I'm facing and issue with "Kubernets Jobs" block. In my code I have defined the namespace as follow :
Copy code
k8s_jb = KubernetesJob(
        ...
        namespace="prefect",
    )
After saving the block I can see in the UI that the namespace is correctly defined. But sometimes, when I run a deployment flow, I get these errors:
Copy code
TTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch is forbidden: User \"system:serviceaccount:prefect:prefect-worker\" cannot create resource \"jobs\" in API group \"batch\" in the namespace \"default\"","reason":"Forbidden","details":{"group":"batch","kind":"jobs"},"code":403}
The error message indicates that I'm in the wrong namespace. If I use the "Retry" button, at some point it will be executed in the "prefect" namespace. Sometimes I don't get the error, sometimes I have to retry 5 times. Does anyone have the same problem?
j

John Kang

08/28/2023, 8:42 PM
I had that same issue while running locally. I had to ensure that I setup the 'prefect' namespace on my local kubernetes and not just the block
Also, ensure your namespace is setup correctly on your work pool:
🙏 1
👀 1
m

Mikaël Ferreira de Almeida

08/29/2023, 7:50 AM
I hadn't configured the work pool namespace, I think it will work perfectly now. Thanks a lot!
Then I'am wondering why it's needed. I have to specify mi image name in both ""Kubernets Jobs" block and my WorkPool manifest 🤔 otherwise my flows will fail because the code environnement it's not the same
c

Christopher Boyd

09/01/2023, 12:08 AM
KubernetesJob blocks shouldn’t be used for workers
👀 1
The work pool replaces the KubernetesJob block
So there were probably multiple configs trying to pick up the flow run and sometimes the right one got it and sometimes it didn’t
m

Mikaël Ferreira de Almeida

09/01/2023, 7:40 AM
I see. So when I use
Deployment.build_from_flow(
, i should not specify the "infra" parameter if I already said that the deployment is on a work_pool ?
c

Christopher Boyd

09/01/2023, 12:23 PM
correct, the infra is part of the worker and that’s what the work pool configuration is
you can still pass in infra_overrides however, like image, or environment variables if they are specific to that flow