Hey ! I’m running the prefect agent on AWS EKS (Kubernetes) and when firing a job (which creates a new pod on the same cluster) I get the error

NoCredentialsError('Unable to locate credentials')

from the job execution. In the past I used the ECS agent and I was able to pass a

task_role_arn

to it. However I see that both the

KubernetesAgent

and the run config for Kubernetes

KubernetesRun

don’t have a “role” parameter to inform. My next thought was that either the role attached to the

EKS cluster

or the

Fargate profile

should do the work, but as a test I gave admin to both and I still get the credentials issue. What am I missing ?

Hey @Maikel Penz, I’m not 100% sure if that is expected to work, but I think it should. Maybe try passing credentials as env vars to KubernetesAgent?

this is the flow not being able to communicate with AWS resources (e.g: S3, EMR, etc..).. The flow starts, so the agent can pull the image and all of that.. But the flow itself doesn’t seem to have permission to hit the AWS API

Yeah I know exactly what you are saying now. Let me look into it a bit. I haven’t seen this situation before

hmm you know I think I found out the problem.. see this: https://docs.aws.amazon.com/eks/latest/userguide/pod-execution-role.html

Just got back to the desk and got a response from our k8s users at Prefect. Yes you’re right with this links that you need to make a service account. I agree that would be helpful Would you be interesting in contributing that to the docs when you get it set up? 😄

@Maikel Penz - I am also running into the same issue. I was wondering if you can share a document on how to solve this problem.

Hey @Amogh Kulkarni you’ll need to follow the 3 steps on the bottom of this page and put as part of your automation

Thanks for sharing this link Maikel. It worked like a charm and it was exactly what I was looking for.

great success !