Great question. The short version: move AWS permissions off the single, broad IRSA and onto per-deployment Kubernetes service accounts. In Prefect 3.x with a Kubernetes work pool, each deployment can specify the

service_account_name

it should run with. That service account is IRSA-bound to an IAM role with just the permissions that flow needs. Here’s a concise migration plan. 1) Decide the boundaries - Group flows by the AWS permissions they need (e.g., “S3-read-only”, “S3+SNS”, “DynamoDB-writer”). - Plan to create one Kubernetes ServiceAccount (SA) per permission set, or per sensitive flow if you need stricter isolation. - Optional but recommended: use separate Kubernetes namespaces per team or permission set to tighten RBAC. 2) Create least-privilege IAM roles and bind them via IRSA - For each permission set: - Create an IAM policy with only the required AWS actions/resources. - Create an IAM role that trusts your EKS OIDC provider and the specific SA and namespace. - Create a Kubernetes ServiceAccount in the target namespace and annotate it with the IAM role ARN. Example (replace ARNs, namespace, and names for your cluster):

# ServiceAccount in the 'data' namespace for S3 read-only flows
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-s3-readonly
  namespace: data
  annotations:
    <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: arn:aws:iam::123456789012:role/prefect-s3-readonly

IAM role trust policy (principal/condition must match your OIDC provider and SA):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.<region>.<http://amazonaws.com/id/EXAMPLEID|amazonaws.com/id/EXAMPLEID>"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.<region>.<http://amazonaws.com/id/EXAMPLEID:sub|amazonaws.com/id/EXAMPLEID:sub>": "system:serviceaccount:data:sa-s3-readonly"
        }
      }
    }
  ]
}

Important: For official EKS IRSA, the role is attached via the ServiceAccount annotation (not pod annotations). 3) Configure Prefect deployments to use the right service account - Keep using your Kubernetes work pool. - For each deployment, set

work_pool.job_variables.service_account_name

(and

namespace

if you’re not using the default). - This ensures each flow run pod assumes the least-privilege IAM role via its SA. Example prefect.yaml with two deployments:

deployments:
- name: flow-s3-reader
  entrypoint: flows/s3_reader.py:flow_s3_reader
  work_pool:
    name: kubernetes
    job_variables:
      namespace: "data"
      service_account_name: "sa-s3-readonly"

- name: flow-ddb-writer
  entrypoint: flows/ddb_writer.py:flow_ddb_writer
  work_pool:
    name: kubernetes
    job_variables:
      namespace: "etl"
      service_account_name: "sa-ddb-writer"

Python deploy example:

from prefect import flow

@flow
def flow_s3_reader():
    ...

if __name__ == "__main__":
    flow_s3_reader.deploy(
        name="flow-s3-reader",
        work_pool_name="kubernetes",
        job_variables={
            "namespace": "data",
            "service_account_name": "sa-s3-readonly",
        },
    )

Notes: - The service account must exist in the same namespace the job runs in. - Avoid relying on a single “worker” SA for AWS access. The worker just submits Jobs; the Job’s pod uses the SA you set above. 4) (Optional) Use multiple work pools for broader isolation - If you also want different node groups, taints, security context, images, or network policies, create separate Kubernetes work pools with custom base job templates (e.g., different nodeSelector/tolerations). - Still set

service_account_name

per deployment for IAM scoping. 5) Migrate incrementally - Start with one flow group: - Create IAM role + SA. - Update its deployment to the new SA. - Run a test; verify: - kubectl get job/pod shows the correct serviceAccountName.

- The flow can access only its intended AWS resources (e.g., test an S3 object it should access and another it should not). Helpful kubectl checks:

kubectl get jobs -n data
kubectl get job <job-name> -n data -o jsonpath='{.spec.template.spec.serviceAccountName}{"\n"}'
kubectl get sa sa-s3-readonly -n data -o yaml

6) Remove broad permissions - After all deployments are migrated, remove the broad AWS permissions from the old IRSA role used by the worker or legacy SA. - Ensure your Prefect worker’s K8s RBAC allows creating Jobs in the target namespaces, but it does not need broad AWS IAM permissions. 7) Ongoing management tips - One SA/role per deployment is the simplest mental model for strict least privilege. - If you want different AWS permissions based on runtime parameters, you’ll typically use different deployments (each with its own SA/role); job variables are not overridden per-run. - Keep policies small and use resource-level permissions and condition keys (e.g., aws:ResourceTag) where possible. Would you like me to draft the exact IAM policies and the ServiceAccount manifests for a couple of your flows if you share: - Flow names and the AWS services/resources they need - Target namespaces - Your work pool name(s) and whether you want to split by team or by permission set?

Short answer: with EKS IRSA you can only attach one IAM role to a pod (via its Kubernetes ServiceAccount). You cannot “combine” two roles at runtime for a single pod. The practical pattern is: - Keep reusable, per-service IAM policies (e.g., PolicyS3ReadOnly, PolicySNSPublish) - Create a single role per deployment that attaches the specific policies it needs (e.g., attach both the S3 and SNS policies) - Bind that role to the ServiceAccount used by the deployment Three viable approaches 1) Recommended: policy-per-service, role-per-deployment - Create managed policies per service (S3, SNS, etc.) - For a flow that needs both, create one IAM role and attach both policies - Annotate a Kubernetes ServiceAccount with that role and point your Prefect deployment at that SA Example:

# Kubernetes ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-s3-sns
  namespace: data
  annotations:
    <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: arn:aws:iam::123456789012:role/prefect-s3-sns

Prefect deployment:

deployments:
- name: flow-s3-sns
  entrypoint: flows/s3_sns.py:flow
  work_pool:
    name: kubernetes
    job_variables:
      namespace: "data"
      service_account_name: "sa-s3-sns"

2) Role chaining (advanced) - Keep separate roles per service - Allow your pod’s primary IRSA role to sts:AssumeRole into the other role(s) - Your code assumes roles as needed for different AWS clients - Trade-offs: added complexity and explicit assume-role logic; useful for cross-account access or strict separation of duties 3) Split the pipeline into subflows with different SAs/roles - If you need strict least-privilege at different steps, deploy those steps as separate deployments - Each subflow/deployment uses its own ServiceAccount/role - Orchestrate with

run_deployment

between stages so each pod has only the permissions needed for that stage Key points - One ServiceAccount → one IRSA role → one pod - Compose permissions by attaching multiple policies to that one role, or assume into other roles from code - In Prefect 3.x, set per-deployment

service_account_name

(and

namespace

) in

work_pool.job_variables

so each deployment runs with its own least-privilege role If you want, share: - Which flows need S3, which need SNS, and which need both - The namespaces you use - Your work pool name I can draft the exact IAM policies, trust policies, and ServiceAccount manifests for those cases. Reference: AWS EKS IRSA (official docs)