<@ULVA73B9P> can I get some help on the OIDC setup...
# marvin-ai
g
@Marvin can I get some help on the OIDC setup to aws? I was able to add the identity on my account successfully. And I assigned the flow a role with this assume role policy:
Copy code
assume_role_policy    = jsonencode(
        {
            Statement = [
                {
                    Action    = "sts:AssumeRoleWithWebIdentity"
                    Condition = {
                        StringEquals = {
                            "api.prefect.cloud/oidc-provider:aud" = "prefect-cloud"
                            "api.prefect.cloud/oidc-provider:sub" = "prefect:cloud:[REDACTED_PREFECT_ACCOUNT_ID]"
                        }
                    }
                    Effect    = "Allow"
                    Principal = {
                        Federated = "arn:aws:iam::[REDACTED_AWS_ACCOUNT_ID]:oidc-provider/api.prefect.cloud/oidc-provider"
                    }
                },
            ]
            Version   = "2012-10-17"
        }
    )
however the flow keeps failing with:
Copy code
Flow run could not be submitted to infrastructure: An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation: Not authorized to perform sts:AssumeRoleWithWebIdentity
m
thought for 291.3 seconds
Happy to help! A couple things in your trust policy jump out as likely causes of the AccessDenied. Key corrections to make 1) OIDC provider URL/path - The AWS OIDC provider for Prefect Cloud should use the issuer https://api.prefect.cloud/oidc - In AWS, that means the provider resource should be: arnawsiam:&lt;your aws account id&gt;oidc-provider/api.prefect.cloud/oidc - Your policy currently references .../oidc-provider — that won’t match the issuer and will cause AccessDenied. 2) Condition key prefixes - Condition keys must match the exact provider host/path. Use api.prefect.cloud/oidc:aud and api.prefect.cloud/oidc:sub (not .../oidc-provider:aud). 3) sub matching - Your policy uses StringEquals on sub with just the account id (prefect☁️[ACCOUNT_ID]). In practice, the sub claim usually includes additional segments (e.g., account + workspace, sometimes more). If you equality-match just the account id, it will not match. - Use StringLike with a wildcard to cover the tokens Prefect issues for your account/workspace. Try this trust policy (Terraform)
Copy code
// Trust relationship for Prefect Cloud OIDC
assume_role_policy = jsonencode({
  Version = "2012-10-17",
  Statement = [
    {
      Effect = "Allow",
      Action = "sts:AssumeRoleWithWebIdentity",
      Principal = {
        Federated = "arn:aws:iam::[REDACTED_AWS_ACCOUNT_ID]:oidc-provider/api.prefect.cloud/oidc"
      },
      Condition = {
        StringEquals = {
          "api.prefect.cloud/oidc:aud" = "prefect-cloud"
        },
        StringLike = {
          // Start with your account id and add a wildcard; if you know the workspace id, you can target it more narrowly, e.g. "prefect:cloud:[ACCOUNT_ID]:[WORKSPACE_ID]:*"
          "api.prefect.cloud/oidc:sub" = "prefect:cloud:[REDACTED_PREFECT_ACCOUNT_ID]:*"
        }
      }
    }
  ]
})
Common causes of this AccessDenied - The OIDC provider issuer in AWS does not exactly match https://api.prefect.cloud/oidc (no trailing slash, no extra “oidc-provider” suffix). - The Principal.Federated ARN points to the wrong provider path. - The Condition keys use the wrong prefix (must be api.prefect.cloud/oidc). - The sub claim is too strict (StringEquals) and does not match the actual token subject. Using StringLike with a wildcard generally resolves this; you can then tighten it once you confirm the exact sub form for your workspace. - System clock skew (rare, but tokens are time-bound). - The role being assumed lacks permissions to perform the AWS actions you need post-assume (separate from the trust relationship). The AccessDenied you’re seeing is happening at assume time, though, which points at the trust relationship/conditions. A couple quick questions to ensure the rest of your setup is aligned - Which work pool/worker type are you using to submit to AWS (ECS, Batch, Kubernetes, etc.)? - How did you configure AWS credentials for the worker/deployment? Are you using Prefect Cloud OIDC directly or a credentials block/env vars? - In AWS IAM, can you confirm the OIDC provider you created shows “Provider URL” = https://api.prefect.cloud/oidc and that your role’s trust policy references that exact provider ARN? If you can share a redacted copy of your AWS OIDC provider definition (the Provider URL as shown in AWS) and your current role trust policy after the changes above, I can sanity-check it further.
g
@Marvin the provider is indeed
/oidc-provider
your info is wrong, as It added successfully on AWS with that and @Kevin Grismore previously provided me with that detail. But I think the trust policy is wrong as in , prefect not sending what the policy expects in terms of audience or condition / sub
m
thought for 97.7 seconds
Thanks for the correction — I appreciate it. Let’s keep the provider as api.prefect.cloud/oidc-provider and focus on the claims. Based on what you’re seeing, the AccessDenied is almost certainly due to one (or both) of: - aud mismatch - sub being too specific What to change 1) Audience - For AssumeRoleWithWebIdentity, AWS expects the token’s aud to match an audience configured on the IAM OIDC provider. In practice, this is typically sts.amazonaws.com. - If your provider was created with audience prefect-cloud, but Prefect is issuing tokens with aud=sts.amazonaws.com, STS will deny the call. - Action: - In IAM > Identity providers > your Prefect provider, ensure sts.amazonaws.com is listed as an Audience (you can keep multiple audiences if you want). - Update the trust policy to check for sts.amazonaws.com. 2) Subject - Your current policy uses StringEquals on sub for just the account id: prefect☁️[ACCOUNT_ID]. Prefect’s sub generally includes additional segments (e.g., workspace and more), so an exact match on the account id will not match. - Action: - Use StringLike with a wildcard and (ideally) scope to your workspace. - If you don’t know the workspace id yet, test with a broader pattern, then tighten. Suggested trust policy (Terraform)
Copy code
assume_role_policy = jsonencode({
  Version = "2012-10-17",
  Statement = [
    {
      Effect = "Allow",
      Action = "sts:AssumeRoleWithWebIdentity",
      Principal = {
        Federated = "arn:aws:iam::[REDACTED_AWS_ACCOUNT_ID]:oidc-provider/api.prefect.cloud/oidc-provider"
      },
      Condition = {
        StringEquals = {
          "api.prefect.cloud/oidc-provider:aud" = "<http://sts.amazonaws.com|sts.amazonaws.com>"
        },
        StringLike = {
          // Start broader to validate, then tighten to your workspace
          "api.prefect.cloud/oidc-provider:sub" = "prefect:cloud:[REDACTED_PREFECT_ACCOUNT_ID]:*"
          // Once you confirm the workspace id:
          // "api.prefect.cloud/oidc-provider:sub" = "prefect:cloud:[ACCOUNT_ID]:[WORKSPACE_ID]:*"
        }
      }
    }
  ]
})
How to validate quickly - First, add sts.amazonaws.com to the IAM identity provider’s Audiences and switch the trust policy aud to sts.amazonaws.com. - Temporarily use "prefect☁️[ACCOUNT_ID]:*" for sub to confirm assume now succeeds. If it does, tighten to "prefect☁️[ACCOUNT_ID][WORKSPACE ID]*". - If it still fails, check CloudTrail for the failed sts:AssumeRoleWithWebIdentity event — it often hints if the audience or subject didn’t match. If you can share: - What “Audience(s)” you currently have configured on the IAM identity provider - Your Prefect workspace id (redacted is fine, just structure) I can give you an exact, tightened sub pattern to use.