I’ve have a scheduled flow that is failing due to a credential issue:

Error downloading Flow from S3: An error occurred (ExpiredToken) when calling the GetObject operation: The provided token has expired.

I’m using AWS SSO credentials to upload the flow to S3:

flow.storage = S3(
    bucket=Config.PREFECT_S3_BUCKET,
    client_options={
        "aws_access_key_id": Config.AWS_ACCESS_KEY_ID,
        "aws_secret_access_key": Config.AWS_SECRET_ACCESS_KEY,
        "aws_session_token": Config.AWS_SESSION_TOKEN,
    },
)

Initially, the flow runs. However, flows scheduled after the token expiry fail with the above error. How do I both use SSO to upload flows to S3 and have them scheduled on Prefect Cloud?

Did you try to do

aws configure

with AWS CLI on the host from which you register your flow? Do you register from your local machine, some shared VM or from CI?

We’re using SSO, not an IAM key pair. These are short lived credentials generated by

aws sso login

or pasted from the SSO UI.

In that case I believe the easiest way would be to run this aws sso login command before you register the flow from your terminal? Would that work for you? Not sure how otherwise approach it, especially given that SSO would require you to continue in the browser, correct?

I can successfully register the flow in question using SSO credentials. The issue occurs when I make use of the “schedule” functionality on Prefect Cloud: when the flow gets scheduled to run after the token expiry, I get the above

ExpiredToken

error.

I see - I understand your issue now. I don't think that using SSO will work for the remote agent. SSO are credentials bound to a person (here: to you). For an agent, you need long-lived programmatic access credentials or an IAM role

Okay. So I can’t use two different types of credentials for uploading flows from my local machine and the agent downloading them?

you totally can, AWS S3 permissions can be set in a variety of ways - for registering from your laptop you may use your SSO credentials and for the agent you may use a totally different IAM user do you have some DevOps engineer in your team who can help you set this up?

Yes, I do. This has already been configured — I use SSO credentials and the Agent has programmatic credentials. With reference to the above code snippet, how do I create a

Flow

instance such that two different sets of credentials can be used for upload/download of the flow?

I'm not sure what's the best way of explaining it is There are two processes: build-time process and runtime process Build-time This is the part where you register your flow. While registering your flow, Prefect may push your flow code to S3 if you want. If you do this from your laptop, then authenticating via SSO and then registering your flow will work and this concludes the build time process Runtime This is the part where your agent deploys scheduled flow runs and while doing that, it needs to pull your flow code from S3. To do that, your agent (your execution layer) needs to have valid credentials. Given that you use the Kubernetes agent, you could configure that using IAM roles for service accounts attached to your Kubernetes cluster.

@Anna Geller, I am in similar situation, I registered flow using AWS temporary credentials. My agent (local agent) is deployed in ECS task. I attached task role with full permission to S3 bucket where Flow is stored. I am still getting ExpiredToken error

Can you post it in a separate thread @Patrick Tan? your issue is very different since it's for ECS, not K8s