Hi, Do you have example of building flow to AWS S3 with credentials like aws token? The example in doc does not have credentials
Copy code
from prefect import Flow
from prefect.storage import S3
flow = Flow("s3-flow", storage=S3(bucket="<my-bucket>"))
k
Kevin Kho
05/04/2022, 1:51 PM
For registering and uploading, you would just authenticate on the machine you’re working on. You can just do
aws configure
and add your info there,
On the pull, you can use secrets. There is a default secret for
AWS_CREDENTIALS
that Prefect can use for AWS connections listed here
Kevin Kho
05/04/2022, 1:53 PM
There is an example there for the syntax of the Prefect secret
a
Anna Geller
05/04/2022, 3:57 PM
what agent type do you use? e.g. Kubernetes agent and ECS agent require a different way of providing S3 permissions than a local agent
p
Patrick Tan
05/05/2022, 12:44 PM
ECS agent
a
Anna Geller
05/05/2022, 2:09 PM
in that case, you need to specify permissions using a task role (IAM role). Check this example and this blog post for more details on how to set this up
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.