Hi there, looking for some help running an ECS Agent with S3 storage. Getting a weird boto error.
from prefect import task, Flow
from prefect.storage import S3
logger = prefect.context.get("logger")
<http://logger.info|logger.info>(f'hello world')withFlow("hello-flow") as flow:
flow.storage = S3(bucket="****")
if __name__ == '__main__':
I was able to get an ECS Service running the agent.
So I do have an agent running under the "Agents" tab in the Prefect Cloud UI.
I was able to get past
Error downloading Flow from S3: Unable to locate credentials
by providing all the possible combinations of AWS env vars in the task definition container.
Now I'm getting something that looks like a boto error:
Submitted for execution: Task ****
Downloading flow from s3://***/hello-flow/2022-03-29t04-10-25-834594-00-00
Error downloading Flow from S3: 'str' object has no attribute 'get'
Failed to load and execute flow run: AttributeError("'str' object has no attribute 'get'")
any help appreciated 🙏
as agent image)
6 months ago
@Sacha Ventura Here are some resources that may help you get started with Prefect and an ECS agent:1. Blog post explaining how to set up an ECS agent as an ECS service
Medium – 1 Nov 21How to Cut Your AWS ECS Costs with Fargate Spot and PrefectServerless Data Engineering Pipelines in Python
Reading time: 11 min read1. This repo has many examples with various storage and run_configuration options - check for those that contain