Nate
03/02/2023, 8:00 PMMarvin
03/02/2023, 8:01 PMFROM prefecthq/prefect:2-python3.9 # example base image
RUN pip install s3fs prefect-aws
You should also allocate enough CPU and memory to your agent, as well as add environment variables to your agent environment such as AWS_MAX_ATTEMPTS
and AWS_RETRY_MODE
, as these will enable Prefect to better access your ECS resources. Setting a work-queue concurrency limit is also recommended. When creating a deployment, you should specify the process
infrastructure type and configure remote Storage (local storage is not supported for ECS tasks).
For more details about this, check out the [Flow deployments - Prefect 2 - Coordinating the world's dataflows](https://docs.prefect.io/tutorials/deployments/) and [Infrastructure - Prefect 2 - Coordinating the world's dataflows](https://docs.prefect.io/concepts/infrastructure/) pages. I hope this helps!Nate
03/02/2023, 8:05 PM