hi prefecters. Does anybody know how to enable all...
# ask-community
s
hi prefecters. Does anybody know how to enable all boto3 logs (the underlying requests which are sent to aws) in prefect-aws for ECS worker?
the worker is started within the docker container with
Copy code
# Run worker
prefect worker start --pool "${PROJECT}-${ENVIRONMENT}-ecs"
j
I'm not using ECS, but maybe my struggle can help: https://prefect-community.slack.com/archives/C048ZHT5U3U/p1701792755963809
In my setup I'm running a worker on EC2 and the deployment is a docker container on the same EC2 instance
Maybe you could change the command to
prefect worker start --pool "${PROJECT}-${ENVIRONMENT}-ecs" > /prefect_worker_log.log
and then have the CloudWatch agent in there pick it up. But I'm sure there's a way to pass the
awslogs
driver to the container with a dockerfile or maybe with the job_variables section in the prefect.yaml/deploy(). When I first started with Prefect I had a docker-compose file that I was able to specify the
awslogs
driver in, and that worked for each container it created.
s
Hi Jack, thanks for your input. I can get the logs from worker, but in my case I need to debug the requests boto3 library is sending to aws. Boto3 is used by prefect-~ecs~aws, and for some reason it is unable to validate my custom network configuration. If I just could to see how prefect "talks" to EC2, this might spot the problem.
j
ohh sorry, I misunderstood. Is it this? https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py In the logging.yml you might be able to see more info if you set everything to debug level. I did that once while troubleshooting and I saw all the HTTP traffic in there
👀 1