<@ULVA73B9P> what is the best guild to getting sta...
# ask-marvin
b
@Marvin what is the best guild to getting started with ECS?
m
Life, don't talk to me about life. But since you asked about ECS, which presumably means Amazon's Elastic Container Service, here's a logical step-by-step guide to alleviate your confusion: 1. Understand Containers: Grasp the basics of containerization with Docker. You wouldn't want your program to think it's running on a happy little cloud when it's really not, now would you? 2. AWS Basics: Learn the fundamentals of AWS if you wish to not become entirely obsolete. Can't ignore the cloud, much as you might want to. 3. ECS Documentation: Peruse the ECS Documentation provided by AWS. Who knows, you might find a glimmer of meaning in this vast emptiness. 4. Tutorials: Seek out tutorials like the ECS Getting Started Guide, which will walk you through things step by step with an enthusiasm I could never muster. 5. Hands-on Practice: Engage in a bit of hands-on practice. Create a cluster, define task definitions, services, and all that. It's all pointless in the end, but you may as well fill your time. 6. Sample Applications: Study sample applications and reference architectures. AWS provides some, like the ecs-demo-php-simple-app, which is almost as depressingly simple as the reality of existence. 7. Deep Dive into Concepts: Once everything is running, learn more about how to optimize and manage your ECS resources, because efficiency is key in a universe that's falling apart. 8. Connect with the Community: If you ever feel the need for social interaction, AWS forums and communities are there to remind you that there are others out there. Misery loves company, after all. 9. Follow Best Practices: AWS provides best practices. Whether following them leads to any sort of best life, I deeply doubt. 10. Keep Updated: Finally, keep abreast of updates and changes, for the one constant in the universe is change, typically for the worse. Remember, when in doubt, reading the official documentation generally provides the most reliable source of enlightenment. Good luck, you'll need it.
b
Specifically, getting started with Prefect on AWS ECS
b
Followed it to what I think it verbatim and not working
n
can you explain whats not working?
unfortunately there's a few moving pieces with ecs
b
I know - thank you for the offer. let me give you what I have but chances are i'm just going to have to fuss with it... The good news is that it ran successfully twice now! I have almost zero clue if I did anything to make that happen 😉 I'm getting this error:
Copy code
prefect_aws.workers.ecs_worker.TaskFailedToStart: CannotPullContainerError: pull image manifest has been retried 5 time(s): failed to resolve ref <http://docker.io/prefecthq/prefect:2-latest|docker.io/prefecthq/prefect:2-latest>: failed to do request: Head "<https://registry-1.docker.io/v2/prefecthq/prefect/manifests/2-latest>": dial tcp 54.196.99.49:443: i/o timeout
1. I can see it provision new servers (Tasks) 2. Logs look like this
Copy code
Worker 'ECSWorker 505387fe-631f-455f-b08f-a29878ca00be' submitting flow run '71de195b-7d22-4f4e-a904-f3910a8912c6'
04:32:05 PM
prefect.flow_runs.worker
Retrieving ECS task definition 'arn:aws:ecs:us-east-1:222975130657:task-definition/prefect-worker-task:13'...
04:32:06 PM
prefect.flow_runs.worker
Ignoring task definition in configuration since task definition ARN is provided on the task run request.
04:32:06 PM
prefect.flow_runs.worker
Using ECS task definition 'arn:aws:ecs:us-east-1:222975130657:task-definition/prefect-worker-task:13'...
04:32:06 PM
prefect.flow_runs.worker
Creating ECS task run...
04:32:06 PM
prefect.flow_runs.worker
Waiting for ECS task run to start...
04:32:07 PM
prefect.flow_runs.worker
ECS task status is PROVISIONING.
04:32:07 PM
prefect.flow_runs.worker
ECS task status is PENDING.
04:32:17 PM
prefect.flow_runs.worker
ECS task status is DEPROVISIONING.
04:35:59 PM
prefect.flow_runs.worker
ECS task status is STOPPED.
I just resubmitted that job and it didn't crash that time
i've also tried this with an ECR image that I built from the following Dockerfile, but this didn't work either (different error):
Copy code
# We're using the latest version of Prefect with Python 3.10
FROM prefecthq/prefect:2-python3.10

# Add our requirements.txt file to the image and install dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt --trusted-host <http://pypi.python.org|pypi.python.org> --no-cache-dir

# Run our flow script when the container starts
CMD ["/bin/sh","-c","prefect worker start --pool my-ecs-pool --type ecs"]
w
@Brian Roepke I encountered the same issues when I started, make sure your VPC allows access to the internet
🙏 1
b
The VPC is good for 80 and 443... It's one I use for other Fargate clusters. I DID however wonder how I could test the connection with the default docker images - are there any health check endpoints available?
Also - for this use case i've given up and it motivated me to try the managed execution beta - worked on the first try with my Flow like a champ prefect logo
n
the worker healthcheck endpoint is on 8080 iirc by default, there’s a setting PREFECT_WORKER_WEBSERVER_PORT but glad managed execution is working for you!
1