This usually means the Prefect CLI script isn’t on your PATH or it was installed for a different Python/user than the one you’re using on the EC2 box. Here’s a quick checklist to debug and fix it. 1) Verify Prefect is actually installed for Python 3.12

python3.12 -m pip show prefect
python3.12 -m pip --version

- If it’s not installed, install it (avoid sudo):

python3.12 -m pip install -U prefect

2) Try the CLI via Python directly (works even if PATH isn’t set)

python3.12 -m prefect --version
python3.12 -m prefect server start

If that works, the issue is just PATH. 3) Find where the

prefect

script was installed - Common user install location on Linux:

~/.local/bin

- You can confirm:

python3.12 -m site --user-base
# If this prints /home/ec2-user/.local, then the scripts live in /home/ec2-user/.local/bin

Or:

python3.12 -c "import sysconfig; print(sysconfig.get_path('scripts'))"

4) Add that directory to your PATH and reload your shell

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
# then
prefect --version

If you installed system-wide (without --user), the script may be in

/usr/local/bin

or the Python 3.12 “bin” directory. Ensure that directory is in PATH too. 5) Avoid installing as root with sudo If you installed with sudo, the script may be owned by root and not visible to your user. Prefer a virtual environment or user install. If needed, uninstall and reinstall as your user:

sudo python3.12 -m pip uninstall prefect
python3.12 -m pip install --user prefect

6) Recommended: use a virtual environment (clean and reliable)

python3.12 -m venv ~/venvs/prefect
source ~/venvs/prefect/bin/activate
pip install -U pip setuptools wheel
pip install prefect
prefect --version

7) Alternative: use pipx (manages PATH for you)

python3.12 -m pip install -U pipx
pipx ensurepath
pipx install prefect
# restart shell if needed, then:
prefect --version

Docs and references: - Prefect 3.x installation guide - Quickstart If it’s still not working, could you share: - How you installed Prefect (pip in venv, pip --user, pipx, or sudo)? - The output of:

echo $PATH
python3.12 -m pip show prefect
python3.12 -m site --user-base
which prefect

With that, I can pinpoint the exact fix for your EC2 setup.

Great! To reliably start Prefect Server on your EC2 instance and access the UI, here are the key steps. 1) Start Prefect Server bound to all interfaces - Run it in your venv (or use python -m if you prefer):

prefect server start --host 0.0.0.0 --port 4200 --ui
# or, if you prefer:
python3.12 -m prefect server start --host 0.0.0.0 --port 4200 --ui

- Quick health check on the instance:

curl -I <http://127.0.0.1:4200>

You should get a 200/301 response, indicating the webserver is up. 2) Make the UI reachable from your laptop You have two options: - Option A: Open the port in your EC2 security group (simplest) - Add an inbound rule allowing TCP 4200 from your IP - Visit: http//<your ec2 public ip>4200 - Option B: Keep the port closed and use an SSH tunnel (more secure)

ssh -i /path/to/key.pem -L 4200:localhost:4200 ec2-user@<your-ec2-public-ip>

Then in your browser on your laptop, visit http://localhost:4200 3) Persist your settings so you don’t need flags every time

prefect config set PREFECT_SERVER_API_HOST=0.0.0.0 PREFECT_SERVER_API_PORT=4200 PREFECT_UI_ENABLED=true

You can verify your active settings:

prefect profile inspect

4) (Optional) Run Prefect Server as a systemd service (starts on boot) - Create a unit file at `/etc/systemd/system/prefect-server.service`:

[Unit]
Description=Prefect Server
After=network.target

[Service]
User=ec2-user
WorkingDirectory=/home/ec2-user
Environment="PREFECT_SERVER_API_HOST=0.0.0.0" "PREFECT_SERVER_API_PORT=4200" "PREFECT_UI_ENABLED=true"
ExecStart=/home/ec2-user/venvs/prefect/bin/prefect server start
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

- Reload and start:

sudo systemctl daemon-reload
sudo systemctl enable --now prefect-server
sudo systemctl status prefect-server
journalctl -u prefect-server -f

5) Troubleshooting tips - If the UI won’t load remotely: - Confirm server is listening on 0.0.0.0:

ss -ltnp | grep 4200

- Confirm your security group allows inbound TCP 4200 (or use the SSH tunnel). - Check logs:

journalctl -u prefect-server -f

- If you don’t see the UI: - Ensure

PREFECT_UI_ENABLED=true

(via

prefect profile inspect

) - Restart the server Reference docs: - Prefect 3.x installation - Quickstart If you’d like, share how you want to expose it (public IP, SSH tunnel, or behind Nginx/ALB with TLS), and I can give you a tailored config.

Perfect—here’s the simplest, secure-ish way to expose Prefect Server over your EC2’s public IP. 1) Start the server bound to the public interface - Run this on the EC2 instance:

prefect server start --host 0.0.0.0 --port 4200 --ui

- Quick local check:

curl -I <http://127.0.0.1:4200>

2) Open the port on your EC2 Security Group - In AWS, add an inbound rule: - Type: TCP - Port: 4200 - Source: Your IP (recommended) or 0.0.0.0/0 (public) - Then access the UI from your browser: - http//<EC2 PUBLIC IP>4200/ 3) Persist settings so you don’t need flags every time

prefect config set PREFECT_SERVER_API_HOST=0.0.0.0 PREFECT_SERVER_API_PORT=4200 PREFECT_UI_ENABLED=true
prefect profile inspect

4) Point any remote clients/workers at your server - On any machine that should talk to your EC2 server:

prefect config set PREFECT_API_URL=http://<EC2_PUBLIC_IP>:4200/api

5) Optional: run it as a service (starts on boot) - Example systemd unit at /etc/systemd/system/prefect-server.service:

[Unit]
Description=Prefect Server
After=network.target

[Service]
User=ec2-user
WorkingDirectory=/home/ec2-user
Environment="PREFECT_SERVER_API_HOST=0.0.0.0" "PREFECT_SERVER_API_PORT=4200" "PREFECT_UI_ENABLED=true"
ExecStart=/home/ec2-user/venvs/prefect/bin/prefect server start
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

- Enable it:

sudo systemctl daemon-reload
sudo systemctl enable --now prefect-server
sudo systemctl status prefect-server

Troubleshooting - Confirm the server is listening:

ss -ltnp | grep 4200

- Check health from your laptop:

curl http://<EC2_PUBLIC_IP>:4200/api/health

- If you can’t reach it, re-check the Security Group and any OS firewalls (ufw/firewalld). Docs - Prefect Server CLI - Settings reference - Self-hosted server guide If you share your Linux user/venv path, I can tailor the systemd unit so it’s copy/paste ready. Also, if you want HTTPS, we can put Nginx in front with TLS and a friendly DNS name.

That popup means the UI is trying to call the API at 0.0.0.0, which a browser can’t reach. You bound the API to 0.0.0.0 (good for listening), but the UI also used that value to build its API URL. Fix by telling the UI the public address of your server. Do this on the EC2 instance, replacing <EC2_PUBLIC_IP>:

# Make the API listen on all interfaces
prefect config set PREFECT_SERVER_API_HOST=0.0.0.0 PREFECT_SERVER_API_PORT=4200

# Tell the UI where to find the API from a browser
prefect config set PREFECT_SERVER_UI_API_URL=http://<EC2_PUBLIC_IP>:4200/api

# (UI is usually on by default, but ensure it)
prefect config set PREFECT_UI_ENABLED=true

# Restart the server
prefect server start

Sanity checks - From your laptop:

curl http://<EC2_PUBLIC_IP>:4200/api/health

Should return a small JSON OK response. - In the browser, hard refresh (or open dev tools > Network and confirm calls go to http//&lt;EC2 PUBLIC IP&gt;4200/api/... not 0.0.0.0). Notes - 0.0.0.0 is valid for binding the server, but not for the UI’s API URL. - If you later use a DNS name or a load balancer, set

PREFECT_SERVER_UI_API_URL

to that (e.g., https://prefect.mycompany.com/api). References - Prefect Server CLI - Settings reference

Short answer: you don’t need additional Prefect Server configuration beyond what you have. The key is making sure your workers and ECS tasks can reach your server’s API at http//&lt;EC2 PUBLIC IP&gt;4200/api. Here’s a minimal, working path to run flows on ECS with a “pull” worker: 1) Install the AWS integration where you’ll run the worker

pip install "prefect[aws]"
# or
pip install prefect-aws

2) Create an ECS work pool

prefect work-pool create my-ecs --type ecs

3) Configure the pool’s base job template (cluster, subnets, roles, image, etc.) - Get the default template, edit, then apply:

prefect work-pool get-default-base-job-template --type ecs --file ecs-template.json
# edit ecs-template.json:
# - cluster
# - launch_type (FARGATE or EC2)
# - subnets, security_groups, assign_public_ip
# - execution_role_arn, task_role_arn
# - cpu, memory
# - image (e.g., your ECR image) and optional extra_pip_packages
# - region, etc.
prefect work-pool update my-ecs --base-job-template ecs-template.json

4) Start the ECS worker (it needs to reach your server)

export PREFECT_API_URL=http://<EC2_PUBLIC_IP>:4200/api
prefect worker start --pool my-ecs

- The worker must have AWS permissions to submit tasks to ECS (ecs:RunTask, iam:PassRole, logs permissions, etc.). Give the machine/container an IAM role or configure AWS credentials. - Ensure the worker has network access to your EC2 server’s API. 5) Deploy a flow targeting that pool - With Prefect 3.x, from your project repo:

from prefect import flow

@flow
def hello():
    print("hi from ECS")

if __name__ == "__main__":
    hello.from_source(".", entrypoint="flows.py:hello").deploy(
        name="hello-ecs",
        work_pool_name="my-ecs",
    )

- Or use

prefect deploy

with a prefect.yaml. Networking considerations - Your ECS tasks must also be able to reach Prefect Server at http//&lt;EC2 PUBLIC IP&gt;4200/api to report state/logs. - If using Fargate in public subnets, set assign_public_ip=true and allow outbound internet. - If in private subnets, make sure there is a NAT gateway/route to the internet (or peer/VPN to the VPC where the server lives). - Keep the EC2 security group open for TCP 4200 from the sources that need it (your browser and the ECS tasks if they use public routing). - The worker will inject PREFECT_API_URL into the task environment; if you customize templates heavily, ensure it’s present. Alternative: ECS “push” (no worker) - If you prefer not to run a long-lived worker, use a push work pool:

prefect work-pool create my-ecs-push --type ecs:push
prefect work-pool provision-infrastructure my-ecs-push

- Deploy flows to

my-ecs-push

and Prefect will submit tasks directly to ECS. Still ensure tasks can reach PREFECT_API_URL. Docs - ECS guide (Prefect AWS integration) - Workers and work pools - Settings and profiles - Create deployments If you share: - ECS cluster name - VPC/subnet IDs and security group - Whether you want Fargate or EC2 - IAM role ARNs (task/execution) - Your ECR image (or if you want to use a Prefect base image + extra_pip_packages) I can fill in a ready-to-use ecs-template.json for your pool and a minimal Dockerfile if needed.

Short answer: use the API endpoint, not the bare UI URL. For a self-hosted Prefect 3 server on EC2, set

PREFECT_API_URL

to:

http://<EC2_PUBLIC_IP>:4200/api

The UI lives at

/

(e.g., http//&lt;EC2 PUBLIC IP&gt;4200), while workers must talk to the API at

/api

on the same port. Tips: - If your ECS tasks can reach the server over a private network, prefer the private IP, an internal DNS name, or an ALB URL: e.g.,

<http://ip-or-hostname:4200/api>

or

<https://your-domain/api>

(if terminating TLS at a load balancer). - Open security group access from the ECS tasks to the server on port 4200. - You can verify connectivity with:

curl -sf http://<host>:4200/api/health

- From any environment where Prefect is configured, you can confirm the effective value with:

prefect config view

Relevant docs: - ECS integration guide - Self-hosted server (Docker) guide - Settings reference (see

PREFECT_API_URL

) If you’re using Prefect Cloud instead of a self-hosted server, the URL is different—happy to help you set that if needed.

Short answer: No. If your self-hosted server has

server.api.auth_string = null

(the default), the API is not protected with Basic Auth and you do not need to set

PREFECT_API_AUTH_STRING

on workers/clients. If you want to enable Basic Auth on your self-hosted API: 1) Choose credentials (avoid colons in username/password): e.g.,

admin:StrongPass!

. 2) Set it on the server and restart the server: - Via environment variable (server process):

export PREFECT_SERVER_API_AUTH_STRING='admin:StrongPass!'
prefect server start

- Or set it in the server’s profile:

prefect config set server.api.auth_string='admin:StrongPass!'
prefect server start

- If using Docker/containers, pass

PREFECT_SERVER_API_AUTH_STRING

into the server container. If using Helm, set

server.api.auth_string

in values. 3) Set the same value on every client/worker that talks to the API: - For ECS, inject it as a secret or env var in the task definition: - Name:

PREFECT_API_AUTH_STRING

- Value:

admin:StrongPass!

- Ensure

PREFECT_API_KEY

is not set (it’s for Prefect Cloud and can interfere). 4) Verify:

curl -sf -u admin:StrongPass! http://<your-host>:4200/api/health

A 200 response confirms auth is working. Docs: - Security settings (self-hosted API auth) - Settings reference (see

server.api.auth_string

and

api.auth_string

) - Manage settings and profiles - Server CLI

Great — since you’ve got an ECS worker up, here’s the quickest way to wire your flow, prefect.yaml, and ECR together and run a test. I’ll show two options: A) simplest: use a Prefect base image + pull your code at runtime B) bake your code into your own ECR image Before you start - Make sure the machine where you’ll run CLI commands has

PREFECT_API_URL

set to your server’s API endpoint:

export PREFECT_API_URL=http://<your-server>:4200/api

- Your deployment’s work_pool.name must match the ECS work pool your worker is polling. - Your ECS task needs permission to pull the image (via the task execution role) and reach your Prefect Server. Option A: Use Prefect base image + pull your code 1) Put your code in a Git repo (or any git-accessible location). At the repo root, include your

flow.py

and

prefect.yaml

. Example flow.py:

from prefect import flow

@flow
def hello():
    print("Hello from ECS!")

if __name__ == "__main__":
    hello()

Example prefect.yaml (project root):

name: ecs-test
prefect-version: ">=3.0.0"

pull:
  - prefect.deployments.steps.git_clone:
      repository: "<https://github.com/your-org/your-repo.git>"
      branch: "main"
  - prefect.deployments.steps.set_working_directory:
      directory: "."

deployments:
  - name: hello-ecs
    entrypoint: "flow.py:hello"
    work_pool:
      name: "your-ecs-pool-name"
      job_variables:
        image: "public.ecr.aws/prefecthq/prefect:3-python3.11"
        cpu: "256"
        memory: "512"
        cluster: "your-ecs-cluster-name"
        subnets:
          - "subnet-abc123"
        security_groups:
          - "sg-abc123"
        execution_role_arn: "arn:aws:iam::<acct-id>:role/ecsTaskExecutionRole"
        task_role_arn: "arn:aws:iam::<acct-id>:role/ecsTaskRole"
        assign_public_ip: false

2) From the repo root, create the deployment:

prefect deploy

3) Run it:

prefect deployment run hello/hello-ecs --watch

This will: - Use the Prefect base image on ECS - Clone your repo at runtime into the container - Run

flow.py:hello

Option B: Bake your code into your own ECR image 1) Create a Dockerfile next to your `flow.py`:

FROM public.ecr.aws/prefecthq/prefect:3-python3.11
WORKDIR /app
COPY . /app
# If you have dependencies:
# COPY requirements.txt /app/
# RUN pip install -r requirements.txt

2) Build and push to your ECR repo:

docker build -t <acct>.dkr.ecr.<region>.<http://amazonaws.com/<repo>:latest|amazonaws.com/<repo>:latest> .
aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <acct>.dkr.ecr.<region>.<http://amazonaws.com|amazonaws.com>
docker push <acct>.dkr.ecr.<region>.<http://amazonaws.com/<repo>:latest|amazonaws.com/<repo>:latest>

3) Prefect.yaml for baked image (no git pull needed):

name: ecs-test
prefect-version: ">=3.0.0"

deployments:
  - name: hello-ecs
    entrypoint: "flow.py:hello"
    work_pool:
      name: "your-ecs-pool-name"
      job_variables:
        image: "<acct>.dkr.ecr.<region>.<http://amazonaws.com/<repo>:latest|amazonaws.com/<repo>:latest>"
        cpu: "256"
        memory: "512"
        cluster: "your-ecs-cluster-name"
        subnets:
          - "subnet-abc123"
        security_groups:
          - "sg-abc123"
        execution_role_arn: "arn:aws:iam::<acct-id>:role/ecsTaskExecutionRole"
        task_role_arn: "arn:aws:iam::<acct-id>:role/ecsTaskRole"
        assign_public_ip: false

4) Deploy and run:

prefect deploy
prefect deployment run hello/hello-ecs --watch

Where everything “goes” - flow.py and prefect.yaml: live together at your project root (ideally in a git repo). - ECR repo: stores your container image, either your own image (Option B) or you can skip and use the Prefect base image (Option A). - Work pool and worker: your deployment’s

work_pool.name

must match the ECS pool your worker is polling. The worker will launch ECS tasks using the

job_variables

you set. Common gotchas - Make sure your task execution role can pull from ECR and write logs to CloudWatch.

- Ensure the cluster/subnets/security groups in

job_variables

match the environment where the ECS tasks should run. - If your API is protected, set

PREFECT_API_AUTH_STRING

on the worker/containers. If you left

server.api.auth_string = null

, you don’t need it. Handy docs: - ECS integration guide - prefect.yaml reference - Customize job variables - Store (and pull) flow code If you share your current

prefect.yaml

and the ECS pool name, I can tailor the job_variables and entrypoint exactly for your setup.

That error means your ECS task definition is missing an execution role. For Fargate, you must set an executionRoleArn so the task can pull from ECR (and write logs to CloudWatch). What to do 1) Create or identify an execution role - If you already have one (often named ecsTaskExecutionRole), grab its ARN and skip to step 2. - Otherwise, create it and attach the AWS-managed policy:

# trust policy (trust.json)
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "<http://ecs-tasks.amazonaws.com|ecs-tasks.amazonaws.com>" },
      "Action": "sts:AssumeRole"
    }
  ]
}

aws iam create-role --role-name ecsTaskExecutionRole --assume-role-policy-document <file://trust.json>
aws iam attach-role-policy --role-name ecsTaskExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

Copy the role ARN:

aws iam get-role --role-name ecsTaskExecutionRole --query 'Role.Arn' --output text

2) Add the role to your Prefect deployment’s ECS job variables - In prefect.yaml under your deployment:

deployments:
  - name: hello-ecs
    entrypoint: "flow.py:hello"
    work_pool:
      name: "your-ecs-pool-name"
      job_variables:
        image: "<acct>.dkr.ecr.<region>.<http://amazonaws.com/<repo>:latest|amazonaws.com/<repo>:latest>"
        cpu: "256"
        memory: "512"
        cluster: "your-ecs-cluster-name"
        subnets: ["subnet-abc123"]
        security_groups: ["sg-abc123"]
        execution_role_arn: "arn:aws:iam::<acct-id>:role/ecsTaskExecutionRole"   # <-- required for ECR pulls
        task_role_arn: "arn:aws:iam::<acct-id>:role/ecsTaskRole"                # optional, for your flow's AWS access

- Alternatively, set it at the work pool level (base job template) so all deployments inherit it. 3) Redeploy and test

prefect deploy
prefect deployment run <FLOW_NAME>/<DEPLOYMENT_NAME> --watch

Quick one-off test (no file edits) You can pass the role just for this run:

prefect deployment run <FLOW_NAME>/<DEPLOYMENT_NAME> \
  --job-variable execution_role_arn='"arn:aws:iam::<acct-id>:role/ecsTaskExecutionRole"' \
  --watch

Note the value is parsed as JSON, so quote it as shown. Notes - execution_role_arn is for pulling images from ECR and sending logs; task_role_arn is for your code accessing AWS services. - If your tasks are in private subnets without NAT or VPC endpoints, Fargate won’t reach ECR even with the right role. Either add NAT or set up VPC endpoints for ECR/ECR DKR. - You can view the ECS template Prefect uses with:

prefect work-pool get-default-base-job-template --type ecs

Docs - Prefect ECS integration guide - Customize job variables - AWS: Task execution IAM role

Nice — sounds like the work pool picked up your run and submitted it to your infrastructure. A few clarifications will help me give exact commands: - Which worker type are you using (Process, Docker, Kubernetes, ECS, Azure ACI, etc.)? - What’s the jobId string you see? - Do you have the flow run ID/link from the UI? Meanwhile: 1) Where to find the submitted job + logs - In the Prefect UI - Open the flow run > Logs tab for Prefect logs (what your code/loggers print). - On the Run details page, check Infrastructure → Job ID. That ID maps to the underlying infra job/container/pod/task. - You can also pull the same logs via CLI:

prefect flow-run logs <flow-run-id>

And inspect run metadata:

prefect flow-run inspect <flow-run-id>

- Depending on your worker type, the underlying job and its native logs live here: - Process worker (local): The job runs as a local process on the machine where the worker runs. Logs are in the Prefect UI and in the worker’s stdout/stderr. - Docker worker: A Docker container on the worker host. Find and view logs:

docker ps -a
    docker logs <container-id-or-name>    # often the jobId is the container ID

Note: if the worker is configured to clean up, the container may be removed right after it exits. Docs: Docker worker guide - Kubernetes worker: A Kubernetes Job/Pod in your cluster/namespace. If you know the flow run ID, you can often find by label; then fetch logs:

kubectl -n <namespace> get pods
    kubectl -n <namespace> logs <pod-name> -c prefect

Docs: Kubernetes worker guide - AWS ECS worker: An ECS task; logs are in CloudWatch for the container named

prefect

by default. Look up the task in your cluster, then open the CloudWatch log stream. Docs: ECS worker guide - Azure ACI worker: An Azure Container Instance; logs via Azure portal or CLI:

az container logs -g <resource-group> -n <container-group-name>

Docs: Azure ACI worker guide 2) About the crash: “The following containers stopped with a non-zero exit code: ['prefect']” - Meaning: The container named

prefect

(the main container Prefect runs your flow in) exited with a non-zero status. This is typical for Docker/ECS/ACI/K8s and indicates the container crashed or the process inside returned an error. - Most common causes you’ll see in the container’s logs: - Import/module errors (your flow or dependencies aren’t in the image) - Wrong image or missing project dependencies - Code raised an exception during startup - Credentials/secrets/ENV variables missing - Network/registry pull issues - OOMKilled or resource limits too low (K8s/ECS) What I suggest next - Share your worker type and the jobId; I’ll give exact commands to retrieve logs. - Grab the underlying container/pod/task logs using the steps above. That will show the real root cause (import error, traceback, OOM, etc.). - If you’re using containers, confirm your deployment image includes your code and dependencies. A quick sanity check is to run the image locally and import your flow code. - If you’re stuck, paste the last ~50 lines of the infra logs here and I’ll help debug. References: - Workers overview: Workers concepts - Docker: Docker worker guide - Kubernetes: Kubernetes worker guide - ECS: ECS worker guide

- Azure ACI: Azure ACI worker guide

Thanks for the detail — this helps a lot. You’ve got a Prefect 3.x ECS work pool polling via SQS and submitting tasks; the container isn’t starting and you don’t have CloudWatch logs yet. Here’s how to find the failing ECS task, enable logs, and the most common fixes. First, can you share: - The jobId shown on your flow run’s “Infrastructure → Job ID” in the UI - Whether you’re using Fargate (most likely) - Your Prefect API URL (where your self-hosted server is reachable from the ECS subnets) 1) Find the ECS task and its stopped reason now Even without CloudWatch logs, the ECS task details will show a “Stopped reason” that’s usually very telling. - AWS Console - ECS → Clusters → my-ecs-cluster → Tasks → filter by “Last 1 hour” and desired status = Stopped - Open the latest task; check “Stopped reason” and each container’s exit code. - Common messages: - CannotPullContainerError (permissions or networking) - Resource requirements (invalid CPU/memory combo) - Essential container in task exited (your

prefect

container crashed) - AWS CLI (if you prefer)

aws ecs list-tasks --cluster my-ecs-cluster --desired-status STOPPED
  aws ecs describe-tasks --cluster my-ecs-cluster --tasks <task-arn>

Look at

stoppedReason

and the

containers[].reason/exitCode

. 2) Enable CloudWatch logs for all future runs (strongly recommended) Right now, your tasks likely have no

logConfiguration

, so there’s nothing in CloudWatch. You have two easy ways to enable logs: Option A — Edit the ECS work pool’s Base Job Template (recommended) - In the Prefect UI: Work Pools → my-ecs-pool → Configuration → Edit Base Job Template - Add a logConfiguration to the container definition. For example:

{
    "variables": {
      "cluster": null,
      "subnets": null,
      "security_groups": null,
      "execution_role_arn": null,
      "task_role_arn": null,
      "cpu": "512",
      "memory": "1024",
      "assign_public_ip": true,
      "awslogs_group": "/prefect/ecs",
      "awslogs_region": "us-east-2",
      "awslogs_stream_prefix": "prefect"
    },
    "task_definition": {
      "family": "prefect-ecs-pool",
      "networkMode": "awsvpc",
      "requiresCompatibilities": ["FARGATE"],
      "cpu": "{{ cpu }}",
      "memory": "{{ memory }}",
      "executionRoleArn": "{{ execution_role_arn }}",
      "taskRoleArn": "{{ task_role_arn }}",
      "containerDefinitions": [
        {
          "name": "prefect",
          "image": "{{ image }}",
          "essential": true,
          "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
              "awslogs-group": "{{ awslogs_group }}",
              "awslogs-region": "{{ awslogs_region }}",
              "awslogs-stream-prefix": "{{ awslogs_stream_prefix }}"
            }
          }
        }
      ]
    },
    "run_task_kwargs": {
      "cluster": "{{ cluster }}",
      "launchType": "FARGATE",
      "networkConfiguration": {
        "awsvpcConfiguration": {
          "subnets": {{ subnets }},
          "securityGroups": {{ security_groups }},
          "assignPublicIp": "{{ 'ENABLED' if assign_public_ip else 'DISABLED' }}"
        }
      }
    }
  }

- Create the CloudWatch log group

/prefect/ecs

first (or let the execution role create it — see IAM note below). Option B — Use a pre-created Task Definition - Create an ECS task definition in AWS that includes the

awslogs

log configuration above. - In your deployment’s

work_pool.job_variables

, set

task_definition_arn

to that task definition so Prefect uses it. CLI tip to view the base job template you start from:

prefect work-pool get-default-base-job-template --type ecs --file ecs-template.json

You can adapt that JSON in your pool’s Configuration. Docs: ECS worker guide and Manage work pools

3) IAM and networking checks that commonly cause “container exited” on ECS - Execution role permissions - The role you set as

execution_role_arn

must have the managed policy

AmazonECSTaskExecutionRolePolicy

attached. This covers ECR pulls and CloudWatch logs. Without it, you’ll see “CannotPullContainerError” and no logs. - Trust relationship should include

<http://ecs-tasks.amazonaws.com|ecs-tasks.amazonaws.com>

. - Networking - If your subnets are private, ensure there’s a NAT so tasks can pull from ECR and reach your Prefect server. If they’re public and you want internet egress, set

assignPublicIp: ENABLED

in the work pool template or

job_variables

. - Security group must allow egress to ECR, CloudWatch Logs, and your Prefect server host/port. - Prefect server reachability - If your self-hosted Prefect API is not reachable from the ECS network, the container will start, fail to connect, and exit quickly. Make sure the Prefect API URL is reachable from those subnets/SGs. - The worker typically injects

PREFECT_API_URL

for flow runs; if in doubt, you can explicitly add it:

work_pool:
      name: my-ecs-pool
      job_variables:
        env:
          PREFECT_API_URL: "http://<your-server-host>:<port>/api"

Replace with your actual server URL accessible from ECS. - CPU/memory and platform - Fargate requires valid cpu/memory combos (e.g., 256/512, 512/1024, 1024/2048, etc.). If you override these, use a valid pair. - Ensure your region and platform version are supported for your settings. - Image and flow code - Confirm the ECR image exists at the tag you deployed and that your flow code is baked into the image at the path you expect. From your

prefect.yaml

, that should be true, but you can test locally:

docker run --rm -it <acct-id>.<http://dkr.ecr.us-east-2.amazonaws.com/prefect-test-1:latest|dkr.ecr.us-east-2.amazonaws.com/prefect-test-1:latest> python -c "import flow; print('ok')"

- Minor note: in your Slack snippet,

<http://logger.info|logger.info>

was linked. Ensure your actual file has

<http://logger.info|logger.info>("Hello from ECS!!")

and not Slack’s linkified text. 4) Small updates to your deployment config that may help - Add

task_role_arn

if your flow needs AWS access at runtime:

work_pool:
    name: my-ecs-pool
    job_variables:
      image: '{{ build_image.image }}'
      cluster: "my-ecs-cluster"
      subnets: ["subnet-..."]
      security_groups: ["sg-..."]
      execution_role_arn: "arn:aws:iam::<acct>:role/PrefectEcsExecutionRole"
      task_role_arn: "arn:aws:iam::<acct>:role/PrefectEcsTaskRole"
      # if your subnets are public and you want internet egress:
      assign_public_ip: true
      env:
        PREFECT_LOGGING_LEVEL: "INFO"

Once you enable CloudWatch logs, re-run a flow and you’ll get container logs under the log group you set (e.g.,

/prefect/ecs

) with stream prefix

prefect

. That will immediately show the traceback or pull/network error if any. If you can, please paste: - The ECS task “Stopped reason” and container “Exit code/Reason” from the task details - The jobId / task ARN from the Prefect UI - Whether you need assignPublicIp or have NAT on your subnets - Which execution role policy attachments you’ve set I’ll help zero in on the exact fix.

✅ thought for 44.2 seconds

Got it — on an M4, your local Docker builds ARM64 images by default. ECS Fargate typically runs x86_64 unless you tell it otherwise. That arch mismatch is exactly what causes: - CloudWatch:

exec /usr/bin/tini: exec format error

- Task: essential container exited (exit 255) Two clean fixes — pick one: Option A (most common): Build your image for x86_64 - Update your prefect.yaml build step to force linux/amd64:

build:
- prefect_docker.deployments.steps.build_docker_image:
    id: build_image
    requires: prefect-docker>=0.3.1
    image_name: <acct-id>.<http://dkr.ecr.us-east-2.amazonaws.com/prefect-test-1|dkr.ecr.us-east-2.amazonaws.com/prefect-test-1>
    tag: latest
    dockerfile: auto
    build_kwargs:
      platform: linux/amd64

- Redeploy to rebuild and push:

prefect deploy

- Trigger a new run. Option B: Keep ARM64 images and run ECS on ARM64 - Edit your ECS work pool’s Base Job Template (Work Pools → my-ecs-pool → Configuration → Edit) and add runtimePlatform to the task definition:

"task_definition": {
  "runtimePlatform": {
    "cpuArchitecture": "ARM64",
    "operatingSystemFamily": "LINUX"
  },
  ...
}

- Save, then run again. Notes - Your execution role is correct to include AmazonECSTaskExecutionRolePolicy; you don’t need the App Runner policy for ECS. - If you haven’t already, add a CloudWatch logConfiguration in the work pool template so logs always land in a predictable group (optional but helpful):

"logConfiguration": {
  "logDriver": "awslogs",
  "options": {
    "awslogs-group": "/prefect/ecs",
    "awslogs-region": "us-east-2",
    "awslogs-stream-prefix": "prefect"
  }
}

- ECS docs for Prefect: ECS worker guide If you prefer, I can provide a full Base Job Template JSON you can paste that includes runtimePlatform and logs. After you try Option A (or B), if it still crashes, paste the new CloudWatch lines and I’ll help pinpoint next steps.

You can absolutely use a prebuilt image. In Prefect 3.x, you don’t need the build/push steps in prefect.yaml—just point your deployment’s

work_pool.job_variables.image

at your image and make sure the container has your code and Prefect installed. What to change: - Remove (or comment out) the

build:

and

push:

sections. - Set the image on the deployment via

work_pool.job_variables.image

. - Ensure the container’s working directory matches where your

flow.py

lives (or adjust the

pull

step accordingly). Example minimal prefect.yaml:

name: ecs-worker-guide

# Optional: set the working directory inside your image to where flow.py lives
pull:
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect/app   # update to your path in the image

deployments:
- name: my_ecs_deployment
  entrypoint: flow.py:my_flow     # must be importable from the working dir
  work_pool:
    name: my-ecs-pool
    job_variables:
      image: "<acct>.<http://dkr.ecr.us-east-2.amazonaws.com/prefect-test-1:my-fixed-tag|dkr.ecr.us-east-2.amazonaws.com/prefect-test-1:my-fixed-tag>"
      cluster: "my-ecs-cluster"
      subnets: ["subnet-..."]
      security_groups: ["sg-..."]
      execution_role_arn: "arn:aws:iam::<acct>:role/PrefectEcsExecutionRole"
      task_role_arn: "arn:aws:iam::<acct>:role/PrefectEcsTaskRole"
      cpu: "512"
      memory: "1024"
      assign_public_ip: true
      env:
        PREFECT_API_URL: "http://<your-prefect-server>/api"
        PREFECT_LOGGING_LEVEL: "INFO"

Notes: - If you don’t need to change directories at runtime, you can remove the

pull:

section entirely. Just ensure

entrypoint

(e.g.,

flow.py:my_flow

) is importable from the container’s default working dir. - Use a fixed tag (avoid reusing

latest

) and make sure the tag in

job_variables.image

matches what you pushed to ECR. - If you’re targeting x86_64 on ECS, make sure the image you pushed is linux/amd64; for ARM64, set your work pool’s Base Job Template

runtimePlatform.cpuArchitecture

to

ARM64

. Docs: - ECS worker guide: ECS worker guide - prefect.yaml reference: prefect.yaml guide - Customizing job variables: Customize job variables If you share the path where

flow.py

lives inside your image, I can confirm the exact

directory

to set (or whether you can drop the

pull

step).