Thanks for the report — that error means the ECS task was able to start the flow run but, at runtime, Prefect could not load the flow code from the deployment’s source/entrypoint. The trailing FileNotFoundError usually points to the exact file/module it expected but did not find. Because it’s intermittent, it’s almost always a transient pull/entrypoint issue rather than a permanent misconfiguration. A quick mental model of what happens on each flow run (Prefect 3.x): - Prefect runner starts in your ECS task - It executes any “pull” steps to fetch your flow code (GitHub/S3/etc.) - It resolves the deployment’s

entrypoint

and imports your flow - If the entrypoint file/module isn’t present (or the pull step failed), you’ll see “Flow can not be retrieved from deployment … FileNotFoundError” Most common causes of intermittent failures on ECS - Network egress hiccups from your ECS subnets to GitHub/S3 (no NAT, flaky NAT, misconfigured VPC endpoints, DNS) - Credentials occasionally failing (expired GitHub token, IAM role assumption issues) - Wrong or brittle entrypoint path (sometimes resolves relative to the wrong working directory) - Missing tools inside the image for your pull method (e.g., git not installed for Git pulls) - S3 permissions not attached to the ECS task role (or only to the execution role) Targeted checks to run 1) Capture the exact not-found path - In the failing task’s logs, note the FileNotFoundError’s path. That will tell us if the file the entrypoint references was actually pulled, and where Prefect looked. - If you can, re-run one failed flow and share the 5–10 lines around the error — it helps pinpoint if we’re failing during a pull step or during entrypoint resolution. 2) Validate entrypoint consistency - Make sure your deployment’s

entrypoint

matches the repository layout after your pull. It should be either: - file path relative to the repo root, plus

:function_name

(e.g.,

flows/my_flow.py:my_flow

) - or a fully-qualified module path (e.g.,

my_pkg.flows.my_flow:my_flow

) - Case sensitivity and folder nesting matter. Reference: Entrypoint reference 3) Verify how you fetch source code - If you use GitHub: - Ensure the token/key is valid and available to the ECS task. - Prefer HTTPS with a PAT over SSH for Fargate unless you’ve baked SSH and known_hosts into the image. - Pin to a specific ref/commit to avoid timing issues on fast-changing branches. - If you use S3/remote storage: - Make sure the ECS task role (not just the execution role) has S3 permissions for the bucket/prefix. - Typical minimal policy includes

s3:ListBucket

,

s3:GetObject

,

s3:GetObjectVersion

, and

s3:HeadObject

. Example S3 policy snippet to attach to the ECS task role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowListBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::my-bucket",
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "my-prefix/*"
          ]
        }
      }
    },
    {
      "Sid": "AllowGetObjects",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:HeadObject"
      ],
      "Resource": "arn:aws:s3:::my-bucket/my-prefix/*"
    }
  ]
}

4) ECS network sanity - If your tasks run in private subnets, ensure there’s outbound internet via a NAT gateway for GitHub/S3 (unless you have VPC endpoints correctly configured). - Security groups should allow egress on 443. - If you’re using VPC endpoints for S3, confirm endpoint policies permit List/Get on your bucket/prefix and that DNS is enabled for the VPC. AWS ECS guide: Prefect ECS guide 5) Ensure the image supports your pull method and working directory - For Git pulls, the container image needs

git

installed.

- Confirm the working directory used by your runner matches where the pull step writes files; otherwise the entrypoint path may be correct but the file exists in a different directory. 6) Make it more robust (two good options) - Bake code into the image. This avoids runtime network pulls entirely and is the most reliable pattern for production ECS runs. - Build your image with your repo included (COPY or a build step), and point the deployment entrypoint at that in-image path. - Or, pin source and add resilience: - Pin Git ref/commit - Ensure tokens/roles don’t expire - If you must pull at runtime, consider simple retry logic in a wrapper step that clones/downloads with retries before starting the Prefect runner. Related docs: - Storing/pulling flow code: Store flow code for deployments - Prefect YAML / deployments: prefect.yaml reference To help you quickly, could you share: - Your Prefect version (assuming 3.x, please confirm) - How your deployment’s source is configured (GitHub URL + entrypoint, or S3 path, etc.) - The exact entrypoint string - The ECS task role vs. execution role you’re using for the task - Whether your ECS subnets have NAT or S3/GitHub endpoints - The exact FileNotFoundError path from the logs With that info, I can pinpoint the failure stage and suggest the minimal change to stop the intermittency. If you prefer the “bake-into-image” approach, I can share a small example Dockerfile + deployment pattern as well.

Thanks — and thanks for the full traceback. Since you’re on Prefect 2.20.22, here’s what your logs tell us: - Prefect is trying to import the flow from the entrypoint

projects/test/flows.py:test_flow

- At runtime inside the ECS task, it looks for that file at

/opt/prefect/projects/test/flows.py

- Intermittently, that path does not exist in the container, so the import fails with FileNotFoundError In 2.x on ECS, this happens when the flow code is not reliably present in the container at the path your entrypoint references. That can be due to: - Code not baked into the image and no/failed pull step (git/S3) at runtime - Git/S3 pulls that sometimes fail due to network/credentials and leave the directory empty - Cloning/downsloading to a different directory than your entrypoint expects (e.g., repo ends up at

/opt/prefect/my-repo/...

vs

/opt/prefect/...

) - Working directory not set consistently before Prefect imports the entrypoint Two stable patterns to fix this (pick one) 1) Bake the code into your image (most reliable for ECS) - Ensure the file exists in the built image at

/opt/prefect/projects/test/flows.py

- Example Dockerfile:

FROM prefecthq/prefect:2.20.22-python3.11
WORKDIR /opt/prefect
COPY . /opt/prefect
RUN pip install -r requirements.txt

- Then keep your entrypoint as

projects/test/flows.py:test_flow

- You can add a minimal pull step just to set the working dir:

pull:
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect

2) Pull code at runtime and align paths - If using Git, explicitly clone to the same directory your entrypoint expects and set the working dir:

pull:
- prefect.deployments.steps.git_clone:
    repository: <https://github.com/ORG/REPO.git>
    branch: main
    clone_dir: /opt/prefect
    # token: "{{ prefect.blocks.github-credentials.MY_GH.token }}"  # if needed
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect

- If your repo lands at

/opt/prefect/REPO/…

, either: - change

clone_dir

to

/opt/prefect

so the repo contents are at that root, or - update your entrypoint to include the repo folder, e.g.

REPO/projects/test/flows.py:test_flow

- If using S3, pull to a known location and set the working dir to that folder:

pull:
- prefect_aws.deployments.steps.pull_from_s3:
    bucket: my-bucket
    folder: my-prefix
    aws_credentials: "{{ prefect.blocks.aws-credentials.my-creds }}"
- prefect.deployments.steps.set_working_directory:
    directory: /opt/prefect/my-prefix

Extra checks to remove intermittency - Verify the file exists in the container: run the image locally and

ls /opt/prefect/projects/test/flows.py

- If using Git pulls: - Ensure

git

is installed in the image - Pin a specific commit/sha (don’t pull a moving target) - Provide a PAT or correct SSH setup; add simple retries if your network is flaky - If using S3 pulls: - The ECS task role (not just the execution role) must have

s3:ListBucket

,

s3:GetObject

,

s3:GetObjectVersion

,

s3:HeadObject

for your bucket/prefix - Ensure your subnets have outbound internet (NAT) or properly configured S3 VPC endpoints - Set the ECS container working directory in your work pool’s job template (if you’re using an ECS worker and job template) to

/opt/prefect

so CWD and entrypoint resolution are consistent - Enable debug logs to see pull step output on failing runs (

PREFECT_LOGGING_LEVEL=DEBUG

) Docs (Prefect 2.x): - ECS guide: https://docs-2.prefect.io/latest/integrations/prefect-aws/ecs_guide/ - Storage and code retrieval: https://docs-2.prefect.io/latest/guides/deployment/storage-guide/ - Deployments overview: https://docs-2.prefect.io/latest/guides/prefect-deploy/ - Docker guide: https://docs-2.prefect.io/latest/guides/docker/

If you can share your

prefect.yaml

(especially the pull section) and how your image/work pool is configured, I can pinpoint exactly where the path mismatch is. Given your entrypoint and error path, the fastest fix is usually one of: - Bake the repo into the image at

/opt/prefect

and keep the entrypoint as-is, or - Ensure your pull step places the repo contents directly under

/opt/prefect

(not in a nested repo folder), and set the working directory to

/opt/prefect

.