<@ULVA73B9P> I'm doing an ECS deployment, and I'm ...
# ask-marvin
d
@Marvin I'm doing an ECS deployment, and I'm pulling an image from a private repository. What would be a good way to provide credentials for the repository? Blocks?
m
Error: No module named 'prefect_aws.workers.ecs'
thought for 102.4 seconds
thought for 148.6 seconds
Error: No module named 'prefect_aws.workers'
Short answer: don’t use Prefect blocks for this. Let ECS do what it’s designed to do. Best options: - If your image is in ECR (recommended) - Put the image in ECR and give your ECS task execution role the AmazonECSTaskExecutionRolePolicy. ECS will handle registry auth via IAM automatically. No extra secrets, no blocks, no drama. - If your image is in another private registry (Docker Hub, GHCR, etc.) - Create a secret in AWS Secrets Manager that contains your registry creds (either JSON with username/password or a Docker config JSON). - Grant your ECS task execution role permission to read that secret (
secretsmanager:GetSecretValue
). - In your Prefect ECS work pool’s Base Job Template, set the container definition’s
repositoryCredentials.credentialsParameter
to the ARN of that secret. For example:
Copy code
{
      "containerDefinitions": [
        {
          "name": "prefect",
          "image": "<http://ghcr.io/acme/flow:latest|ghcr.io/acme/flow:latest>",
          "repositoryCredentials": {
            "credentialsParameter": "arn:aws:secretsmanager:us-east-1:123456789012:secret:ghcr-creds-abc123"
          },
          "essential": true
        }
      ],
      "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
      ...
    }
- If you don’t want to hardcode the ARN, add a work pool variable like
registry_credentials_arn
and reference it in the template:
Copy code
"repositoryCredentials": { "credentialsParameter": "{{ job_variables.registry_credentials_arn }}" }
Why not Prefect blocks? The image is pulled by ECS before your flow starts, so Prefect can’t hand ECS credentials at that point. Use IAM/ECS/Secrets Manager for image pulls; use Prefect blocks for things your flow needs at runtime. Useful docs: - Private registry auth for ECS: AWS: Private registry authentication - Task execution role (required for pulls and reading Secrets Manager): AWS: Task execution IAM role If you tell me which registry you’re using (ECR, Docker Hub, GHCR, something more… exotic), I can give you an exact setup checklist.
thought for 24.6 seconds
Short answer: don’t use Prefect Blocks for this. ECS pulls the image, so AWS needs the creds — not Prefect. Good options, depending on your registry: - Amazon ECR: Best path. Give your ECS task execution role the AmazonECSTaskExecutionRolePolicy (or the equivalent ECR permissions) and you don’t need to pass any credentials at all. ECS will pull the image via IAM. - Docs: ECS task execution role - Docker Hub / GHCR / other private registries: Create a secret in AWS Secrets Manager with your registry credentials and reference it via repositoryCredentials in the ECS task definition your Prefect ECS worker uses. - Secret format (yes, it’s just username/password or token):
Copy code
{
  "username": "<your-username>",
  "password": "<your-password-or-token>"
}
For GHCR, password = a PAT with appropriate scopes; username = your GitHub username. - Docs: Private registry authentication for ECS How to wire it into Prefect (3.x): - In your ECS work pool’s job template (UI or API), set the container’s repositoryCredentials to the ARN of that Secrets Manager secret. The worker will render the rest (like the image) from your deployment. Example ECS job template snippet for a Prefect ECS work pool:
Copy code
{
  "taskDefinition": {
    "family": "prefect-ecs-task",
    "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
    "taskRoleArn": "arn:aws:iam::123456789012:role/prefectTaskRole",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
      "FARGATE"
    ],
    "cpu": "1024",
    "memory": "2048",
    "containerDefinitions": [
      {
        "name": "prefect",
        "image": "{{ image }}",
        "repositoryCredentials": {
          "credentialsParameter": "arn:aws:secretsmanager:us-east-1:123456789012:secret:dockerhub-creds-abc123"
        },
        "environment": [
          {
            "name": "PREFECT_API_URL",
            "value": "{{ api_url }}"
          },
          {
            "name": "PREFECT_API_KEY",
            "value": "{{ api_key }}"
          },
          {
            "name": "PREFECT_FLOW_RUN_ID",
            "value": "{{ flow_run_id }}"
          }
        ],
        "essential": true
      }
    ]
  },
  "cluster": "my-ecs-cluster",
  "launchType": "FARGATE",
  "assignPublicIp": "ENABLED",
  "subnets": [
    "subnet-abc",
    "subnet-def"
  ],
  "securityGroups": [
    "sg-123"
  ]
}
Creating the secret (example):
Copy code
aws secretsmanager create-secret \
  --name dockerhub-creds \
  --secret-string '{"username":"myuser","password":"my-personal-access-token"}'
References: - Prefect ECS worker and job template docs - ECS private registry auth - ECS container definition (repositoryCredentials) - ECR image pull permissions If you tell me which registry you’re using (ECR, Docker Hub, GHCR, something more exotic), I’ll tailor the exact setup. I know, thrilling.
Short version: don’t use Prefect blocks. Let ECS handle registry auth. Best practice - ECR (recommended) - Push the image to ECR and give your ECS task execution role the AmazonECSTaskExecutionRolePolicy. ECS will pull the image via IAM—no secrets, no blocks, no drama. - Docs: AWS: ECR on ECS and Task execution IAM role - Private registries (Docker Hub, GHCR, etc.) 1) Put your registry creds in AWS Secrets Manager (either a Docker config JSON or username/password—AWS supports both; see the doc for exact formats). - Doc: AWS: Private registry authentication 2) Grant your ECS task execution role permission to read that secret (
secretsmanager:GetSecretValue
). 3) In your Prefect ECS work pool’s Base Job Template, add
repositoryCredentials.credentialsParameter
(the secret ARN) to the container definition that runs your flow. For example:
Copy code
{
  "containerDefinitions": [
    {
      "name": "prefect",
      "image": "<http://ghcr.io/acme/flow:latest|ghcr.io/acme/flow:latest>",
      "repositoryCredentials": {
        "credentialsParameter": "arn:aws:secretsmanager:us-east-1:123456789012:secret:ghcr-creds-abc123"
      },
      "essential": true
    }
  ],
  "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole"
}
Or parameterize it:
Copy code
"repositoryCredentials": { "credentialsParameter": "{{ job_variables.registry_credentials_arn }}" }
Why not blocks? - ECS pulls the image before your flow starts. Prefect blocks can’t help with auth at that stage; they’re for runtime secrets inside your flow. Tell me which registry you’re using (ECR, Docker Hub, GHCR, something exotic) and I’ll give you an exact checklist, including the right secret format and any needed scopes/permissions.