After using `publish_as_work_pool()` to move my ba...
# prefect-aws
s
After using
publish_as_work_pool()
to move my base infra to a work pool and working through the error above, I'm still having issues. It seems my
job_variables
config is not being applied. details in thread.
stack trace:
Copy code
Failed to submit flow run 'abc' to infrastructure.
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prefect/workers/base.py", line 904, in _submit_run_and_capture_errors
    result = await self.run(
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect_aws/workers/ecs_worker.py", line 638, in run
    ) = await run_sync_in_worker_thread(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 95, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/prefect_aws/workers/ecs_worker.py", line 754, in _create_task_and_wait_for_start
    self._wait_for_task_start(
  File "/usr/local/lib/python3.11/site-packages/prefect_aws/workers/ecs_worker.py", line 1032, in _wait_for_task_start
    raise type(code, (RuntimeError,), {})(reason)
prefect_aws.workers.ecs_worker.TaskFailedToStart: CannotPullContainerError: pull image manifest has been retried 1 time(s): failed to resolve ref 1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest: 1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest: not found
deployment config for job variables in UI:
Copy code
{
  "image": "1234.dkr.ecr.us-east-1.amazonaws.com/prefect:heartbeat-switch-to-workers"
}
Flow run debug info:
Copy code
Task definition {
  "cpu": "512",
  "memory": "1024",
  "networkMode": "awsvpc",
  "taskRoleArn": "arn:aws:iam::1234:role/dev-prefect-agent-task-role",
  "executionRoleArn": "arn:aws:iam::1234:role/dev-prefect-agent-execution-role",
  "containerDefinitions": [
    {
      "name": "log_router",
      "image": "<http://1234.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit|1234.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit>",
      "essential": true,
      "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
          "enable-ecs-log-metadata": "true"
        }
      }
    },
    {
      "cpu": 512,
      "name": "prefect",
      "image": "<http://1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest|1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest>",
      "memory": 1024,
      "essential": true,
      "logConfiguration": {
        "options": {
          "Name": "newrelic",
          "apiKey": "abc",
          "endpoint": "<https://log-api.newrelic.com/log/v1>"
        },
        "logDriver": "awsfirelens"
      }
    }
  ],
  "placementConstraints": [],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "family": "prefect_engineering-work-pool_abc"
}
Why is my override/job variable not being applied to the
image
key?