Stephen Lloyd
04/29/2024, 7:29 AMpublish_as_work_pool()
to move my base infra to a work pool and working through the error above, I'm still having issues. It seems my job_variables
config is not being applied. details in thread.Stephen Lloyd
04/29/2024, 7:30 AMFailed to submit flow run 'abc' to infrastructure.
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/prefect/workers/base.py", line 904, in _submit_run_and_capture_errors
result = await self.run(
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/prefect_aws/workers/ecs_worker.py", line 638, in run
) = await run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/prefect/utilities/asyncutils.py", line 95, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/prefect_aws/workers/ecs_worker.py", line 754, in _create_task_and_wait_for_start
self._wait_for_task_start(
File "/usr/local/lib/python3.11/site-packages/prefect_aws/workers/ecs_worker.py", line 1032, in _wait_for_task_start
raise type(code, (RuntimeError,), {})(reason)
prefect_aws.workers.ecs_worker.TaskFailedToStart: CannotPullContainerError: pull image manifest has been retried 1 time(s): failed to resolve ref 1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest: 1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest: not found
Stephen Lloyd
04/29/2024, 7:30 AM{
"image": "1234.dkr.ecr.us-east-1.amazonaws.com/prefect:heartbeat-switch-to-workers"
}
Stephen Lloyd
04/29/2024, 7:32 AMTask definition {
"cpu": "512",
"memory": "1024",
"networkMode": "awsvpc",
"taskRoleArn": "arn:aws:iam::1234:role/dev-prefect-agent-task-role",
"executionRoleArn": "arn:aws:iam::1234:role/dev-prefect-agent-execution-role",
"containerDefinitions": [
{
"name": "log_router",
"image": "<http://1234.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit|1234.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit>",
"essential": true,
"firelensConfiguration": {
"type": "fluentbit",
"options": {
"enable-ecs-log-metadata": "true"
}
}
},
{
"cpu": 512,
"name": "prefect",
"image": "<http://1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest|1234.dkr.ecr.us-east-1.amazonaws.com/prefect:latest>",
"memory": 1024,
"essential": true,
"logConfiguration": {
"options": {
"Name": "newrelic",
"apiKey": "abc",
"endpoint": "<https://log-api.newrelic.com/log/v1>"
},
"logDriver": "awsfirelens"
}
}
],
"placementConstraints": [],
"requiresCompatibilities": [
"FARGATE"
],
"family": "prefect_engineering-work-pool_abc"
}
Stephen Lloyd
04/29/2024, 7:33 AMimage
key?