Hi i’m having issues running an ecs task from a worker pool in an ECS service.
@flow
def my_flow():
logger = get_run_logger()
<http://logger.info|logger.info>("Hello from ECS!!")
My flow configuration looks like:
###
### A complete description of a Prefect Deployment for flow 'my-flow'
###
name: my-flow
description: null
version: c11a68e0ce6739c76dc99de079edeb29
# The work queue that will handle this deployment's runs
work_queue_name: default
work_pool_name: ecs-pool
tags: []
parameters: {}
schedule: null
is_schedule_active: true
infra_overrides: {}
###
### DO NOT EDIT BELOW THIS LINE
###
flow_name: my-flow
manifest_path: null
infrastructure:
type: ecs-task
env: {}
labels: {}
name: null
command: null
aws_credentials:
aws_access_key_id: null
aws_secret_access_key: null
aws_session_token: null
profile_name: null
region_name: null
aws_client_parameters:
api_version: null
use_ssl: true
verify: true
verify_cert_path: null
endpoint_url: null
config: null
block_type_slug: aws-credentials
task_definition_arn: null
task_definition: null
family: null
image: prefecthq/prefect:2-python3.10
auto_deregister_task_definition: true
cpu: null
memory: null
execution_role_arn: some-execution-role-arn
configure_cloudwatch_logs: true
cloudwatch_logs_options: {}
stream_output: null
launch_type: FARGATE
vpc_id: some-vpc-id
cluster: cluster-worker-arn
task_role_arn: null
task_customizations:
- op: add
path: /networkConfiguration/awsvpcConfiguration/securityGroups
value:
- some_security_group
task_start_timeout_seconds: 120
task_watch_poll_interval: 5.0
_block_document_id: 99c69452-38cb-4ce0-a0f9-65035b2db175
_block_document_name: default-ecs-job
_is_anonymous: false
block_type_slug: ecs-task
_block_type_slug: ecs-task
storage: null
path: /opt/prefect/flows
entrypoint: minerva/test.py:my_flow
parameter_openapi_schema:
title: Parameters
type: object
properties: {}
required: null
definitions: null
timestamp: '2023-09-27T14:08:05.642381+00:00'
triggers: []
enforce_parameter_schema: null
The error I’m getting:
Failed to submit flow run '180f4415-b052-40d4-9f7c-21c1b6220835' to infrastructure.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/tenacity/_init_.py", line 382, in _call_
result = fn(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/prefect_aws/workers/ecs_worker.py", line 1524, in _create_task_run
return ecs_client.run_task(**task_run_request)["tasks"][0]
File "/usr/local/lib/python3.10/site-packages/botocore/client.py", line 535, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.10/site-packages/botocore/client.py", line 980, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.ClusterNotFoundException: An error occurred (ClusterNotFoundException) when calling the RunTask operation: Cluster not found.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect_aws/workers/ecs_worker.py", line 724, in _create_task_and_wait_for_start
task = self._create_task_run(ecs_client, task_run_request)
File "/usr/local/lib/python3.10/site-packages/tenacity/_init_.py", line 289, in wrapped_f
return self(f, *args, **kw)
File "/usr/local/lib/python3.10/site-packages/tenacity/_init_.py", line 379, in _call_
do = self.iter(retry_state=retry_state)
File "/usr/local/lib/python3.10/site-packages/tenacity/_init_.py", line 326, in iter
raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x7f997b4445b0 state=finished raised ClusterNotFoundException>]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/workers/base.py", line 843, in _submit_run_and_capture_errors
result = await self.run(
File "/usr/local/lib/python3.10/site-packages/prefect_aws/workers/ecs_worker.py", line 567, in run
) = await run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
return await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/site-packages/prefect_aws/workers/ecs_worker.py", line 728, in _create_task_and_wait_for_start
self._report_task_run_creation_failure(configuration, task_run_request, exc)
File "/usr/local/lib/python3.10/site-packages/prefect_aws/workers/ecs_worker.py", line 819, in _report_task_run_creation_failure
raise RuntimeError(
RuntimeError: Failed to run ECS task, cluster 'default' not found. Confirm that the cluster is configured in your region.
What am I missing? It seems like the task is trying to deploy into the default cluster which doesn’t exist although I have specified where I’d like to deploy the task into.
