Hey :wave: , I am new to Prefect Cloud and have h...
# prefect-community
Hey ๐Ÿ‘‹ , I am new to Prefect Cloud and have hit a task definition snag which I can't seem to solve. We have an ECS Agent deployed on EC2 with the correct API tokens which shows up in our Cloud UI. We have a ECS cluster which we declare when launching the agent but leave the launch type as default (Fargate). Our example Flow is below:
Copy code
default_client = docker.from_env()
FLOW_NAME = "hello-flow"
flow_schedule = CronSchedule("0 8 * * *")
flow_storage = Docker(
flow_run_config = ECSRun(
    run_task_kwargs={"requiresCompatibilities": ["FARGATE"], "compatibilities": ["FARGATE"]}

with Flow(
    ) as flow:

if is_serializable(flow):
    flow.register(project_name="Test", registry_url=flow_storage)
    raise TypeError("Flow did not serialise.")
We are getting the following error from our task logs:
Copy code
An error occurred (InvalidParameterException) when calling the RunTask operation: Task definition does not support launch_type FARGATE.
In an attempt to resolve this issue I tried adding
as above but with no luck. Does anyone have any pointers? (I can see from the ECS task definition panel that the
is set to EC2 and
Requires compatibilities
is blank and from this thread that could be the cause...)
Also, deploying the agent as so returns the same error:
Copy code
prefect agent ecs start --cluster prefect-cluster --launch-type EC2
Hmm @Kieran are you deploying to the
cluster or do you have another one created? I feel like Iโ€™ve seen this when the cluster the agent is trying to deploy to isnโ€™t permissioned properly to use Fargate ๐Ÿค”
Hey @josh we have provisioned a cluster for prefect (called prefect-cluster). Our best guess at the minute is that permissions (IAM roles) are off slightly. What is confusing is that memory and CPU limits can be set within ESCRun() and are correctly bring provided to the task definition but we seem unable to pass compatibilities kwargs to declare the task as a FARGATE launch type. Other threads in here seem to suggest that the error is a slight red herring and we need to declare the task and execution roles in a custom task definition yaml file to overcome this issue... (Or possibly/likely I'm just confused with the deployment details)
Yeah I think it could be IAM related as well, I saw someone internal to Prefect run into an issue like this last week but everything worked fine when they went back to using the default cluster. As for the compatibilities kwargs, yes currently you would need to define a custom task definition for the ECSRun. In implementing this run config we chose to only expose the bare minimum and are adding new ones as user requests come in! If you think this is something that would be helpful for users to expose as a direct kwarg feel free to open an issue for the suggestion ๐Ÿ™‚
๐Ÿ™Œ 1
@josh @Kieran apologies for jumping in between your thread, I am also getting the same challenge in creating ECSRun agent. I am getting an error
Copy code
RROR - agent | Failed to infer default networkConfiguration, please explicitly configure using `--run-task-kwargs`

ValueError: Failed to infer default networkConfiguration, please explicitly configure using `--run-task-kwargs`
I have already implemented a task_definition.yaml as below as suggested by another community member in another thread but still no relief
Copy code
networkMode: awsvpc
cpu: 1024
memory: 2048
  - name: flow
taskRoleArn: <ecs_task_role_arn>
executionRoleArn: <insert_ecsTaskExecutionRole_arn>
For anyone who reads this thread with the same issue as me. A customer task_definition file which declares FARGATE as the required compatible launch type has fixed my issue.
โœ… 2
Hi @Kieran, Can you share the minimum contents of the task_definition file? Thank you in advance.
Hi @Jay Sundaram We use the folloiwing:
Copy code
networkMode: awsvpc
cpu: 256
memory: 512
  - name: flow
executionRoleArn: arn:aws:iam::xxxxxxxxxx:role/tf-iam-role-prefect-ecs-tasks <--- change to whatever you need
๐Ÿ™ 1