I’m using prefect 2.20.9 and prefect-aws 0.4.0 with python 3.12. I’m creating an ecs work-pool with a base job configuration I set in a shell script, and I’m building my deployment with Deployment.build_from_flow() from prefect.deployments. For 3 days (between my other work), I’ve been going through 10s (100s? it feels like) of permutations of the json structures to set task_run_request with my taskDefinitionArn, or task_definition_arn, or defining a new task_definition, in the base job config of the work pool, or in the infra_overrides parameter of the deployment. Inspecting the workpool confirms the keys and values are there but they never seem to work. Every time I get the below error from my work-pool when I trigger a flow run. Any suggestions for how I could get this working (e.g., an example structure that would work that resolves the below error)? (as a side note/feedback I’m not a data engineer, I’m a data scientist; I was hoping prefect would work “out of the box”, and was looking forward to the observability of flow execution, but there have been so many versions to sift through and align, and different documentation sources, import paths, objects to try to understand, and errors… I’m about to go to plan b and just manage my ecs jobs with custom scripts.) pydantic.error_wrappers.ValidationError: 1 validation error for ECSJobConfiguration root A task definition must be provided if a task definition ARN is not present on the task run request. (type=value_error)

hi @luke b - sorry you've hit trouble are you using the ECS push pool? that requires significantly less setup than the hybrid ECS work pool, the latter you should only need if you have specific requirements on how the worker should run have you seen this?

@Nate Thanks for the pointer to the push work pools. I’m still trying to puzzle through the issues with my current set up though. I got past the last error--it was a particular json key name and structure issue in the base-job-template used when creating the work-pool. Now moving onto tackle the next error, something about the network configuration between ecr and ecs. Will reach out again if I can’t get it sorted.

👍