Follow up to setting up an ECS worker but does anyone know i Prefect Community #ask-community

Follow-up to setting up an ECS worker - but does a...

Samuel Hinton

08/15/2024, 10:20 AM

Follow-up to setting up an ECS worker - but does anyone know if theres a way when creating a worker in an ecs typed worker pool to set up defaults for networking, vpcs, etc. Ideally via terraform, but also possibly via command line or environment variables. In the old (deprecated) way of doing things, we passed a host of kwargs to the ECSTask that were common to every task, and I'm not sure what the best practise way is now that theres an ECS worker running flows intead of an ECSTask

Bianca Hoch

08/17/2024, 5:37 PM

Hi Samuel! Setting networking defaults can be done in your ECS work pool. You'll be able to specify your vpc, subnets, and security groups in the base template. It'd look a bit like this in the Network Configuration field.

Bianca Hoch

08/17/2024, 5:39 PM

And you should be able to set up your work pools using the terraform provider.

Bianca Hoch

08/17/2024, 5:40 PM

https://github.com/PrefectHQ/terraform-provider-prefect

Samuel Hinton

08/19/2024, 2:16 AM

Hey @Bianca Hoch, thanks a ton for the response! Just to confirm, it looks like the terraform examples for the work pool assume that its only compatible with prefect cloud? If there isnt a local endpoint terraform available for this, do you know if we could configure a work pool template using the python client?

Samuel Hinton

08/19/2024, 3:44 AM

Can I also ask what document I should be referencing to figure out how to change the base template? I ran

Copy code

prefect work-pool get-default-base-job-template --type ecs > template.json

But this has the top level

Copy code

"network_configuration": "{{ network_configuration }}",

I'm not sure what this means, where is the top level network_configuration defined if its getting inserted into the base template. (ie how did you get the subnet, public ip and security group attributes from?)\ Is there a different thing I should be using to generate the template? Similarly I have an exection_role_arn and task_role_arn made in terraform to set, but unsure how to create these as a default and not ruin whatever prefect replace is happening. EDIT: After digging more, I think its coming from https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ecs-service-awsvpcconfiguration.html I will try things out and report back with what solution did or didnt work

Mitch Nielsen

10/28/2024, 5:18 PM

connecting the thread here too, https://github.com/PrefectHQ/terraform-provider-prefect/pull/288 clarifies that the work pools resource is not cloud-only

Open in Slack

Previous Next