Hi everyone, We are trying to run our prefect flow...
# ask-communitys
Sagar Arora
08/09/2023, 2:15 PMHi everyone, We are trying to run our prefect flows in an ECS cluster and we need to run different flows with different secrets and different container configs is there a way we can achieve this using prefect.yaml
j
Jamie Zieziula
08/09/2023, 3:55 PMyou should be able to define the specific configuration you need either in the deployment or as an override to the workpool
s
Sagar Arora
08/09/2023, 3:56 PMWe tried doing that but when we see our ecs task definition the configuration does not replicate there
Sagar Arora
08/09/2023, 5:00 PMthis what our deployment looks like
Sagar Arora
08/09/2023, 5:01 PMSo the secrets section and the ephemeral storage does not replicate to the ecs task that is created using this deployment
Sagar Arora
08/09/2023, 5:03 PMthe work around we found for this was to edit the advanced section of the work pool manually and add these things under job_configuration and save it.
j
Jamie Zieziula
08/09/2023, 5:06 PMah yes exactly!
s
Sagar Arora
08/09/2023, 5:07 PMJust to confirm that it can be done manually only we can’t use prefect.yaml file for this?
j
Jamie Zieziula
08/09/2023, 10:59 PMyes you can! you can update your work pool job variables (at the deployment level) with the specified config you want to override. You’ll also need to update your workpool for ECS under the advanced tab to expose the variables for the secret // ephemeral storage (under
variablesjob_configurationJamie Zieziula
08/09/2023, 10:59 PMso something like this:
Jamie Zieziula
08/09/2023, 11:02 PM Copy code
{
"variables": {
"type": "integer",
"title": "sizeInGiB",
"description": "description"
...
},
"job_configuration": {
"task_definition": {
"ephemeralStorage": {
"sizeInGiB": "{{ sizeInGiB }}"
}
}
}
}Jamie Zieziula
08/09/2023, 11:02 PM(abbreviated)