https://prefect.io logo
Powered by
Title
b

Braun Reyes

03/28/2023, 9:24 PM
hey all....do we know of a way to add secrets to the container definition of the ECSTask block? I tried this, but it just creates a new container to the task. 
task_definition={
                "containerDefinitions": [
                    {
                        "secrets": [
                            {
                                "name": i.envar_name,
                                "valueFrom": f"arn:aws:secretsmanager:{AWS_REGION}:{account_id}:secret:{i.secret_name}",
                            }
                            for i in self.secrets
                        ],
                        "name": slugified_flow_name,
                        "image": image_name
                    }
                ],
                "ephemeralStorage": {"sizeInGiB": self.ephemeral_storage_gb},
            }
I thought it would override the main container with my values here.
"containerDefinitions": [
        {
            "name": "common-utils-flow-1",
            "image": "<http://410318598490.dkr.ecr.us-east-1.amazonaws.com/data-flows-prefect-envs:common-utils-dev-py-3.10|410318598490.dkr.ecr.us-east-1.amazonaws.com/data-flows-prefect-envs:common-utils-dev-py-3.10>",
            "cpu": 0,
            "portMappings": [],
            "essential": true,
            "environment": [],
            "mountPoints": [],
            "volumesFrom": [],
            "secrets": [
                {
                    "name": "MY_SECRET_JSON",
                    "valueFrom": "arn:aws:secretsmanager:us-east-1:410318598490:secret:/my/secretsmanager/secret"
                }
            ]
        },
        {
            "name": "prefect",
            "image": "<http://410318598490.dkr.ecr.us-east-1.amazonaws.com/data-flows-prefect-envs:common-utils-dev-py-3.10|410318598490.dkr.ecr.us-east-1.amazonaws.com/data-flows-prefect-envs:common-utils-dev-py-3.10>",
            "cpu": 0,
            "portMappings": [],
            "essential": true,
            "environment": [],
            "mountPoints": [],
            "volumesFrom": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-create-group": "true",
                    "awslogs-group": "prefect",
                    "awslogs-region": "us-east-1",
                    "awslogs-stream-prefix": "common-utils-flow-1-dev-test"
                }
            }
        }
    ]
j

James Gatter

03/29/2023, 5:12 PM
I might be misunderstanding what you're trying to do with the secrets, but maybe you could instead use boto3 within the flow to pull these secrets from secrets manager at runtime?
b

Braun Reyes

04/04/2023, 5:40 PM
that is a good idea...for now I am just creating my own task definition
#prefect-awsJoin Slack
Powered by