https://prefect.io logo
r

Romain Vincent

08/31/2023, 4:35 PM
Hello everyone, I'm trying to inject aws credentials as env variables into my ecs tasks using an ecs work pool env variables override. I'm collecting the credentials from an existing aws credentials block like so :
Copy code
{
  "AWS_ACCESS_KEY_ID": "{{ prefect.blocks.aws-credentials.<my-aws-block>.aws_access_key_id }}",
  "AWS_SECRET_ACCESS_KEY": "{{ prefect.blocks.aws-credentials.<my-aws-block>.aws_secret_access_key }}"
}
Unfortunately, my ecs tasks crashes at submission time with this error :
Copy code
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/prefect/workers/base.py", line 831, in _submit_run_and_capture_errors
    configuration = await self._get_configuration(flow_run)
  File "/usr/local/lib/python3.10/site-packages/prefect/workers/base.py", line 906, in _get_configuration
    configuration = await self.job_configuration.from_template_and_values(
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/workers/base.py", line 125, in from_template_and_values
    populated_configuration = await resolve_block_document_references(
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/utilities/templating.py", line 205, in resolve_block_document_references
    updated_value = await resolve_block_document_references(
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/utilities/templating.py", line 205, in resolve_block_document_references
    updated_value = await resolve_block_document_references(
  File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 51, in with_injected_client
    return await fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/prefect/utilities/templating.py", line 228, in resolve_block_document_references
    block_type_slug, block_document_name = (
ValueError: too many values to unpack (expected 2)
I guess it has to do with the formatting, but I could not find the related documentation. Any help would be appreciated!
Well. Since I couldn't get this to work, I went the very ugly way:
Copy code
BOTO3_SESSION = (
    AwsCredentials.load('<my-credentials-block>')
    .get_boto3_session()
)

os.environ['AWS_ACCESS_KEY_ID'] = BOTO3_SESSION.get_credentials().access_key
os.environ['AWS_SECRET_ACCESS_KEY'] = BOTO3_SESSION.get_credentials().secret_key