https://prefect.io
Join Slack
```RUN export PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS=$(echo `(aws secretsmanager get-secret-valu...
# ask-community
c
Copy code
RUN export PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS=$(echo `(aws secretsmanager get-secret-value --secret-id ${prefect_aws_secret_name} --region ${region}) | jq -r '.SecretString'`)
I'm having a ton of trouble using ARG/ENV/RUN with a dockerfile, what I want is for PREFECT__CONTEXT__SECRETS_AWS_CREDENTIALS to show up as an env variable in the final image. Does anyone have experience setting env variables in a dockerfile from command output?
✅ 1
k
I haven’t tried this. Do you get an ENV variable or no env variable at all? Are you using ECS run?
What is the use case here? Cuz you can just give a role to the container and then you bypass this?
c
Depending on whether I have ENV set before or after the RUN command, I will get an ENV variable but it is either blank or equal to the string "PREFECT__CONTEXT__SECRETS__AWS_CREDENTIALS"_
this dockerfile is built using ecs and I can verify that it is able to pull the secrets from secretsmanager. The trouble is having the secret stay as an env variable in the final image. The use case is that I'm a developer and I want to debug a script locally, my script uses one or more of the Prefect AWS Tasks, but I'm unable to authenticate because of the missing env variable
we don't want to use ~/.aws/credentials, anything hardcoded into a file
k
I personally can’t see anything wrong with the attempt based on this
m
maybe a better way is to provide envs in run docker -e or in docker compose? not to hardcode it into dockerfile, which usually is pushed into repo ?
c
would i be able to provide the aws secretsmanager command using -e?
k
I don’t think so from the Prefect CLI
a
Marcin gave a great suggestion and I would go even as far as saying that you shouldn't use AWS key pair for anything else other than local development - for local development you can mount your home .aws folder to your root container folder And for production you should use IAM roles instead
c
@Dave Thomas @Anna Geller agreed, but we'd like to access the key pair using aws secretsmanager inside the image, rather than distributing different key pairs to each dev user individually
a
in that case: • for development, you should bind mount the credential file • for production, you should assign the IAM role allowing secrets manager access
the easiest is by attaching it to your Docker agent:
Copy code
prefect agent docker start --label AGENT_LABEL --volume ~/.aws:/root/.aws
definitely, as @Marcin Grzybowski correctly mentioned, baking the credentials directly into the image would be a security risk, but mounting as shown here would be a slightly better approach as you wouldn't risk those credentials would be baked into image directly and pushed to some registry
🙌 1
8 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by