https://prefect.io logo
j

jcozar

01/27/2021, 5:01 PM
Hello all, I am running a prefect ECS agent as a service in AWS ECS. Now I am trying to register a flow run in Prefect Cloud using the Docker storage. I created a
ECSRun
run configuration for the
run_config
argument in the Flow. If I use the
env
argument to provide the
AWS_ACCESS_KEY_ID
and
AWS_SECRET_ACCESS_KEY
it works! However, I don’t want to put my credentials in the source code. I am trying to use the
task_definition_arn
argument, but I am not sure if it is the correct way, because the image of the task should be the Flow docker image. Can you give me any tip or advise? Thank you very much!
j

josh

01/27/2021, 5:30 PM
Hi @jcozar are you setting AWS credentials on your ECSAgent? I may be missing something but I’m not sure how setting them on the run config would effect the agent being able to create the task 🤔
j

jcozar

01/27/2021, 5:46 PM
Thank you very much for your fast response! In order to simplify the problem, let’s assume that I am using AWS S3 Storage. The prefect ecs agent is running in ECS using a custom task definition. The task role has enough privileges to access S3 and to run new tasks in ECS. I register a new Flow in Prefect Cloud using a ECSRun Run Configuration. When I run a new flow run manually, the task in my ECS cluster appears. However: • If I don’t provide the access and secret access key to the
RunConfig
, then, the new task raises the exception
NoCredentialsError('Unable to locate credentials')
when trying to download the flow from S3 (last info log message is
Downloading flow from s3://...
• If I provide the access and secret access key to the
RunConfig
, it has permissions to download the flow and execute the flow run. I read in the documentation that if the
task_definition
is not specified in the
RunConfig
, the default configuration for the task_definitions is used (one per each flow run version). The default configuration uses None for the task_execution_role, and that’s why it cannot download the flow definition from S3. The same issue if I use Docker and AWS ECR registry. I tried to use the
execution_role_arn
to use a full privileged role, but the problem is not creating the task, is trying to download the flow (storage) from S3 in runtime. I did’t try the combination of the Docker Storage and the
execution_role_arn
with full privileged (maybe in this case the task image is the one from AWS ECR). I’m gonna try it!
j

josh

01/27/2021, 6:09 PM
Ah yeah the
task_role_arn
(either set as default on the agent or on the flow’s run config) is what allows the flow to use the S3 storage inside the job kicked off by the agent. This is how you avoid having to set the access keys directly in the run config. I also think that the
execution_role_arn
is what is needed to pull the ECR image however that wouldn’t be a responsibility of the job itself and instead would be something the agent would need (or if you provide access keys to the agent it would accomplish the same behavior)
j

jcozar

01/27/2021, 6:13 PM
Ok thank you! Indeed, if I use the Docker storage and provide the
excution_role_arn
then the flow run is executed correctly! However, in the case that I need to use S3 storage, how can I provide the
task_role_arn
? The only way I see is to specify a task definition, which makes sense to me 🙂 Is this right? Thank you!
j

josh

01/27/2021, 6:14 PM
The flow’s ECSRun allows for setting the
task_role_arn
https://docs.prefect.io/api/latest/run_configs.html#ecsrun
You can either set it there directly or in the task definition if you have a fully custom definition you want to use
j

jcozar

01/27/2021, 6:18 PM
Thank you! I read all the arguments but I missread that one!
5 Views