Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Did some digging. Does prefect override my selected VPC subnets when running the task?
<https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py#L924|https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py#L924>

Would that cause issues with private subnets?
<https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py#L1406|https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py#L1406>

Are you looking for <https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py#L62-L74>

I have a custom task definition arn being used. Do I still need to define those?

I think most of the network config needs to happen at runtime not task definition time?

ie the network configuration field in <https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs/client/run_task.html#run-task>

You are correct. But that’s already defined in the running service, no?

I was under the impression that if I set a static task definition arn it won’t run another task? Which does seem to happen. But for some reason when I run a flow, it can’t read secrets. The agent does start up correctly and is able to read tasks. 

Or am I thinking about this all wrong?

I’m not really sure what you’re asking

I used this for my starter <https://github.com/PrefectHQ/prefect-recipes/tree/main/devops/infrastructure-as-code/aws/tf-prefect2-ecs-agent|https://github.com/PrefectHQ/prefect-recipes/tree/main/devops/infrastructure-as-code/aws/tf-prefect2-ecs-agent>

And added datadog as the logging mechanism (with the api key as a secret). When I run the agent it works fine saying that it’s waiting for flow runs. But when a flow actually runs it can’t access the secret

Hey <@U028YB2M70F> :wave: The recipe specifically <https://github.com/PrefectHQ/prefect-recipes/blob/main/devops/infrastructure-as-code/aws/tf-prefect2-ecs-agent/main.tf#L38-L56|allows the agent’s execution role to access only the API key secret>. You may need to attach a permission to allow reading your new secret to the role that your task uses.

Thank you <@U01CEUST9B5> <@U02JKLM1C2K>! What I failed to realize is that the ECS agent spawns additional ECS tasks, with configuration from the block!

&gt; Are you looking for <https://github.com/PrefectHQ/prefect-aws/blob/main/prefect_aws/ecs.py#L62-L74>
Thank you!! I just changed the VPC to our company's main VPC after following dataflowops template and specifying the subnet here allowed my container to access ECR again! Solved:
```prefect_aws.ecs.TaskFailedToStart: ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post "<https://api.ecr>.&lt;region&gt;.<http://amazonaws.com/|amazonaws.com/>"```