https://prefect.io logo
c

Christopher

02/10/2022, 5:58 PM
I've got an
ECSAgent
running (in ECS, using the base prefect image with
prefect agent ecs start
), but the tasks it spins up are in the wrong subnet so I want to customise the task definition. It looks like I can pass in a task definition path but that's a bit troublesome because now I need to store file somewhere accessible to the agent. Is there a way to pass a task definition ARN instead? It looks like I can pass that to
ECSRun
but the subnet ID is generated by Terraform so I can't figure out how to get it into the Python...
k

Kevin Kho

02/10/2022, 6:14 PM
Have you seen this ? You can pass it to the agent. Or am I missing something?
c

Christopher

02/10/2022, 6:28 PM
I think that's the Arn for the task role, not the task definition? Or there's the --task-definition Param but that takes a path and I was trying to avoid uploading a file separately
k

Kevin Kho

02/10/2022, 6:37 PM
Ah my bad. You are right. Let me think about this
Yes you are right you can not pass the task definition arn, only a file.
Is the subnet generated on the fly?
c

Christopher

02/10/2022, 9:32 PM
it's not totally dynamic, but we have multiple environments with different subnets, so I can't just bake it into the ECSRun
I've actually realised that the subnet isn't specified as part of the task definition anyway, so now I have no idea how to get the task launched in the correct subnet
k

Kevin Kho

02/10/2022, 10:25 PM
Oh yeah that is true. You need subnet to go in
run_task_kwargs
or on the agent
run_task_kwargs
Like this
You might have you separate out the agents into different subnets and then register the flows with different labels to handle it like that?
c

Christopher

02/11/2022, 8:51 AM
I've got it to work by putting the run_task_kwargs in S3 and reading it from there. I'm happy to share my solution in case it helps future readers - is there a good place for me to do that?
k

Kevin Kho

02/11/2022, 1:52 PM
Yes! Slack is too temporary so I’d appreciate a post with your setup here so that I can link it to future people
Also, if ever you make a blog, we’d publicize on our end
9 Views