https://prefect.io logo
Title
s

Stephen Herron

09/21/2022, 10:53 AM
I’m experimenting with the new ECSTask in 2.4.0, as we run prefect 1. One option in prefect 1 was the ability to pass in
networkConfiguration
and thus
awsvpcConfiguration
, to allow the flow to run under specific subnets/security groups but this doesn’t seem possible in prefect2 using this task at the moment?
1
a

Anna Geller

09/21/2022, 12:39 PM
it absolutely is, you can specify networkMode on your task definition - default is awsvpc; then you can set a custom vpc_id on your ECSTask block, even from the UI
the effect is the same but with the difference that you don't need to worry about specifying
networkConfiguration
since in 90% it's always awsvpc and that's what we already use as default (and if you use EC2 launch type, the default is bridge); and
awsvpcConfiguration
was cumbersome because usually you only care about to which VPC your ECS task gets deployed (custom or default VPC) and you don't want to worry about which subnet to pick -- in the current setup, we automatically query all available subnets in the VPC ID you provided and ECS decides to which one it's best to run a specific ECS task with your flow run so all use cases from 1.0 are possible, but way easier - feel free to open a GitHub issue on the
prefect-aws
repo if you struggle to configure something - given that this block is still experimental, we are open to feedback to adjust this block if needed
s

Stephen Herron

09/21/2022, 12:59 PM
Ahh - this doesn’t work when the subnets for the custom VPC (the VPC itself even) - are owned by another account. removing VPC config from the block and it will run on the default VPC. Specifying say the company designated VPC = this error:
botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: At least one security group must be supplied when specifying subnets that are owned by a different account.
created this for reference - https://github.com/PrefectHQ/prefect-aws/issues/112
a

Anna Geller

09/21/2022, 3:12 PM
I answered o the issue
let's keep the discussion there