Hi everyone,
I have a problem defining the network configuration in work pool. (Possibly code issue). What are the alternative if the Network Configuration in workpool does not seem to work?
I am using ECS Workpool. Flow executes in ECS Task based on the configuration defined in the
Infrastructure Overrides in Deployment.
I need to restrict the flow to execute in specific Subnets so I am defining them in
Network Configuration defined in the work pool.
However flow execution throws error: ValueError: Subnets [’subnet-abc] not found within VPC with ID vpc-MYVPC. Please check that VPC is associated with supplied subnets.
I have double
checked that subnet exist in the vpc. There are no permission issues to call
describe_subnets method.
I found this line of code
https://github.com/PrefectHQ/prefect-aws/blob/525c9173a7db0e7cbe337a2252204b2d076124cf/prefect_aws/workers/ecs_worker.py#L1354 which tries to match the existing subnets in VPC to subnets id in network configuration has issues.
ec2_client.describe_subnets method correctly to retrieve the subnets from AWS, however that retrieves lot more information of each Subnet and not Just Subnet ID which is generally mentioned the network configuration (awsvpcconfiguration.
If I change the code from
if not all([conf_sn in sn.values() for conf_sn in config_subnets for sn in subnets]):
raise ValueError(...)
to
if not all(conf_sn in [sn['SubnetId'] for sn in subnets] for conf_sn in config_subnets):
raise ValueError(...)
then it compares then correctly.
Appreciate if someone here points what are the options
if I want to restrict this to run in specific subnets when Network Configuration seems to have issue.