Hi I am passing a networkConfiguration yaml file a...
# ask-communityr
Renzo Becerra
03/25/2021, 6:21 PMHi I am passing a networkConfiguration yaml file at agent start, yet running into this botocore error on flow execution: I'm unable to start an ecs agent w/out a path to networkConfig (i get an infer default config error) however flows are failing to run. any insight here? thanks
botocore.exceptions.ParamValidationError: Parameter validation failed:
Unknown parameter in input: "networkConfiguration", must be one of: family, taskRoleArn, executionRoleArn, networkMode, containerDefinitions, volumes, placementConstraints, requiresCompatibilities, cpu, memory, tags, pidMode, ipcMode, proxyConfiguration, inferenceAccelerators
s
Sean Talia
03/25/2021, 6:43 PMcan you share what your yml file looks like?
r
Renzo Becerra
03/25/2021, 8:50 PMwhen we specify our networkConfiguration file-- Because we don't have a default networkconfig, prefect is failing to infer a default. When we pass the netConfig.yaml file at agent start --run-task-kwargs, the agent starts normally but is failing @ register_task_definitions()
Renzo Becerra
03/25/2021, 8:52 PMnetworkConfiguration:
awsvpcConfiguration:
assignPublicIp: 'DISABLED'
subnets:
- subnet-******
securityGroups:
- sg-*************
- sg-*************
Renzo Becerra
03/25/2021, 8:52 PM@Sean Talia
s
Sean Talia
03/25/2021, 8:54 PMthat's odd, that's exactly how my file is structured
Sean Talia
03/25/2021, 8:54 PMare you working with the version of boto that's installed with prefect?
r
Renzo Becerra
03/25/2021, 8:54 PMyes
Renzo Becerra
03/25/2021, 8:55 PMwhich version are you using?
s
Sean Talia
03/25/2021, 8:55 PM Copy code
boto3==1.17.15
botocore==1.20.15Sean Talia
03/25/2021, 8:56 PMi unfortunately have to sign off for the day but if you're still having issues tomorrow i can possibly try to help diagnose, it seems like it's something quite petty though 😎
r
Renzo Becerra
03/26/2021, 6:02 PM@Darren
m
Mariia Kerimova
03/26/2021, 6:14 PMHello Renzo! It looks like you are trying to add
networkConfigurationr
Renzo Becerra
03/26/2021, 8:29 PM@Mariia Kerimova how can I pass network configuration upon agent start?
Renzo Becerra
03/26/2021, 8:30 PMfailing to do so results in a 'failed to infer default network configuration'
m
Mariia Kerimova
03/26/2021, 8:37 PMThis PR is not merged yet, but I provide an example of creating a service for the agent
https://github.com/PrefectHQ/prefect/pull/4302/files
Mariia Kerimova
03/26/2021, 8:39 PMYou can pass network configuration when you create a service. Do you use default VPC or did you create a new one? In any case, you can follow that example, just update your token, subnet, security group. Let me know if you'll have any questions
r
Renzo Becerra
03/26/2021, 11:33 PM@Mariia Kerimova we don't have a default VPC, so we have to pass vpcConfig to include subnets and sec groups
Renzo Becerra
03/26/2021, 11:43 PMWhy is it that I have to create a task definition and service when prefect fails to infer a default vpc? I have a default vpc config set on my personal aws acct and everything works perfectly
m
Mariia Kerimova
03/29/2021, 8:57 PMCurrently the agent can infer the default VPC, but not custom vpc. I'm talking with one of our engineers to expand this behavior to custom vpcs too 👍
r
Renzo Becerra
03/31/2021, 3:24 PM@Mariia Kerimova I added to the src code and I'm able to set the net config w/out any issues. I'm happy to submit a MR, lmk.
m
Mariia Kerimova
03/31/2021, 3:27 PMsounds awesome! Yes, we appreciate contributions a lot! 🙂
r
Renzo Becerra
03/31/2021, 3:31 PMis there an issue assigned to this?
m
Mariia Kerimova
03/31/2021, 3:34 PMThere should be an issue as I remember, but you can open PR without referencing the issue for now. I’m going to find it in a bit, and mention it in your PR or post it here if you would prefer that
r
Renzo Becerra
03/31/2021, 4:20 PM@Mariia Kerimova Update ~ ECS Fargate Agent: +network configuration param #4323
Renzo Becerra
03/31/2021, 4:20 PMm
Mariia Kerimova
03/31/2021, 4:27 PMFabulous, thank you so much! We are going to review it shortly! @Jim Crist-Harif
👍 1
5 Views