eric lumsden
04/05/2021, 5:30 PMValueError: Failed to infer default networkConfiguration, please explicitly configure using `--run-task-kwargs`
Kevin Kho
04/05/2021, 5:35 PMeric lumsden
04/05/2021, 5:43 PMKevin Kho
04/05/2021, 5:45 PMeric lumsden
04/05/2021, 5:47 PMKevin Kho
04/05/2021, 5:50 PMeric lumsden
04/05/2021, 5:53 PMKevin Kho
04/05/2021, 5:54 PMeric lumsden
04/05/2021, 5:56 PMKevin Kho
04/05/2021, 5:57 PMeric lumsden
04/05/2021, 6:01 PMKevin Kho
04/05/2021, 6:02 PMaws ecs create-cluster
try this?eric lumsden
04/05/2021, 6:11 PMKevin Kho
04/05/2021, 6:15 PMeric lumsden
04/05/2021, 6:15 PMKevin Kho
04/05/2021, 6:17 PMeric lumsden
04/05/2021, 6:25 PMKevin Kho
04/05/2021, 6:27 PMRUN_CONFIG = ECSRun(execution_role_arn="arn:aws:iam::1234:role/ecsTaskExecutionRole",
run_task_kwargs={
"networkConfiguration": {
"awsvpcConfiguration": {
"assignPublicIp": "ENABLED",
"subnets": ['subnet-123456', 'subnet-123456', 'subnet-123456'],
"securityGroups": ["sg-123456"],
}
}
},
)
run_config=RUN_CONFIG
. get the VPC configuration from the default subnetseric lumsden
04/05/2021, 6:30 PMKevin Kho
04/05/2021, 6:44 PMFlow()
call.
RUN_CONFIG = ECSRun(execution_role_arn="arn:aws:iam::1234:role/ecsTaskExecutionRole",
run_task_kwargs={
"networkConfiguration": {
"awsvpcConfiguration": {
"assignPublicIp": "ENABLED",
"subnets": ['subnet-123456', 'subnet-123456', 'subnet-123456'],
"securityGroups": ["sg-123456"],
}
}
},
)
with Flow(name="ecs-test", storage=STORAGE, run_config=RUN_CONFIG, executor=EXECUTOR) as flow:
print_log()
flow.run()
eric lumsden
04/05/2021, 7:42 PMKevin Kho
04/05/2021, 7:50 PMflow.py
?prefect
inside?print_log
with print("1")
to test this block. You don’t have the STORAGE and EXECUTOR defined though so this will error. Do you have the original snippet that caused the error? Maybe we can take a look at thateric lumsden
04/05/2021, 8:02 PMKevin Kho
04/05/2021, 8:04 PMSTORAGE
is to point to where your code lives. For ECS it might point to the Docker image location. Something like:
STORAGE = Docker(
registry_url="<http://1234.dkr.ecr.us-east-2.amazonaws.com|1234.dkr.ecr.us-east-2.amazonaws.com>",
image_name="ecs-test",
)
eric lumsden
04/05/2021, 8:04 PMKevin Kho
04/05/2021, 8:05 PMeric lumsden
04/05/2021, 8:06 PMKevin Kho
04/05/2021, 8:06 PMeric lumsden
04/06/2021, 3:38 PM--run-task-kwargs
I'm guessing I need to set the networkMode to an existing vpc. Unfortunately my environment shifted it's hosting region overnight so I am pointed somewhere with no VPCs. How do I point to the new region and select a VPC?Kevin Kho
04/06/2021, 3:41 PMnetworkMode: awsvpc
cpu: 256
memory: 512
containerDefinitions:
- name: flow
requiresCompatibilities:
- FARGATE
executionRoleArn: arn:aws:iam::xxxxxxxxxx:role/tf-iam-role-prefect-ecs-tasks <--- change to whatever you need
eric lumsden
04/06/2021, 3:58 PMKevin Kho
04/06/2021, 4:00 PMeric lumsden
04/06/2021, 4:00 PMKevin Kho
04/06/2021, 4:21 PMrun_task.yaml
 with this:
networkConfiguration:
awsvpcConfiguration:
subnets:
- 'subnet-1234'
securityGroups:
- 'sg-1234'
assignPublicIp: 'ENABLED'
run prefect agent ecs start --run-task-kwargs /ecs/run_task.yaml
eric lumsden
04/06/2021, 4:37 PMKevin Kho
04/06/2021, 4:59 PMeric lumsden
04/06/2021, 5:12 PMprefect agent ecs start --run-task-kwargs <s3://bucketname>
prefect.utilities.exceptions.AuthorizationError: No agent API token provided.
Kevin Kho
04/06/2021, 5:21 PMprefect agent ecs start
on your local machine and you want the local machine to kick off ECS jobs right?eric lumsden
04/06/2021, 5:53 PMKevin Kho
04/06/2021, 5:55 PMeric lumsden
04/06/2021, 6:02 PMKevin Kho
04/06/2021, 6:08 PMeric lumsden
04/06/2021, 6:30 PMKevin Kho
04/06/2021, 6:36 PM