eric lumsden
04/05/2021, 5:30 PMValueError: Failed to infer default networkConfiguration, please explicitly configure using `--run-task-kwargs`
Kevin Kho
eric lumsden
04/05/2021, 5:43 PMKevin Kho
eric lumsden
04/05/2021, 5:47 PMKevin Kho
eric lumsden
04/05/2021, 5:53 PMeric lumsden
04/05/2021, 5:53 PMeric lumsden
04/05/2021, 5:54 PMKevin Kho
eric lumsden
04/05/2021, 5:56 PMKevin Kho
Kevin Kho
eric lumsden
04/05/2021, 6:01 PMKevin Kho
Kevin Kho
aws ecs create-cluster
try this?Kevin Kho
eric lumsden
04/05/2021, 6:11 PMeric lumsden
04/05/2021, 6:13 PMeric lumsden
04/05/2021, 6:14 PMKevin Kho
eric lumsden
04/05/2021, 6:15 PMKevin Kho
Kevin Kho
eric lumsden
04/05/2021, 6:25 PMKevin Kho
RUN_CONFIG = ECSRun(execution_role_arn="arn:aws:iam::1234:role/ecsTaskExecutionRole",
run_task_kwargs={
"networkConfiguration": {
"awsvpcConfiguration": {
"assignPublicIp": "ENABLED",
"subnets": ['subnet-123456', 'subnet-123456', 'subnet-123456'],
"securityGroups": ["sg-123456"],
}
}
},
)
Kevin Kho
run_config=RUN_CONFIG
. get the VPC configuration from the default subnetsKevin Kho
eric lumsden
04/05/2021, 6:30 PMeric lumsden
04/05/2021, 6:41 PMKevin Kho
Flow()
call.
RUN_CONFIG = ECSRun(execution_role_arn="arn:aws:iam::1234:role/ecsTaskExecutionRole",
run_task_kwargs={
"networkConfiguration": {
"awsvpcConfiguration": {
"assignPublicIp": "ENABLED",
"subnets": ['subnet-123456', 'subnet-123456', 'subnet-123456'],
"securityGroups": ["sg-123456"],
}
}
},
)
with Flow(name="ecs-test", storage=STORAGE, run_config=RUN_CONFIG, executor=EXECUTOR) as flow:
print_log()
flow.run()
Kevin Kho
eric lumsden
04/05/2021, 7:42 PMKevin Kho
flow.py
?Kevin Kho
prefect
inside?Kevin Kho
Kevin Kho
print_log
with print("1")
to test this block. You don’t have the STORAGE and EXECUTOR defined though so this will error. Do you have the original snippet that caused the error? Maybe we can take a look at thateric lumsden
04/05/2021, 8:02 PMKevin Kho
STORAGE
is to point to where your code lives. For ECS it might point to the Docker image location. Something like:
STORAGE = Docker(
registry_url="<http://1234.dkr.ecr.us-east-2.amazonaws.com|1234.dkr.ecr.us-east-2.amazonaws.com>",
image_name="ecs-test",
)
eric lumsden
04/05/2021, 8:04 PMeric lumsden
04/05/2021, 8:05 PMKevin Kho
eric lumsden
04/05/2021, 8:06 PMKevin Kho
Kevin Kho
Kevin Kho
Kevin Kho
eric lumsden
04/06/2021, 3:38 PM--run-task-kwargs
I'm guessing I need to set the networkMode to an existing vpc. Unfortunately my environment shifted it's hosting region overnight so I am pointed somewhere with no VPCs. How do I point to the new region and select a VPC?Kevin Kho
networkMode: awsvpc
cpu: 256
memory: 512
containerDefinitions:
- name: flow
requiresCompatibilities:
- FARGATE
executionRoleArn: arn:aws:iam::xxxxxxxxxx:role/tf-iam-role-prefect-ecs-tasks <--- change to whatever you need
Kevin Kho
eric lumsden
04/06/2021, 3:58 PMKevin Kho
eric lumsden
04/06/2021, 4:00 PMeric lumsden
04/06/2021, 4:20 PMKevin Kho
Kevin Kho
run_task.yaml
 with this:
networkConfiguration:
awsvpcConfiguration:
subnets:
- 'subnet-1234'
securityGroups:
- 'sg-1234'
assignPublicIp: 'ENABLED'
run prefect agent ecs start --run-task-kwargs /ecs/run_task.yaml
Kevin Kho
Kevin Kho
eric lumsden
04/06/2021, 4:37 PMKevin Kho
Kevin Kho
eric lumsden
04/06/2021, 5:12 PMprefect agent ecs start --run-task-kwargs <s3://bucketname>
eric lumsden
04/06/2021, 5:12 PMprefect.utilities.exceptions.AuthorizationError: No agent API token provided.
eric lumsden
04/06/2021, 5:13 PMeric lumsden
04/06/2021, 5:19 PMKevin Kho
Kevin Kho
prefect agent ecs start
on your local machine and you want the local machine to kick off ECS jobs right?Kevin Kho
Kevin Kho
Kevin Kho
eric lumsden
04/06/2021, 5:53 PMKevin Kho
eric lumsden
04/06/2021, 6:02 PMKevin Kho
Kevin Kho
eric lumsden
04/06/2021, 6:30 PMKevin Kho