Lana Dann
03/23/2022, 3:43 PMKevin Kho
Lana Dann
03/23/2022, 3:53 PMjust specify your num_workers and threads otherwise it will default to 2to clarify, are you suggesting i try
LocalDaskExecutor
before DaskExecutor
and just configure my workers/threads? do i need to update any IAM policy to run LocalDaskExecutor
?
You would give your ECS task a role that would give it the resources to spin up a clusterwould my ECS task spin up the cluster, or the ECS service running the ECS agent?
Kevin Kho
Lana Dann
03/23/2022, 4:25 PMKevin Kho
Lana Dann
03/23/2022, 4:54 PMflow.executor = LocalDaskExecutor(scheduler="threads", num_workers=8)
does it need anything else? i see when i try to kick it off now, in the ECS task logs:
/bin/sh: 1: prefect: not found
Kevin Kho
Lana Dann
03/23/2022, 5:00 PMflow
image with prefect installedKevin Kho
Lana Dann
03/23/2022, 5:05 PMRescheduled by a Lazarus process. This is attempt 1.
Kevin Kho
Lana Dann
03/23/2022, 5:13 PM/bin/sh: 1: prefect: not found
i’ll try removing and retrying nowKevin Kho
Lana Dann
03/23/2022, 5:36 PMKevin Kho
Lana Dann
03/23/2022, 5:39 PMKevin Kho