Scott Aefsky
03/30/2022, 6:54 PMAnna Geller
Scott Aefsky
03/30/2022, 6:56 PMECSRun(
image=ECR_IMAGE,
execution_role_arn=f'arn:aws:iam::{account}:role/prefect-ecs-execution-role',
task_role_arn=f'arn:aws:iam::{account}:role/prefect-ecs-task-role',
run_task_kwargs={
"cluster": "PrefectCluster",
},
labels=[account]
)
Anna Geller
Scott Aefsky
03/30/2022, 6:57 PMnetworkMode: awsvpc
cpu: 1024
memory: 2048
containerDefinitions:
- name: flow
image: not_a_real_image
logConfiguration:
logDriver: awslogs
options:
awslogs-group: prefect_ecs_task_log_ACCOUNT
awslogs-region: us-east-1
awslogs-stream-prefix: ecs-prefect
awslogs-create-group: "True"
def dynamic_executor():
if prefect.context.parameters["use_fargate"] == 'true':
return FargateCluster(
image= ECR_IMAGE,
n_workers= 5, # Must specify n_workers
cluster_arn= fr"arn:aws:ecs:us-east-1:{account}:cluster/PrefectCluster",
task_role_arn= fr"arn:aws:iam::{account}:role/prefect-ecs-task-role",
vpc= vpc,
subnets= [subnet],
security_groups= [sg],
cloudwatch_logs_group=f'prefect_ecs_task_log_{account}'
)
else:
return LocalCluster()
flow1.executor = DaskExecutor(
cluster_class=dynamic_executor
)
Anna Geller
with Flow("yourflow", executor=DaskExecutor(
cluster_class=dynamic_executor
)) as flow:
Scott Aefsky
03/30/2022, 7:02 PMwith Flow('flow', executor=myExecutor) as f:
and
with Flow('flow') as f:
<tasks>
f.executor=myExecutor
are functionally equivalent?Kevin Kho
from some_other_file import flow1
flow1.storage = ..
flow1.executor = ..
flow1.register()
the executor is not stored because it’s pulled from the Flow definition in storage. Is it in the same file for you?Anna Geller
with Flow('flow') as f:
<tasks>
if __name__ == "__main__":
f.executor=DaskExecutor()
and main is not evaluated when flow is already registered.
a likely reason why your flow runs out of memory when using a local dask cluster is that it only has 1 CPU and 2 GB of memory so it sounds reasonable that in this setting your flow run can get OOM errorsScott Aefsky
03/30/2022, 7:06 PMKevin Kho
Scott Aefsky
03/30/2022, 7:12 PMScott Aefsky
03/30/2022, 7:13 PMKevin Kho
Scott Aefsky
03/30/2022, 7:20 PMKevin Kho
Scott Aefsky
03/30/2022, 7:25 PMnetworkMode: awsvpc
cpu: 1024
memory: 2048
containerDefinitions:
- name: flow
environment:
- name: MALLOC_TRIM_THRESHOLD_
value: 65536
image: not_a_real_image
logConfiguration:
logDriver: awslogs
options:
awslogs-group: prefect_ecs_task_log_ACCOUNT
awslogs-region: us-east-1
awslogs-stream-prefix: ecs-prefect
awslogs-create-group: "True"
Kevin Kho
Scott Aefsky
03/30/2022, 7:30 PMArseniy Aseev
03/31/2022, 4:32 PM