egk
04/14/2022, 5:39 PMKevin Kho
04/14/2022, 5:40 PMcheckpointing
for the task and you can’t use DaskExecutor
because both will attempt to serialize it.egk
04/14/2022, 5:42 PMKevin Kho
04/14/2022, 5:49 PMwith Flow(...) as flow:
socket = task_a()
b = task_b(socket)
c = task_c(socket)
will work as long as task_a
checkpointing is turned offfegk
04/14/2022, 5:51 PMDockerExecutor
, right?
Are Executors per Flow or per Task?Kevin Kho
04/14/2022, 5:53 PMLocalExecutor
,LocalDaskExecutor
and DaskExecutor
. Docker is a Run Configuration. Both Run Configuration and Executor are per Flow.
All executors will then run relative to the new container spun up for DockerRun.egk
04/14/2022, 5:54 PMKevin Kho
04/14/2022, 5:56 PMegk
04/14/2022, 5:59 PMKevin Kho
04/14/2022, 6:00 PMegk
04/14/2022, 6:15 PMKevin Kho
04/14/2022, 6:31 PMegk
04/14/2022, 6:35 PMKevin Kho
04/14/2022, 6:36 PM