Shiyu Gan
03/23/2022, 3:24 AMKevin Kho
client.submit()
. The exception is mapping where if you have A -> B -> C, then A1 -> B1 -> C1 are submitted.
The Prefect DAG still has features the Dask DAG doesn’t have. For example, Prefect is still responsible for stuff like caching across flow runs or triggers to determine whether or not to run a task.Shiyu Gan
03/23/2022, 3:32 AMKevin Kho
Shiyu Gan
03/23/2022, 3:39 AMIf tasks are submitted immediately, how is the "dependency wait" honoured?Copy codeTasks are submitted immediately using client.submit().
Kevin Kho
Shiyu Gan
03/23/2022, 3:44 AMKevin Kho
Shiyu Gan
03/23/2022, 3:48 AMflow_runner.py
's get_flow_run_state
is the "block" logic?Kevin Kho
Shiyu Gan
03/23/2022, 3:56 AMexecutor.submit()
= client.submit()
. If using LocalExecutor, executor.submit() = regular function call
Shiyu Gan
03/23/2022, 4:49 AMKevin Kho
Shiyu Gan
03/23/2022, 5:25 AMTomer Cagan
03/23/2022, 7:11 AMShiyu Gan
03/24/2022, 8:17 AMKevin Kho