Scott Moreland11/24/2020, 2:42 PM
Jenny11/24/2020, 3:57 PM
Scott Moreland11/24/2020, 6:57 PM
Jenny11/24/2020, 7:02 PM
Scott Moreland11/24/2020, 7:40 PM
Wondering if there is a simple way to just run
with Flow('my flow') as flow: first_output = first_task() second_output = second_task(first_output) third_output = third_task(second_output) if __name__ == '__main__': flow.run()
, such that it reads
from the checkpoint (on disk), performs the computation defined by
, and then exits with a success code. I guess more generally, it would be nice to run a contiguous subset of the flow graph by defining start and stop tasks if such a thing is possible with a DAG.
Jenny11/24/2020, 8:30 PM
which could contain the state of a previous flow run that failed (and has all of the task states attached). Then the runner would only rerun the failed task and forward BUT that would also require taking the previous failed state and putting it into a Scheduled state / setting the tasks to rescheduled manually. The reason we added that button in the UI is so it can do all that logic for you!
Scott Moreland11/24/2020, 8:39 PM
Jenny11/24/2020, 8:41 PM