Tobias Schmidt
11/18/2019, 9:47 AMA -> B -> C
. I want task A to be skippable via a command line argument to the Python script that implements that flow. What's the best way to do this? Can I mark A as skipped, set skip_on_upstream_skip
on B and C (or just B?) to False and then run the flow as usual? Or is there a more idiomatic way of doing this?emre
11/18/2019, 9:57 AMwith Flow('name') as flow:
upstream_tasks_b = []
if not skip_a:
a_result = A()()
upstream_tasks_b.append(a_result)
b_result = b()(
upstream_tasks=upstream_tasks_b
)
c_result = c()(b_result)
I’m personally satisfied with the result, as it keeps flow clean depending on my config, but this can get wonky pretty quick. I’m as much interested about hearing how people approach this problem of flow customization.Jeremiah
11/18/2019, 1:39 PMraise prefect.engine.signals.SKIP()
inside task A depending on a Parameter
passed to it at runtime. A would mark itself as skipped and B or C could respond appropriately (through skip_on_upstream_skip=False
, for example)