alex07/07/2020, 4:27 PM
In the schematic and gantt chart I have a dag with all the
with Flow("My flow", schedule=schedule) as flow: for source in all_sources: source.bind( ## parameters for my run() function here ) res = [source() for feed in all_sources]
leading to a list, which is expected but I also duplicated
tasks without any edges
Zachary Hughes07/07/2020, 4:33 PM
alex07/07/2020, 4:40 PM
call resolved that issue, I think maybe I was just using it incorrectly?
Zachary Hughes07/07/2020, 4:59 PM
creates a keyed edge-- I'm wondering if there's some peculiarity in visualization between keyed edges and non-keyed edges.
alex07/07/2020, 5:29 PM
class DerivedMeta(SignatureValidator, type(ABC)): pass class BaseSource(Task, metaclass=DerivedMeta): # has run method w custom parameters class DerivedClasses(BaseSource): # no run implementation, uses parent's
is a list[DerivedClasses]
Zachary Hughes07/07/2020, 6:16 PM
, you create a separate set of tasks without edges. That would explain the duplicate set of tasks you were seeing.
res = [source() for feed in all_sources]
alex07/07/2020, 7:11 PM
, what would the best way to achieve what I am trying to do be? Would just using
instead of bind() be ok?
res = [source(**run_kwargs) for feed in all_sources]
Jim Crist-Harif07/07/2020, 7:34 PM
can be useful, but we recommend using the functional api when possible.