Michael Reeves
08/19/2020, 10:16 PMwith Flow('file') as flow:
name = Parameter('name')
lines = read_file(name=name)
for l in lines:
do_line(l)
finish()
I know theres a prefect Map
functionality, but I didn't see an easy way to make it so each task for each line depends on its previous line task in an iterative mannerChris White
08/19/2020, 11:00 PMLocalExecutor
with mapping, then your tasks will execute sequentially as you expect.
I’m going to keep thinking about this though - I am intrigued by the idea of dynamic dependencies on dynamic tasksMichael Reeves
08/19/2020, 11:01 PMChris White
08/19/2020, 11:02 PMMaxwell Dylla
08/19/2020, 11:03 PMTask.map
, and then Flow.add_edge
to introduce new execution dependencies between the mapped tasks?Chris White
08/19/2020, 11:04 PMMichael Reeves
08/19/2020, 11:05 PMChris White
08/19/2020, 11:05 PMMichael Reeves
08/19/2020, 11:08 PMChris White
08/19/2020, 11:09 PMMichael Reeves
08/19/2020, 11:10 PMChris White
08/19/2020, 11:11 PMMichael Reeves
08/19/2020, 11:18 PMChris White
08/19/2020, 11:22 PM--show-flow-logs
flag on the agent alsoMichael Reeves
08/19/2020, 11:24 PMChris White
08/19/2020, 11:24 PMMichael Reeves
08/19/2020, 11:25 PMpython3 flows.py > out.log
Chris White
08/19/2020, 11:26 PMMichael Reeves
08/19/2020, 11:26 PMChris White
08/19/2020, 11:27 PMMichael Reeves
08/19/2020, 11:27 PMChris White
08/19/2020, 11:29 PMMichael Reeves
08/19/2020, 11:30 PM