Dzmitry Aliashkevich
10/30/2022, 2:32 PM@task
def prepare_request(batch_size=200):
# querying two databases to get last update ids, then
return list(range(first_id, second_id))
with Flow('import-scheduled') as flow:
load_ids = prepare_request()
wflr = None
for id in load_ids:
flr = create_flow_run()
if wflr:
flr.set_upstream(wflr)
wflr = wait_for_flow_run()
wflr.set_upstream(flr)
the problem here is that it seems I can’t just iterate task result, and I can’t use just method without task decorator as using db connection secrets from contextAnna Geller
10/30/2022, 3:37 PMDzmitry Aliashkevich
10/31/2022, 6:03 AMAnna Geller
10/31/2022, 11:38 AMDzmitry Aliashkevich
10/31/2022, 11:42 AM