Argemiro Neto
09/18/2019, 4:28 PM==== START ====
=== starting the flow
== generate called
= status: [1, 2, 3]
== task runner called with 1
== task runner called with 2
== task runner called with 3
==== END ====
Code:
@task
def generate_task_list() -> list:
print("== generate called")
return [1, 2, 3]
@task(max_retries=3, retry_delay=timedelta(seconds=0))
def task_runner(x: int) -> None:
print('== task runner called with {}'.format(x))
@task
def print_status(text: any) -> None:
print('= status: {}'.format(text))
with Flow("random-mapping") as f:
print_status('=== starting the flow')
values = generate_task_list()
print_status(values)
final = task_runner.map(x=values)
print('==== START ====')
f.run()
print('==== END ====')
By running this multiple times I get different results but never what I expected. What am I missing? Thank you!Chris White
09/18/2019, 4:35 PM== generate called
= status: [1, 2, 3]
== task runner called with 1
== task runner called with 2
== task runner called with 3
is always printed in the correct order, but
=== starting the flow
shows up randomly, is that accurate?Argemiro Neto
09/18/2019, 4:37 PM==== START ====
== generate called
== task runner called with 1
== task runner called with 2
== task runner called with 3
= status: [1, 2, 3]
==== END ====
Chris White
09/18/2019, 4:38 PMf.visualize()
you’ll see how Prefect views your Flow: the print_status
task doesn’t have any dependencies (upstream or downstream), and so can run at anytime; if you want to ensure it always runs prior to generate_task_list
, you can update your code to:
status = print_status('=== starting the flow')
values = generate_task_list(upstream_tasks=[status])
generate_task_list
has two completely independent downstream dependencies: print_status
and `task_runner`; if you want them to run in a certain order you have to explicitly tell Prefect their dependency structureArgemiro Neto
09/18/2019, 4:41 PMChris White
09/18/2019, 4:43 PMMarvin
09/18/2019, 4:43 PMChris White
09/18/2019, 4:43 PM