The latest version of Orion introduces

.submit()

, eg:

@flow
def greetings(names: List[str]) -> None:
    for name in names:
        say_hello.submit(name)
        say_goodbye.submit(name)

but the flow still works without using

.submit()

. What are the implications of not using

.submit()

?

without .submit tasks are being executed sequentially like if you didn’t decorate them with @task at all.

@Oliver Mannion you can read more here https://discourse.prefect.io/t/what-is-the-default-taskrunner-executor/63#prefect-20-1

And what if you combine tasks with and without

submit

in a flow (if that’s even possible)?

yes, it should be, if not @Matthias please report back

So then, some tasks are send to the taskrunner and some are not? Sounds awesome!

Exactly 💯 the idea is that task runner is not always needed. It's primarily needed when you want to move to more advanced use cases such as parallel execution with async, Dask or Ray, but this is not always required, so now only when you explicitly submit a task to a task runner, it will be considered for this type of execution, otherwise it will run sequentially directly in a flow run process

Maybe stupid next question: in light of what you explained, what is then the purpose of the sequential task runner?

debugging maybe. a more lockstep environment.

what is then the purpose of the sequential task runner?

@Matthias I believe the purpose is: if you want to operate on state dependencies rather than returned data (and even exceptions) from tasks, and you still want to use sequential execution, you need Sequential Task Runner:

@task
def task_a():
    pass

@task
def task_b():
    pass

@task
def task_c():
    pass

@task
def task_d():
    pass

@flow(task_runner=SequentialTaskRunner())
def my_flow():
    a = task_a.submit()
    b = task_b.submit()
    # Wait for task_a and task_b to complete
    c = task_c.submit(wait_for=[a, b])
    # task_d will wait for task_c to complete
    # Note: If waiting for one task it must still be in a list.
    d = task_d(wait_for=[c])

Thanks for the explanation! Will that be in the documentation somewhere?

100%, if not please ping us in the next week or two 😄

Personllay I find this .submit change a little annoying... One of the "selling" points of Prefect 2.0 was, as stated by the introduction to Prefect 2.0 article:

automatically parallelize your synchronous Python script with one of our concurrent executors (enabled by default)

This automatic and easy parallelization that we could automatically achieve for all the tasks in a flow is now lost in last beta versions... From now on we need to provide the .submit for each task if we want all of the tasks in our script to run in parallel and handled by Dask.

There are always trade-offs but it will get even more interesting and configurable - follow the #CKNSX5WG3