https://prefect.io logo
Title
k

Keith

08/06/2022, 12:59 AM
Have a general question about migrating from Prefect 1.0 to 2.0. In 1.0 there was a generic
upstream_tasks
parameter that you could pass to tasks so that each task knew to wait for the previous one to run. Through my reading of the documentation it seems like this is not necessary anymore b/c everything should run like it would in Python so it basically defaults to a sequential executor. Is this the correct logic? Obviously this story changes a bit when adding in the different
Task Runners
but just wanted to confirm that using default code blocks that tasks run in sequence and won't run the next task until the previous one is complete.
āœ… 3
n

Nico Neumann

08/06/2022, 1:04 AM
In prefect 2.0 it was unified to
wait_for
As for as I know default should be to run task in sequential order, so there is no need to include
wait_for
:thank-you: 1
šŸ™Œ 2
Just tried it:
from prefect import flow, get_run_logger, task


@task
def task1():
    logger = get_run_logger()
    while True:
        <http://logger.info|logger.info>("task1")


@task
def task2():
    logger = get_run_logger()
    while True:
        <http://logger.info|logger.info>("task2")


@flow(name="Test Flow")
def flow_run():
    task1()
    raise Exception()
    task2()


flow_run()
It prints task1 forever and never raises the Exception because it waits forever for the first task to finish
@Keith Just had a look in the documentation: The default task_runner is
ConcurrentTaskRunner
https://docs.prefect.io/api-ref/prefect/task-runners/#prefect.task_runners.ConcurrentTaskRunner Depends on your use case you might want to switch to
SequentialTaskRunner
@Anna Gellerā€™s message explains it well that it only runs in parallel when explicitly calling with .submit or .map: https://prefect-community.slack.com/archives/CL09KU1K7/p1659095834495739?thread_ts=1659086994.265399&amp;cid=CL09KU1K7
šŸ™Œ 1
šŸ™ 1
:upvote: 1
:thank-you: 1
šŸ‘ 1