@Marvin Is it possible to have builds depend on eachother? I have two large separate python scripts and want a flow/build that will run both of them in sequence

It is possible to have tasks and flows depend on each other in Prefect to run in a sequence. You can achieve this by specifying upstream dependencies when defining your tasks and flows. The

SequentialTaskRunner

can enforce tasks to run sequentially. Additionally, you can manually define upstream dependencies to ensure a task waits for another to complete before running. You can set the task dependency using the

wait_for

argument:

python
from prefect import flow, task

@task
def upstream():
    # your upstream task logic

@task
def downstream(wait_for):
    # your downstream task logic

@flow
def flow_of_tasks():
    upstream_result = upstream.submit()
    downstream_result = downstream.submit(wait_for=[upstream_result])
    return downstream_result

If you have separate Python scripts that you want to run in sequence within a Prefect flow, you can wrap the script execution within tasks and use similar

wait_for

logic to sequence them. You can find more details about running tasks sequentially in the Task Runners documentation and about specifying upstream dependencies in the Specifying Upstream Dependencies guide.

by "large" I assume you mean you'd want them to run on separate infra, like separate containers?

hi Kevin! Just long as in many lines of code, running on the same container is fine. I am new to Prefect moving from a CRON job structure and would like to keep these two python scripts independent, but execute in sequence

I think it depends on what exactly you mean by independent. if they both have a main/entrypoint style function, you could decorate them both with

@flow

and call them both in sequence from within a flow, and they'll both show up as fully fledged flows in the UI

So the main flow would be something like?:

@flow
def mainflow():
    scriptflow1_result = scriptflow1.submit()
    scriptflow2_result = scriptflow1.submit(wait_for=[scriptflow1_result ])
    return scriptflow2_result

In this example, would creating a third py script be the best way to organize/build this?

if the scripts are in other files, you'll need to import them and call them like any other function

got it. Thank you for the help! Will be piecing this together now 🙂