https://prefect.io
Join Slack
hey is there a way to run a subset of a flow? ideally I could pick a single task in a flow and any d...
# ask-community
j
hey is there a way to run a subset of a flow? ideally I could pick a single task in a flow and any dependent tasks for that specific flow would also be run
z
this might be what youre looking for https://docs.prefect.io/core/idioms/flow-to-flow.html
👍 1
k
Hey @joshua mclellan, the short answer is not really. You can make more than one flow object in a script though so you can make another flow that is the subset.
Copy code
@task()
def add_ten(x):
    print(f'starting {x}')
    sleep(10)
    print(f'ending {x}')
    return x * 2
    
with Flow('simple map') as flow:
    t = add_ten.map([1, 2, 3])
    s = add_ten.map([10, 20, 30])

with Flow('simple map2') as flow2:
    t = add_ten.map([1, 2])

flow.register('project')
flow2.register('project')
Yes Zach is right though that maybe you want the Flow of Flows setup. A lot of users use this to trigger subsections of the full logic at a time. This also has the added benefit that different subflows can run on different Executors
For running an individual task, you can use 
task.run()
also, but this won’t run all the dependencies attached.
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by