Thread
#prefect-community
    m

    Michael Reeves

    2 years ago
    If I want my flow to run a series of tasks in an exact order ignoring data dependencies, is there an easier way to do that other than manually adding an edge between all the tasks?
    I'm having an issue where a parameter is causing tasks to be executed multiple times
    nicholas

    nicholas

    2 years ago
    Hi @Michael Reeves - you can manually set upstream dependencies (independent of data dependencies) with the
    upstream_tasks
    kwarg, which accepts an array of tasks, like this:
    my_task(upstream_tasks=[my_upstream_task])
    m

    Michael Reeves

    2 years ago
    I did that and it still ran twice due to being downstream of a parameter task
    it seemed to almost be making two copies of a task
    nicholas

    nicholas

    2 years ago
    Ah, usually what's going on there is you're passing independent references to the task, which is telling Prefect you want that task to run in two separate instances. To explicitly tell Prefect to use a single instance of a task, instantiate it independently and pass in that reference like this:
    my_upstream_task_ref = my_upstream_task()
    
    task_1(upstream_tasks=[my_upstream_task_ref])
    task_2(upstream_tasks=[my_upstream_task_ref])
    (and continue that for all dependencies that rely on that instantiation)
    m

    Michael Reeves

    2 years ago
    AH thats it! thanks!
    nicholas

    nicholas

    2 years ago
    😄
    m

    Michael Reeves

    2 years ago
    gosh that was very frustrating, but that makes so much sense now, I was passing in references to the original tasks not the instantiated tasks
    nicholas

    nicholas

    2 years ago
    Sorry about that! It's easy to forget that this is normal Python sometimes since it's doing such cool things 💪
    m

    Michael Reeves

    2 years ago
    well I'm also a c programmer so should have thought of this as the first issue lolol