https://prefect.io logo
Title
m

Michael Reeves

08/21/2020, 3:43 PM
If I want my flow to run a series of tasks in an exact order ignoring data dependencies, is there an easier way to do that other than manually adding an edge between all the tasks?
I'm having an issue where a parameter is causing tasks to be executed multiple times
n

nicholas

08/21/2020, 3:55 PM
Hi @Michael Reeves - you can manually set upstream dependencies (independent of data dependencies) with the
upstream_tasks
kwarg, which accepts an array of tasks, like this:
my_task(upstream_tasks=[my_upstream_task])
👍 1
m

Michael Reeves

08/21/2020, 3:59 PM
I did that and it still ran twice due to being downstream of a parameter task
it seemed to almost be making two copies of a task
n

nicholas

08/21/2020, 4:02 PM
Ah, usually what's going on there is you're passing independent references to the task, which is telling Prefect you want that task to run in two separate instances. To explicitly tell Prefect to use a single instance of a task, instantiate it independently and pass in that reference like this:
my_upstream_task_ref = my_upstream_task()

task_1(upstream_tasks=[my_upstream_task_ref])
task_2(upstream_tasks=[my_upstream_task_ref])
(and continue that for all dependencies that rely on that instantiation)
m

Michael Reeves

08/21/2020, 4:16 PM
AH thats it! thanks!
n

nicholas

08/21/2020, 4:16 PM
😄
m

Michael Reeves

08/21/2020, 4:16 PM
gosh that was very frustrating, but that makes so much sense now, I was passing in references to the original tasks not the instantiated tasks
n

nicholas

08/21/2020, 4:17 PM
Sorry about that! It's easy to forget that this is normal Python sometimes since it's doing such cool things 💪
👍 1
m

Michael Reeves

08/21/2020, 4:18 PM
well I'm also a c programmer so should have thought of this as the first issue lolol