So I have a bunch of tasks which create and read f...
# ask-community
j
So I have a bunch of tasks which create and read from files. The downstream tasks in my flow don't really depend on the actual Python return value of the upstream tasks, only that the files have been created. Is there any way to explicitly declare that one task relies on another? Ideally I'd have one parameter for my flow that would be passed to each task. But if I do that out-of-the-box, then using a Dask executor each function will try to run immediately despite the necessary files not being created.
a
@John Jacoby you can declare these as upstream tasks by declaring them explicitly with the
upstream_tasks
keyword argument, e.g.
Copy code
with Flow('my_flow') as flow:
    unrelated = unrelated_task()
    other = other_task(upstream_tasks=[unrelated])
This guarantees that
unrelated
finishes first even if
other
doesn’t need any values from it.
upvote 2
j
Thank you! I was looking for something like that in the docs.
Does this work with mapped tasks? During debugging it seems that a task that shouldn't run based on a failed earlier task is still running.