https://prefect.io logo
Title
m

Malavika S Menon

05/11/2023, 3:22 PM
Hi, I want to create a DAG using prefect and run the tasks such that all dependencies are satisfied, essentially in a topologically sorted manner. I referrred to https://discourse.prefect.io/t/how-to-define-dag-like-flow-in-prefect-2-0/2069/7 but here are some constraints I am facing.
my_network_graph = {
    "task_1": [],
    "task_2": [],
    "task_3": ["task_1", "task_2"],
    "task_4": ["task_3"],
    "task_5": ["task_4"],
}
This is the graph mentioned in the example, if I change this to include
my_network_graph = {
    "task_5": ["task_4"],
    "task_1": [],
    "task_2": [],
    "task_3": ["task_1", "task_2"],
    "task_4": ["task_3"],
}
then task 5 would be executed first regardless of its dependency on task 4. I want to give a dictionary with deps, but the order of keys in the dictionary shouldnt matter. This becomes especially important in my use case where I have 100+ nodes in the graph, and its complicated to chart out the deps such that all upstream deps are mentioned before in the dictionary. How can I solve this?