https://prefect.io logo
#prefect-ui
Title
# prefect-ui
y

YSF

09/02/2022, 2:25 AM
Having a difficult time interpreting how this flow definition turns into this graphic. Can anyone help? How would I make it show that task 2 depends on task 1, and log_task depends on task 1 & task 2, and task 4 depends on log_task? I'm really unclear on what this radar view is trying to tell me
o

Oscar Björhn

09/02/2022, 7:20 AM
Hey, You need to use .submit if you want to visualize a dependency graph.
plus one 1
j

Joao Moniz

09/02/2022, 8:52 AM
Yeah I had the same issue and the solution was what Oscar proposed. But I think this should be fixed somehow if possible, would be surprised if this is the intended behaviour
o

Oscar Björhn

09/02/2022, 9:28 AM
Agreed!
t

Taylor Curran

09/02/2022, 1:34 PM
Hi again Yusuf! These docs should give you a good example of how to use
.submit()
syntax in your flow code. https://docs.prefect.io/tutorials/execution/?h=.submit#result
y

YSF

09/02/2022, 3:14 PM
Thank you for the comments, all! This is how it looks now: The answer here in the discourse may need to be updated.
z

Zanie

09/02/2022, 4:39 PM
We do track dependencies without using
submit
but it depends on the data that you’re passing
🙏 2
k

Keith Hickey

09/06/2022, 5:00 PM
@YSF I recently ran into this same confusion and explored how it works in detail over here: https://prefect-community.slack.com/archives/C0192RWGJQH/p1661609573510969
f

Florian Giroud

10/05/2022, 8:10 PM
Same here, just want to shared feedback : I’m having a very hard time trying to understand the radar, and how to show dependencies between tasks visually. I believe a UI is also some sort of documentation on what a flow does. I was desperately looking for documentation and tutorial about the radar, but I guess I finally have a clue with this thread and the submit() method. Weird though that the radar doesn’t show dependency natively Tx
This is my final workaround, submit alone doesn’t work. We also need to specify wait_for. It’s a lot of useless code just in order to have a radar that displays the tasks in order. @Zanie , I’m curious to see if you have a better way to track dependencies without using .submit
Copy code
@flow(task_runner=SequentialTaskRunner())
def rfdm():
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"I am an INFO message")
    t1 = ingest_data.submit()
    t2 = dbt_run.submit(wait_for=[t1])
    t3 = a.submit(wait_for=[t2])
    t4 = b.submit(wait_for=[t3])
t

Todd de Quincey

11/22/2022, 8:57 PM
To chime in here, I feel like whomever designed this radar was trying to be a little too edgy perhaps (hopefully that doesn’t come across too rude). I think this would be far better it it were redesigned to be a simpler “DAG-style” graph. Not mentioning any competitors, but this is the part of Prefect that I really find lacking. If you have simple flows it isn’t an issue. But if you have even a semi-complicated graph of dependencies, this visualization is critical IMO. Especially if you want to start eg parsing large dbt files and visualise the dependencies. Does anyone know if improvements to this are on the horizon? This is one of my main gripes to adopting Prefect.
5
z

Zanie

11/22/2022, 9:00 PM
cc @justabill
🙏 1
a

Anna Geller

11/22/2022, 11:05 PM
thanks for the feedback, Todd, we have a work in progress to add alternative visualizations meanwhile, for dbt, you could use dbt diagram and if you want to parse it into separate tasks, check out this blog post series https://medium.com/the-prefect-blog/how-to-build-a-modular-data-stack-data-platform-with-prefect-dbt-and-snowflake-89f928974e85
t

Todd de Quincey

11/23/2022, 5:07 AM
Thanks, @Anna Geller. Using the dbt diagram means that you have only a fragmented view of the process if there are upstream and downstream non-dbt tasks. So it doesn’t really provide the solution I am looking for or expecting. Any rough ETAs for these new visualisations or a beta program for them that I can assist with? More than happy to trial run them and provide feedback, identify bugs etc.
a

Anna Geller

11/23/2022, 1:31 PM
that's kind of you, I appreciate that. We'll keep you posted via #announcements and we'll keep in mind that you're interested as early tester
b

Brandon Reid

11/28/2022, 4:11 PM
@YSF Am I right in assuming you expect this flow to end up looking something like this?
y

YSF

11/28/2022, 4:23 PM
Yeah exactly. The thing that's immediately clear here is that log_task has a dependency on both task 1 and task 2. In the way it renders in the radar chart as in my screen grab above after I used the correct syntax it still takes a lot of visual effort and its still not unmistakeably clear. If I didn't know better I'd have thought that task 1 was generating either two outputs or the same output and then was being joined to the output of task 2 before that single object was fed into log_task. Although full disclosure we just ended up signing a contract with Prefect 1 and will switch in the future.
b

Brandon Reid

11/28/2022, 4:26 PM
Ahh, interesting. Nice, thank you.
o

Oscar Björhn

11/28/2022, 5:10 PM
If you run your tasks using submit and wait_for, it will build the graph the way you're expecting it to work. I'm not sure why submit is required, perhaps that's something that could be improved.
gratitude thank you 1
z

Zanie

11/28/2022, 5:45 PM
Running your tasks linearly does not mean there is necessarily a dependency between them
If you pass data between tasks, we’ll show a relationship
If you run your tasks concurrently (i.e. with submit) and want to have a task wait for another task, you can use
wait_for
to indicate an implicit relationship.
o

Oscar Björhn

11/28/2022, 6:08 PM
Makes sense. Could it be made to work without submit, though? I mean, maybe wait_for could be all that's required to build the implicit relationship?
z

Zanie

11/28/2022, 6:14 PM
wait_for
doesn’t have the same meaning outside of
submit
since things are already running sequentially, we can definitely investigate a way to indicate implicit relationships in sequential code though!
o

Oscar Björhn

11/28/2022, 6:15 PM
I think that would be pretty neat!
plus one 1
t

Todd de Quincey

01/19/2023, 8:38 PM
Is the timeline view per this release the proposed alternative visualisation to solve the above problem? Or are there additional visualisations in the pipeline as well? https://prefect-community.slack.com/archives/CKNSX5WG3/p1673555396413039
z

Zanie

01/19/2023, 8:38 PM
Additional visualizations are being planned as well
👌 1
🔥 1
f

Florian Giroud

01/19/2023, 8:41 PM
wow that’s really cool !