shown below, as I've been getting a warning when passing a large dataframe to prefect tasks.
The docs mention the following:
>
prefect.utilities.annotations.quote
will disable task dependency tracking for the wrapped object, but likely will increase performance
What exactly is task dependency tracking? Is that the little flow chart arrows shown in the prefect UI?
✅ 1
n
Nate
03/09/2024, 10:51 PM
hey @Matthew Bell - yep, dependency tracking in general is the APIs understanding of what task runs depend on each other based on how they pass their results around
Copy code
@flow
def foo():
foo_result = foo_task()
bar(foo_result) # bar takes foo's inputs, so bar "depends" on foo
and yea, in the UI that's visualized with the arrows between task or flow runs
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.