https://prefect.io logo
Title
d

David Hlavaty

03/13/2023, 11:16 PM
Hi, I have a question about how caching results in memory work with sub-flows. Considering this code snippet:
@task
def large_data() -> str:
    return "this is very very very large data returned by Prefect task"


@flow
def sub_flow() -> int:
    l = large_data()
    length = len(l)
    # Statement 1: output of task `large_data` is cached in-memory inside `sub_flow` context
    return length


@flow
def example():
    length = sub_flow()
    # Statement 2: output of task `large_data` is NO longer cached in-memory
    #      only output of `sub_flow` is cached in-memory at this point
Are the two statements in comments correct? More generally: Once (sub)flow returns, all results cached in-memory (other than the return value(s) of the (sub)flow itself) are released and can be garbage collected? When inspecting flow run context in debugger, I didn't see any state that would suggest above is not true. Thanks!