David Hlavaty
03/13/2023, 11:16 PM@task
def large_data() -> str:
return "this is very very very large data returned by Prefect task"
@flow
def sub_flow() -> int:
l = large_data()
length = len(l)
# Statement 1: output of task `large_data` is cached in-memory inside `sub_flow` context
return length
@flow
def example():
length = sub_flow()
# Statement 2: output of task `large_data` is NO longer cached in-memory
# only output of `sub_flow` is cached in-memory at this point
Are the two statements in comments correct? More generally:
Once (sub)flow returns, all results cached in-memory (other than the return value(s) of the (sub)flow itself) are released and can be garbage collected?
When inspecting flow run context in debugger, I didn't see any state that would suggest above is not true.
Thanks!