question about memory management - I have a flow along the lines of:

with Flow() as f:
   big_obj = gen_obj1()
   big_obj2 = gen_obj2(big_obj) #never need to use big_obj again, want to reclaim memory
   big_obj3 = gen_obj3(big_obj2)    # finished with big_obj2
   ...etc

what's the canonical way of freeing up these objects within the flow, once they've been consumed and are no longer needed?

Hey @Philip MacMenamin, for larger datasets, the canoncial way is to persist them inside the task and then pass the location downstream, then open it in the downstream task so that they don’t need to be held in memory.

as in https://docs.prefect.io/core/concepts/persistence.html#persisting-output

No because the Result would persist it but still hold it. More like

@task
def abc():
    result = SomeResult()
    df = ...
    result.write(df, location = "xxx")
    return result.location

You also don’t need the result interface to do this. The downstream task would then:

@task
def bcd(location):
    result = SomeResult()
    df = result.read(location = "xxx")
    return

ok - thank you!