Hi, can anyone tell me if prefect 1 keeps task res...
# ask-community
r
Hi, can anyone tell me if prefect 1 keeps task results in memory for the duration of the flow? Especially with DaskExecutor? We have a flow that handles some large outputs/inputs at and early stage in the flow, but they have no utility for the end result of the flow. It seems like they might be held in memory throughout the entire duration of the flow. I notice there is an option
cache_result_in_memory=False
in prefect 2, but does anybody know if there is a way to disable memory caching in prefect 1?
1
c
With DaskExecutor specifically, the dask scheduler manages memory
I believe there was an option to be more aggressive with memory reclamation, I’ll have to see if I can find it again
gratitude thank you 1
r
Thanks Christopher will look into this