Chris
04/29/2024, 1:48 PMNate
04/29/2024, 3:39 PMChris
04/29/2024, 3:40 PM@flow(cache_result_in_memory=False)
and my memory usage seems much better. Is there another config for not storing/caching?Nate
04/29/2024, 3:41 PMcache_result_in_memory=False
should avoid taking up extra space in memoryChris
04/29/2024, 3:42 PMNate
04/29/2024, 3:45 PMcache_result_in_memory
is a lever you have there. is there a specific problem you're still hitting?
other general tips would be:
⢠pass references to really large data
⢠use prefect.annotations.quote(really_big_input_param)
to avoid inspection of some massive task parameter (if its just static data, and not futures that need to be resolved, since quote will also skip that)Chris
04/29/2024, 3:52 PM