Lennert Van de Velde

11/21/2022, 3:05 PM
Hey, We are developing a flow where a lot of JSON records get queried (in a Task). To save on memory resources, I've configured Result Persistence on the Task, but I'm now noticing a build up of memory usage of a serializer object. When are results serialized and when is this memory released?
Build up of memory usage by
/usr/local/lib/python3.9/site-packages/prefect/ size=70.3 MiB, count=427649, average=172 B
/usr/local/lib/python3.9/site-packages/prefect/ size=73.4 MiB, count=450775, average=171 B
Task decorator:

Kalise Richmond

11/22/2022, 5:49 PM
Hey @Lennert Van de Velde how did you configure the result persistence on the task? What is the output of your
when you run
prefect config view

Lennert Van de Velde

11/23/2022, 8:41 AM
Hi Kalise, I've already put up an issue on Github, so it is close to being solved I believe 🙂