Hey everyone A few months ago I posted a discussion on optim

Question

Hey everyone A few months ago I posted a discussion on optimizing for high memory usage in the < C03D12VV4NN|best practices coordination plane> We re still running into this issue Idk if this apparent build up of memory usage is a bug with prefect or if it is something we can control

Joshua Grant · Answer

I m just some guy not associated with prefect org but what TaskRunner are you using

Kelvin DeCosta · Answer

Hey < Joshua Grant> I m using the `SequentialTaskRunner` I ve tried setting `cache result in memory` as `False` but that only moves the problem to the Disk Usage Flows now crash with `OSError code 28 No space on device`

Kelvin DeCosta · Answer

More context on the tasks About 20 30 inputs at a time `task a map` gt `task a states` gt `task a results` `task a results` are used by `task b` `map` to give `task b result` Finally `task c` takes in `task b results` and returns `None`

Kelvin DeCosta · Answer

I previously used the `ConcurrentTaskRunner` and found that the memory usage builds up even though the the input is already consumed by the end of `task c` and the new batch must begin