Are there any known issues in prefect 1.0 of failed tasks, when retrying, causing memory leaks (LocalDask executor)?
Anders Segerberg
05/02/2023, 5:03 PM
We've run into recurring issues where, when mapping over a large number of tasks (which otherwise should fit well within memory), if some of them start to fail, and then retry, there will be cascading failures which eventually trigger the gQL server going down, as well as the flow -- and then ultimately the VM we're running on, which runs out of swap space and has other extremely-high memory and CPU usage issues.
I recognize this is a rather broad question, but, generally speaking, I'm wondering if there was at any point, known issues in this area, related to resource usage and contention
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.