One of my flows is struggling to process: agent is on AKS and the CPU usage, through lens, shows a value of 1 for about 6 hours before dropping to negligible amounts (0.002, etc) while still maintaining high levels of RAM usage (30-40gb).
At first I assumed it was stalling, but looking further it seems that the flow is only using 1/8 cores at 100%. This explains the continued RAM use.
Any suggestions on how to utilize the full compute power of the aks vm? I'm assuming DaskTaskRunner is the route to go, and I'm currently experimenting with it, but not having much luck with configurations.