karpor icon indicating copy to clipboard operation
karpor copied to clipboard

Performance Issue: Cluster Management audit/score/topology api response timeout

Open CirillaQL opened this issue 1 year ago • 1 comments

What happened?

I tried to check information about my cluster in Cluster Management, but the audit, score, and topology responses timed out. 微信截图_20240801144709 微信截图_20240801144721

What did you expect to happen?

apis response successfully

How can we reproduce it (as minimally and precisely as possible)?

import a large Kubernetes cluster and enter into cluster management.

Anything else we need to know?

I use Postman to get api, topology api use 43.5s and audit api use 38.5s, maybe find some ways to fix this performance issue. 微信截图_20240801145638 微信截图_20240801145924 And this is my cluster size: 微信截图_20240801145948

Karpor version

v0.4.4
$ karpor -V
# paste output here

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here
alpine Docker

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

Helm Chart

CirillaQL avatar Aug 01 '24 07:08 CirillaQL

@CirillaQL Thanks for feedback, what you said looks like a performance problem. Usually, the first execution of the audit api will scan all resources in the set cluster. The calculation result will record in the cache, but this will cause the first time to open the page too slowly when there are more resources, which is indeed a problem.

I think I can optimize performance at this point:

  • Do not scan all resources, or asynchronously scan resources

Do you have any ideas?

elliotxx avatar Aug 01 '24 08:08 elliotxx