TairHash icon indicating copy to clipboard operation
TairHash copied to clipboard
verified publisher icon tair-opensource

Potential performance inconsistency

Open ashtonian opened this issue 3 years ago • 3 comments

Hey guys,

Just rolled that time series PR into prod so, its working and seems to be working well for the most part. We are doing around ~300ops/s. It seems that some of the operations are not consistent time though. Any suggestions on profiling this further?

image

image

ashtonian avatar Aug 22 '22 22:08 ashtonian

You mean the 120 ms latency caused by exhgetallwithver ?

chenyang8094 avatar Aug 29 '22 06:08 chenyang8094

yea focusing on reads. So we are writing (to hashmaps via tairhash) across the cluster at about 240ops/s. We have a couple of apps doing very light reads, and then user driven traffic makes up the rest. I think most exhgetallwithver calls are <1ms and I think its a bit weird that sometimes its >10ms and occasionally >100ms.

ashtonian avatar Sep 07 '22 17:09 ashtonian

Do you think all exhgetallwithver is accessing the same key? Maybe some tairhash is very big (bigkey) and exhgetallwithver is very slow. exhgetallwithver is an O(n) complexity command.

chenyang8094 avatar Sep 08 '22 00:09 chenyang8094