pd
pd copied to clipboard
Random connect PD instance for the MemberList grpc call
Feature Request
Random connect PD instance for the MemberList call
Describe your feature request related problem
When PD looses leadership, all TiKV/TiDB pods start to call MemberList on PD/ETCD to find the leader. During incident we found that all these calls always go to a particular PD instance. It seems we always sort the PD members, thus when TiKV/TiDB trying to get the PD leaders they always reach to a specific PD pod which caused disproportional GRPC calls increase on the PD instance.
func (c *RaftCluster) Members() []*Member {
c.Lock()
defer c.Unlock()
var ms MembersByID
for _, m := range c.members {
ms = append(ms, m.Clone())
}
sort.Sort(ms)
return []*Member(ms)
}