pinot
pinot copied to clipboard
[federation] Add multi-cluster routing support for federated broker queries
This PR is a part of the Cross Cluster Federation in Pinot as proposed here: https://github.com/apache/pinot/issues/16456
Overview
Introduces multi-cluster routing capability that allows a broker to query both local and remote Pinot clusters, enabling federated query execution across cluster boundaries. Subsequent PRs will hook these routing managers with the larger federation flow.
Changes
- Added
MultiClusterRoutingManager- orchestrates routing across local and remote Pinot clusters - Added
RemoteClusterBrokerRoutingManager- manages routing for individual remote clusters with periodic sync - Modified
BrokerRoutingManagervisibility to support inheritance (5 fields/methods: private → protected) - Added
REMOTE_CLUSTER_BROKER_ROUTING_CALCULATION_TIME_MSmetric toBrokerTimer
These are new classes which are not hooked to any production flow currently, and hence do not impact any regular functionality.
Testing
Tested as a part of the draft PR: https://github.com/apache/pinot/pull/17145. Added unit tests in this PR and integration tests will be added subsequently.