pinot icon indicating copy to clipboard operation
pinot copied to clipboard

[federation] Add multi-cluster routing support for federated broker queries

Open shauryachats opened this issue 2 months ago • 1 comments

This PR is a part of the Cross Cluster Federation in Pinot as proposed here: https://github.com/apache/pinot/issues/16456

Overview

Introduces multi-cluster routing capability that allows a broker to query both local and remote Pinot clusters, enabling federated query execution across cluster boundaries. Subsequent PRs will hook these routing managers with the larger federation flow.

Changes

  • Added MultiClusterRoutingManager - orchestrates routing across local and remote Pinot clusters
  • Added RemoteClusterBrokerRoutingManager - manages routing for individual remote clusters with periodic sync
  • Modified BrokerRoutingManager visibility to support inheritance (5 fields/methods: private → protected)
  • Added REMOTE_CLUSTER_BROKER_ROUTING_CALCULATION_TIME_MS metric to BrokerTimer

These are new classes which are not hooked to any production flow currently, and hence do not impact any regular functionality.

Testing

Tested as a part of the draft PR: https://github.com/apache/pinot/pull/17145. Added unit tests in this PR and integration tests will be added subsequently.

shauryachats avatar Dec 01 '25 21:12 shauryachats