apache
DrillSideways and early termination
Description
As I was upgrading an application using Lucene 8.10 to Lucene 10.x, I've hit an unexpected situation with DrillSideways. In the past, when using DrillSideways with a collector that would early terminate, the whole collection would terminate and facet collection would also exit (with incomplete counts). It seems like now, DrillSideways#search code will wrap the hitCollector with a MultiCollector, and because of this, early termination in one collector doesn't translate into early termination in the other (the FacetCollector), so the collection will complete to the end. While I haven't test it, I expect FacetCollectorManager#search to behave similarly, since it's also wrapping the hitCollector in a MultiCollector.
Looking at the 8.x code, it seems like the behavior wasn't 100% consistent (i.e when "drillDownDims" was empty we would the and likely hit the same behavior) so this could have been a feature or a bug in 8.10 and before.
That said, I think this is a valid feature. I'm thinking we could have DrillSideways/FacetCollectorManager use a different implementation equivalent to MultiCollector that optionally exits collection whenever it sees the hitCollector throw a CollectionTerminatedException. Alternatively, if this is useful for features other than facet, we could change MultiCollector to optionally bubble up the CollectionTerminatedException instead of removing the particular collector that wants to terminate.
Version and environment details
No response
Thanks @tflobbe for raising this. It's a bit of an interesting challenge to figure out the right thing to do with multi-collection. The current approach is that each wrapped collector can independently terminate, but those terminations don't impact the ability for other sibling collectors to continue. So in this case, if your main collector is terminating early, facet collection will continue (as you point out).
Have you looked at the DrillSideways#search method that was added ~1 year ago that lets you provide your own drill-down and drill-sideways facet collector implementations? I think this should give you the flexibility to do what you're looking for if you create a version of MultiCollectorManager that terminates all collection as soon as one collector terminates?
