opensearch-project
[FEATURE] Multi Terms Aggregation has no option to add missing field value.
Multiterms Aggregation builder takes a MultiTermsLookUp List as the parameter for terms which has a field function but if the field value is missing for that field, the data is not retuned. Basically there is no missing function which exixts for Terms Aggregation Builder.
A function eaxctly like which is present for TermsAggregation Builder would solve the problem I suppose, TermsAggregatio Builder has a method to add missing field value but no such method appears to Exist for MultiTerms. Java docs link. I've tried to use Terms Aggregation Builder but that only takes on field at a time but I want to use multiple terms.
List<MultiTermLookup> termLookupList = new ArrayList<>();
termLookupList.add(new MultiTermLookup.Builder().field("sample.id").build());
termLookupList.add(new MultiTermLookup.Builder().field("sample.name").build());
termLookupList.add(new MultiTermLookup.Builder().field("sample.mediaCode").build());
MultiTermsAggregation multiTermsAggregation = new MultiTermsAggregation.Builder()
.terms(termLookupList)
.size(10000)
.build();
This is the way I am using the multi terms aggregation. But when the 'mediacode' field is missing for a particular document. That creates a problem and data is not returned for that document. I am further using the multiTerms aggregation to do some calculations like this. :
FieldSort fieldSort = new FieldSort.Builder()
.field("aggregatedValue")
.order(SortOrder.Desc)
.build();
SortOptions sortOptions = new SortOptions.Builder()
.field(fieldSort)
.build();
BucketSortAggregation sortAggregation = new BucketSortAggregation.Builder()
.sort(sortOptions)
.size(10)
.build();
Aggregation aggregationForSum = new Aggregation.Builder().sum(sumAggregation).build();
Aggregation aggregationForReverseNesting = new Aggregation.Builder().reverseNested(reverseNestedAggregation).build();
Aggregation aggregationForBucketSort = new Aggregation.Builder().bucketSort(sortAggregation).build();
Aggregation rankingAggregation = new Aggregation.Builder()
.multiTerms(multiTermsAggregation)
.aggregations("missing",aggregationForMissing)
.aggregations("aggregatedValue",aggregationForSum)
.aggregations("dealCount",aggregationForReverseNesting)
.aggregations("aggregatedValue_bucket_sort",aggregationForBucketSort)
.build();
I have also tried using MissingAggregation Builder but that does not seem to work with multiterms when I'm adding that in my ranking aggregation.
I'm not sure that multi_terms supports a missing parameter on OpenSearch itself: https://opensearch.org/docs/latest/aggregations/bucket/multi-terms/
Do you have a query you've used on the OpenSearch Dashboards Dev Tools that does work as expected?
If not then it would be worth opening a feature request on https://github.com/opensearch-project/OpenSearch
Yes I do have sample on dev tools where I used multi terms aggregation with a missing parameter.
"aggregations": {
"ranking": {
"multi_terms": {
"terms": [
{
"field": "sampleField.id"
},
{
"field": "sampleField.name",
"missing": ""
},
{
"field": "sampleField.mediaCode",
"missing": ""
}
],
"size": 10000
}
Here simply If the name or mediacode field was missing it was replaced with an empty string. But if I removed the missing parameter, the data was skipped for aggregation if one of the field for it did not exist.
Elastic search API reference: MultiTermsAggregation.Builder, The terms take a type of MultiTermLookup value which has the missing parameter in its builder.