Elasticsearch Java Api client single metric aggregation deserializer

Question

I am using elasticsearch-java api client (8.10.3) and facing a small issue and I hope someone has already encountered this issue before.

I am trying to perform a min aggregation on documents that may not contain the field I try to aggregate. In case that all the documents that match my query don't contain the field I try to aggregate, the "value": null is being deserializing to 0 via the JsonpDeserializer.doubleOrNullDeserializer(0) which is defined inside the setupSingleMetricAggregateBaseDeserializer SingleMetricAggregateBase class

https://github.com/elastic/elasticsearch-java/blob/main/java-client/src/main/java/co/elastic/clients/elasticsearch/_types/aggregations/SingleMetricAggregateBase.java

In my opinion, this approach may lead to confusion in our product and I am searching a way to avoid this deserialization but I couldn't find so far. It's ok to return null in that case and I would prefer to achieve this.

any ideas? :)

I tried to add also value count aggregation in order to determine if there are values in the field I try to aggregate and it worked but it is a little weird in my opinion.

G0l0s · Accepted Answer · 2024-03-26 11:34:55Z

You can filter documents without a field by the exist query

Sample documents

POST /filter_without_field/_bulk
{"create":{"_id":1}}
{"comment":"document with field","field": 1}
{"create":{"_id":2}}
{"comment":"document without field"}
{"create":{"_id":3}}
{"comment":"document with field","field": 10}

Query with a filter and an aggregation

GET /filter_without_field/_search?filter_path=aggregations
{
    "query": {
        "bool": {
            "filter": [
                {
                    "exists": {
                        "field": "field"
                    }
                }
            ]
        }
    },
    "aggs": {
        "min_field_value": {
            "min": {
                "field": "field"
            }
        }
    }
}

Response

{
    "aggregations" : {
        "min_field_value" : {
            "value" : 1.0
        }
    }
}

Or you can use the missing parameter with a great number

POST /filter_without_field/_search?filter_path=aggregations
{
    "aggs": {
        "min_field_value": {
            "min": {
                "field": "field",
                "missing": 1000
            }
        }
    }
}

Response is the same

Convert a favorite query to Java code

thanks for your comment! It's correct that I can use the ExistsQuery, this will reslove my issue. however, my team is developing a service which is a proxy to an elasticsearch cluster. our current design is that once we are creating an aggregation via the library we cannot change the query. but I think we can do such a change. I think using the missing funcitonallity is a little problematic in case of average. overall, the native behavior of elasticsearch is to ignore documents that are missing the aggregated field and may return null. its stange that the library doesn't allow this.

Collectives™ on Stack Overflow

Elasticsearch Java Api client single metric aggregation deserializer

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related