Finding documents lacking a field in ElasticSearch

Question

I'm new to ES and currently evaluating how it would work for us as a backend for storing log data/events. One of the requirements is to bulk update all indexed events and, e.g. add a field.

I looked at the query DSL but I can't find an example for a query that would retrieve all docs which do NOT contain a certain field. The DSL is very expressive when it comes to describing values, but not fieldnames and field existence.

Maybe this is an unusual usecase for typical ES users but I already ran into a real need for it when playing with the ElasticSearch ELK stack. When I inserted data into ES which had the wrong format for Kibana, I got loads of exceptions. Now I wanted to correct the format but I can't retrieve the corrupted documents. Is there a way?

¨this might be what you're looking for elasticsearch.org/guide/en/elasticsearch/reference/current/… — Touchpad
– Touchpad, Commented Apr 7, 2014 at 14:13

keety · Accepted Answer · 2014-04-08 03:23:38Z

2

You should be able to use the "exists" filter in conjunction with the "not" filter mentioned above by @touchpad or simply the "missing" filter

  {
   "filter": {
      "not": {
         "exists": {
            "field": <field_name>
         }
      }
   }
}

edited Apr 8, 2014 at 3:23

answered Apr 7, 2014 at 14:34

keety

17.5k4 gold badges53 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Finding documents lacking a field in ElasticSearch

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related