Elasticsearch Query Returning Zero Results for Substring

Question

I created my first AWS ElasticSearch cluster and uploaded some data to it (shown below).

When I search for a domain such as example.com, I get zero results.

Is this a search query or indexing issue?

# curl -XGET -u username:password 'https://xxxxx.us-east-1.es.amazonaws.com/hosts/_search?q=example.com&pretty=true'
{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

I confirmed that a match_all query does return all the records.

match_all

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "hosts",
        "_type" : "_doc",
        "_id" : "KK0PcnMBqk4TBzxZPeGU",
        "_score" : 1.0,
        "_source" : {
          "name" : "mail.stackoverflow.com",
          "type" : "a",
          "value" : "10.0.0.3"
        }
      },
      {
        "_index" : "hosts",
        "_type" : "_doc",
        "_id" : "J60PcnMBqk4TBzxZPeGU",
        "_score" : 1.0,
        "_source" : {
          "name" : "ns1.guardian.co.uk",
          "type" : "a",
          "value" : "10.0.0.2"
        }
      },
      {
        "_index" : "hosts",
        "_type" : "_doc",
        "_id" : "Ka0PcnMBqk4TBzxZPeGU",
        "_score" : 1.0,
        "_source" : {
          "name" : "test.example.com",
          "type" : "a",
          "value" : "10.0.0.4"
        }
      }
    ]
  }
}

Bulk Upload Command

curl -XPUT -u username:password https://xxxxx.us-east-1.es.amazonaws.com/_bulk --data-binary @bulk.json -H 'Content-Type: application/json'

bulk.json

{ "index" : { "_index": "hosts" } }
{"name":"ns1.guardian.co.uk","type":"a","value":"10.0.0.2"}
{ "index" : { "_index": "hosts" } }
{"name":"mail.stackoverflow.com","type":"a","value":"10.0.0.3"}
{ "index" : { "_index": "hosts" } }
{"name":"test.example.com","type":"a","value":"10.0.0.4"}

This answer might help: stackoverflow.com/questions/34887458/… — Val
– Val, Commented Jul 21, 2020 at 15:59

Bhavya · Accepted Answer · 2020-07-22 05:12:36Z

You can use the Path hierarchy tokenizer that takes a hierarchical value like a filesystem path, splits on the path separator, and emits a term for each component in the tree.

Index Mapping:

{
  "settings": {
    "analysis": {
      "analyzer": {
        "path-analyzer": {
          "type": "custom",
          "tokenizer": "path-tokenizer"
        }
      },
      "tokenizer": {
        "path-tokenizer": {
          "type": "path_hierarchy",
          "delimiter": ".",
          "reverse": "true"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "analyzer": "path-analyzer",
        "search_analyzer": "keyword"
      }
    }
  }
}

Analyze API

In the index mapping above,reverse is set to true which will emit the tokens in reverse order. (reverse is by default set to false)

POST /hosts/_analyze
{
  "analyzer": "path-analyzer",
  "text": "test.example.com"
}

This will produce three tokens:

{
"tokens": [
    {
        "token": "test.example.com",
        "start_offset": 0,
        "end_offset": 16,
        "type": "word",
        "position": 0
    },
    {
        "token": "example.com",
        "start_offset": 5,
        "end_offset": 16,
        "type": "word",
        "position": 0
    },
    {
        "token": "com",
        "start_offset": 13,
        "end_offset": 16,
        "type": "word",
        "position": 0
    }
]

}

Search Query:

    {
  "query": {
    "term": {
      "name": "example.com"
    }
  }
}

Search Result:

"hits": [
  {
    "_index": "hosts",
    "_type": "_doc",
    "_id": "d67gdHMBcF4W0YVjq8ed",
    "_score": 1.3744103,
    "_source": {
      "name": "test.example.com",
      "type": "a",
      "value": "10.0.0.4"
    }
  }
]

@A_B did you get a chance to go through my answer, looking forward to get feedback from you ?

Collectives™ on Stack Overflow

Elasticsearch Query Returning Zero Results for Substring

1 Answer 1

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Linked

Related