spring-data-elasticsearch - registering custom analyser

Question

I'm trying to use ElasticSearch in my application for full text search and at this time I'm trying use autocomplete analyser:

{
    "settings": {
        "number_of_shards": 1,
        "analysis": {
            "filter": {
                "autocomplete_filter": {
                    "type": "edge_ngram",
                    "min_gram": 1,
                    "max_gram": 20
                }
            },
            "analyzer": {
                "autocomplete": {
                    "type": "custom",
                    "tokenizer": "standard",
                    "filter": [
                        "lowercase",
                        "autocomplete_filter"
                    ]
                }
            }
        }
    }
}

As my application was constructed with Spring, I've decided use Spring-data-elasticsearch and mapped my entity this way:

@Document(indexName = "estabelecimento")
@Setting(settingPath = "/elasticsearch/autocomplete-analyser.json")
public class ESEstabelecimento {

    private Long id;
    @Field(type = FieldType.String, indexAnalyzer = "autocomplete")
    private String nome;
    private String razaoSocial;
    private String tipoEstabelecimento;
    @Field(type = FieldType.Object)
    private ESCidade cidade;
}

However, elasticsearch isn't loading the custom analyser:

[DEBUG] org.elasticsearch.action.admin.indices.mapping.put - [Magus] failed to put mappings on indices [[estabelecimento]], type [esestabelecimento] org.elasticsearch.index.mapper.MapperParsingException: Analyzer [autocomplete] not found for field [nome] at org.elasticsearch.index.mapper.core.TypeParsers.parseField(TypeParsers.java:220) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.index.mapper.core.StringFieldMapper$TypeParser.parse(StringFieldMapper.java:153) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:290) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:214) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:136) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:211) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:192) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:434) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.cluster.metadata.MetaDataMappingService$4.execute(MetaDataMappingService.java:505) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:365) ~[elasticsearch-1.5.2.jar:na] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:188) [elasticsearch-1.5.2.jar:na] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:158) [elasticsearch-1.5.2.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_77] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_77] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77] [ERROR] org.springframework.data.elasticsearch.repository.support.AbstractElasticsearchRepository - failed to load elasticsearch nodes : org.elasticsearch.index.mapper.MapperParsingException: Analyzer [autocomplete] not found for field [nome]

I can't identify the cause of the problem, I don't know if the analyser is invalid or elasticsearch even found the autocomplete-analyser.json file. How can I solve this?

Did you remove the index between trying? It will only create the index if it was not there yet. If the index was already there it will not update the settings. First remove the index and let spring recreate it. You can also check the settings using GET /estabelecimento/_settings, maybe that gives some insight. — Jettro Coenradie
– Jettro Coenradie, Commented Mar 27, 2016 at 11:39

Richa · Accepted Answer · 2016-03-26 18:47:29Z

7

I guess you need to remove the field "settings": from json file and put only the content .

Instead of using :

{
"settings": {
    ....
   }
}

Just use :

{ 
"index": {
"number_of_shards": 1,
"analysis": {
  "filter": {
    "autocomplete_filter": {
      "type": "edge_ngram",
      "min_gram": 1,
      "max_gram": 20
    }
  },
  "analyzer": {
    "autocomplete": {
      "type": "custom",
      "tokenizer": "standard",
      "filter": [
        "lowercase",
        "autocomplete_filter"
      ]
     }
    }
   } 
  }
}

Hope it helps you..

edited Mar 26, 2016 at 18:47

answered Mar 26, 2016 at 18:34

Richa

7,6796 gold badges30 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

brevleq Over a year ago

same result... Is there a way I can debug @Setting? I think the problem could be there...

Richa Over a year ago

Just to confirm,Is your json file located at the same location as mentioned in settingPath ?

brevleq Over a year ago

yes. the file is in src/main/resources/elasticsearch/autocomplete-analyser.json. I've tried with complete path: /home/hudson/... but the result is the same

Richa Over a year ago

Remove the analyzer part and see how many shards are being created. If it is 1 which you mentioned in settings then we will be sure that your json file is being read and there is some problem in analyzer. If number of shards created is 5 (default) ,then this will infer that your json file is not being read.

brevleq Over a year ago

It is not reading the file: [DEBUG] org.elasticsearch.indices.memory - [Nightside] recalculating shard indexing buffer (reason=[[ADDED]]), total is [354.3mb] with [5] active shards, each shard set to indexing=[70.8mb], translog=[64kb]

|

brevleq · Accepted Answer · 2016-03-28 21:28:43Z

5

Solved

After some research I've discovered the problem was caused by this declaration:

@Bean
public ElasticsearchOperations elasticsearchTemplate(Client client) {
    return new ElasticsearchTemplate(client, new CustomEntityMapper());
}

I removed this declaration and it began get the configuration.

answered Mar 28, 2016 at 21:28

brevleq

1,94310 gold badges55 silver badges101 bronze badges

Comments

Vinod · Accepted Answer · 2020-07-27 09:49:02Z

I had a similar issue and this can occur due to multiple reasons:

Make sure that your json file located at the same location as mentioned in the settingPath and its being getting read properly. To verify this as Richa said in the above just remove the analyzer from the json and test the number of shards created.
In case the location is correct but still json file is not read then just check did you add @Document(indexName = "indexName", createIndex = false) annotation in the model class then simply remove the createIndex = false from document annotation and then try again.
Always remember to remove the old index and then test it again.

Collectives™ on Stack Overflow

spring-data-elasticsearch - registering custom analyser

3 Answers 3

9 Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

9 Comments

Comments

Comments

Related