30

I use spring-data-elasticsearch framework to get query result from elasticsearch server, the java code like this:

SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery()).withSearchType(SearchType.COUNT)
.addAggregation(new MinBuilder("min_createDate").field("createDate"))
.build();

List<Entity> list = template.queryForList(searchQuery, Entity.class);

While how can I know the raw http query sent to elasticssearch server? How can I enable the logging, I tried add log4j, but it seems the spring-data-elasticsearch doesn't log the query.

Improve this question
1

8 Answers 8

Reset to default
33

After digging through the spring data code i found this helpful little logger called "tracer" (name not very unique)

By setting the following in application.properties

logging.level.tracer=TRACE

It will print out a full curl statement for the request along with full JSON the response from Elasticsearch.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

it didn't work for me. KIndly, can you share you application.properties (only the part enabling spring-data-elasticsearch logs)?
worked when adding to logback-test.xml <logger name="tracer" level="TRACE" />. Using springboot 2.2.10 + spring-data-elasticsearch 3.2.10.RELEASE
Indeed, also the only solution that worked for me (spring-data-elasticsearch 4.2.4, elasticsearch java client 7.12.1)
The only solution that worked for me as well, using it for junit tests so added in the test package application.yml.
16

This one is quite old, but I'd still like to share the solution that worked for me. To log Spring Data Elasticsearch queries executed through the Repository, you need to enable DEBUG logging for the package org.springframework.data.elasticsearch.core.*, e.g. as follows:

logging:
  level:
    org:
      springframework:
        data:
          elasticsearch:
            core: DEBUG

After that, queries will appear in logs:

{
  "from" : 0,
  "size" : 1,
  "query" : {
    "bool" : {
      "should" : [ {
        "query_string" : {
          "query" : "John Doe",
          "fields" : [ "entityName" ],
          "default_operator" : "and"
        }
      }, {
        "query_string" : {
          "query" : "John Doe",
          "fields" : [ "alias" ],
          "default_operator" : "and"
        }
      } ]
    }
  },
  "post_filter" : {
    "bool" : { }
  }
}

One would expect an elegant solution similar to JPA, but it seems that it doesn't simply exist.

Tested with Spring Boot 1.4.0 and Spring Data Elasticsearch 1.7.3.

Improve this answer

5 Comments

Which version of spring data elasticsearch you are using? It's not work for me in 2.0.5.
@TanMaiVan thanks for letting know. I was using it with SD ES 1.7.3 that was going along with SB 1.4.0M1. There must be package change or logging traces change in >2.x.x.
@VladimirSalin. Do you know what I do in order to print the commands sent from Spring-data to Elasticsearch during Srpingboot startup? I applied your suggestion and I got DEBUG 23872 --- [nio-8080-exec-1] o.s.data.elasticsearch.core.QUERY : {"from":0,"size":2,"query":{"match_all":{"boost":1.0}},"version":true} when I search. So far so good. Nevertheless I created the index during Springboot startup and I want to see these commands but your solution seems not taking effect during Springboot initialization
@JimC you'd need to investigate a little bit what packages are involved into bootstrapping process, as they can differ from org.springframework.data.elasticsearch.core mentioned above. Once found, list the package in a similar way with the DEBUG level.
@VladimirSalin, how to do that for logback.xml file ? I tried org.springframework.data.elasticsearch.core.* in debug level but didn't work for .XML file configuration.
11

This works on Spring Boot 2.3.3.RELEASE

logging.level.org.springframework.data.elasticsearch.client.WIRE=trace

Improve this answer

1 Comment

Works on Spring Boot 3 as well, thanks!
10

If you are using spring boot you can set the following in your application.properties:

logging.level.org.elasticsearch.index.search.slowlog.query=INFO
spring.data.elasticsearch.properties.index.search.slowlog.threshold.query.info=1ms
Improve this answer

2 Comments

1: Is the log written to application log file? 2: How can I set this when not using spring boot but springs ElasticsearchTemplate?
spring.data.elasticsearch.properties.index.search.slowlog.threshold.query.info deprecated in boot 2.3
9

I don't have an answer for Spring Data Elasticsearch, but in ES itself you can bump up the default settings for slow query logging and see all the queries in the slow log. More details about slow log here.

As to how to change the thresholds, a command like this should be used:

PUT /_settings
{
  "index.search.slowlog.threshold.query.info": "1ms"
}

1ms is kindof the smallest value you can set.

Improve this answer

2 Comments

Update: In 2021, seems that you can set 0ms :)
Note, that it sets the threshold for the whole Elastic cluster, not only for your application. If you have a microservice mesh, like we do, it will set the logging for all microservices, which is not exactly you want to achieve.
4

I encountered the same problem, In the ElasticsearchTemplate only a few method have log debug level, E.g: 

public <T> Page<T> queryForPage(CriteriaQuery criteriaQuery, Class<T> clazz) {
    QueryBuilder elasticsearchQuery = new CriteriaQueryProcessor().createQueryFromCriteria(criteriaQuery.getCriteria());
    QueryBuilder elasticsearchFilter = new CriteriaFilterProcessor().createFilterFromCriteria(criteriaQuery.getCriteria());
    SearchRequestBuilder searchRequestBuilder = prepareSearch(criteriaQuery, clazz);

    if (elasticsearchQuery != null) {
        searchRequestBuilder.setQuery(elasticsearchQuery);
    } else {
        searchRequestBuilder.setQuery(QueryBuilders.matchAllQuery());
    }

    if (criteriaQuery.getMinScore() > 0) {
        searchRequestBuilder.setMinScore(criteriaQuery.getMinScore());
    }

    if (elasticsearchFilter != null)
        searchRequestBuilder.setPostFilter(elasticsearchFilter);
    if (logger.isDebugEnabled()) {
        logger.debug("doSearch query:\n" + searchRequestBuilder.toString());
    }

    SearchResponse response = getSearchResponse(searchRequestBuilder
            .execute());
    return resultsMapper.mapResults(response, clazz, criteriaQuery.getPageable());
}
Improve this answer

Comments

1

@2280258 is correct, and here comes the official doc:

<logger name="org.springframework.data.elasticsearch.client.WIRE" level="trace"/>

Here is the reason: in org.springframework.data.elasticsearch.client.ClientLogger, spring data elasticsearch creates a logger named "org.springframework.data.elasticsearch.client.WIRE"：

    private static final Logger WIRE_LOGGER = LoggerFactory
            .getLogger("org.springframework.data.elasticsearch.client.WIRE");
Improve this answer

Comments

0

Just to add my two cents to @AndreiStefan: Now you can set 0ms instead of 1ms. It seems that some very fast queries can be captured using this method.

Simply do:

PUT /_settings
{
  "index.search.slowlog.threshold.query.info": "0ms"
}
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.