Skip to content

ScrollHelper makes additional requests per hit instead of per batch #2556

Closed
@dbackeus

Description

@dbackeus

Describe the bug/error/problem

When iterating through hits using ScrollHelper, an additional search (with scroll) request is made to ElasticSearch for each resulting hit:
https://github.com/elastic/elasticsearch-ruby/blob/main/elasticsearch/lib/elasticsearch/helpers/scroll_helper.rb#L50

This leads to massive latency overhead and makes the ScrollHelper pretty much unusable.

Expected behavior

Additional search (with scroll) requests should be made in only after completing iteration of all current elements in the @docs array, to see if there are more hits to scroll to.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions