Skip to content

VictoriaLogs: optimize sort offset limit performance #96

@shimmer1995

Description

@shimmer1995

Is your feature request related to a problem? Please describe

#119

Describe the solution you'd like

I attempted to use VictoriaLogs and ran the Benchmark Suite to paginate log data sorted by the _time field using sort (_time desc) offset N limit M. I found that when the log data volume is quite large (over tens of millions), the query efficiency is somewhat low. The sorting operation based on the _time field took much longer than I expected. Additionally, the overall query time seems to increase almost linearly with the increase in offset or limit values. For example, when the offset or limit increases from 100 to 1000, the time taken increases from 1.5 seconds to 6 seconds. This should be a relatively common query logic, and I'm unsure if there is room for optimization.

Describe alternatives you've considered

No response

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions