Pushdown for LIKE (LIST)

[julian-elastic]

Improved performance of LIKE (LIST) by a factor of 5 by pushing an Automaton to do the evaluation down to Lucine

Before:
{"took":629,"documents_found":10000000,"values":[[26419752]]}
After:
{"took":122,"documents_found":2222222,"values":[[26419752]]}

Measured by this script, look at test like list with and without this change

passwd="redacted"
curl -sk -uelastic:$passwd -HContent-Type:application/json -XDELETE http://localhost:9200/test
curl -sk -uelastic:$passwd -HContent-Type:application/json -XPUT http://localhost:9200/test -d'{
    "settings": {
        "index.refresh_interval": -1
    },
    "mappings": {
        "properties": {
            "f": {
                "type": "keyword"
            }
        }
    }
}'
counter=1
for a in {1..10000}; do
    rm -f /tmp/bulk*
    for b in {1..1000}; do
        echo '{"index":{}}' >> /tmp/bulk
        printf '{"f":"text %d"}\n' "$counter" >> /tmp/bulk
        counter=$((counter + 1))
    done
    #ls -l /tmp/bulk*
    printf %05d: $a
    curl -sk -uelastic:$passwd -HContent-Type:application/json -XPOST http://localhost:9200/test/_bulk?pretty --data-binary @/tmp/bulk | grep errors
done
curl -sk -uelastic:$passwd -HContent-Type:application/json -XPOST http://localhost:9200/test/_forcemerge?max_num_segments=1
curl -sk -uelastic:$passwd -HContent-Type:application/json -XPOST http://localhost:9200/test/_refresh
echo
curl -sk -uelastic:$passwd http://localhost:9200/_cat/indices?v

test_like() {
    echo -n "test like"
    curl -sk -uelastic:$passwd -HContent-Type:application/json -XPOST 'http://localhost:9200/_query?pretty' -d'{
        "query": "FROM test | WHERE f like \"text 2*\" or f like \"text 3*\" or f like \"text 224*\" or f like \"text 225*\" or f like \"text 226*\"| STATS SUM(LENGTH(f))",
        "pragma": {
            "data_partitioning": "shard"
        }
    }' | jq -c '{took, documents_found, values}'
}

test_like_list() {
    echo -n "test like list"
    curl -sk -uelastic:$passwd -HContent-Type:application/json -XPOST 'http://localhost:9200/_query?pretty' -d'{
        "query": "FROM test | WHERE f like (\"text 2*\", \"text 3*\", \"text 224*\", \"text 225*\", \"text 226*\") | STATS SUM(LENGTH(f))",
        "pragma": {
            "data_partitioning": "shard"
        }
    }' | jq -c '{took, documents_found, values}'
}


for a in {1..100}; do
    test_like
    test_like_list
done

[elasticsearchmachine]

Hi @julian-elastic, I've created a changelog YAML for you.

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	8.19	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 129557

[julian-elastic]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Questions ?

Please refer to the Backport tool documentation