Shay Banon

@kimchy

the dude abides @elastic

thedudeabides.com

Joined June 2007

Shay Banon
@kimchy
Jul 9
happy, following up on all the work we did for tsdb/metrics in ES, we are formalizing our learnings to a more generic columnar mode in ES, on an index level. Personally, still remember us adding doc values (columnar) to Lucene so many years ago (2012!), and so much since
Elastic
@elastic
Jul 9
We're introducing Elasticsearch Columnar Mode: A new index mode that stores data once, in columnar form, with no redundant copies and no indexes the workload doesn't need. Not replacing the document model. Adding a second way to organise data alongside it, for the workloads
Shay Banon
@kimchy
May 19
When turboquant came out, it got a lot of justified excitement. We took it for a Spin, and our year long work on Elasticsearch BBQ (better binary quantization) holds up better, especially in the abundant world of CPUs. * also, we need to get Thom an upgrade from M2 ;)
Elastic
@elastic
May 14
🧵 Our BBQ at 1-bit/doc beats TurboQuant at 4-bit/doc on shifted data on ranking accuracy. At 1/5 the storage. We center on the segment centroid before quantization, so the bits go where they are actually needed for ranking. TurboQuant's Hadamard rotation can't exploit that
Shay Banon
@kimchy
May 8
Impressive work by our team making Elasticsearch a kick ass columnar metrics store
Elastic
@elastic
May 8
30x faster than Prometheus on gauge averages sounds like a typo. It's not. Elasticsearch 9.4 rebuilt TSDS and ES|QL into a fully columnar metrics engine. Vectorized time series aggregations, zero-copy decoding, doc value skippers replacing inverted indices. The result at 1.4M
Shay Banon
@kimchy
Apr 23
our journey with making Elasticsearch one of the best vector databases in the world continues, impressive and innovative work by our team
Elastic Dev
@elastic_devs
Apr 23
95 ns/vector vs 412 ns, Elasticsearch simdvec VS jvector scoring float32 at 32,500 vectors on x86. But the number isn't really the story here. The story is that at scale, past L3 cache, memory latency beats compute every time and simdvec is built just for that. At this depth,
Shay Banon
@kimchy
Apr 16
Native promql support for metrics in Elasticsearch, fused right into our ES|QL so you can optionally fuse it. Pretty cool and tons of more metrics news (perf! Moar columnar!) coming soon!
Elastic Dev
@elastic_devs
Apr 16
If your team lives in PromQL, you no longer have to context-switch to query your Elasticsearch metrics. The new PROMQL source command in ES|QL lets you run PromQL queries directly against Elasticsearch time series data, whether ingested from Prometheus Remote Write,