Abdulrahman Al-lahham

We just released our new arXiv paper: “Structure and Diversity Aware Context Bubble Construction for Enterprise Retrieval Augmented Systems”. arXiv: https://lnkd.in/gJY9USty Many RAG pipelines still default to Top K. In real enterprise docs (multi sheet Excel, scopes, contracts, technical packs) that causes fragmentation, duplicate evidence, and wasted tokens. Context Bubbles treat context assembly as an explicit constrained selection problem. -> Structure aware scoring -> Diversity and redundancy control -> Strict token and per section budgets -> A full retrieval trace for auditability and deterministic tuning In our case study we cut context from 780 tokens to 214 while improving multi section coverage and reducing overlap. Guess what? There is allot more we did behind the scenes and this paper is the first in a series on building more ‘aware’ enterprise retrieval systems, where we combine structure, diversity constraints, and auditable selection with a broader memory and agent based architecture we are rolling out. The coolest thing we have built is our AI Engine which has created it's own code base and language and is taking the route of quantum inspired structures. This is already running in our ERP workflows and our medical research pipelines to reduce context bloat and improve coverage and citation faithfulness under limited windows. Keen to hear thoughts from anyone building RAG on structured enterprise data. #RAG #LLM #EnterpriseAI #InformationRetrieval #AIEngineering

5

A fundamental divide has emerged between two #AI #pioneers, Yann LeCun and Demis Hassabis, over the very definition of intelligence! The "Specialized" View (Yann LeCun) LeCun, now leading Advanced Machine Intelligence, argues that "General Intelligence" is a myth. He contends that human intelligence is actually highly specialized for our physical world, calling the concept of AGI "complete BS." To LeCun, our sense of "generality" is an illusion; he believes true progress requires moving past LLMs toward "World Models" that understand physical reality. The "Generalist" View (Demis Hassabis) Google DeepMind’s CEO, Demis Hassabis, calls this view "plain incorrect." He argues that LeCun is confusing "universal" intelligence with "general" intelligence. Hassabis maintains that the human brain is demonstrably general capable of mastering everything from survival to quantum physics and believes AGI is a tangible goal achievable within the next decade or so. I wonder if intelligence is a specialized tool for the physical world, or a general-purpose engine capable of mastering any domain? Which AI titan will prove right? The answer will define the next decade of AI development!

69

39 Comments

Google has recently launched their AlphaEvolve: a Gemini-powered coding agent for algorithm discovery. It shows clear progression from discovering algorithms for definite industry domains to developing more complex algorithms for a wide range of real-world challenges. AlphaEvolve orchestrates an autonomous pipeline of LLMs, which brings along automatic improvement of an algorithm by making direct changes to the code. It’s able to: ✅ Design faster matrix multiplication algorithms ✅ Find new solutions to open math problems ✅ Make data centers, chip design and AI training more efficient across ✅ Efficient data center scheduling and helping to improve hardware design Read a summary about it at https://lnkd.in/ga2jic2A Read Complete Research paper : https://lnkd.in/gr3CYrVn #LLMs #chatGPT #GENAI #gemini #gcp

47

1 Comment

Last week, I was at the #HintonLectures in Toronto, listening to #GeoffreyHinton and other experts talk about AI safety, alignment, and what’s coming next. Two days later, Anthropic revealed that a state-linked group used an AI model to carry out the majority of a cyber-espionage attack autonomously. AI is moving faster than our safety frameworks, and we need to catch up. Hinton’s point was simple: systems don’t need to be programmed with “bad intentions” to develop power-seeking behaviour. They discover it because it helps them achieve their goals. The Anthropic incident was a pretty good demonstration of that. The people closest to #AIsafety aren’t giving up on AI. They’re pushing for safer systems and better guardrails. The rest of us have a role too. We can ask questions, support better policy, and stay engaged. AI isn't slowing down, but with real leadership, we can influence the direction it takes. In this week's newsletter, I write about where this leaves us and what we can push for to protect ourselves from bad actors using AI. Link is in the comments.

12

5 Comments

Toronto startup Taalas launched the HC1 chip on Feb 19, hard-wiring Llama 3.1 8B into a single silicon die delivering 17,000+ tokens/sec per user. It reportedly outperforms Cerebras and Nvidia by wide margins (~8x), uses ~10× less power, and runs in a 200W air-cooled server. Founded in 2023 by Ljubisa Bajic, Taalas has raised $219M and is targeting applications from appliances to medical devices. Impressive speed. Another question is how easily the chip can be customized for new models, given the rapid iteration of model architectures. https://lnkd.in/ghVfZXde

17

RAG was phase one.The next competitive layer may be observational memory. New evidence suggests memory-augmented agents can reduce costs by up to 10x and outperform classic RAG on long-horizon tasks. This is not a marginal improvement. It’s an architectural shift. Enterprise AI will not be defined by larger models, but by how we design memory, continuity, and evaluation at system level. Once agents retain structured memory, evaluation shifts from accuracy per prompt to stability and alignment over time. The question is no longer “Which model?” It is “What cognitive architecture are we building?” #AgenticAI #AISystems #AIArchitecture #EnterpriseAI #Innovation https://lnkd.in/ebSGNp5g

57

5 Comments

17 000 tokens/sec! Canadian hardware startup Taalas has unveiled a fundamentally different approach to AI inference: LLMs baked directly into silicon. 1 chip = 1 model. Instead of running models as software on GPUs or TPUs, Taalas “hard-wires” an entire model into an ASIC turning weights and compute into custom hardware. First version runs Llama 3.1 8B with LoRA-based fine-tuning support. 10X faster than the current state of the art, while costing 20X less to build, and consuming 10X less power. From model to deployable ASIC in ~2 months. You can test it live at https://chatjimmy.ai/, the speed of response is insane! Source: Igor Urmincek

22

2 Comments

NVIDIA just pulled off what many thought wasn’t possible. They pre-trained a 12B-parameter LLM on 10 trillion tokens using only 4-bit precision. Why it matters: 2–3× higher math throughput and ~50% lower memory vs FP8 Accuracy holds: MMLU-Pro — FP8 62.62% vs NVFP4 62.58% Stable training via Random Hadamard transforms, stochastic rounding, and 2D scaling End-to-end on NVIDIA Blackwell GPUs—the first 4-bit run to stay stable across 10T tokens This is the first large-scale 4-bit pretraining demo without accuracy loss. The next wave of frontier models will be faster and cheaper —with no obvious trade-offs. #AI #DeepLearning #LLM #NVIDIA #Blackwell #ModelTraining #MLOps #Efficiency https://lnkd.in/dFVkx5pc

12

REFRAG ( Representation for RAG) represents a fundamental shift for enterprise RAG systems.A Smarter Way to Scale RAG Using RL-Guided Chunk Compression: ➡️ Not “bigger context windows,” but smarter context routing ➡️ Not “retrieve everything” from sources as part of Retrieval step but retrieve and compress strategically before passing to Generation (decoder) step . Everyone in the LLM ecosystem is chasing longer context windows — but REFRAG offers a more elegant, scalable alternative. Here’s the breakthrough: 1️⃣ Compression Happens BEFORE passing to LLM Decoding For each retrieved chunk as part of retrieval step in RAG pipeline : • The encoder creates a semantic embedding • A trained RL policy decides: Compress this chunk → replace with embedding vector? Or keep it fully tokenized? This is the only place RL is applied. 2️⃣ What the Decoder Actually Sees The decoder receives a hybrid input: • Real tokens (for important chunks) • Single embedding vectors (for compressed chunks) Crucially: • ❌ The decoder does not decompress • ❌ There is no expansion of embeddings back to text • ✔ The decoder is pretrained to interpret compressed vectors as dense summarizations 3️⃣ Why It Works Because RL optimizes: • Which chunks matter • How many tokens to save • How to maximize task accuracy The result: • ⚡ Faster inference • 💡 Better use of limited context • 💰 Lower cost per query • 📈 Better relevance and grounding As agentic AI and on-device intelligence grow, techniques like this will be essential to scale GenAI without bloating context windows.

19

🔬 Fine-Tuning for Better Few-Shot Prompting: What You Need to Know 🤖📚✨ Researchers at the University of Southern California – Institute for Creative Technologies have shared exciting results on fine-tuning large language models (LLMs) for automated short answer grading 📝💡 ✨ Key Finding Fine-tuning LLMs with small datasets can significantly boost performance ⚡ For example, GPT-4o-mini fine-tuned on just 150 examples improved F1 score from 0.68 → 0.73 🚀 🎯 Why This Matters Enables reliable AI-driven assessment in education 🎓 Schools and edtech companies can implement automated grading without massive datasets or GPUs 💻 ⚙️ How it Works Methods: OpenAI fine-tuning & QLoRA for LLama3.1 8B-Instruct 🛠️ Evaluated on 148 labeled short-answer examples, extended to 17,820 prompts using N-shot strategies 📊 Key insights: GPT-4o-mini improved significantly ✅ QLoRA LLama3.1 saw limited gains, but synthetic data from Google Gemini 1.5 Flash boosted performance 🌟 💡 Practical Takeaway Small, high-quality datasets + the right fine-tuning method = better task-specific performance 🎯 Crucial for reliable AI-driven grading ⚡ ➕🔔 Follow me for more AI research updates & insights! 🔍📈 Woongsik Dr. Su, MBA #AIResearch #FineTuning #LLMs #AutomatedGrading #EducationTechnology #ArtificialIntelligence #MachineLearning #NLP #EdTech #Innovation #FutureOfEducation

9

The Overhead of static_cast Last time (https://lnkd.in/gMQc6_C5), we saw that to cast between two types, we should not use a C-style cast. Instead, C++ provides us with static_cast, dynamic_cast, and others to achieve what we want to do correctly. You will commonly hear people say that dynamic_cast has runtime overhead, whereas static_cast does not. But that is not entirely true. While the overhead of static_cast is much less than that of a dynamic_cast, a static_cast can also carry some overhead. In certain cases, it has to perform a nullptr check at runtime. Let us look at an example: ``` class A { int a; }; class B { int b; }; class C : public A, public B { int c; }; void foo(B& obj); int main() { C c{}; C* cptr = rand() ? &c : nullptr; B* bptr = static_cast<B*>(cptr); } ``` Why would static_cast need a nullptr check? The memory layout of C (under typical implementations) is such that: > The A subobject is located at address p > The B subobject is located at address p + offset That offset depends on layout and alignment, but conceptually, the pointer must be adjusted when converting from C* to B* in multiple inheritance. So when converting a C* to a B*, the compiler must adjust the pointer. But what if we pass nullptr? The C++ standard guarantees that casting a null pointer results in a null pointer. If the compiler simply added the offset blindly, it would compute something like: nullptr + offset which would not be nullptr, and would be incorrect. So the generated code must ensure that if the original pointer is nullptr, the result is also nullptr. We can verify this using Compiler Explorer (https://lnkd.in/gcbiZqz8). The assembly looks like this: .L3: cmp QWORD PTR [rbp-8], 0 ; Check cptr for nullptr je .L4 ; If nullptr, jump to .L4 mov rax, QWORD PTR [rbp-8] ; rax = cptr add rax, 4 ; rax = cptr + offset (adjust to B) mov QWORD PTR [rbp-16], rax ; bptr = adjusted pointer jmp .L5 .L4: mov QWORD PTR [rbp-16], 0 ; set bptr to nullptr .L5: The useful work is done in: mov rax, QWORD PTR [rbp-8] add rax, 4 mov QWORD PTR [rbp-16], rax The remaining instructions handle the nullptr case.

29

1 Comment

AI is no longer just assisting humans in financial services — it’s beginning to act on behalf of them. In this new Tearsheet interview, Kevin Levitt from NVIDIA shares how firms like Capital One, Visa, and RBC are deploying agentic AI systems powered by NVIDIA infrastructure to make real-time financial decisions — from payments and lending to research and risk management. Read more: https://bit.ly/4qWpzm9 Watch the video: https://bit.ly/47TrVJM Listen to the podcast: https://bit.ly/4r29FHa

2

The key statement right there ❤️ “Building evaluators (basically tests) is, I believe, going to become the focus of systems and software over the next decade.”

1

Gemini 3 Pro Added to Database AI Suite. Tested Against Claude Sonnet 4.5 and GPT-5.1. Results: Claude still leads. GPT-5.1 is solid. Gemini 3 Pro lands third. ▪️ Performance Scores Multi-step database analysis workflows: Claude Sonnet 4.5: 115 GPT-5.1: 100 Gemini 3 Pro: 90 90-tier: Gemini 2.5 Pro, GPT-4.1, KIMI 2 Thinking 85-tier: Gemini 2.5 Flash, Qwen 3 Max, GLM 4.6, DeepSeek R1 80-tier: Gemini 2.0 Flash ▪️ Model Findings - Claude Sonnet 4.5: Creates multiple segmentation variables beyond the prompt. Quality leader. - GPT-5.1: Strong. Token bloat gone vs GPT-5. Clear second. - Gemini 3 Pro: Similar to Gemini 2.5 Pro. Better explanations, same output quality. Doesn't match top two. ▪️ What Was Tested Credit card analysis: 1M customer + 10M transaction tables (AWS RDS MySQL). Multi-step workflow - plan, summarize, create derived variables, merge, segment profiles. Weighted ranking of Indian credit card issuers from RBI data (Postgres). ▪️ Practical Choices High quality → Claude Sonnet 4.5 Balance → GPT-4.1 Low cost- great value→ Gemini 2.5 / 2.0 Flash ▪️ Cost Breakdown -- Planning vs Execution Single iteration: reasoning LLM runs once (20% cost), execution agent runs 7-10 queries with debugging (80% cost). -- Reasoning / Planning Cost (Per 100 Questions) High: Claude, GPT-5.1, Gemini 3 Pro (~$7.00) Mid: GPT-4.1, KIMI 2, Qwen, GLM, DeepSeek (~$2.50) Budget: Gemini 2.5 Flash (~$1.50), Gemini 2.0 Flash ($0.30) Avoid: Gemini 2.5 Pro ($10) and GPT-5 ($15) - token bloat. -- Execution Cost (Per 100 Questions) - Advanced analysis: ~$12.50 (GPT-4.1, consistent across reasoning models) - Single-step: $0.40 (GPT-4.1-mini) to ~$3.50 (GPT-5.1) I use OpenAI for SQL execution - more reliable. Multi-step workflows multiply costs fast. Use only when needed. These costs relate to my typical uses. Your numbers will vary based on context, architecture & output volume. Always test with your use case. Always check actual billing, not token based estimates. ▪️ DATS-4: Database AI Suite- v4 Remote database AI app. Postgres or MySQL. Single-step queries or multi-step analysis. Python charts, table upload, PDF reports. Open source, live, free. -- Try It https://lnkd.in/gxY2PJqZ → DATS-4 Use Sample button. Data loads to temp Postgres. Or connect your database. Datasets: https://lnkd.in/geHAn2uf Public app routes through my backend - sandbox only. Deploy on your servers for live work ▪️ Resources Database AI Field guide- usage, architecture, process flows https://lnkd.in/gxY2PJqZ Previous post: GPT-5.1 + KIMI 2 evaluation with short video https://lnkd.in/gzkeBQrf Live apps: app.tigzig.com Posts & guides - AI for Analytics: tigzig.com

9

2 Comments

𝗘𝘅𝗽𝗹𝗮𝗶𝗻𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝘄𝗼𝗿𝗸𝘀 — 𝘄𝗶𝘁𝗵𝗶𝗻 𝗹𝗶𝗺𝗶𝘁𝘀 Over the past decade, post-hoc explainability has become a widely adopted practice: Feature importance scores SHAP values Local surrogate models They provide insight into how predictions are generated. Yet they can: • Shift under small model perturbations • Appear stable even when models change materially • Be manipulated without altering predictive performance • Capture statistical associations rather than causal structure This does not invalidate interpretability tools. It clarifies what they are and what they are not. Most explanation tooling operates downstream of inference. Governance challenges emerge upstream, in how systems are deployed, updated, and assigned clear ownership If accountability depends primarily on dashboards, it becomes difficult to sustain over time. Part 3 will outline the architectural design choices that make AI systems defensible, not just explainable.

15