Badal G.

Quantization in Large Language Models (LLMs) As LLMs scale, intelligence is no longer the bottleneck — memory, latency, and cost are. Quantization turns out to be one of the most powerful (and misunderstood) techniques enabling real-world deployment of large models. In this blog, I break down: • Why LLMs are surprisingly quantization-friendly • INT8 vs INT4 vs NF4 — what actually works in practice • How modern architectures absorb low-bit noise • Where quantization fails and how to design around it Quantization is no longer an optimization step — it’s foundational infrastructure for production LLM systems. Link: https://lnkd.in/gqGje75G #LLMs #AIEngineering #Quantization #MachineLearning #GenAI #MLOps

9

2 Comments

Sarvam-M: A 24B Parameter Multilingual AI Model is finetuned Mistral-Small with Indic language desi nuances covering 11 major languages from india 🇮🇳 Key Features: • Dual-mode interface: Quick "non-think" responses + detailed "think" mode for complex reasoning 🧠 • Strong performance on math (GSM-8K) and coding (SWE-Bench) benchmarks 📊 • Supports both native Indic scripts and romanized text ✍️ • Built for real-world multilingual conversational agents 💬 I think it's a good start , instead of training model from scratch Sarvam focused on building indic dataset and model like in video shows good translingual conversational capability . Try it out via the Colab link - https://lnkd.in/gzcW4hru Playground - https://lnkd.in/gXijZrkY

27

1 Comment

Few of the agentic AI implementations I am doing currently has some of the following learnings : 1. Agentic AI implementation is driven by a use case 2. while executing such implementation complexity resides in understanding the product and its UI/UX in terms of what fields and attributes we have to ask the agents and LLMs to populated automatically which ones will be the input to the agents 3. For the fields which are input to the agents we need the RAG to be created which is vector DB in this case pinecone from AWS and for each attribute you need descriptions to be put into the vector DB so that genAI can pick right understanding of the prompted texts 4. One agent not sufficient to do UI/UX integration and automation but minimum 5 agents master agent node agent build agent validator agent aggregator agent in some case we may have to device a listener agent 5. Use of crew.ai is good for agents creation is preferred over langchain due to better features 6. Claude sonnet is used as LLM 7. I have driven this project with lots of design documents which include scope screen designs, Agents mapping design and multi test case documents 8. On AWS we need to think about deployment options and regions. For claude sonnet we need to consider wider regions so that models gets exposed to larger public data and gives better responses This much a good write up for this saturday. See u next week.

18

3 Comments

For ingesting events (json) into Snowflake SNS (prefer non-fifo to fifo) -> data firehose -> snowflake can't be beat. It might be overkill for small volumes but for the larger ones it just eats it up AND the best part, the killer feature: no issues, no downtime, no outages, no maintenance needed once put into production. It's as close to the perfect data ingestion tool that there is. There you go. Enjoy that factoid. #aws #snowflake #streaming

3

One of the biggest trade-offs in production RAG systems is retrieval quality vs latency. You can improve results by adding: • reranking • larger candidate sets • richer context But every additional step adds latency. A 2–3 second response may be fine for research assistants. But it can break the experience in customer-facing applications. The best AI architectures are not the most sophisticated ones. They’re the ones that fit within the latency budget of the product and the domain. #AIEngineering #GenAI #RAG #MachineLearning #ArtificialIntelligence

12

3 Comments

Having worked with a wide variety of LLMs, this is a great roadmap to follow for anyone who wants to truly learn NLP

9

TRANSITION MATRIX to find errors in Agent :- When you have an agent with many steps (e.g., 'Plan', 'Search', 'Code', 'Finalize'), it's hard to know which step is failing most. A Transition Failure Matrix helps you find the hotspots. How it works: 1.Define states: List all the possible steps or 'states' your agent can be in. 2.Create a matrix: Create a grid where the rows are the 'From' state and the columns are the 'To' state. 3.Count failures: For each failure, identify the last successful transition that happened before the error. Populate the count of failures in the grid cells accordingly. #AI #DataScience

5

Over the years working with large datasets, I’ve realized something simple: 👉 Most performance issues don’t start with queries, they start with design. That’s why I’m starting a new series on SQL Optimization & Scalability — sharing practical lessons on schema design, indexing, partitioning, and more. Here’s 💡 Part 1: Faster SQL Starts with Smarter Schema Design & Indexing. While working in today’s evolved data industry, I’ve realized one thing very clearly: 👉 A well-designed schema solves 90–95% of business problems before we even start writing complex queries. Why? Because a robust schema and efficient database design can make or break how quickly we can turn raw data into insights — whether for day-to-day reporting, powering ML models, or feeding dashboards. At first, schema design feels simple when you’re only handling a few thousand rows. But as data grows exponentially, Data Definition Language (DDL) decisions suddenly become critical. The right schema design = faster query retrieval. Efficient DDL = smoother Data Manipulation Language (DML) operations at scale. Together, they set the foundation for every downstream use case — from analytics to AI. This series will explore how schema design, indexing, partitioning, and other optimizations unlock query speed and scalability. 💡 My takeaway so far: Schema is not just a database step — it’s a business impact multiplier. Indexing: The Shortcut That Makes SQL Fly When dealing with millions of rows, scanning data sequentially (O(N)) quickly becomes a bottleneck. That’s where indexing comes in — reducing lookups to O(logN) by letting SQL jump directly to the relevant data. Indexes build sorted, searchable structures (like B-Trees, Bitmaps, Columnstores), acting as smart catalogs that avoid expensive full table scans. 🔹 Clustered Index Defines the physical row order in a sorted manner. 👉 Think of it like a diary arranged strictly by date. If you want to find March entries, you flip straight to March — without reading Jan and Feb. CREATE CLUSTERED INDEX cix_fact_transact_date on dbo.fact_transactions(transaction_date). And QUERY SELECT * from dbo.fact_transactions WHERE transaction_date between BETWEEN '2024-01-01' AND '2024-01-31';   🔹 Non-Clustered Index Creates shortcuts for specific query patterns (great for filters like city + date). 👉 It works like a table of contents — pointing directly to the page you need. CREATE NON-CLUSTERED INDEX nix_fact_city_date on dbo. fact_transactions(city_id,transaction_date) INCLUDE (price); Query SELECT transaction_date, price FROM dbo.fact_transactions WHERE city_id = 1001 AND transaction_date >= '2024-01-01' 💡 Takeaway: 🔹 Indexes aren’t just “performance tweaks.” They’re query accelerators that transform how databases scale. 🔹 Use clustered indexes for natural order. 🔹 Use non-clustered indexes for targeted lookups. #SQL #QueryOptimization #Database #DataScience #DataEngineering

8

🌾 𝐈𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐢𝐧𝐠 𝐒𝐚𝐦𝐚𝐫𝐭𝐡 — 𝐈𝐧𝐝𝐢𝐚’𝐬 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐭 𝐐&𝐀 𝐒𝐲𝐬𝐭𝐞𝐦 𝐟𝐨𝐫 𝐀𝐠𝐫𝐢𝐜𝐮𝐥𝐭𝐮𝐫𝐚𝐥 & 𝐂𝐥𝐢𝐦𝐚𝐭𝐞 𝐃𝐚𝐭𝐚 “𝐖𝐞 𝐡𝐚𝐯𝐞 𝐭𝐡𝐞 𝐝𝐚𝐭𝐚. 𝐖𝐞 𝐣𝐮𝐬𝐭 𝐜𝐚𝐧’𝐭 𝐮𝐬𝐞 𝐢𝐭.” That’s what a problem statement told me about India’s agricultural datasets. Thousands of high-quality datasets. Decades of rainfall records. Detailed crop production stats. All public. All free. Yet answering a simple, cross-domain question still takes days of manual Excel work. That frustration led me to build something. 🌿 𝐌𝐞𝐞𝐭 𝐒𝐚𝐦𝐚𝐫𝐭𝐡 (सामर्थ्य = Capability) An AI system that makes India’s open government data actually useful and queryable in plain English. Ask ➜ “𝐂𝐨𝐦𝐩𝐚𝐫𝐞 𝐭𝐡𝐞 𝐚𝐯𝐞𝐫𝐚𝐠𝐞 𝐚𝐧𝐧𝐮𝐚𝐥 𝐫𝐚𝐢𝐧𝐟𝐚𝐥𝐥 𝐢𝐧 𝐌𝐚𝐡𝐚𝐫𝐚𝐬𝐡𝐭𝐫𝐚 𝐚𝐧𝐝 𝐆𝐮𝐣𝐚𝐫𝐚𝐭 𝐟𝐨𝐫 𝐭𝐡𝐞 𝐥𝐚𝐬𝐭 𝟏𝟎 𝐲𝐞𝐚𝐫𝐬 𝐚𝐧𝐝 𝐥𝐢𝐬𝐭 𝐭𝐡𝐞 𝐭𝐨𝐩 𝟑 𝐜𝐫𝐨𝐩𝐬 𝐛𝐲 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐢𝐧 𝐛𝐨𝐭𝐡 𝐬𝐭𝐚𝐭𝐞𝐬.” Get back ➜ 📊 Structured insights 📈 Temporal trends 🔗 Exact dataset citations — all within 5 seconds. 💡 𝐖𝐡𝐲 𝐢𝐭 𝐦𝐚𝐭𝐭𝐞𝐫𝐬 → Policymakers can make faster, data-backed decisions → Researchers spend less time cleaning, more time analyzing → Farmers (via extension workers) get evidence-based recommendations Because good policy deserves good data. ⚙️ 𝐔𝐧𝐝𝐞𝐫 𝐭𝐡𝐞 𝐇𝐨𝐨𝐝 — 𝐓𝐡𝐞 𝐓𝐞𝐜𝐡 𝐁𝐞𝐡𝐢𝐧𝐝 𝐒𝐚𝐦𝐚𝐫𝐭𝐡 🧩 Stack: Flask • LangChain • OpenAI (GPT-3.5-turbo) • FAISS • BM25 • Cross-Encoder Reranking • SQLite • Docker 🧠 Approach: A 5-stage Retrieval-Augmented Generation (RAG) Pipeline built for Indian government data 1️⃣ 𝐐𝐮𝐞𝐫𝐲 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐦𝐞𝐧𝐭 → HyDE + entity extraction expand natural language queries 2️⃣ 𝐇𝐲𝐛𝐫𝐢𝐝 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 → FAISS (dense) + BM25 (sparse) with Reciprocal Rank Fusion 3️⃣ 𝐑𝐞𝐫𝐚𝐧𝐤𝐢𝐧𝐠 → Cross-Encoder + MMR ensures relevance & diversity 4️⃣ 𝐂𝐨𝐧𝐭𝐞𝐱𝐭 𝐂𝐨𝐦𝐩𝐫𝐞𝐬𝐬𝐢𝐨𝐧 → Dedupes and extracts only key evidence snippets 5️⃣ 𝐀𝐧𝐬𝐰𝐞𝐫 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 → GPT synthesizes cited, factual responses — no hallucination 📈 𝐑𝐞𝐬𝐮𝐥𝐭𝐬: ✅ 88% factual accuracy ✅ 98% citation correctness ✅<6s average response time 🧩 𝐊𝐞𝐲 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠𝐬 → The hardest part isn’t the AI — it’s handling messy, inconsistent public data. → Indian datasets often have missing values, mixed units, and inconsistent naming. The best AI is invisible — it just works quietly behind the scenes. 🌍 𝐎𝐩𝐞𝐧 𝐒𝐨𝐮𝐫𝐜𝐞. 𝐅𝐫𝐞𝐞 𝐭𝐨 𝐮𝐬𝐞. 🚀 Experience it live: https://lnkd.in/dJi4yfyx 💻 Source code: https://lnkd.in/dVgScGtZ I’d love to hear your feedback, thoughts, and ideas! Let’s keep pushing the boundaries of what AI can achieve. #ImpactTech #AI4Good #OpenData #PolicyTech #LangChain #MachineLearning #InnovationForBharat #FAISS #RAG #DataForGood #SocialImpact #AI #Innovation #India #buildforbharat

19

2 Comments

🚀 Partitioning vs Clustering in Data Engineering — Made Simple In large datasets, smart data organization can turn slow queries into lightning-fast results. Two common techniques: Partitioning and Clustering. 1️⃣ Partitioning – “Separate folders” We split a table into chunks based on a column value. Example: 📂 /sales/country=USA/... 📂 /sales/country=UK/... Query: SELECT * FROM sales WHERE country = 'USA'; ✅ Only scans the USA folder — skips all UK data. Best for: Columns with few unique values (e.g., Country, Year). 2️⃣ Clustering – “Organized shelves” We keep the data in the same file but group similar rows together. ✅ Without Clustering The data might be stored unsorted, like: file1.parquet → (2024-01-03, USA), (2024-01-01, USA), (2024-01-02, UK)... file2.parquet → (2024-01-02, USA), (2024-01-01, UK), (2024-01-03, UK)... ✅ With Clustering by order_date The data inside each file is grouped and sorted by order_date: file1.parquet → 2024-01-01, USA, 100 2024-01-01, USA, 150 2024-01-02, UK, 200 2024-01-02, USA, 180 file2.parquet → 2024-01-03, UK, 220 2024-01-03, USA, 300 ✅ Queries can skip irrelevant ranges inside the file. Best for: Columns with many unique values (e.g., Order Date, User ID). 3️⃣ Together = Maximum Efficiency In BigQuery or Snowflake, you can do: PARTITION BY DATE(order_date) CLUSTER BY country This way: Partitioning skips entire days. Clustering skips irrelevant countries inside each day. 💡 Rule of Thumb: Partition for broad pruning (skip whole files) Cluster for fine-grained pruning (skip parts of files) 🔍 How do you use partitioning and clustering in your data projects? #DataEngineering #BigQuery #Snowflake #Partitioning #Clustering #SQLPerformance #DataModeling

15

🚀 Just Launched: Amazon Review Explorer – An end-to-end NLP-powered Streamlit web app to analyze Amazon product reviews through topic modeling, sentiment analysis, and dynamic image integration. 🔍 What I Built: 📌 Web Scraping & Data Collection → Collected live product reviews and metadata directly from Amazon using Selenium and BeautifulSoup. 📌 Topic Modeling → Cleaned and processed review text using standard NLP techniques. → Trained an LDA model to identify key discussion topics. → Selected optimal topic count using coherence scores and refined them manually for clarity. 📌 Sentiment Analysis → Used VADER to classify sentiments as Positive, Neutral, or Negative. → Integrated sentiment tags for each review to enable deep filtering and visual exploration. 📌 Image Extraction & Mapping → Extracted image URLs for products and downloaded them locally. → Mapped each product to its correct image using fuzzy string matching. → Product image is dynamically displayed upon selection in the app. 📌 Interactive Dashboard with Streamlit → Filters by product, topic, and sentiment. → Visualizes topic distribution, sentiment breakdowns, and sample reviews. → Product stats and images enhance the storytelling experience. 🧪 Technologies Used Python · Selenium · BeautifulSoup · pandas · gensim · NLTK · VADER · Streamlit · matplotlib · seaborn · scikit-learn · difflib 🚀 Deployment The app is currently focused on a narrow category to demonstrate the full pipeline. More products and categories will be added over time. 🔗 Live App: https://lnkd.in/gTqaMrJg 📂 GitHub Repo: github.com/nikhil0523 This project was a deep dive into building an end-to-end NLP pipeline — from live data scraping to interactive visualization. I’d love to hear your thoughts, feedback, or ideas for extending this further! Let’s connect if you’re working on something similar or have ideas to collaborate. #DataScience #NLP #MachineLearning #WebScraping #Streamlit #TopicModeling #SentimentAnalysis #Python #PortfolioProject

14

4 Comments

🚀 Building a Time Series Forecasting Project - EDA Analysis Complete   Continuing the structured time series analysis and forecasting project using real market data (MSFT stock prices).   After completing Notebook 01 (Data Inspection), this notebook dives into deeper time series diagnostics to understand the statistical structure before moving into modeling.   Previous Post: https://lnkd.in/dxgEatzB   This step establishes the statistical groundwork for feature engineering and model development.   𝐖𝐡𝐚𝐭 𝐭𝐡𝐢𝐬 𝐩𝐡𝐚𝐬𝐞 𝐟𝐨𝐜𝐮𝐬𝐞𝐝 𝐨𝐧:        • Horizon selection based on market and data regimes        • Multi-horizon trend analysis (weekly/monthly smoothing)        • Decomposition (classic & STL) with critical interpretation        • Stationarity diagnostics of log returns        • ACF/PACF analysis of log-returns to assess linear memory        • Volatility clustering and rolling risk behaviour        • Calendar effect testing in mean and variance   𝐊𝐞𝐲 𝐓𝐚𝐤𝐞𝐚𝐰𝐚𝐲𝐬:        • Price levels are non-stationary and trend-dominated.        • Log-returns provide a more stable modeling foundation.        • Volatility is time-varying (heteroskedastic), not constant.        • Log-returns show weak linear autocorrelation — suggesting limited short-term memory.        • No statistically significant calendar effect in mean returns   𝐍𝐞𝐱𝐭 𝐮𝐩 → Notebook 03 intent to construct feature representations that explicitly capture temporal dependence and volatility structure for downstream modeling.   Tools: Python | Pandas | NumPy | Matplotlib | Seaborn | Statsmodels | SciPy | yFinance   Full repository: https://lnkd.in/d7idKJVT   Notebook attached as PDF for reference.   + Follow Shashank Garewal for the more such data science content and insights.   #TimeSeries #DataScience #Python #Forecasting #QuantFinance #MachineLearning #Analytics #StockForecasting #Microsoft

7

1 Comment

n the context of e-commerce, why is relying solely on word matching of query and products considered limited, and what factors should be considered for more effective item retrieval? What are the additional signals or features that can be incorporated into the embeddings of Two-Tower Models to enhance scalability and accuracy? We answer it all and go through the process of building a Two-Tower Search-Item Retrieval model from scratch using an open-source Amazon dataset in details, allowing one to witness firsthand how this architecture can create a highly effective and scalable retrieval system. Find out more by watching the video I released on it. : ) Video Link: youtu.be/o-pZk5R0TZg Channel Link: youtube.com/@datatrek

3

📌 Why Transformers Need Positional Embeddings — And Why Sine & Cosine?    Transformers revolutionized NLP by removing recurrence and convolutions — but in doing so, they also lost any natural sense of word order.    So, how does a Transformer know that “The dog chased the cat” is different from “The cat chased the dog”? 👉 Positional Embeddings.    But here’s the interesting part:  🌀 The original Transformer model uses sine and cosine functions (not learned vectors) to encode token positions. Why?    🔍 Because sine and cosine:  1. Are smooth and differentiable (great for optimization), 2. Work across multiple frequencies to capture both local and global position,  3. Naturally allow the model to compute relative positions using trigonometric identities,  4. Are fixed (no training needed) and generalize well to longer sequences,  5. Stay bounded (values between -1 and 1), which stabilizes learning.    ✅ Most importantly:  They give the model a way to “feel” the distance between words, not just where they are.    It’s one of those elegant engineering decisions — simple math, massive impact.    💡 Curious? Try plotting the sinusoidal embeddings across dimensions   Image source: https://lnkd.in/ggZJgKMt #MachineLearning #DeepLearning #Transformers #NLP #PositionalEmbeddings #AI #MLCommunity #GenAI

36

7 Comments

🔧 How RLHF Is Built (Step-by-Step) 1️⃣ Pretrained Model Start with a large language model trained on massive data. Good at patterns — not yet aligned with human preference. 2️⃣ Supervised Fine-Tuning (SFT) Humans create example responses the model should learn to imitate. 3️⃣ Human Preference Data For each prompt, generate several responses. Humans rank them from best to worst. 4️⃣ Train a Reward Model This model learns to predict which answer humans prefer. 5️⃣ Reinforcement Learning (e.g., PPO) The main model is optimized to maximize the reward score while staying close to the original knowledge. The result is behavior that aligns with human expectations. 🌍 Real-Life Company Example: OpenAI’s ChatGPT -OpenAI uses RLHF to train ChatGPT so that its responses are more helpful, safe, and aligned with human preferences — not just statistically likely text. RLHF is the reason ChatGPT feels conversational, context-aware, and user-friendly rather than purely predictive. This isn’t just theory — it’s what makes modern assistants like ChatGPT useful in products across customer support, education, and enterprise workflows. ❤️ Why RLHF Matters Without RLHF: -Models are smart but can be tone-deaf. -They may produce harmful or unhelpful responses. With RLHF: -Responses feel natural and safe. -Behavior reflects human expectations. -User experience improves dramatically. RLHF doesn’t make models smarter — it makes them aligned. And in real-world AI products, alignment wins. #AI #MachineLearning #LLM #RLHF #Product #AIAlignment

4

Myths that stops Data Scientists moving from SBC to PBC 🚫 1. PBCs only hire from top-tier colleges. I’ve seen people from tier-3 colleges land offers at Swiggy, Fractal, etc. Because strong portfolios and problem-solving skills always speak louder than a college tag. 2. “I’ll prepare for 6 months and then start applying.” Don’t make this mistake. If you’re aiming for a product company, build real-time use cases, keep your fundamentals strong, and apply in parallel. And this one’s my favorite: 3. “SBC experience isn’t valued in PBCs.” Not true at all. Experience is experience. What matters is the tech you’ve worked with, the projects you’ve built, and the impact you can prove. So if you’re in an SBC right now and dreaming of that switch: Stop doubting your background. Start sharpening your skills. Start building. Start applying. And a perfect place to start your journey would be Bosscoder Academy’s Data Science program. Check it here → bcalinks.com/iWliFcQ They’ve helped 2200+ professionals switch into Data & ML roles at top tech companies by offering: ✅Structured curriculum covering ML, Deep Learning & GenAI ✅Industry projects with real world datasets that build your portfolio ✅1:1 mentorship from Data Scientist at top product companies ✅Placement support to help you land your dream data role #myths #datascience #sponsoredpost Image Credit: Google Banana model 🙈

66

30 Comments

Wrote an article on Byte Pair Encoding (subword Tokenization) and how it works which was potentially used in the the earlier versions of GPT / traditional Language Learning model paradigms. Took a sample of Hindi Language corpus text data to showcase the same. Article below - https://lnkd.in/gCu7Bgpf

33

As a student at IIT, Shyoji noticed a pattern: Smart engineers don’t like doing the same thing twice. They’ll script, automate, and systemize just to earn time back — To build, to ship, to play. That same mindset built PromptBI. Because data engineers today still waste hours on: ❌ Ad hoc SQL ❌ Broken ETL pipelines ❌ Debugging dashboards ❌ Manually inspecting data anomalies So we asked — What if your analytics tool was an AI sandbox? Ask questions. Get inspectable Python. Clean logic. Visual insights. Export, reuse, automate. That’s what we’ve built. And if you’re a data engineer, you’ll get it instantly. Try it → https://promptbi.com P.S. If you’re a lazy-smart builder (like us), this is for you. #PromptBI #DataEngineering #Python #AItools #Analytics #Founders #ETL #DataOps #LazySmart #ProductLaunch

26