Raja Biswas

In this 20-minute talk, PolyMage Labs founder Uday Reddy Bondhugula talks about why and how one should build effective compilers for AI programming frameworks: https://lnkd.in/gWUQNUsF

79

Physics-Informed Neural ODEs with Scale-Aware Residuals for Learning Stiff Biophysical Dynamics Kamalpreet Singh Kainth, Prathamesh Dinesh Joshi, Raj Abhijit Dandekar, Rajat Dandekar, Sreedat Panat https://lnkd.in/dtRD_ide This paper tackles a critical challenge in using Neural ODEs for modeling stiff biophysical systems, specifically the difficulty in achieving stable and accurate long-term predictions. The core idea is to improve training stability and accuracy by incorporating physics-informed regularization with scale-aware residual normalization within a Neural ODE framework. The authors propose "PI-NODE-SR," which combines two key elements: 1. Physics-Informed Regularization: They leverage the known governing equations (e.g., Hodgkin-Huxley) to define a physics-informed loss term that penalizes deviations from the expected behavior. This is a standard PINN approach, but the key is how they handle the residual. 2. Scale-Aware Residual Normalization: This is the novel contribution. Stiff systems often involve state variables evolving on vastly different timescales. Directly applying a physics-informed loss can lead to one timescale dominating the training process, hindering convergence and accuracy. To address this, they normalize the residual associated with each state variable by a scaling factor. This factor is chosen to balance the contributions of each variable to the overall loss, preventing the faster dynamics from overwhelming the slower ones. They use a low-order explicit solver (Heun method) to calculate the residual. The authors demonstrate the effectiveness of PI-NODE-SR on the Hodgkin-Huxley equations, showing that it can learn from a single oscillation and extrapolate accurately over longer time horizons. Importantly, they show that the method can recover morphological features in the gating variables that are typically only captured by higher-order solvers. ----- This work directly addresses a significant limitation of Neural ODEs in scientific applications: their struggle with stiff systems. By introducing scale-aware residual normalization, the authors provide a principled way to stabilize training and improve the accuracy of long-term predictions. This is particularly relevant for: - Biophysical modeling: Accurately simulating neuronal dynamics, cardiac electrophysiology, and other complex biological processes. - Chemical kinetics: Modeling reaction networks with widely varying reaction rates. - Multi-physics simulations: Situations where different physical processes evolve on different timescales. The ability to use lower-order solvers with neural correction to achieve accuracy comparable to higher-order methods has significant implications for computational efficiency. While the method's sensitivity to initialization is noted, the overall approach offers a promising direction for developing more robust and efficient Physics-Informed Neural ODEs for a wide range of scientific applications.

146

1 Comment

There is no doubt that everyone agrees that AI is the revolutionary technology, and Jensen is strategically expanding the market of GPUs with his cash. Eventually, we may only need maybe 2 super intelligent LLMs running in 3-5 data centers to support all the applications. Who will be the winners is not settled yet. When the winner comes out, Nvidia will need the software market to make up the loss of hardware market.

3

1 Comment

Soumith Chintala, 𝗰𝗿𝗲𝗮𝘁𝗼𝗿 of PyTorch and one of the most respected builders in 𝗺𝗼𝗱𝗲𝗿𝗻 𝗔𝗜, has joined Mira Murati ’s fast-rising startup, Thinking Machines Lab. After 11 years at Meta (Facebook), where PyTorch grew from a 𝗻𝗶𝗰𝗵𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁 𝘁𝗼 𝗽𝗼𝘄𝗲𝗿𝗶𝗻𝗴 𝗴𝗹𝗼𝗯𝗮𝗹 𝗔𝗜 𝗶𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻, Chintala’s move signals a new wave in human–𝗔𝗜 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻. Thinking Machines Lab isn’t just gathering top talent from Meta, OpenAI, Anthropic, and leading universities—they’re offering 𝗴𝗮𝗺𝗲-𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘀𝗮𝗹𝗮𝗿𝗶𝗲𝘀 (up to $500K), 𝗮 𝗺𝗮𝘀𝘀𝗶𝘃𝗲 $𝟮 𝗯𝗶𝗹𝗹𝗶𝗼𝗻 𝘀𝗲𝗲𝗱 𝗿𝗼𝘂𝗻𝗱, and are aiming for a $50 billion market valuation. Their new product, Tinker, is already in use at Princeton, Stanford, and by major enterprises. Meta, meanwhile, is regrouping under the 𝗻𝗲𝘄 𝗦𝘂𝗽𝗲𝗿𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝗟𝗮𝗯𝘀 division led by Alexandr Wang, with other AI leaders reportedly on the move. Chintala’s reasoning: time to pursue “𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙨𝙢𝙖𝙡𝙡, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙣𝙚𝙬, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙪𝙣𝙘𝙤𝙢𝙛𝙤𝙧𝙩𝙖𝙗𝙡𝙚.” His legacy—the PyTorch tool now thriving in classrooms from 𝗠𝗜𝗧 𝘁𝗼 𝗿𝘂𝗿𝗮𝗹 𝗜𝗻𝗱𝗶𝗮—𝗶𝘀 𝘀𝗲𝗰𝘂𝗿𝗲. #AI #TechNews #PyTorch #Meta #ThinkingMachines #Startup #AIJobs #TalentWar #Innovation #BigFunding

8

Why does spectral clustering outperform K-means? How do eigenvectors reveal complex cluster structures? Rukshan Pramoditha presents a thorough explainer on a powerful machine learning approach.

18

Powerful takeaways from today’s “Building AI Agents for Enterprise” session with Ajay Shenoy and Abhijith Neerkaje The discussion went well beyond surface-level RAG patterns. We unpacked Retrieval-Augmented Generation from first principles through production realities—why chunking is not a preprocessing detail but a design decision, how embedding spaces influence recall vs. precision, the role of retrieval strategies under latency constraints, and why evaluation frameworks are the hardest (and most critical) part of making RAG reliable in production. A key reminder: fundamentals scale. These concepts aren’t academic—they directly determine whether AI agents can reason autonomously, retrieve the right context under ambiguity, and make defensible decisions in enterprise systems where correctness, cost, and observability matter. The Q&A stood out. Seeing seasoned practitioners deconstruct real-world failure modes—hallucinations, poor grounding, retrieval drift, and evaluation blind spots—and reason through solutions live was invaluable. This is what effective technical learning looks like: clarity rooted in experience, not abstractions. Agentic AI is no longer a future concept. As systems evolve from prompt-response to planning, tool-use, memory, and self-correction, a solid grasp of RAG fundamentals becomes non-negotiable for anyone building production-grade AI. Grateful for sessions like this—where theory meets operational reality, and learning is anchored in what actually works at scale. #AgenticAI, #LLM,#RAG, #AgenticRAG

25

2 Comments

Reinforcement Fine Tuning Reinforcement Fine-Tuning (RFT) is a technique for adapting LLMs to complex reasoning tasks like mathematics and coding. RFT leverages reinforcement learning (RL) to help models develop their strategies for completing a task, rather than relying on pre-existing examples as in traditional supervised fine-tuning. One RL algorithm, called Group Relative Policy Optimization (GRPO), is beneficial for tasks with verifiable outcomes and can work well even when you have fewer than 100 training examples. Using RFT to adapt small, open-source models can lead to competitive performance on reasoning tasks, giving you more options for your LLM-powered applications. This deeeplearning.ai provides an overview of GRPO with simple examples. Worth having a look 😀 #grpo #llm #reasoning #rl #rlhf #sft #predibase

2

𝗦𝗲𝗲 𝗼𝘂𝗿 𝗹𝗮𝘁𝗲𝘀𝘁 𝘄𝗼𝗿𝗸 𝗼𝗻 𝗤𝘂𝗮𝗻𝘁𝘂𝗺 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 (𝗤𝗠𝗟) We have been thinking deeply about Quantum Machine Learning (QML) at the Institute of Applied Artificial Intelligence and Robotics (IAAIR) over the past few months and have penned down our thoughts on where the field might be headed. In this paper (preprint) authored by myself, Jason Fisher and Hung Du, we reflect on recent experimental results that hint at quantum advantage, discuss some of the key limitations holding QML back, and outline a possible roadmap for how it could make its way into applied research and enterprise systems over the next decade. Here is the link: https://lnkd.in/ggFdCauv If you're working in this space or simply curious about where QML might be heading, we’d love for you to take a look: #QML #IAAIR #QuantumComputing #FutureOfQML

34

Introducing Generative Adversarial Distillation (GAD): a novel GAN-style formulation and framework that facilitates both on-policy and black-box distillation of large language models (LLMs). GAD is the first technique to enable block-box on-policy distillation from proprietary teachers where internal logits or parameters are inaccessible, or distillation between teacher and student LLMs with incompatible vocabularies. GAD expands our prior work on white-box on-policy distillation (i.e., MiniLLM), pioneering block-box on-policy distillation for LLM training. Specifically, GAD frames the student LLM as a generator and trains a discriminator to distinguish its responses from the teacher LLM’s, creating a minimax game. The discriminator acts as an on-policy reward model that co-evolves with the student, providing stable, adaptive feedback. Experimental results show that GAD consistently surpasses the commonly used sequence-level knowledge distillation.  In particular, Qwen2.5-14B-Instruct (student) trained with GAD becomes comparable to its teacher, GPT-5-Chat, on the LMSYS-Chat automatic evaluation. The results establish GAD as a promising and effective paradigm for black-box LLM distillation. Our team has been conducting fundamental research in knowledge distillation with wide adoptions across the industry. -         MiniLM: We introduced multi-head attention distillation, establishing the most effective distillation method for BERT-style models. The open-source MiniLM models (e.g., 6x384) have become the most widely utilized small encoder models on the Hugging Face. -         MiniLLM: Our proposed Reverse KLD is recognized as one of the most effective, de facto on-policy distillation approaches for modern LLM training, which has been widely used by Thinking Machines, Gemma, and many other teams and models. -         BitDistill: We proposed BitNet Distillation to finetune off-the-shelf full-precision LLMs (e.g., Qwen) into 1.58-bit precision (ternary weights {-1, 0, 1}), achieving performance parity with the full-precision counterparts on specific downstream tasks. -         GAD: The development of Generative Adversarial Distillation (GAD) now allows for black-box on-policy distillation, overcoming two major prior limitations: (1) Distillation from proprietary teachers where internal logits or parameters are inaccessible; (2) Distillation between teacher and student LLMs with incompatible vocabularies. https://lnkd.in/gMaP2c7w

125

2 Comments

In collaborative machine learning (CML), early contributors take on more risk and encourage participation from wait-and-see parties, but existing CML frameworks treat every party as if they join simultaneously. The #NeurIPS2025 paper of Jiangwei Chen, Nguyen Pham, Rachael Sim, Arun Verma, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low proposes a time-aware CML framework that rewards parties not just for what they contribute, but when they contribute: •⁠ ⁠8 incentives extending fairness with time-aware notions that incentivize early participation while preventing low-quality early submissions. •⁠ ⁠2 reward distribution methods that theoretically satisfy all incentives and recover the #ShapleyValue in time-agnostic cases. •⁠ ⁠A practical recipe for constructing valuation functions and reward realization, with empirical validation on synthetic and real-world data. Find our more at our posters! #NeurIPS2025 Thu, Dec 4, Exhibit Hall C, D, E #1104 #EurIPS2025 Wed, Dec 3, Poster Stand #56 Paper: https://lnkd.in/g_yM6j39 #FederatedLearning

50

🧠📘 Deeply impressed by Mathematics for Machine Learning by Tivadar Danka. Packt and Ankur Mulasi provided the book. As someone working at the intersection of advanced computing and applied mathematics, I’ve read my share of math-for-ML resources. But this book stands out for its clarity, depth, and elegance in how it presents the core mathematical foundations that underpin modern machine learning. ✅ Linear algebra, multivariable calculus, probability theory, information theory, and optimization — all covered with the right blend of intuition and formalism. No hand-waving, no shortcuts. Just solid math, cleanly explained. 📈 Each chapter builds progressively, layering concepts in a pedagogically sound way. The exposition doesn’t just aim to “get you through” the math—it actually helps you understand it. You’ll find: - Elegant derivations (e.g., matrix calculus done properly) - Practical insights tied directly to machine learning techniques - Well-chosen code snippets that ground the math in application - Thoughtful visualizations that bridge geometry and algebra - Exercises that genuinely reinforce conceptual understanding 🧩 Whether you’re refreshing your background or aiming to move from practitioner to researcher, this book equips you, not just prepares you. If you’re serious about building a robust mathematical foundation for ML, this is one of the most valuable books out there. 👏 Kudos to Tivadar Danka for writing a book that is both mathematically rigorous and practically relevant. #MachineLearning #Mathematics #DeepLearning #MLTheory #AI #MathForML #LinearAlgebra #Optimization #MathematicsEducation #TivadarDanka

26

2 Comments

I recently wrote this blog with Mahimna Darji, where we explain how DSPy changes the way developers build and optimize language model systems. The post explores how DSPy moves beyond prompt engineering by turning model behavior into structured, programmable logic that improves with data. It also covers how DSPy works internally, where it fits in real projects, and why many organizations are adopting it for reliable and reproducible workflows. #AI #MachineLearning #DataScience #DataEngineering #LLM #LargeLanguageModels #DSPy #Data #PromptEngineering #AmazonWebServices #Google #StanfordUniversity #Medium #TechInnovation #GenerativeAI #AIApplications #IndianaUniversityBloomington #IUB #IU #Data #Tech #LangChain #LangGraph #WomenInTech

14

Announcing our EMNLP 2025 paper, CAARMA: Class Augmentation with Adversarial Mixup Regularization  With Xiang Li, Hao Chen, Syed Abdul Hannan, Rita Singh, Bhiksha Raj. We present CAARMA, a novel framework that redefines zero-shot speaker verification by generating synthetic classes through adversarial mixup in the embedding space. Unlike traditional methods that focus on generating more samples per class, CAARMA enhances class diversity with an adversarial refinement mechanism, ensuring synthetic classes are indistinguishable from real ones. This approach boosts zero-shot generalization, achieving up to 8% improvement over strong baselines across multiple speaker verification benchmarks and zero-shot speech analysis tasks. Explore the paper and code below: Paper: https://lnkd.in/gT_fnEbN Published in EMNLP Findings Code: https://lnkd.in/ge-utV8R #EMNLP2025 #SpeakerVerification #ZeroShotLearning #CAARMA #AI #MachineLearning

99

3 Comments

NeurIPS 2026 call for papers is live now. Make your AI agents get to work. Papers won't write themselves. Tell them to hurry! /s

1