Mudit Tiwari

🚨 NeurIPS 2025 Spotlight Paper Alert 🚨 ➡️Paper Title: STITCH-OPE: Trajectory Stitching with Guided Diffusion for Off-Policy Evaluation 🌟Few pointers from the paper 🎯Off-policy evaluation (OPE) estimates the performance of a target policy using offline data collected from a behavior policy, and is crucial in domains such as robotics or healthcare where direct interaction with the environment is costly or unsafe. 🎯Existing OPE methods are ineffective for high-dimensional, long-horizon problems, due to exponential blow-ups in variance from importance weighting or compounding errors from learned dynamics models. 🎯To address these challenges, authors of this paper proposed “STITCH-OPE”, a model-based generative framework that leverages denoising diffusion for long-horizon OPE in high-dimensional state and action spaces. 🎯Starting with a diffusion model pre-trained on the behavior data, STITCH-OPE generates synthetic trajectories from the target policy by guiding the denoising process using the score function of the target policy. 🎯STITCH-OPE proposes two technical innovations that make it advantageous for OPE:  (1) prevents over-regularization by subtracting the score of the behavior policy during guidance, and  (2) generates long-horizon trajectories by stitching partial trajectories together end-to-end. 🎯They provide a theoretical guarantee that under mild assumptions, these modifications result in an exponential reduction in variance versus long-horizon trajectory diffusion. 🎯Experiments on the D4RL and OpenAI Gym benchmarks show substantial improvement in mean squared error, correlation, and regret metrics compared to state-of-the-art OPE methods. 🏢Organization: University of Toronto, University of Toronto Robotics Institute, Toyota Research Institute, Vector Institute 🧙Paper Authors: Hossein Goli, Michael Gimelfarb, Nathan Samuel de Lara, Haruki Nishimura, Masha Itkina, Florian Shkurti 📝 Read the Full Paper here: https://lnkd.in/gz2-DSQn 🗂️ Project Page: https://lnkd.in/gV5vdcF4 🧑‍💻 Code: https://lnkd.in/g9AvhmpE 🎥 Be sure to watch the attached Technical Summary Video - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️REPOST and teach your network something new Follow me 👣, Naveen Manwani, for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements. #NeurIPS2025

11

Soumith Chintala, 𝗰𝗿𝗲𝗮𝘁𝗼𝗿 of PyTorch and one of the most respected builders in 𝗺𝗼𝗱𝗲𝗿𝗻 𝗔𝗜, has joined Mira Murati ’s fast-rising startup, Thinking Machines Lab. After 11 years at Meta (Facebook), where PyTorch grew from a 𝗻𝗶𝗰𝗵𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁 𝘁𝗼 𝗽𝗼𝘄𝗲𝗿𝗶𝗻𝗴 𝗴𝗹𝗼𝗯𝗮𝗹 𝗔𝗜 𝗶𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻, Chintala’s move signals a new wave in human–𝗔𝗜 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻. Thinking Machines Lab isn’t just gathering top talent from Meta, OpenAI, Anthropic, and leading universities—they’re offering 𝗴𝗮𝗺𝗲-𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘀𝗮𝗹𝗮𝗿𝗶𝗲𝘀 (up to $500K), 𝗮 𝗺𝗮𝘀𝘀𝗶𝘃𝗲 $𝟮 𝗯𝗶𝗹𝗹𝗶𝗼𝗻 𝘀𝗲𝗲𝗱 𝗿𝗼𝘂𝗻𝗱, and are aiming for a $50 billion market valuation. Their new product, Tinker, is already in use at Princeton, Stanford, and by major enterprises. Meta, meanwhile, is regrouping under the 𝗻𝗲𝘄 𝗦𝘂𝗽𝗲𝗿𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝗟𝗮𝗯𝘀 division led by Alexandr Wang, with other AI leaders reportedly on the move. Chintala’s reasoning: time to pursue “𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙨𝙢𝙖𝙡𝙡, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙣𝙚𝙬, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙪𝙣𝙘𝙤𝙢𝙛𝙤𝙧𝙩𝙖𝙗𝙡𝙚.” His legacy—the PyTorch tool now thriving in classrooms from 𝗠𝗜𝗧 𝘁𝗼 𝗿𝘂𝗿𝗮𝗹 𝗜𝗻𝗱𝗶𝗮—𝗶𝘀 𝘀𝗲𝗰𝘂𝗿𝗲. #AI #TechNews #PyTorch #Meta #ThinkingMachines #Startup #AIJobs #TalentWar #Innovation #BigFunding

8

🚀 New Post on Incomplete Distillation (subscribe/share) Fine-tuning LLMs using Reinforcement Learning – II: From RLOO to GRPO In my previous post, I looked at how reinforcement learning (RL) can be used to fine-tune large language models (LLMs) — starting from first principles with a hands-on implementation of REINFORCE for a code generation task. This week, I continue that exploration — moving from REINFORCE with Leave-One-Out (RLOO) to Group Relative Policy Optimization (GRPO), the algorithm powering many recent “reasoning-style” LLMs. The story builds progressively: 🔹 RLOO gives us a critic-free, simple baseline, but it’s sample inefficient. 🔹 PPO introduces importance sampling and clipped objectives to reuse expensive rollouts, but adds a critic network and tuning overhead. 🔹 GRPO borrows PPO’s clipped surrogate and KL-regularization, yet stays critic-free by using group-relative advantages — combining sample efficiency, stability, and simplicity in one framework. I also walk through a GRPO training loop in PyTorch, illustrating how to: ✅ Reuse rollouts safely using importance ratios ✅ Stabilize updates with PPO-style clipping ✅ Prevent drift with a KL penalty to the SFT reference ✅ Use group-normalized advantages instead of a critic As before, all experiments are built around a small model (Qwen2.5-Coder-0.5B-Instruct) fine-tuned on HumanEval, focusing on clarity over performance. 🧠 The key insight: critic-free, group-based reinforcement learning can scale surprisingly well when combined with smart regularization — it’s what makes GRPO such an elegant middle ground between REINFORCE and PPO. 📖 Read the full post here: https://lnkd.in/gvQUPbXG #MachineLearning #LLMs #ReinforcementLearning #AIAlignment #FineTuning #IncompleteDistillation #GRPO #PPO #RLOO #AI

113

1 Comment

Serving LLMs on TPUs with SGLang ! I've been exploring the JAX ecosystem for inference recently and came across SGL-JAX, a high-performance, JAX-based inference engine for LLMs, specifically optimized for Google TPUs. Some highlights: 🌳 The "Radix Tree" KV Cache: SGL-JAX implements a Radix Tree (similar to PagedAttention) to manage memory. This enables efficient prefix sharing for multi-turn chat or agentic workloads where the system prompt remains constant. ⚡ JAX-Native FlashAttention: It integrates a high-performance FlashAttention kernel directly into the JAX compute graph. This is critical for faster, more memory-efficient attention usage, particularly when dealing with long sequence lengths on TPUs. 🧩 Native Tensor Parallelism: It handles the sharding logic natively using JAX distributed primitives. It supports distributing large models (like the Qwen 3 MoE series) across multiple TPU devices without needing complex, manual mesh definitions. ⏳ Continuous Batching: It dynamically schedules incoming requests to maximize TPU core saturation. You aren't waiting for a fixed batch size to fill up while tail latency spikes. 🔧 Drop-in Compatibility: It exposes an OpenAI-compatible API standard. You can literally point your existing LangChain or LlamaIndex setup at the new endpoint and switch hardware backends seamlessly. For more info about the architecture and how it works, check out the repo below! #JAX #MachineLearning #TPU #LLMOps #GoogleCloud #AIEngineering

46

1 Comment

You're in an interview for a Senior ML Engineer role at Google DeepMind. The interviewer asks: "Our new reasoning model is great, but it uses a 2000 token Chain of Thought even for simple questions like 'What is 2+2?'. This is killing our inference budget. How do you fix this without sacrificing its ability to solve complex problems?" Most candidates say: "I'd train two models, a small, fast one for simple queries and our large reasoning model for hard ones." Wrong approach. Now you have a complex routing problem, double the hosting costs, and a maintenance nightmare. The reality: You don't need two models. 𝐘𝐨𝐮 𝐧𝐞𝐞𝐝 𝐨𝐧𝐞 𝐦𝐨𝐝𝐞𝐥 𝐰𝐢𝐭𝐡 𝐭𝐰𝐨 𝐦𝐨𝐝𝐞𝐬. Your reasoning model 𝘢𝘭𝘳𝘦𝘢𝘥𝘺 knows '2+2=4'. It's just been trained to always show its work, like an over-eager student. You just need to teach it when not to. This isn't an RL problem. This is a production-level SFT problem. The solution is "𝐓𝐡𝐢𝐧𝐤𝐢𝐧𝐠 𝐌𝐨𝐝𝐞 𝐅𝐮𝐬𝐢𝐨𝐧": Step 1. Take your final, fully-trained reasoning model. Step 2. Create a new fine-tuning dataset with special instruction tags. Step 3. For hard problems, format it: <𝘵𝘩𝘪𝘯𝘬> [𝘘𝘜𝘌𝘚𝘛𝘐𝘖𝘕] <𝘊𝘰𝘛> ... [𝘈𝘕𝘚𝘞𝘌𝘙] Step 4. For simple problems, format it: <𝘯𝘰_𝘵𝘩𝘪𝘯𝘬> [𝘘𝘜𝘌𝘚𝘛𝘐𝘖𝘕] [𝘋𝘐𝘙𝘌𝘊𝘛 𝘈𝘕𝘚𝘞𝘌𝘙] You fine-tune the 𝘴𝘪𝘯𝘨𝘭𝘦 model on this mixed dataset. Now, your inference-time logic is trivial. For simple queries, just add the <𝘯𝘰_𝘵𝘩𝘪𝘯𝘬> tag to the prompt. The model will suppress its Chain of Thought and answer directly. No routing, no extra VRAM, same set of weights. 𝐓𝐡𝐞 𝐚𝐧𝐬𝐰𝐞𝐫 𝐭𝐡𝐚𝐭 𝐠𝐞𝐭𝐬 𝐲𝐨𝐮 𝐡𝐢𝐫𝐞𝐝: "Don't build a multi-model routing system - build a single, controllable fusion model. Routing adds cost and complexity. Fusion adds control and capability. With 𝐓𝐡𝐢𝐧𝐤𝐢𝐧𝐠 𝐌𝐨𝐝𝐞 𝐅𝐮𝐬𝐢𝐨𝐧 you teach one model to decide when to think and when to act." #AI #MLEngineering #LLM #DeepLearning #Inference #SystemsDesign #DataScience

514

73 Comments

Projecr Titel: Strassen’s Algorithm for matrix multiplication Contributor: Rajeev Singh Sisodiya, ORCID: 0009-0003-6585-4744 Strassen’s algorithm is a divide-and-conquer algorithm for matrix multiplication that reduces the computational complexity compared to the standard (naïve) method. Instead of performing 8 multiplications when dividing matrices into blocks, Strassen reduces it to 7 multiplications by increasing the number of additions/subtractions. Strassen’s algorithm is much deeper than “just a faster multiplication trick.” It is fundamentally about tensor rank reduction, which connects it directly to fast linear algebra and even quantum algorithms. The standard algorithm corresponds to tensor rank = 8, because it uses 8 scalar multiplications. After Strassen, research focused on minimizing the tensor rank of matrix multiplication. Key milestones: 1.Strassen (1969) 2.Coppersmith–Winograd (1987) 3.Alman–Vassilevska Williams (2021) The relationship between HHL, block-encoding, and QSVT (Quantum Singular Value Transformation) is essentially the evolution of quantum linear-algebra algorithms from a specialized construction to a general operator-transformation framework. You can think of it historically and structurally: HHL (2009) → first quantum linear-system solver Block-encoding (2015–2019) → unified operator representation QSVT (2018–2019) → universal matrix-function compiler https://lnkd.in/gHqfydKT

1

“Before, sellers would take 28 days to fully onboard. With Catalog Transformer and Anyscale, we reduced that to a few hours in most cases.” — Arthur Delaitre, Manager of Data Science, Mirakl Mirakl scaled LLM-powered catalog onboarding with Ray on Anyscale, delivering 3× lower inference cost, 20x throughput, and reliable peak handling. See the blueprint: architecture, scaling patterns, and cost wins. https://lnkd.in/gzFaitj4

21

The most ironic part of all this is, the subject they deem only theoretical often define our real world. AI functioning on matrices and theorems? That's not something which you can build or realise. It's a concept, a method, a thought process. And that very thought process now impacts the whole world. Pure Science, Mechanical Engineering, Aeronautics, Food Technology, Philosophy, Psychology, all these supposed niche fields require huge brain power, intellectual capability and creativity. The applied fields and concepts are nothing without their theoretical part. We must remember that.

6