Sadeep Jayasumana

I have been getting a lot of questions about Meta’s recent Nvidia announcement, particularly the scale of CPU deployments. A large CPU buildout does not indicate any shift away from GPUs or weakening AI demand. It reflects AI moving from concentrated training into broad production. GPUs remain the core of training and the most intense inference workloads. CPUs run the surrounding infrastructure that makes production scale possible. That includes scheduling, data pipelines, storage, networking, security, ads systems, and ranking engines. Once AI features are deployed across billions of users, the platform becomes CPU heavy even as GPU investment stays central. This move is not meaningfully tied to the Manus AI acquisition. Orders of this magnitude reflect platform wide expansion rather than the integration of a single company. The supply chain impact is broad. More CPUs mean increased advanced node wafer demand, higher ABF substrate usage, greater DDR5 attach, rising enterprise SSD needs, and expanded networking and rack integration. The bottom line is that this is expansion, not substitution. GPUs remain at the core. CPUs are scaling around them because AI workloads are becoming deeply embedded across Meta’s entire platform.  #meta #nvidia #cpus #DRAM #DDR5 #GPUs #marketintelligence #technews

9

In this 20-minute talk, PolyMage Labs founder Uday Reddy Bondhugula talks about why and how one should build effective compilers for AI programming frameworks: https://lnkd.in/gWUQNUsF

81

Two days. Two LLM launches. Same pattern. Early February: MiniMax M2.5 Early benchmarks position it close to Claude Opus 4.6 — at reportedly ~20x lower cost. Comparable reasoning. Fraction of the price. Yesterday: GLM-5 Another model narrowing the gap with ChatGPT, Claude, and Gemini. And here’s the real shift. Having an LLM with performance close to the 800-pound gorillas of the arena running on your own infrastructure is no longer theoretical. It’s reality. Yes, there is hardware cost. Yes, there is power and maintenance. But we’re not talking about hyperscaler-only territory anymore. We’re talking about serious reasoning capabilities available to companies willing to invest in a limited GPU setup. News like this now drops almost weekly. The moat isn’t model access anymore. It’s integration, product thinking, and governance. Especially in regulated environments, this changes the equation. The question is no longer: “Are these models good enough?” It’s: “What are we building with them?” #AI #LLM #CTOInsights #RegulatedSoftware #OpenSourceAI #MedTech Image taken from the z.ai blog https://z.ai/blog/glm-5

10

7 Comments

Just came across RLP (Reinforcement Learning Pretraining) by NVIDIA — a paper that fundamentally rethinks when and how we introduce reasoning into LLMs. 💡 The Core Problem: Traditional pretraining relies solely on next-token prediction, which doesn't explicitly encourage reasoning. Reinforcement learning has been reserved for the final post-training phase, after supervised fine-tuning. But is this optimal? 🔍 What RLP Does Differently: • Brings RL forward to the pretraining stage itself • Treats chain-of-thought (CoT) as an exploratory action before predicting each token • Rewards thoughts based on information gain — how much they improve next-token prediction • Uses a verifier-free, dense reward signal across all tokens • Works on ordinary web text, not just curated reasoning datasets 📊 Key Results: • 19% improvement on Qwen3-1.7B across math & science benchmarks • Gains persist and compound after post-training (8% better than baseline) • Scales effectively to 12B parameters (35% improvement on Nemotron-Nano) • Strong cross-domain transfer — improves both math AND science reasoning 🎯 Why This Matters for AI Architects: RLP demonstrates that reasoning can be woven into the fabric of pretraining rather than bolted on at the end. The dense, position-wise reward signal teaches models to "think before predicting" — a more natural learning pattern that mirrors human comprehension. Unlike methods requiring verification oracles or narrow datasets, RLP extracts reasoning signals from diverse web corpora. This makes it practical for real-world pretraining pipelines. 🔮 Looking Ahead: As we push toward more capable reasoning systems, the boundary between pretraining and post-training may blur. RLP suggests that reinforcement-driven exploration belongs earlier in the training lifecycle than we thought. Curious how this information-gain approach might influence future foundation models and architectural choices? The implications for scaling reasoning without external verifiers are significant. 📄 Attached: The original research paper for those who want to dive deeper into the methodology and experimental results. #AI #MachineLearning #LLM #ReinforcementLearning #AIResearch #DeepLearning #AIArchitecture

24

For the past year the edge AI conversation has mostly been about whether models could run locally at all. Could useful language models fit on devices? Would quantization destroy accuracy? Were mobile NPUs actually capable of meaningful inference? At this point many of those questions have answers. Models run locally. Small language models are surprisingly capable. Modern edge silicon can sustain real inference workloads. The question is shifting. Running a model on one device is easy. Running and operating AI across fleets of devices is something very different. Memory limits, runtimes, quantization strategies, model size, thermal constraints, and lifecycle management all start to shape what is actually practical. I wrote a new piece exploring that layer of the stack. It looks at model tiers that realistically fit on edge hardware, the runtimes that execute them, the role of quantization and distillation, and what it takes to manage models across real device fleets. If you are evaluating edge AI projects, these are the engineering decisions that start to determine whether a proof of concept becomes something operational. AI at the Edge: Models, Runtimes, and the Execution Stack https://lnkd.in/gTR_tiPs

9

1 Comment

The clearest sign a closed-source model is expensive to run is when the lab starts building its own chips. Anthropic just hired Amir Salek, the engineer who built and ran Google's TPU program for seven generations. The mandate: an in-house inference chip. That's not a side project. That's an admission that renting Nvidia and Broadcom capacity at scale doesn't pencil out long term. Vertical integration is what companies do when the unit economics of the current model don't work. Open-weight models don't need their own silicon roadmap to stay viable.

1

A new RL algorithm that fixes a hidden flaw in PPO The authors propose CE-GPPO (“Coordinating Entropy via Gradient-Preserving Policy Optimization”), a variant of PPO that restores gradient contributions from clipped actions in the policy update. They argue that traditional clipping discards useful gradient signals from low-probability tokens, which play an important role in controlling the agent’s entropy during training. By bounding those gradients in a controlled way, CE-GPPO maintains exploration–exploitation balance more stably than prior methods. They provide a theoretical analysis showing that CE-GPPO mitigates entropy instability, and empirically test it on mathematical reasoning benchmarks. Their results indicate consistent improvement over strong baselines across different model scales, demonstrating that preserving clipped gradients can lead to better performance in reinforcement learning for reasoning tasks. https://lnkd.in/gYZsJ-8f

1