Andrew Ng

New research from Deep Alignment My paper, Mirror-Neuron Patterns in AI Alignment, is now live on arXiv. The work explores how cooperative learning can produce shared neural representations, as an analog of affective empathy in artificial agents. These shared patterns may offer a structural basis for AI alignment, where systems learn to understand others' states as part of their own. Using a custom game environment called Frog & Toad, the model learned to "help" another agent recover from energy loss - a simple but striking example of how alignment-relevant behavior can emerge from shared internal states rather than explicit programming. Read the paper: https://lnkd.in/eT3cszR7 More context: https://deepalignment.ai #AIAlignment #AIEthics #MachineLearning #DeepAlignment #EmpathyInAI #Research

2

🚀 The Unicorn Paradox: Metrics for Inverse Selection in LLM Systems Hiring The demand for senior AI/LLM architects has grown exponentially, yet common evaluation methods — technical storytelling, short answers, rhetorical signaling — create a structural gap between the profile needed and the one selected. 📊 This work formalizes the gap using QLCM metrics: phi_i: Intentional Coherence H_s: Semantic Fidelity INCS: Non-Collapse Index and introduces Psi_h (Susceptibility Index to Inverse Selection) as a quantifiable diagnostic tool. 🔍 Analysis of 73 real job postings reveals: Organizations with Psi_h > 3.0 select narrative-driven candidates with low operational evidence (phi_i < 0.35, H_s < 0.62). Organizations with Psi_h < 1.2 capture architects with verifiable expertise (phi_i > 0.68, H_s > 0.89). 💡 Conclusion: The “unicorn” isn’t a shortage of talent — it’s a fiction generated by evaluation mechanisms structurally misaligned with the role. ⚙️ The real challenge: designing evaluation systems that recognize true expertise without oversimplification. #AI #LLM #TalentSelection #Hiring #Architecture #Metrics #Innovation #QLCM #InverseSelection #CareerDevelopment

3

Good to be on The Information's TITV again today, talking all things NVIDIA. One of Jensen's comments in the keynote about how NVIDIA won't stop working on open source models and that Nemotron 3 will be followed by Nemotron 4 caused me to reflect on the company's position in the open source AI model firmament in the future....

13

We're excited to share that Michal Valko has joined Merrai as an advisor. Michal was Principal Llama Scientist at Meta GenAI, where he built the online reinforcement-learning stack for Llama 3, working closely with Yann LeCun and Mark Zuckerberg. Before that he co-created Google DeepMind Paris. Today he's a founding researcher at Isara Labs (backed by OpenAI) and a tenured researcher at Inria. From day one we've been building toward one thing: AI that already knows what to do, and simply does it. It works across every tool you use and evolves with you. Crucially, it's owned by you, not locked inside someone else's platform. Michal sees what we see. Memory becomes core AI infrastructure as the stack evolves, and personal intelligence is still a wide-open track. That's why he joined, and why his work on recursive learning is exactly where we're headed: AI that compounds with you, and is yours to own. If you're testing every new assistant, switching models every month, and copy-pasting yourself into each one: this is for you. Stop re-introducing yourself to AI. Your context, one layer, every tool. And if you're a team running five AI tools, you're paying five times to rebuild the same context, in tokens and in time. Come talk to us. Our app is live today, free for the first 100 people with the code ownmyintelligence. Start now and when our assistant lands, or a new AI tool lands, it will already know you. Welcome aboard, Michal. 🚀 🚀

162

22 Comments

Just came across RLP (Reinforcement Learning Pretraining) by NVIDIA — a paper that fundamentally rethinks when and how we introduce reasoning into LLMs. 💡 The Core Problem: Traditional pretraining relies solely on next-token prediction, which doesn't explicitly encourage reasoning. Reinforcement learning has been reserved for the final post-training phase, after supervised fine-tuning. But is this optimal? 🔍 What RLP Does Differently: • Brings RL forward to the pretraining stage itself • Treats chain-of-thought (CoT) as an exploratory action before predicting each token • Rewards thoughts based on information gain — how much they improve next-token prediction • Uses a verifier-free, dense reward signal across all tokens • Works on ordinary web text, not just curated reasoning datasets 📊 Key Results: • 19% improvement on Qwen3-1.7B across math & science benchmarks • Gains persist and compound after post-training (8% better than baseline) • Scales effectively to 12B parameters (35% improvement on Nemotron-Nano) • Strong cross-domain transfer — improves both math AND science reasoning 🎯 Why This Matters for AI Architects: RLP demonstrates that reasoning can be woven into the fabric of pretraining rather than bolted on at the end. The dense, position-wise reward signal teaches models to "think before predicting" — a more natural learning pattern that mirrors human comprehension. Unlike methods requiring verification oracles or narrow datasets, RLP extracts reasoning signals from diverse web corpora. This makes it practical for real-world pretraining pipelines. 🔮 Looking Ahead: As we push toward more capable reasoning systems, the boundary between pretraining and post-training may blur. RLP suggests that reinforcement-driven exploration belongs earlier in the training lifecycle than we thought. Curious how this information-gain approach might influence future foundation models and architectural choices? The implications for scaling reasoning without external verifiers are significant. 📄 Attached: The original research paper for those who want to dive deeper into the methodology and experimental results. #AI #MachineLearning #LLM #ReinforcementLearning #AIResearch #DeepLearning #AIArchitecture

24

Thrilled to see the new paper that could change how we fine-tune Large Language Models. The work, "Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning" (https://lnkd.in/gt6upTER), challenges the dominance of Reinforcement Learning (RL) and presents a powerful, and surprisingly simple, alternative: Evolution Strategies (ES). You might remember there was a 2017 paper by OpenAI, with Ilya Sutskever as a coauthor, titled "Evolution strategies as a scalable alternative to reinforcement learning" (https://lnkd.in/ggreRuwf). I myself wrote a review back then (https://lnkd.in/g9ugwkVv) and expected neuroevolution would become a major force. That didn't happen, at least at the scale I wanted, but the recent paper by a joint team from Cognizant and MIT gives me a feeling it still may happen! The authors show that ES-fine-tuned models are less prone to "going off the rails" (reward hacking), require significantly less data (often <20% of what RL needs), and produce consistent results every time—a huge win for deploying robust AI systems. 🚀 For our technical audience, the paper is a masterclass in engineering—it's simple and beautiful. The authors successfully scale Evolution Strategies (ES) to fine-tune the full parameters of multi-billion parameter models like LLaMA and Qwen, completely bypassing backpropagation. 🧠 Their memory-efficient, massively parallelizable implementation directly explores the parameter space, achieving superior performance on reasoning and conciseness tasks. The results are striking: ES not only outperforms state-of-the-art RL methods in accuracy but also shows 15.5x lower variance in performance, suggesting a much smoother and more stable optimization landscape. This work opens a compelling new direction for AI alignment. Check out our full breakdown and the original paper here: https://lnkd.in/geNpSMhv Kudos to the authors: Xin Qiu, Yulu Gan, Conor F. Hayes, Qiyao Liang, Elliot Meyerson, Babak Hodjat, Risto Miikkulainen. #AI #LLM #MachineLearning #FineTuning #EvolutionaryAlgorithms #ReinforcementLearning #DeepLearning #AIResearch #TechInnovation

43

2 Comments

https://lnkd.in/gF6XXbV2 A recent interview from Jeff Dean. Quite inspirational discussion around the Pareto frontier of capabilities vs efficiency materialized in different dimensions, model sizes, latency, weight knowledge vs retrieval, one model for all vs bespoke models, etc.

27

Sharing a great thought piece from our founding engineer Jay Liang, who breaks down why 𝙨𝙩𝙖𝙗𝙞𝙡𝙞𝙩𝙮 deserves as much attention as accuracy when evaluating AI systems: 🔗 𝗙𝗿𝗼𝗺 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝘁𝗼 𝗦𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 — 𝗥𝗲𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗣𝗿𝗼𝗴𝗿𝗲𝘀𝘀 𝗶𝗻 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 https://lnkd.in/eeq8F_NZ — In traditional software engineering, tests focus on the binary: does the output match the expected result? We rarely question how the module got there because we control the entire logic. In AI-first systems, that “how” is often a black box. The same prompt can produce correct results one moment, and wildly different ones the next. Even when outputs are accurate, inefficiencies in latency, token use, or reasoning paths can make systems unstable at scale. — Thanks to leadership from our engineers, stability (i.e., how consistently and efficiently a model reaches its goal) is becoming a principle of responsible, scalable AI engineering at Vibranium Labs. #AIEngineering #AgenticAI #ResponsibleAI #ModelStability #VibraniumLabs

20