Kenneth S. Goodman

Barbarians at the Gate (https://lnkd.in/gkEBX79s) is a very interesting new paper from some great folks at Berkeley, with some exciting results showing the potential for AI in systems research. But I think the authors aren't quite asking the hardest problem about where this takes systems as a field. I wrote a new blog post about it: https://lnkd.in/gxS3s4v4 The core point of the paper is that AI can be effective at automatically driving the 'number go up' part of systems research. Given an evaluator, AI can automate progress on implementation. I believe this is 100% correct. The paper also talks about how more focus will have to go into developing evaluators. Again, I 100% agree, but think they're understating their case. Building evaluators (basically tests) is, I believe, going to become the focus of systems and software over the next decade. Which leaves researchers with the hard problem of choosing which problems to go after. That's never been easy in academic systems, and is only going to become more of a challenge. But the best systems research is driven by choosing problems well! This vision for AI could lead to lots more of the least interesting systems research. The "number go up" kind. Which could make it harder than ever to identify the truly innovative and visionary work. We're already seeing that elsewhere in science. So, mostly, I think the paper is right, but not yet asking the biggest question: what should systems research be?

276

12 Comments

Most studies describing what appears in AI-generated search answers are observational. They rarely explain what is causal. So, I published a piece on Substack this week that tries to close that gap. This article looks at why certain content structures surface more often in LLM-based search, and when they stop working. The analysis is grounded in model architecture and mathematics, focusing on attention, entropy, and decoding behavior after retrieval. A few things the piece covers: - Why middle-of-funnel prompts often yield lists and summaries - Why illustrative "for example" mentions behave differently than list inclusion - How competition inside a category changes visibility dynamics - Why schema and JSON-LD are often misunderstood in AI search - I also use a deliberately simple example involving animals with very large appetites to make the mechanics concrete without drifting into tactics. The goal is to provide a causal, systems-level mental model for AI search behavior rather than relying on folklore. And if you're interested, in my first post I used first principles to show that classical SEO and LLM-based search are fundamentally different optimization problems. Classical search ranks documents. LLM-based systems generate answers by probabilistically reconstructing evidence. Substack is going to be the space where I dive deep. If you have any topics or questions you'd like me to go into from a mathematical and research lens, feel free to message me. I'll add the link to my Substack page and my recent article in the comments. I'd really appreciate if you'd share and subscribe!

21

5 Comments

In June 2017, Google published "Attention is All You Need," introducing the transformer architecture. Not yet a decade ago! Six months later, Jeremy Howard and Sebastian Ruder published ULMFit, the fine-tuning breakthrough that quietly underpins most of what we call AI today. I had been building with NLP and ML since before TensorFlow shipped, but I was lucky enough to be in the room at the University of San Francisco Data Institute when Jeremy and Rachel Thomas ran the first in-person fast.ai course that same year. They taught practical deep learning to anyone willing to put in the work. Top-down, code-first, no PhD required. That room felt electric. Since then I've built mostly in open source intelligence and software intelligence, organized some of the biggest pre-COVID AI and data events, and watched a small community of true believers build the world we're living in now. Less than a decade from the transformer paper to agents writing production code. There is so much more road ahead. But something has been quietly breaking. Teams are shipping agents that don't know the architecture, ignore the standards, and leave no trace of what they decided or why. Spend spirals. Context rots. Consequential changes land in production-adjacent code with no guardrails and no audit trail. The people responsible for governance are flying blind, not because they don't care, but because the tooling was never built for this. The gap between AI policy and AI enforcement is enormous, and growing. That's what we're building at ChooChoo. We continuously evaluate how your agents operate against your actual codebase, trace every decision they make, and translate all of it into something your whole team can act on. Not just engineers. Everyone building, growing, and governing the product. We work with YC AI startups and Fortune 500 enterprises alike. If you're building at the frontier and need your agents to stay on track, let's talk. It's Day One at ChooChoo. We start with context, so you can ship with confidence.

5

Had a good/long talk with the team about vibe coding. Keep in mind that LLMs don't hallucinate 5% of the time. They hallucinate 100% of the time, we just agree with them 95% of the time. Without a human reviewer you are just asking for trouble. Think of it as hiring absolutely whip-smart junior devs, who are insanely fast at cutting and pasting from Stack Overflow. That's good, except they don't understand your business, business processes, security needs, etc... etc. So if you don't review the code, no one will. h/t to Casey Ellis for the graphic idea.

45

9 Comments

Kay is one of the most impressive human beings I’ve had the opportunity to get to know and the fortune to build a company with! When we started building Genspark.ai , we asked ourselves a deceptively simple question: what does it actually take for an AI system to get real work done - not just answer questions, but complete complex, multi-step tasks over hours or even days? That question shaped everything about how we build. This week at #NVIDIAGTC, our co-founder and CTO Kay Zhu takes the stage to share exactly what we've learned the hard way. Session S82413: How AI Systems Evolve Into Long-Horizon Agents isn't a product pitch - it's a technical deep-dive into the architectural trade-offs that separate AI demos from AI that actually ships. The four stages Kay will walk through: 🔍 AI Search → the starting point ⚙️ Asynchronous Workflows → parallelism unlocked 🤖 Agents → reasoning + action combined 🌐 Long-Horizon Agents → tasks that run for hours, reliably, at scale Most teams can build a cool agent prototype. Very few can make it reliable in production. That gap - between demo-ready and production-ready - is where Genspark has invested the most. And we're opening the playbook. The future isn't just smarter AI. It's AI that can be trusted to do your work while you focus on what matters most. If you're at GTC, don't miss this one. 👇 📍 Session S82413 — Thursday, March 19 | 12:00 PM PDT Who's in San Jose? Drop a comment - I'd love to connect in person. 🤝 #NVIDIAGTC #AIAgents #LongHorizonAgents #Genspark #ArtificialIntelligence #AIInfrastructure #AgenticAI

40

3 Comments

Northflank is partnering again with Cerebral Valley, PyTorch, CoreWeave (and more) for the OpenEnv Hackathon in San Francisco, March 7–8. Participants will be building RL environments and post-training base models across 5 themes, solving the most pressing problems in RL and agentic orchestration. We'll be supporting teams in deploying their models through the Northflank platform with compute from our wonderful partners, CoreWeave (Matthew Lu, Jacob Feldman). $100K+ prize pool and and a stacked lineup of judges and mentors from Meta, Hugging Face, University of California, Berkeley, Cursor, Scale AI, and more. You can submit an application at the link in comments.

56

3 Comments