Activity
9K followers
-
Ion Stoica shared thisWe introduce AdaMAST, a framework that helps AI agents understand and fix their own mistakes by learning system-specific failure taxonomies from raw traces. Leveraging AdaMast, we improve SWE-bench Verified Mini to 70.7% and TerminalBench 2.0 to 89.9%. Read more about how we achieved this in the thread below, and try out the library via "pip install adamast". Great work by the whole team!Ion Stoica shared thisWhat are failure taxonomies, and how can they improve AI agents? In our new paper “Fantastic Failure Taxonomies and How to Use Them”, we introduce AdaMAST, which is a framework to learn system-specific failure taxonomies directly from agent traces and reuse them across runs. Unlike fixed taxonomies, AdaMAST only fixes three high-level axes (categories): system-level, role-specific, and domain-specific. The actual failure codes in these axes, definitions, and evidence patterns are induced from the behavior of the target agent system. We then use the learned taxonomy in three ways: 1. As runtime feedback for coding agents 2. As structured mutation feedback when optimizing agent systems 3. As a test-time scaling signal for best-of-N trajectory selection Some results: - Claude Code (with Haiku) improves from 64.0% to 70.7% on SWE-bench Verified Mini. - When optimizing an agent system using optimizers with evolutionary algorithms AdaMAST improves on existing unstructured guidance steps and it pushes post-search accuracy on all 5 benchmarks by +3.4 to +7.5 points. - On Terminal-Bench 2.0, AdaMAST reaches 89.9% with the Opus 4.6 / ForgeCode harness using a best-of-N judge. We have also released AdaMAST as an installable library: pip install adamast Check out the paper and the code below! Paper: https://lnkd.in/ghs7KBjF Blog: https://lnkd.in/gxm2xMpZ Project: https://lnkd.in/gFMH8NxU Code: https://lnkd.in/gxqN_RX4 Docs: https://lnkd.in/gQZYkdix This was a very fun collaboration with Andrei Cojocaru, Melissa Pan, Shu Liu, Shubham Agarwal, Alex Krentsel, Xiaojun (Jay) Tang, Kannan Ramchandran, Joseph Gonzalez, Matei Zaharia, Alex Dimakis, and Ion Stoica. #AIAgents #AgenticAI #MachineLearning #OpenSource
-
Ion Stoica reposted thisIon Stoica reposted this☑️ Your Unity Catalog managed tables can now stay current with best-practice features automatically. Auto Upgrades observes how each table is used, verifies workload compatibility, and applies eligible GA features when the table is ready. That means better performance, reliability, interoperability, and cost savings without manual compatibility checks or ALTER TABLE work. Every upgrade remains visible, reversible, and configurable per table. https://lnkd.in/ghRwR_-e
-
Ion Stoica shared thisBig congratulations to the team on emerging from stealth today with over $20M raised. Amazing to see SkyPilot evolve from an early project into a platform solving AI compute fragmentation for frontier organizations building custom intelligence.Ion Stoica shared thisToday, SkyPilot is out of stealth. Building custom intelligence is now existential. We help frontier AI teams build intelligence faster by removing their biggest bottleneck: AI compute fragmentation. Frontier teams like Applied Compute, Abridge, Hippocratic AI, H Company, and Nubank already run on SkyPilot, with 10x faster time-to-intelligence and double-digit increase in GPU utilization. AI teams today get compute anywhere they can. They then firefight compute fragmentation across providers. Researchers burn time on workload setup. Infra gets paged when GPUs go down. Frontier teams build slowly even on the fastest compute. SkyPilot turns your fragmented compute into one AI supercomputer, so you run frontier workloads faster. Many users manage 10,000+ GPUs across providers with SkyPilot. GPU hours consumption has grown 6x in the last 6 months. 1/ We're launching SkyPilot Platform — the AI compute platform for frontier AI teams to manage large GPU fleets and accelerate building custom intelligence. Optimized for fleet management, team governance, and frontier workloads — pretraining, post-training, multi-cluster serving, and sandboxes. SkyPilot open source users can switch to the platform with a server URL change. 2/ We've raised over $20M led by Lux Capital (Brandon Reeves), with participation from Amplify Partners (Mike Dauber, Lenny Pruss), Coatue Management, Foundation Capital (Ashu Garg, Jaya Gupta), Race Capital, The House Fund, and top operators like Ali Ghodsi (CEO, Databricks), Jeff Dean (Chief Scientist, Google), Guillermo Rauch (CEO, Vercel), Amjad Masad (CEO, Replit), Clem Delangue 🤗 (CEO, Hugging Face) and more. We're hiring across Engineering and GTM to deliver the platform for the next decade of AI. Above all, I'm excited to be building with the incredible team we've assembled, along with my cofounders Zhanghao, Romil, Scott, and Ion. If you firefight AI compute, let's build.
-
Ion Stoica reposted thisIon Stoica reposted thisEnterprises are moving from tokenmaxxing to valuemaxxing. A year ago, everyone wanted to use the largest, most expensive model for every single task. Today, organizations realize they can't burn budget like that. They want the best business outcome per dollar, which means having the freedom to route the right task to the right model, securely and cost-effectively. To help them do this, we are raising a strategic round of funding at a $188 billion valuation. We are going to use this capital to double down on the three pillars of our AI strategy: 1️⃣ Unity AI Gateway - our multi-AI governance solution that helps control costs. 2️⃣ Genie - our AI coworkers that actually understand your business data. 3️⃣ Lakebase - our serverless Postgres database specifically for AI agents. https://lnkd.in/gmr468PvDatabricks is Raising a Strategic Round of Funding at a $188 Billion ValuationDatabricks is Raising a Strategic Round of Funding at a $188 Billion Valuation
-
Ion Stoica reposted thisDay 0 and vLLM already runs Inkling at full feature parity. ⚡ We worked with Thinking Machines Lab to bring their 1T-parameter multimodal model to vLLM on launch day — text, image, and audio in, with up to 1M context. Performance: up to 380 tok/s/user with MTP on 4× GB200s. Getting there was a real lift — sconv-aware TP sharding, low-latency fused collectives, and direct integration of TML's new FA4 sheared-bias kernel. Huge shoutout to the team at Inferact that pulled this off on a Day-0 timeline. 🙌 Write-up + repro instructions 👇 https://lnkd.in/gp2h3D9dIon Stoica reposted thisToday, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. https://lnkd.in/gY4NvS5h Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. Cost and latency are important in real-world use cases. Inkling's continuous thinking effort lets you pick your point on the cost/performance curve — reaching the same score with a fraction of the tokens. Inkling natively understands and reasons across text, audio, and images. It’s strong on audio in particular, ranking among the strongest open-weights models on VoiceBench, MMAU, and AudioMC. We’re grateful to our partners for their day-0 support across the open-source ecosystem: NVIDIA, Together AI, Fireworks AI, Databricks, Unsloth AI, Modal, Baseten, LightSeek Foundation, Inferact on VLLM, RadixArk on SGLang. Inkling is the first in a family. We’ve included some details of Inkling-Small, a lighter-weight model trained on a similar recipe, with full weights to follow. We hope you enjoy Inkling, and as always we’re keen to see what you build.
-
Ion Stoica shared thisBringing a 27B-class model to a phone is an impressive achievement. Congratulations!Ion Stoica shared thisToday, we’re announcing Bonsai 27B: the first 27B-class model to run on a phone. A 27B model brings a meaningful capability tier: multi-step reasoning, structured tool use, multimodal understanding, and agentic workflows that stay coherent across many steps. Until now, that class of model has been impractical to run locally. Bonsai 27B, based on Qwen3.6 27B, changes that with two variants: • Ternary Bonsai 27B: 5.9 GB, optimized for laptop-class quality • 1-bit Bonsai 27B: 3.9 GB, optimized for phone-class footprint Across 15 benchmarks spanning reasoning, math, coding, tool use, instruction following, knowledge, and vision, Ternary Bonsai retains 95% of the full-precision baseline. The 1-bit variant retains 90%. This matters because modern AI workflows are increasingly loops, not single prompts. Agents may reason, call tools, read outputs, and update state hundreds of times. When that loop runs locally, cost, latency, and data movement stop compounding with every step. Private files and intermediate states can stay on-device. It also enables a more efficient hybrid attention architecture: run privacy-sensitive and non-frontier work locally, and reserve frontier cloud models for the hardest steps. Raw capability determines what a model can do. Intelligence density determines where it can do it. By intelligence density, 1-bit Bonsai 27B delivers more than 10x the capability per gigabyte of the full-precision baseline. Both models are available today under the Apache 2.0 license.
-
Ion Stoica reposted thisIon Stoica reposted thisFor the second year in a row, Gartner has recognized Databricks as the #1 Leader in the Magic Quadrant for AI Platforms for Data Science and Machine Learning. We were positioned highest in Ability to Execute and furthest in Completeness of Vision. This recognition validates our long-standing belief that you can’t have an AI strategy without a data strategy. And you can’t scale either without a governance strategy. Today, many companies are stitching together fragmented tools for data, models, agents, and governance. This doesn’t work. It’s time-intensive, clunky, and inconsistent. Databricks solves this problem by delivering one unified platform. That means one copy of your data, one governance layer across data and AI, and one consistent way to build, monitor, and control agents in production. By simplifying this complexity, we’re helping customers deliver AI outcomes faster. None of this would be possible without the relentless execution of the entire Databricks team, and the trust of our incredible customers and partners. Thank you to everyone who played a role! https://lnkd.in/eJ4iNR-q
-
Ion Stoica reposted thisIon Stoica reposted thisIt’s awesome to see Gartner recognize Databricks in TWO Magic Quadrants in the last month! This week we made our inaugural appearance in the MQ for Analytics and Business Intelligence Platforms. We landed as a Visionary, receiving the 2nd-highest Completeness of Vision score of all vendors. At the end of June, we were named the #1 Leader in the MQ for AI Platforms for Data Science and Machine Learning. We received the highest score in both Ability to Execute and Completeness of Vision. This is a huge validation for the Databricks unified platform and the incredible impact of conversational analytics with Genie. We are redefining what's possible for data and AI. Congratulations to our team, our partners, and our customers who made this happen. Let’s go! https://lnkd.in/gaX9nVa7
-
Ion Stoica reposted thisIon Stoica reposted thisWe benchmarked coding agents on our own internal tasks at Databricks and learned a lot! https://lnkd.in/gjBCkQpP There are many surprising opportunities to lower cost, and many models including open source are now competitive. Why did we build an internal coding benchmark? Public benchmarks like SWE-Bench are often over-tuned for, so we took real tasks our engineers did and curated test suites for them to see which agents can solve them end-to-end. These are the results on OUR sample of OUR codebase, so they are not meant to be comprehensive, but we think many companies can do a similar internal benchmark. Several findings immediately stood out: 1) Many models are now competitive at the top tier, including open source. 2) GLM 5.2 in particular was a major step forward in open source coding agent performance, even in our own codebase that is VERY different from SWE-Bench and TerminalBench (lots of Scala, Go, Rust, Java, TypeScript, Protobuf, Jsonnet, etc). 3) Harnesses make a huge difference in cost-performance. The very simple Pi harness got the same success rate as harnesses from the LLM vendors with Opus and GPT 5.5, but at 2x less cost! Seems to be mainly due to smaller inputs to the LLM. 4) Cheaper per-token does not imply cheaper per-task. For example, Sonnet 5 costs less per token than Opus 4.8 but used more tokens, resulting in higher cost and lower quality. Interestingly my ex-student Lingjiao Chen had also found this on other tasks: https://lnkd.in/gcVsRR3F. Read more in the blog how we built the benchmark and what we're doing with the findings. This is partly why we built Omnigent as a "meta-harness" to let developers switch and compose agents, and Unity AI Gateway to analyze and gate LLM usage centrally.
Ion Stoica commented on a post
4y
Michael Hoff sorry to see you leaving. Thank you for all your contributions over the years. It has been a real pleasure knowing you and working with you. All the best wishes to you and your family!
-
Ion Stoica liked thisIon Stoica liked thisI have a confession. I do all my real work in Genie Agents (One) from my phone now. I might have a slightly more advanced version in our internal deployment, but this thing cooks. Research and content development that I used to have to convince a team is SMEs to work on for weeks, Genie does in 2 minutes. For example, I wanted to build a system of alerts and REST triggers for the SQL Warehouse that monitors the system tables and triggers sizing events up or down. I call it the Warehouse Autosizer. But I also wanted to determine what the impact would be on the fleet and how valuable it would be to customers. This would have taken a team of SME Bricksters weeks to research and create. Genie did it in ~15 minutes of prompting, thinking, and tuning. I’m finishing up the blog and testing the code in some of our demo environments. What would you Genie if you could?
-
Ion Stoica liked thisIon Stoica liked thisToday, SkyPilot is out of stealth. Building custom intelligence is now existential. We help frontier AI teams build intelligence faster by removing their biggest bottleneck: AI compute fragmentation. Frontier teams like Applied Compute, Abridge, Hippocratic AI, H Company, and Nubank already run on SkyPilot, with 10x faster time-to-intelligence and double-digit increase in GPU utilization. AI teams today get compute anywhere they can. They then firefight compute fragmentation across providers. Researchers burn time on workload setup. Infra gets paged when GPUs go down. Frontier teams build slowly even on the fastest compute. SkyPilot turns your fragmented compute into one AI supercomputer, so you run frontier workloads faster. Many users manage 10,000+ GPUs across providers with SkyPilot. GPU hours consumption has grown 6x in the last 6 months. 1/ We're launching SkyPilot Platform — the AI compute platform for frontier AI teams to manage large GPU fleets and accelerate building custom intelligence. Optimized for fleet management, team governance, and frontier workloads — pretraining, post-training, multi-cluster serving, and sandboxes. SkyPilot open source users can switch to the platform with a server URL change. 2/ We've raised over $20M led by Lux Capital (Brandon Reeves), with participation from Amplify Partners (Mike Dauber, Lenny Pruss), Coatue Management, Foundation Capital (Ashu Garg, Jaya Gupta), Race Capital, The House Fund, and top operators like Ali Ghodsi (CEO, Databricks), Jeff Dean (Chief Scientist, Google), Guillermo Rauch (CEO, Vercel), Amjad Masad (CEO, Replit), Clem Delangue 🤗 (CEO, Hugging Face) and more. We're hiring across Engineering and GTM to deliver the platform for the next decade of AI. Above all, I'm excited to be building with the incredible team we've assembled, along with my cofounders Zhanghao, Romil, Scott, and Ion. If you firefight AI compute, let's build.
-
Ion Stoica liked thisDay 0 and vLLM already runs Inkling at full feature parity. ⚡ We worked with Thinking Machines Lab to bring their 1T-parameter multimodal model to vLLM on launch day — text, image, and audio in, with up to 1M context. Performance: up to 380 tok/s/user with MTP on 4× GB200s. Getting there was a real lift — sconv-aware TP sharding, low-latency fused collectives, and direct integration of TML's new FA4 sheared-bias kernel. Huge shoutout to the team at Inferact that pulled this off on a Day-0 timeline. 🙌 Write-up + repro instructions 👇 https://lnkd.in/gp2h3D9dIon Stoica liked thisToday, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. https://lnkd.in/gY4NvS5h Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. Cost and latency are important in real-world use cases. Inkling's continuous thinking effort lets you pick your point on the cost/performance curve — reaching the same score with a fraction of the tokens. Inkling natively understands and reasons across text, audio, and images. It’s strong on audio in particular, ranking among the strongest open-weights models on VoiceBench, MMAU, and AudioMC. We’re grateful to our partners for their day-0 support across the open-source ecosystem: NVIDIA, Together AI, Fireworks AI, Databricks, Unsloth AI, Modal, Baseten, LightSeek Foundation, Inferact on VLLM, RadixArk on SGLang. Inkling is the first in a family. We’ve included some details of Inkling-Small, a lighter-weight model trained on a similar recipe, with full weights to follow. We hope you enjoy Inkling, and as always we’re keen to see what you build.
Other similar profiles
Explore more posts
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content