Antoni Baum

New course: Building Coding Agents with Tool Execution, taught by Tereza Tizkova and Fra Zuppichini from E2B. Most AI agents are limited to predefined function calls. This short course teaches you to build agents that write and execute code to accomplish tasks, accessing entire programming language ecosystems instead of being restricted to a fixed set of tools. You'll learn to run agent-generated code safely in sandboxed cloud environments that protect your systems from harmful operations. Skills you'll gain: - Build agents that write and execute code, manage files, and handle errors autonomously through feedback loops - Run agent code safely in E2B cloud sandboxes and understand tradeoffs between local, containerized, and cloud execution - Create a data analyst agent that explores visualizes data with Pandas - Create a full-stack agent that builds complete Next.js web applications Join and build agents that code their way through complex tasks: https://lnkd.in/gmbUriZf

3,477

138 Comments

This paper on the current state of AI Agents is worth reading. Main points: 1. Add memory to agents. 2. Build agents as loops, not pipelines. 3. Go for RL only after the Agent's behavior is reliable. 4. Specify which tool to use when (don't dump 50 tools into a prompt and hope for magic). 5. Multi-agent systems fail unless the roles of planner, executor, and critic are explicit. ------------ Some of these lessons feel obvious, but it's good to know that engineers from Meta, Amazon, and Google agree with each other. Cheers. https://lnkd.in/gncRy-9f

146

6 Comments

The full agenda for AI Dev 25 x NYC is ready. Developers from Google, AWS, Vercel, Groq, Mistral AI, SAP, and other exciting companies will share what they've learned building production AI systems. Here's what we'll cover: Agentic Architecture: When orchestration frameworks help versus when they accumulate errors. How model-driven agents and autonomous planning handle edge cases. Context Engineering: Why retrieval fails for complex reasoning tasks. How knowledge graphs connect information that vector search misses. Building memory systems that preserve relationships. Infrastructure: Where hardware, models, and applications create scaling bottlenecks. Semantic caching strategies that cut costs and latency. How inference speed enables better orchestration. Production Readiness: Moving from informal evaluation to systematic agent testing. Translating AI governance into engineering practice. Building under regulatory constraints. Tooling: MCP implementations that work. Context-rich code review systems. Working demos you can adapt for your applications. I'll share my perspective on where AI development is heading. Looking forward to seeing you there! https://lnkd.in/gMafG9aG

1,596

91 Comments

The complexity of PPO leads practitioners to avoid online RL in favor of RL-free or offline algorithms (e.g., DPO), but why not just use simpler versions of online RL? TL;DR: REINFORCE and RLOO have been shown to work well for training LLMs. And, they do not require a value model, which drastically reduces cost and complexity. Therefore, instead of completely avoiding online RL due to the overhead associated with PPO, we can just use algorithms that provide the benefits of online RL without the unnecessary complexity. Policy gradient: RL optimizers operate by computing the gradient of the RL objective (maximizing rewards) w.r.t. policy parameters—this is called the policy gradient—and performing gradient ascent. A basic policy gradient is easy to compute (see figure) but has high variance. So, we introduce some extra complexity to get stable gradients / training. Actor-critic: We can reduce variance by incorporating the advantage function (instead of the raw reward for a trajectory) into the policy gradient estimate. When we do this, the value function acts as a normalizer / baseline to the reward estimate for a state. PPO estimates advantage with an additional copy of the LLM—called a value model or critic—that is trained via an MSE loss to predict the final reward for every token in a sequence. The critic is on-policy (i.e., it depends on the current parameters of the policy during training) and is trained alongside the policy itself during RL. This is called an actor-critic framework. REINFORCE: Due to keeping and extra copy of the LLM in memory that is trained, actor-critic frameworks come with additional cost and complexity. As an alternative, we can use the much simpler REINFORCE algorithm, which estimates the value function via an average of rewards (either a moving average of all rewards or an average of rewards in a batch) throughout training. Estimating the value in this way is low cost and removes the need for a critic. REINFORCE does yield higher variance relative to actor-critic methods, but recent work has shown that this does not matter in the context of LLMs. Algorithms like PPO were developed in a prior generation of DeepRL research, where neural networks were primarily trained from scratch. In the LLM domain, we are finetuning pretrained models with a strong prior—unstable gradients are much less of a worry. For this reason, even simple online RL algorithms like REINFORCE work well for training LLMs despite their higher variance. RLOO: To reduce variance relative to REINFORCE, we can also use REINFORCE leave one out (RLOO). This algorithm reduces variance by sampling multiple completions for every prompt and estimating the value function as the average of rewards for other completions—excluding the completion for which the value is being estimated—to the same prompt.

230

6 Comments

"Just because AI can write your tests ...should it?" ⬆️ That's the the talk I gave today at PyBay 2025! Here's the 5 min version: LLMs are generalists, so they *can* write passing tests, but they don't follow best practices and use the optimal testing tools. The problems and solutions: 😢 Problem: LLM-written tests are often high in redundancy. ☺️ Solution: In pytest, use parametrize for parameter variation and fixtures for commonly used test data 😢 Problem: LLMs generate sample data that is overly simplistic and not reflective of the actual diversity of the human world, like for fake names ("John Doe") and addresses ("1 Main St"). ☺️ Solution: Use Faker() to generate real-world fake data 😢 Problem: LLM-written assertions often only do a partial check of returned data from an API endpoint, because the LLM doesn't know what it should look like. ☺️ Solution: Use snapshot testing with pytest-snapshot to capture the output from API endpoints, so that you'll know when *any* field changes and be effectively asserting on every field. 😢 Problem: LLM-written tests do not cover 100% of the code lines. ☺️ Solution: Use coverage to measure line coverage, and write tests for missing lines. 😢 Problem: Even 100% coverage does not mean that all edge cases have been explored! ☺️ Solution: Use hypothesis for property-based testing of programs, and schemathesis for property-based testing of API endpoints. In conclusion: Yes, you can use an AI to write tests, but if you do, you should set the LLM up for sucess! Once you decide what best practices and tools you want to use in your tests, write up a precise prompt with those guidelines, and the LLM can write better tests with higher coverage and less redundancy. Check the slides here, where I managed to interleave my love of native California bees: https://lnkd.in/gbzaYz3z I also gave out seeds at the very end to attendees- three fanny packs full! I'll post a link to recording on YouTube when available. It was a great conference as always, so glad I came back this year!

165

7 Comments

Context: I am being asked about it a lot by smart people. 📊 Why I chose C++23 over Rust for my DB25 SQL parser Just shared my technical reasoning with the Rust community - not to start a language war, but to discuss real-world trade-offs in high-performance systems. The context: DB25 achieves 20M+ tokens/sec with 4.5x SIMD speedup Key factors that tipped the scales: SIMD optimization maturity (SSE4.2/AVX2/AVX-512) Zero-copy arena allocation control Direct integration with existing SQL ecosystem Decades of C++ compiler vectorization What I admire about Rust: Memory safety by default Fearless concurrency Cargo package management Strong community focus on safety The best tool depends on your constraints. For DB25's extreme performance needs, C++23 was right. For my next project emphasizing safety over raw speed? Rust is absolutely on the table. Both languages solve real problems. Having multiple excellent systems languages makes us all better developers. What's your experience with language choice for performance-critical projects? #SystemsProgramming #CPlusPlus #RustLang #Performance #SQL #TechDecisions 📄 Full technical discussion: attached 🔗 Projects: https://lnkd.in/gqemydKY

64

1 Comment

The mystery of how LLMs actually work has been solved. Thanks to a new textbook "Foundations of Large Language Models" from Tong Xiao and Jingbo Zhu, I finally get it. And now, so will you. No PhD required. Just 3 minutes of your time. Here’s what you need to know 👇 It all starts with pre-training Instead of teaching a model to classify tweets or answer questions directly, we throw it a mountain of raw, unlabeled text. Then we let it learn language by itself. This is called self-supervised learning. There are 3 training styles: 1. Unsupervised: find patterns without labels 2. Supervised: learn from labeled tasks 3. Self-supervised: generate labels from text itself LLMs use the third. It's insanely effective. How does it work? Take the sentence: “the early bird catches the worm.” Mask some words: “the [MASK] bird catches the [MASK]” Ask the model to fill in the blanks. Boom learning without labels. This leads to 3 model types: 🔑 Encoder-only (like BERT): read & understand text 🌟 Decoder-only (like GPT): generate next word 📣 Encoder-decoder (like T5): input → output Each has superpowers: BERT: best for classification GPT: best for writing/generating T5: flexible for any text task Let’s go deeper. Decoder-only (GPT): Trained to predict the next token. “the cat sat on the [MASK]” ➔ “mat” This is causal language modeling. Encoder-only (BERT): Looks at full sentence, masks some words. Uses context on both sides. Better for understanding than generating. Encoder-decoder (T5): Treats everything as text-to-text. - Translation - Summarization - Sentiment analysis One model. Many jobs. After pre-training, then what? 🧃 Fine-tune it: train on specific tasks 🔗 Prompt it: craft clever inputs Prompting is why LLMs feel magical. Example: "I love this movie. Sentiment:" Model says: "positive" Add a few examples before, and it learns the pattern. That's in-context learning. Prompting strategies: - Chain of thought: "Let's think step by step..." - Self-reflection: critique and revise - Problem decomposition - Retrieval-Augmented Generation (RAG) It’s all possible because these models predict the next word really well. Over and over. At scale. But prediction isn't enough. We need alignment. How? - SFT (Supervised Fine-Tuning): learn from human-written responses - RLHF: optimize responses using human feedback This is how ChatGPT was aligned. Newer method: DPO (Direct Preference Optimization) Less brittle. More stable. Same goal: helpful, harmless, honest AI. Want to go deeper? Read the incredible textbook: "Foundations of Large Language Models" by Tong Xiao and Jingbo Zhu https://lnkd.in/e6uNeB4P This book changed how I understand LLMs. It might do the same for you. P.S. Learn how to: • Use ChatGPT, Claude, and other LLMs like a pro • Automate your workflow • Build & grow with AI That’s what 140,000+ people get from our free newsletter plus 7 free courses: https://lnkd.in/e2pmFDJ5

575

48 Comments

💡With so many new Open Source LLMs in the market, it’s time for us to learn how to run them locally Scenario: You want to try running a GLM 4.6 locally and test it out - but you don't have the hardware for it! Deploying large language models (LLMs) isn't just about running them — it’s about making sure they work well, even when resources are limited. As models move to edge devices or serverless environments, here’s the real question: How can we scale LLMs without depending on OpenAI or Anthropic? 🔹 Smaller hardware: Not everyone has a massive compute setup. 🔹 Performance: We need LLMs that run fast but don’t eat up all the power. 🔹 Scalability: How can we make sure these models scale without losing quality? This is where things like quantization and frameworks like GPTQ, AWQ, and GGUF come in — they make models smaller, faster, and more efficient, all open source. If you're curious to dive deeper into these concepts and how to apply them, check out my course on Maven, Agent Engineering Bootcamp: Developers Edition, where we explore these techniques in detail. Next cohort starts on Nov 1, 2025. Course link: https://bit.ly/47rGy7g

86

4 Comments

🧠 TOON - The Smarter, Lighter JSON Alternative for LLMs When I started experimenting with TOON (Token-Oriented Object Notation), it completely changed how I think about putting structured data into LLM prompts. If you’ve ever tried to include large JSON structures inside a prompt, you already know how painful it gets, too many tokens, too much duplication, and too many brackets. TOON solves exactly that problem. 🚀 What TOON Is: TOON stands for Token-Oriented Object Notation, a compact, human-readable way to represent structured data with dramatically fewer tokens than JSON or YAML. It was introduced publicly on November 2, 2025, in the article “TOON (Token-Oriented Object Notation) - The Smarter, Lighter JSON for LLMs” on dev.to. The concept and implementation are credited to Johann Schopplich and open-source contributors. TOON is designed for LLMs, not web APIs. It optimizes structure to reduce token overhead and make data easier for models to understand. 💡 Why JSON Doesn’t Work Well for LLMs JSON is great for machines, but terrible for token efficiency. Every bracket, comma, and repeated key becomes extra tokens in your prompt. For LLMs, that’s wasted space and worse, it can confuse the model when it sees the same key hundreds of times. I learned this first-hand while building an AI agent for a client. The data was small enough to fit in the model’s context window, so we didn’t need a RAG (Retrieval Augmented Generation) system. But the JSON format made the prompt bloated and noisy. The model started misreading fields and losing track of structure, even though everything fit. 🔄 Then I Tried YAML Switching to YAML was a small breakthrough. It removed a lot of quotes and braces, and the prompt became shorter. The token count dropped noticeably, and readability improved. But YAML still repeats keys for each object, so it only took me so far. ⚙️ Then Just Recebtly I Tried TOON That’s where TOON really stood out. By defining the structure once and removing all redundant syntax, TOON reduced my token count by nearly half. It meant I could include twice as much real data in the same context window and the model understood it more cleanly. No RAG. No data trimming. Just a smarter format for the same context space. The responses became more stable, less confused, and overall more accurate. And because tokens dropped, cost and latency dropped too. 📊 Quick Comparison Example data with two users: JSON: { “users”: [ { “id”: 1, “name”: “Alice” }, { “id”: 2, “name”: “Bob” } ] } YAML: users: • id: 1 name: Alice • id: 2 name: Bob TOON: users[2]{id,name}: 1,Alice 2,Bob It’s like turning your JSON into a lightweight table, perfect for LLMs that interpret structured data. #LLM #AI #PromptEngineering #MachineLearning #ArtificialIntelligence #AIDevelopment #OpenAI #JSON #YAML #TOON #DataEngineering #AICommunity #LLMOptimization #TokenEfficiency #AIAgents #DeveloperTools #AIEngineering #NLP #DataScience #Productivity #AIInsights

969

124 Comments

The real frustration of working with LLMs & Agents in production. (And no, it’s not just about “which model should I use?” 👇) Let’s say you’ve finally built the infra: ✅ VectorDBs in place ✅ Retrieval flows ready ✅ Tool orchestration wired in ✅ Monitoring set up Now comes the actual challenge: You’re at the mercy of your LLM. And the moment it hallucinates - everything breaks + lots of validation errors. Yes, you can use a bigger model, more context, better instructions… But here’s the catch: → Bigger model? Adds latency and costs. → More context? Again blows up latency and costs. → Better instructions? Still not 100% reliable. Adding structured output also adds to the latency. In production when you’re serving millions of requests it becomes a nightmare. So what do we do? Think: → How can we shift more logic away from the LLM? e.g. can we make atleast half of the contextual process a bit deterministic or rule driven? → How can we make systems more deterministic, predictable? And definitely you end up designing a hybrid system: 🔀 Deterministic rules & retrieval pipelines 🔁 Minimal LLM usage where it’s really needed Yes, it’s less “cool” than building a 10-agent system with memory and reflection. But it’s what actually ships. Also… let’s not even start on GDPR, audit logs, compliance, etc. 😅 Anyone has the same feeling here? -- ♻️ Repost to help others 🤗 ➕ Follow me - Shantanu for Production AI - ML - MLOps content and Career tips!

133

26 Comments

LLM APIs are not complete document parsers There’s been a lot of interest on how LLMs can replace existing OCR solutions altogether, and a baseline is screenshotting the page and feeding it to your favorite LLM API (OpenAI/Sonnet/Gemini). My argument is that 1) yes more traditional OCR solutions are dead because LLMs are really good, but 2) relying on LLM APIs on their own will not get you a complete parsing solution that you can deploy across every user. Here’s why:  💡 There’s a lot of table-stakes metadata that will be missing. This includes confidence scores - needed if you need a human-in-the-loop step to validate the confidence of a page. This also includes bounding boxes, text matches, and reasoning - needed for citations, and fine-grained segmentation information (e.g. separating out paragraphs/charts) for chunking and other heuristic processing. 💡There’s still a long-tail of accuracy issues. Screenshotting the page oftentimes misses out on relevant information directly found within the file binary. Even though models are getting much better (e.g. Gemin 2.5, Sonnet 4.0), there’s still areas where they fail. 💡You need to standardize the prompts - there’s a big opportunity cost. Any LLM is only as good as the prompt you give it. Inevitably as you handle more document types, you are going to need to tweak the prompt/architecture and standardize it so it works across more doc types, and every team can use it. At that point you’re basically building/maintaining your own parsing solution. 💡 Misc issues: Rate-limits, content filters, and reliability issues (e.g. when LLM is down) are all too common. Requires human effort to help maintain. Our point of view around LlamaCloud, our document processing solution, to wrap the latest LLM APIs so you get all their capabilities, but add on all the nice features above so you don’t have to build/maintain it yourself! Come check it out: https://lnkd.in/gPJjeZZP

252

19 Comments

Most production LLMs still run in bfloat16 (BF16). It halves FP32 memory while keeping full accuracy, which is why models from Llama-3 to DeepSeek default to it. FP8 can go lighter, but even “near-lossless” 8-bit quantization introduces drift that must be re-validated for every compliance-sensitive workload. DFloat11 (DF11) from xMAD.ai tackles the same memory wall without touching accuracy: • DF11 keeps the sign bit and all seven mantissa bits exactly as trained. • Only the 8-bit exponent stream is compressed. Because most weights fall between −1 and 1, just a few exponent patterns dominate. A Huffman coder replaces those patterns with shorter codes, cutting average storage to about 11 bits per weight. • Storage splits into two parts: an 8-bit array of sign-plus-mantissa and a packed exponent bitstream with a tiny header. A CUDA kernel inflates blocks inside GPU L2 right before each tensor is used, so nothing crosses PCIe. That delivers roughly 30 percent smaller checkpoints that reconstruct to bit-for-bit identical BF16 values. The saved VRAM keeps entire models on-GPU, eliminates off-load stalls, stretches context windows, and even lets a 405-billion-parameter Llama run on a single 8 × H100 node. Credits to my friend Oscar Wu and the xMAD.ai team for driving DF11 forward - and great to see the format already getting serious traction in enterprise pilots. Technical deep dive: https://xmad.ai/blog

84

1 Comment

🚀 Fast-dLLM v2: Parallel Block-Diffusion Decoding for LLMs ⚡️ Highlights 🌟   - Blockwise bidirectional context via complementary masks   - Hierarchical caches (block + sub-block)   - Parallel sub-block decoding + token-shift training Results 📊 - ~2.5× faster vs. standard AR decoding on A100   - 102.5 tok/s (bs=1), 201.0 tok/s (bs=4) — >2× Qwen2.5   - SOTA efficiency–quality trade-off among diffusion LLMs Paper/Code 🌐: https://lnkd.in/g3AhGp96

263

Just getting started with LLMs? Don’t let the jargon slow you down. This 5-part LLM Glossary is your shortcut to mastering the fundamentals👇 1️⃣ Model Types • Foundation Model: Pretrained on large-scale, diverse datasets to learn general capabilities. • Instruction-Tuned Model: Further trained to follow user instructions precisely. • Multi-modal Model: Handles multiple input/output formats like text, images, and audio. • Reasoning Model: Optimized for logic, problem-solving, and step-by-step thinking. • Small Language Model (SLM): Lightweight models for fast, efficient, on-device tasks. 2️⃣ Training & Fine-Tuning • Pretraining: Initial learning phase using massive, diverse data sources. • RLHF (Reinforcement Learning with Human Feedback): Aligns model behavior with human preferences. • DPO (Direct Preference Optimization): Uses ranked preferences instead of rewards for training. • Synthetic Data: AI-generated data used to supplement real datasets. • Fine-Tuning: Targeted retraining on specific domains or tasks. • LoRA / QLoRA: Techniques for efficient low-resource fine-tuning. • Guardrails: Rules or filters applied to enforce safety, ethics, and compliance. 3️⃣ Prompt Engineering • System/User Prompts: Define model behavior and user context in a session. • Chain-of-Thought (CoT): Prompts that guide the model to reason step-by-step. • Few-Shot / Zero-Shot Learning: Demonstrating tasks with few or no examples. • Prompt Tuning: Optimizing prompts via training for specific outcomes. • Context Window: The total number of tokens the model can "remember" per interaction. 4️⃣ Inference & Generation • Temperature: Adjusts output randomness; lower values are more deterministic. • Max Tokens: Limits the number of tokens in a generated response. • Seed: Controls reproducibility of outputs. • Latency: Time taken by the model to respond. • Hallucination: When the model generates factually incorrect but plausible-sounding information. 5️⃣ Retrieval-Augmented Generation (RAG) • Retrieval: Fetching relevant data from external sources before generation. • Semantic Search: Search based on meaning, not just keywords. • Embeddings: Vector representations of text used for similarity matching. • Chunks: Dividing documents into smaller segments for better retrieval. • Vector Databases (VectorDBs): Stores embeddings for fast and accurate search. • Reranking: Rescoring retrieved documents to prioritize relevance. • Indexing: Structuring data for efficient retrieval during generation. 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources) channel on WhatsApp: https://lnkd.in/dCTCEKKc 📌Follow Habib Shaikh For more such content.

232

36 Comments

New article: TimeXer for Time Series Forecasting with Exogenous Features TimeXer is a Transformer-based model, specifically built to take into account exogenous features. Thus, it does not only rely on past values of the series, but also on external information that may help forecast our target. This is especially useful with time series that are impacted by external factors. For example, the demand for a product can change if there is a holiday, or the demand for electricity is impacted by the temperature outside. In this article, we discover the architecture and inner workings of TimeXer and apply it in a small forecasting project using the available implementation in neuralforecast from Nixtla. Enjoy the read! #timeseries #forecasting #datascience #machinelearning #deeplearning #artificialintelligence #python #programming https://lnkd.in/eymfjmh8

90

4 Comments

There’s a new RL algorithm in town… and it’s already shaking up the LLM research community. 🤯 Group Sequence Policy Optimization (GSPO) was created by the research team at Qwen and is purpose built for operating at the sequence level rather than the token level. This means it evaluates, clips, and optimizes entire response sequences by their overall likelihood. The important upshot is that it directly aligns the optimization process with how rewards are actually assigned in real world scenarios! Some key reasons to be excited! 1. Better stability. GSPO improves training stability and helps avoids model collapse that can happen with algorithms like GRPO. 2. Scalable RL for mixture of experts (MoE) models. GSPO inherently solves stability challenges teams face when scaling RL for MoE architectures. Large scale RL becomes more practically feasible! 3. Simplifies Infra. Sequence-level optimization is inherently less sensitive to small numerical differences between training and inference systems compared to token-level methods, which should lead to more robust RL pipelines! We can already see signs that this has paid off in big ways for the open-source community. Qwen3 models are doing really well on benchmarks, boasting competitive performance with top models like Gemini-2.5 Pro on GPQA, AIME25, and LiveCodeBench! https://lnkd.in/gCZMQvUj P.S. 👀 If you want to run RL on your own Qwen3 model with GRPO/GSPO, keep an eye on our open-source post-building repo 🧡 : https://lnkd.in/gXZ32qkm #AI #LLM #ReinforcementLearning

83

10 Comments

🆕 The Ultimate Guide to Open-Source RL Libraries for LLMs: 9 Frameworks Compared 🎯 Three primary RL applications for LLMs: 1️⃣ RLHF for human preference alignment 2️⃣ reasoning models for problem-solving 3️⃣ multi-turn agentic interactions [via Philipp Moritz of Anyscale ] https://lnkd.in/gcF2dNGq

57

1 Comment

vLLM is one of the most popular LLM inference engines in production. However, with more and more optimizations being baked in, the codebase has become too complex to learn from. Fresh breather from DeepSeek team - nano-vllm - only 1200 lines of Python as of now. 🚀 Fast offline inference - Comparable inference speeds to vLLM 📖 Readable codebase - Clean implementation in ~ 1,200 lines of Python code ⚡ Optimization Suite - Prefix caching, Tensor Parallelism, Torch compilation, CUDA graph, etc. Some users even note *reduced* latency vs vLLM on A100 GPUs. Supports only offline inference atm. https://lnkd.in/gAXcF43w

42