Rohit Prasad

Here are 6 design patterns shaping the next generation of AI agents: ➤ Sequential Architecture Tasks are processed step-by-step using a defined flow. Great for predictable, structured automation. ➤ Tool-Enhanced Memory LLMs access external tools to recall, transform, and reason with long-term data. Think smarter context, better answers. ➤ Multi-Agent Collaboration Multiple agents work together — each with specific roles. From researchers to planners, they form a digital workforce. ➤ Shared Tool Interfaces Agents use common tools (e.g., browsers, code editors) to stay aligned. Enables coordination and parallel task handling. ➤ Hierarchical Agent Design Supervising agents delegate tasks to sub-agents. Allows scalable, modular execution. ➤ Human-in-the-Loop AI + Human judgment = best outcomes. Crucial for oversight, ethics, and decision quality. 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources) channel on WhatsApp: https://lnkd.in/dCTCEKKc 📌Follow Habib Shaikh For more such content.

197

33 Comments

In a new blog by Smallest AI, we talk about issues in large fused kernels, particularly problems with: 1. Dynamic Input Shapes 2. Synchronization and Deadlocks 3. L1 cache latency compounding 4. Cross SM coordination 5. Warp context bottlenecks Further, we segway into what would it take to build a sub 50ms(and lower) speech model. Link and author name in comments, let us know what you think!

135

8 Comments

Among other things, my team has been focusing on AI research for agents. We just blogged about a recent project: the Test-Time Diffusion Deep Researcher (TTD-DR). This framework uses a Deep Research agent to draft and revise its own drafts using high-quality retrieved information. This approach achieves new state-of-the-art results in writing long-form research reports and completing complex reasoning tasks. The work achieves better results than any other deep research agent we compared to. It also shows that self-evaluation and self-improvement are must-have components of a successful agent. https://lnkd.in/gBZpVMQy

276

8 Comments

Most AI agents don’t fail because they can’t do something. They fail because they can’t do it reliably. Today, spinning up an agent is easy. With LangGraph, CrewAI, LlamaIndex, or OpenAI’s Agent SDK, you can wire up tools, memory, and orchestration in hours. The real challenge? 𝗥𝗲𝗹𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆. → Can the agent consistently complete actions without stalling midway? → Can it pick the right tool for the task, not just a “good enough” one? → Can it avoid hallucinations, PII leaks, and unsafe decisions under pressure? → And most importantly, can you trust it enough to put it in front of users or customers? ✅ What you actually need is an agent you can trust end-to-end. And this is where the industry gap is huge: Gartner predicts 40% of agentic AI projects will be canceled by 2027 due to reliability issues. The teams that win treat reliability as a first-class metric, not an afterthought. They measure it the same way they track accuracy, latency, or throughput. That’s why I think Galileo’s Agent Reliability Platform is worth paying attention to. It introduces purpose-built metrics like: → 𝗧𝗼𝗼𝗹 𝗦𝗲𝗹𝗲𝗰𝘁𝗶𝗼𝗻 𝗤𝘂𝗮𝗹𝗶𝘁𝘆 → 𝗧𝗼𝗼𝗹 𝗘𝗿𝗿𝗼𝗿 𝗗𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻 → 𝗔𝗰𝘁𝗶𝗼𝗻 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗺𝗲𝗻𝘁 → 𝗔𝗰𝘁𝗶𝗼𝗻 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻 … and more The goal is to diagnose and benchmark reliability across real-world, domain-specific workflows to measure and improve agent performance. All designed to systematically benchmark and improve how agents perform in real-world workflows. 👉 If you’re serious about deploying agents in production, check it out here:  https://lnkd.in/dG23PS4P #GalileoPartner

321

66 Comments

🧠 𝗔𝗜 𝗿𝗮𝗻𝗱𝗼𝗺𝗻𝗲𝘀𝘀? 𝗠𝗮𝘆𝗯𝗲 𝗻𝗼𝘁 𝘀𝗼 𝗿𝗮𝗻𝗱𝗼𝗺 𝗮𝗳𝘁𝗲𝗿 𝗮𝗹𝗹... Kyrtin Atreides noticed something odd: Five major LLMs — ChatGPT, Claude, Gemini, Perplexity, Meta — were all asked the same simple prompt: “Guess a number between 1 and 50.” Every single one guessed… 27. Coincidence? Not likely. This isn’t a failure of intelligence — it’s a clear example of pattern recognition at work. These models aren’t reasoning like humans — they’re predicting the most statistically probable output based on their training data. And apparently, “27” shows up a lot in that context. 🔐 𝗪𝗵𝘆 𝗱𝗼𝗲𝘀 𝘁𝗵𝗶𝘀 𝗺𝗮𝘁𝘁𝗲𝗿? Because what looks like a harmless glitch could have serious implications — especially in cybersecurity. Predictable outputs can be exploited for social engineering, prompt injection, or more subtle vulnerabilities. 💡 It’s more than a fun AI quirk — it’s a real reminder: How much randomness can we actually expect from LLMs? 📢 𝗧𝗼 𝘁𝗵𝗲 𝘁𝗲𝗰𝗵 𝗮𝗻𝗱 𝗔𝗜 𝗰𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆: How should we address this? Do we need better prompt design, external randomness injection — or simply more awareness around predictable patterns?

127

46 Comments

Direct Preference Optimization (DPO) is simple to implement but complex to understand, which creates misconceptions about how it actually works… LLM Training Stages: LLMs are typically trained in four stages: 1. Pretraining 2. Supervised Finetuning (SFT) 3. Reinforcement Learning from Human Feedback (RLHF) 4. Reinforcement Learning from Verifiable Rewards (RLVR) For DPO, we focus on the RLHF stage, which aligns LLMs with human preferences (i.e., DPO is an alternative alignment approach) through three main steps: 1. Collect a human preference dataset. 2. Train a separate reward model (RM) on this dataset. 3. Finetune the LLM with RL (usually PPO), using the RM as the reward signal. This third step is usually online: the policy generates outputs, which the RM scores during training. RLHF aims to maximize the LLM's reward while minimizing deviation from a reference model (e.g., SFT), measured by KL divergence. Challenges of RLHF: Despite its effectiveness, PPO-based RLHF is hard to implement efficiently. In addition to being an online RL algorithm, it requires four LLM copies (policy, reference policy, RM, value function), which consumes significant GPU memory. Numerous implementation details also need tuning for optimal performance. Where does DPO fit in? DPO is an alternative alignment method. Instead of using RL and a separate RM, DPO optimizes the RLHF objective via basic gradient ascent, without explicit RL or reward modeling. This makes DPO much more stable and resource-efficient than PPO-based RLHF, lowering the barrier for LLM alignment research. How does DPO work? A common misconception is that DPO skips reward modeling entirely. In reality, DPO is built on reward modeling—it just uses an implicit reward. DPO reparameterizes the reward, deriving it directly from the policy itself rather than a separate model. We train our LLM to optimize this implicit reward over an offline preference dataset, similarly to an RM. The RLHF-optimal policy is indirectly derived from training this implicit RM! Where does DPO come from? To see how DPO relates to RLHF and RMs, we can derive its training loss from scratch as follows: 1. Start with the RLHF objective and find its closed-form optimum. 2. Rearrange this to express the implicit reward. 3. Plug this reward into the Bradley-Terry preference model. 4. Train the LLM to match this implicit preference model. DPO trains an implicit RM and recovers the RLHF-optimal policy (in closed-form) from this RM, which is why the DPO paper is titled “Your language model is secretly a reward model.

415

18 Comments

Not all large language models are created equal. The architecture behind an LLM determines how fast it runs, how well it scales, and where it shines. Here are 6 architectures every AI builder should know: ➤ 1. Encoder-Only → Focus: Deep understanding, embeddings → Trained on: Masked tokens → Ideal for: Search, classification Examples: BERT, RoBERTa ➤ 2. Decoder-Only → Focus: Next-token prediction → Trained on: Left-to-right context → Ideal for: Chat, code, agents Examples: GPT-4, Claude, LLaMA ➤ 3. Encoder-Decoder (Seq2Seq) → Focus: Input → Output mapping → Trained on: Paired data → Ideal for: Translation, summarization Examples: T5, BART ➤ 4. Mixture of Experts (MoE) → Focus: Sparse routing for scale → Trained on: Expert selection → Ideal for: High performance, lower cost Examples: GLaM, DeepSeek-V2 ➤ 5. State Space Models (SSMs) → Focus: Long context, fast compute → Trained on: State transitions → Ideal for: Large documents, low latency Example: Mamba ➤ 6. Hybrid Models → Focus: Best of all worlds → Trained on: Mixed paradigms → Ideal for: Balanced performance Examples: Jamba, Grok 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources)  channel on WhatsApp: https://lnkd.in/dCTCEKKc 📌Follow Habib Shaikh For more such content.

221

24 Comments

With growing investment in agentic AI, we need shared frameworks to set expectations and guide responsible progress. This paper (https://lnkd.in/gca56zaE) does exactly that, much like how staged autonomy levels helped advance self-driving when the problem was too complex to solve all at once. There are still open questions about how crisp the boundaries are between levels and how certification might work in practice, but these are exactly the kinds of challenges worth tackling now. A valuable step forward at the right time.

138

You see abstract AI agent architectures everywhere, But how to build them in practice? We often use terms like AI tool, chatbot, and #AIagent interchangeably. But underneath the surface, there’s a major architectural difference and it’s reshaping how we build and interact with #intelligentsystems. 🔴 𝗪𝗵𝗮𝘁 𝗶𝘁 i𝘀 𝗻𝗼𝘁: A standard #LLM pipeline is reactive. It takes input → generates a response → maybe uses a tool → and returns output. There’s no memory. No goal-directed behavior. No feedback loops. It’s powerful — but passive. 🔴 𝗪𝗵𝗮𝘁 𝗶𝘁 𝗶𝘀: #AIagents are software systems that use AI to pursue goals and complete tasks on behalf of users. They reason, plan, and act — with memory and autonomy. And they operate in a continuous loop: 1️⃣ Think – Process data and context 2️⃣ Plan – Decide how to achieve the goal 3️⃣ Act – Execute via tools, APIs, or interfaces 4️⃣ Reflect – Evaluate results and adapt This feedback loop makes agents adaptive, iterative, and capable of learning. It’s not just generating text — it’s thinking in steps, remembering context, choosing tools, and deciding what to do next. It has three key traits: 𝟭.𝗠𝗲𝗺𝗼𝗿𝘆 – Retains past actions and results to inform future steps 𝟮.𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻-𝗺𝗮𝗸𝗶𝗻𝗴 – Determines what to do, not just what to say 𝟯.𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 𝗹𝗼𝗼𝗽𝘀 – It learns and adapts within a single task or across multiple sessions 💡 This shift from static responses to autonomous reasoning is foundational in: • Autonomous coding agents • Workflow orchestrators • Multi-modal assistants • Task-solving copilots The age of prompt in → answer out is fading. We’re entering the era of goal in → strategy → action → result. And the difference between the two? It’s not just output. It’s autonomy. 𝗛𝗼𝘄 𝗔𝗴𝗲𝗻𝘁𝘀 𝗪𝗼𝗿𝗸: ➜ You delegate a task ➜ The agent takes autonomous action ➜ It connects to tools, APIs, or the web — uses memory, adapts to input ➜ You’re still in control — but it runs on its own Think of it as a smart intern that never sleeps — and keeps improving. 𝗧𝘆𝗽𝗲𝘀 𝗼𝗳 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁𝘀: Different agents, different strengths — just like any team: ➜ Simple Reflex Agents = rule-based triggers ➜ Model-Based = uses memory to guide decisions ➜ Goal-Based = acts with outcomes in mind ➜ Utility-Based = weighs options and tradeoffs ➜ Learning Agents = continuously improve You wouldn’t run a business with just one intern — same goes for agents. #𝗔𝗴𝗲𝗻𝘁𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲𝘀 : How you structure your agents matters just as much as what they can do: ➜ Single Agent = task-specific assistant ➜ Multi-Agent = agents coordinate and collaborate ➜ Human-Machine = agents work with humans in the loop And this is where most enterprises still struggle — not with the technology, but with governance, security, and trust. As this paradigm evolves, companies that understand and apply these concepts will redefine efficiency, automation, and intelligence at scale.

227

8 Comments

New research from Aymara tested 20 leading language models for real-world safety risks – from misinformation and impersonation to privacy concerns. Amazon Nova models were recognized among the top performers. It’s great validation of our team’s focus on building AI that’s both powerful and responsible. Safety isn’t a nice-to-have – it’s core to how we build.   It's still early days and there's plenty more to do, but proud of the progress and our team behind it. You can read the full report here: https://lnkd.in/eDi-kvN6

656

21 Comments

Why The AI Future is Agentic? A raw large language model has no persistence. Every prompt you send is processed in isolation, except for the temporary context window that lets it stay coherent within a single conversation. To turn an #LLM into an agent, you need memory, not just one kind, but five distinct types, each playing a specific role. LLMs don't remember past sessions, but #AIagents do. 1. Short-Term Memory (STM) - Keeps recent context so the agent can stay coherent in multi-turn conversations. - Think of it as your working memory that manages temporary interactions within a session. 2. Long-Term Memory (LTM) - Stores and retrieves knowledge across sessions, enabling true persistence over days, weeks, or years. - This is what allows agents to remember you and your preferences between conversations. 3. Episodic Memory - Logs past events, actions, and outcomes. - This lets agents "recall" what they've done before and learn from successes or mistakes, building experience over time. 4. Semantic Memory - Stores structured facts, concepts, and relationships for precise reasoning and knowledge retrieval. - This enables agents to maintain consistent understanding of the world. 5. Procedural Memory - Remembers how to perform tasks, from multi-step processes to automated workflows. - This allows agents to execute complex procedures reliably and consistently. The magic happens when these #memorysystems work together. The most powerful AI applications aren't just LLMs, they're agents with sophisticated memory systems that bridge the gap between stateless models and persistent, intelligent assistants. The following amazing tools making this possible: Mem0 for universal memory layers, Pinecone & Weaviate for vector storage, LangChain for orchestration, Neo4j for knowledge graphs, OpenAI Assistants API for integrated memory, LangGraph for multi-agent workflows.

59

1 Comment

Is there truly a gap in performance between online and offline RL training for LLMs? Here’s what the research says… TL;DR: There is a clear performance gap between online and offline RL algorithms, especially in large-scale LLM training. However, this gap can be minimized by incorporating on-policy samples into offline RL algorithms, creating an iterative or semi-online approach. RL for LLMs: A few variants of RL (i.e., RLHF and RLVR) are used for training LLMs. These techniques differ in how they derive a reward, but they both operate by generating completions for a set of prompts, computing rewards for these completions, and using those rewards to update the LLM’s parameters via an RL optimizer. RL optimizers: We usually use an (online) policy gradient-based RL optimizer (e.g., REINFORCE, PPO, or GRPO) to derive the policy update for an LLM. PPO-based RL has been the de facto choice in the past, but PPO: - Is memory intensive due to storing four copies of the model (policy, reference, reward and critic / value models). - Tends to be unstable and requires a lot of domain knowledge to set up correctly. Plus, online RL is difficult due to the complexity of generating on-policy completions during training. Instead, we could use a simpler offline algorithm like DPO, as well as a semi-online algorithm like iterative DPO or rejection sampling. (1) OOD data: In [1], the authors compare DPO with online RL algorithms, finding a clear bias in DPO. When DPO encounters out-of-distribution (OOD) or unseen responses, we see in [1] that the resulting LLM becomes biased toward these responses and behaves unpredictably. In contrast, PPO handles OOD data more robustly and learns a correct policy. We can mitigate these issues with DPO by performing iterative / semi-online DPO, where we: 1. Collect new, on-policy completions from the current LLM. 2. Score them with a reward model to create preference pairs. 3. Run DPO on the new data. 4. Repeat. (2) On-policy + negative gradient: Authors in [2] identify two key properties of alignment algorithms that lead to competitive performance: 1. On-policy sampling. 2. Negative gradients. Online RL algorithms naturally contain on-policy sampling. We can also incorporate on-policy samples into offline or RL-free algorithms by including on-policy samples in the training data. Negative gradients refer to the idea of pushing down the probability of “bad” completions. This naturally happens in PPO-based RL and DPO, but not in maximum likelihood algorithms like SFT. (3) DPO vs. PPO at scale: Authors in [3] perform an extensive comparison between DPO and PPO-based RLHF with large-scale LLMs (i.e., 70B parameters), whereas prior work mostly focuses on smaller LLMs in experimental settings. In this work, we see a clear gap between the performance of DPO and PPO at scale. However, authors find that the quality of training data still has a bigger impact on LLM performance than algorithm choice (i.e., online vs. offline).

151

6 Comments

I started to think about #LLM #memory long time ago, but probably more structurally from reading this early 2024 paper from Princeton: Cognitive Architectures for Language Agents(https://lnkd.in/gaM8mtKa). It outlines different types of memory system: procedural, semantic, episodic etc -- laying a foundation for so much that has come since. We see solutions like #Mem0 and #Zep, which have brought structured, graph-based memory to LLMs. This approach is a clear win, making temporal tracking possible, mimicking how our own brains work. Of course, graph is an enhancement on top of what we commonly use. The field is evolving at an incredible pace, with recent papers like #MemOS, #A-Mem, and #MemTree, pushing the boundaries even further. This makes me wonder: what's next for memory systems? One promising path is creating specialized memory for different types of knowledge, which mirrors human cognition. We all want memory that is universal applicable, personalized and that prioritizes the information we access most frequently. But what does this really mean underneath? The more I think about this, this future looks a lot like a new kind of "#GoogleSearch" — one that not only aggregates unstructured and structured data. The key difference, though, isn't just about reading, it is about continuous writing, reconciliation and consolidation . What if this engine can observe, learn, and write to the our own private data source? We all perceive and listen a lot more before expressing ourselves. Imagine an LLM that doesn't just browse the web as a read-only tool but actively learns and updates its own knowledge base. Building a "Google Search" is absolutely extremely hard, but imagine a new type of "knowledge engine" that put equal weights to "#search" and "#assimilation". That's a powerful next step.

33

Why #SmallLanguageModels are the real future? ⚡ Most agents route every task, no matter how trivial - through GPT-4 or Claude → That’s expensive, slow, and overkill. ⚡ NVIDIA’s new paper argues→ #SLMs models small enough to run on consumer hardware are faster, cheaper, and just as capable for most agentic tasks. ⚡ Prioritize SLMs for cost-effective deployment → Cut latency, energy use, and infra costs, especially for real-time or on-device inference. ⚡ Design modular agentic systems → Use SLMs for routine, narrow tasks; reserve #LLMs only for complex reasoning. ⚡ Leverage SLMs for rapid specialization → Fine-tune small models quickly for evolving use cases, enabling faster iteration and easier adaptation. ⚡Agentic workloads are rarely open-ended conversations. They’re structured, repetitive, and scoped: ✸ Summarize a doc. ✸ Extract a field. ✸ Call an API. ✸ Generate boilerplate code. For these, you don’t need a trillion-parameter giant. A well-tuned 2–7B model often does the job just as well or better. ⚡ Proof in Numbers: ✸ Phi-3 Small (7B) rivals 70B on reasoning + code. ✸ DeepSeek-R1-Distill (7B) beats Claude 3.5 + GPT-4o on reasoning. ✸ SmolLM2 (≤1.7B) matches 70B models from 2 years ago. ✸ Nemotron-H (2–9B) delivers 30B-level tool use at a fraction of compute. ✸ Toolformer (6.7B) outperforms GPT-3 (175B) by learning APIs. Smaller ≠ weaker. ⚡The Efficiency Edge: ✸ 10–30× cheaper to run. ✸ Lower energy use + faster response. ✸ Overnight fine-tuning with LoRA/QLoRA. ✸ More reliable with strict formats (JSON, XML, Python). ✸ Deployable locally for cost savings + data control. ⚡Smarter Architectures: Don’t build monolithic agents. ✸ Use SLMs by default. ✸ Call LLMs only when absolutely necessary. This modular approach is cheaper, more controllable, and easier to debug. ⚡Migration path (LLM → SLM): ✸ Log usage data. ✸ Cluster recurring tasks. ✸ Fine-tune SLMs. ✸ Replace LLM calls. ✸ Iterate + refine. ⚡Real-world Results: ✸ MetaGPT → 60% of LLM calls replaceable. ✸ Cradle → 70%. ✸ Open Operator → 40%. ⚡ Why hasn’t Flipped yet: ✸ Billions sunk into LLM infra. ✸ Benchmarks biased toward generalist models. ✸ Less hype for SLMs But none of these are technical blockers. ⚡ The Takeaway: ✸ The future of agents isn’t bigger models. ✸ It’s smarter architectures built on SLMs. ✸ SLMs give you speed, control, and affordability.

55

6 Comments

Memory in LLMs functions as the substrate that determines whether these systems remain parlor tricks or evolve into enduring collaborators. Too often, the conversation collapses into “context windows” or “retrieval hacks,” as if memory equates to more tokens or smarter indexes. True memory operates at a higher layer: the continuous negotiation between storage, relevance, temporal decay, and reasoning. Retrieval and windowing provide mechanical scaffolds that surface facts. Memory, properly conceived, drives dynamics. What deserves retention? What should expire? How does an assistant reconcile contradictions across time? When does a pattern carry more weight than an isolated fact? These challenges extend beyond clever engineering; they demand systems that reason about the validity of information as it shifts. A next-gen memory system advances beyond scale & search. It develops temporal awareness, validates knowledge against reality, and forgets with as much intention as it remembers. Assistants that only archive facts remain brittle. Assistants that balance storage, decay, and reasoning evolve into adaptive partners capable of growing alongside the world they inhabit. Read my latest take for a deeper dive into how I see the memory layer shaping: https://lnkd.in/e_g9am39

31

3 Comments