Sri Satish Ambati

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Today, let's learn: - Function calling by building custom tools for Agents. - How MCPs help by building a local MCP client with mcp-use and using tools from Browserbase MCP server. In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point: MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable, without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. In this setup, to build a local MCP client, I used mcp-use because it lets us connect any LLM to MCP servers & build private MCP clients, unlike Claude/Cursor. - Compatible with Ollama & LangChain - Stream Agent output async - Built-in debugging mode, etc Find the mcp-use GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

269

11 Comments

Just getting started with LLMs? Don’t let the jargon slow you down. This 5-part LLM Glossary is your shortcut to mastering the fundamentals👇 1️⃣ Model Types • Foundation Model: Pretrained on large-scale, diverse datasets to learn general capabilities. • Instruction-Tuned Model: Further trained to follow user instructions precisely. • Multi-modal Model: Handles multiple input/output formats like text, images, and audio. • Reasoning Model: Optimized for logic, problem-solving, and step-by-step thinking. • Small Language Model (SLM): Lightweight models for fast, efficient, on-device tasks. 2️⃣ Training & Fine-Tuning • Pretraining: Initial learning phase using massive, diverse data sources. • RLHF (Reinforcement Learning with Human Feedback): Aligns model behavior with human preferences. • DPO (Direct Preference Optimization): Uses ranked preferences instead of rewards for training. • Synthetic Data: AI-generated data used to supplement real datasets. • Fine-Tuning: Targeted retraining on specific domains or tasks. • LoRA / QLoRA: Techniques for efficient low-resource fine-tuning. • Guardrails: Rules or filters applied to enforce safety, ethics, and compliance. 3️⃣ Prompt Engineering • System/User Prompts: Define model behavior and user context in a session. • Chain-of-Thought (CoT): Prompts that guide the model to reason step-by-step. • Few-Shot / Zero-Shot Learning: Demonstrating tasks with few or no examples. • Prompt Tuning: Optimizing prompts via training for specific outcomes. • Context Window: The total number of tokens the model can "remember" per interaction. 4️⃣ Inference & Generation • Temperature: Adjusts output randomness; lower values are more deterministic. • Max Tokens: Limits the number of tokens in a generated response. • Seed: Controls reproducibility of outputs. • Latency: Time taken by the model to respond. • Hallucination: When the model generates factually incorrect but plausible-sounding information. 5️⃣ Retrieval-Augmented Generation (RAG) • Retrieval: Fetching relevant data from external sources before generation. • Semantic Search: Search based on meaning, not just keywords. • Embeddings: Vector representations of text used for similarity matching. • Chunks: Dividing documents into smaller segments for better retrieval. • Vector Databases (VectorDBs): Stores embeddings for fast and accurate search. • Reranking: Rescoring retrieved documents to prioritize relevance. • Indexing: Structuring data for efficient retrieval during generation. 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources) channel on WhatsApp: https://lnkd.in/dCTCEKKc 📌Follow Habib Shaikh For more such content.

232

36 Comments

Stop chasing FLOPs. The real gains in AI inference are elsewhere. SemiAnalysis team and their InferenceMax report : The next leap in inference efficiency = hardware × software × orchestration Running Llama 3.3 70B FP8 across (AMD and NVIDIA) GPUs 1. It’s memory-bound, not compute-bound. H200 and B200 outperform older chips because bandwidth and interconnects matter more than peak FLOPs. 2. Software doubles the value of hardware. TensorRT + FP8 nearly doubles throughput—stack optimization now equals silicon innovation. 3. Serving is the new bottleneck. Multi-user loads flatten curves; smarter batching and schedulers are the next frontier..

270

8 Comments

Among other things, my team has been focusing on AI research for agents. We just blogged about a recent project: the Test-Time Diffusion Deep Researcher (TTD-DR). This framework uses a Deep Research agent to draft and revise its own drafts using high-quality retrieved information. This approach achieves new state-of-the-art results in writing long-form research reports and completing complex reasoning tasks. The work achieves better results than any other deep research agent we compared to. It also shows that self-evaluation and self-improvement are must-have components of a successful agent. https://lnkd.in/gBZpVMQy

297

8 Comments

Very impressive results by HSBC and IBM on the potential value of current quantum computers for solving real-world problems in algorithmic bond trading. HSBC press release: https://lnkd.in/eyQQ7jE7 Philip Intallura Ph.D , HSBC Group Head of Quantum Technologies, said: “This is a ground-breaking world-first in bond trading. It means we now have a tangible example of how today’s quantum computers could solve a real-world business problem at scale and offer a competitive edge, which will only continue to grow as quantum computers advance.“ “We have been relentlessly focused on the near-term application of quantum technology, and given the trial delivered positive results on current quantum computing hardware, we have great confidence we are on the cusp of a new frontier of computing in financial services, rather than something that is far away in the future.” https://lnkd.in/echEDmvN

74

3 Comments

Having worked on Agentic systems and open source multi agent deployments for last 2.5+ years (Starting with Langchain and moving to Langgraph, Autogen, Google ADK and many others) two common theme that has come across from deploying Agentic solutions and building using Agentic ecosystems in large scale production systems are - Theme 1 -> Agentic AI is shifting from experiment to enterprise backbone—but its ROI hinges on mastering cost containment and correctness at scale. In multi‑agent ecosystems, uncontrolled reasoning calls lead to spiraling spend with unpredictable accuracy (hence its very important to balance the choice of models, fault tolerant modes and accuracy metrics being defined both at Agent, Tool and Orchestrator levels). The winning CIO strategy is layered orchestration: lightweight models for routing, specialized LLMs for high‑value reasoning, and evaluator agents for correctness. This approach converts raw compute consumption into governed, auditable AI “throughput,” reshaping AI from an R&D cost center into a measurable productivity engine. Theme 2- > Standards like Google’s ADK and the Agent‑to‑Agent (A2A) protocol now provide the backbone for this transformation. By enabling a common language for agent communication, context‑sharing, and validation, they allow enterprises to modularize their AI stacks—aligning model choice with both business logic and budget. The value add: reduced infrastructure burn, accelerated time‑to‑market, and AI systems with built‑in accountability. For CIOs, this means a direct path to higher ROI curves where innovation can also take into account the true business value and cost of running these multi-agentic systems in production. #AgentAI #AgentOrchestration #CIOInsights #AIfrastructure #ADK #A2AProtocol

171

12 Comments

Investment is moving from pre-training to post-training. Early LLM budgets focused on training ever-larger base models. Post-deployment, models were mostly fixed... improvements came via scaling next-gen models. Now labs split resources between pre-training and RL fine-tuning with careful inference optimization. In post-training, models learn reasoning, follow prefs and refine outputs for specific tasks. Models update more frequently, weekly or monthly vs yearly. Each iteration collects new data, extends RL training and adjusts rewards to improve reasoning. New economic models charge for updated reasoning engines vs static models. Serving costs and latency are key so providers favor efficient pro models with most accuracy at lower compute. 3 more takeaways from ICML: 1️⃣ RL is only beginning to cross into “unverifiable” domains. Traditional RL used tasks with clear auto checks (like code compilers or math calcs) for rewards. Domains like Math Olympiad or legal arguments have complex solutions that can't be auto-verified. These need more complex reward models scoring reasoning steps, argument clarity or persuasiveness vs just correctness. With solid reward models, RL could teach coherent proofs, experiment design or legal strategies. This is still early work - current systems often break tasks into verifiable parts or use human prefs, but broader paths are emerging. 2️⃣ Personalization is a safer near-term goal than true continual learning. Interest in adapting models to users is growing with 2 key approaches. Personalization tailors outputs by learning user-specific rewards from few feedbacks or adding curiosity rewards that prompt questions about user tastes. These adjust behavior without changing model weights, improving perceived helpfulness and empathy. Continual learning updates parameters in real time from all user input. But continual learning poses safety and privacy risks like overfitting, bias amp & data leakage. Personalization and context windows (remembering interactions without weight changes) are more practical and responsible. 3️⃣ RL scaling will follow multiple paths. → One path applies current RL methods (PPO, guided reward models, diffusion reg) to more domains. This incremental path needs no new algos, just better rewards, bigger varied datasets & optimization tweaks. → The second tackles sparse-reward probs with long feedback delays. Success depends on credit assignment in long seqs and off-policy learning using varied experiences. Off-policy RL supports multi-datacenter training with clusters handling acting, collection and learning. → The third involves continual learning with frequent updates using user feedback and new data. Each has trade-offs: incremental and safe, risky but domain-expanding, adaptive but complex w/ safety issues. What'd I miss?

50

5 Comments

Found an interesting paper today. AI agent LLMs need robust tool evaluation. Current benchmarks struggle with diverse MCP tools, complex parameter reasoning, varied API responses, and accounting for real-world tool success rates. MCPToolBench++ addresses this: A large-scale, multi-domain benchmark for AI agent MCP tool use. It leverages over 4,000 MCP servers from 40+ categories, featuring both single and challenging multi-step questions. Data generation uses an automated pipeline, including tool sampling, query generation with "Code Dictionaries" for specific inputs, and rigorous validation steps. Evaluation uses two key metrics: 1. Abstract Syntax Tree (AST) Score for static call accuracy, and 2. Pass@K Accuracy for actual tool execution success. A critical finding is that AST and Pass@K rankings often diverge. This means a model might correctly infer the tool and parameters (high AST) but fail during real-world execution (low Pass@K) due to factors like inconsistent tool success rates or parameter errors. Root cause analysis reveals common failures like "Parameter Errors," "API Error," and domain-specific issues (e.g., invalid map coordinates). MCPToolBench++ is crucial for developing more reliable AI agents. Arxiv link: https://lnkd.in/g6fEBM9D #AI #LLMs #AIAgents #MCP #Benchmark #ToolUse

89

3 Comments

Strong start yesterday — great mix of Datacolor AI engineers, UTD students, and AWS/Informatica mentors diving deep into what Agentic AI really means at an enterprise scale. We spent the day iterating on technical patterns, how to build multi-agent orchestration pipelines ,use AWS Bedrock, Claude, and LangGraph, while linking to Informatica’s metadata and data-governance APIs for trusted, context-aware execution. Students from The University of Texas at Dallas brought serious curiosity — some experimenting with the Model Context Protocol (MCP) runtime to connect agents to real business systems For us, this hackathon isn’t about demos — it’s about giving back: sharing frameworks, mentoring students, and helping build the next generation of AI engineers who understand both the model and the system.

71

5 Comments

We're introducing Stax, an experimental developer tool to streamline LLM evaluation with custom and pre-built autoraters. Stax is an AI evaluation platform that allows you to assess and benchmark generative AI models and applications against your specific needs and criteria. In contrast to general leaderboards, Stax helps you understand how an LLM or prompt will perform for your use cases. Testing AI is challenging due to the absence of a single "right" answer and it's non-deterministic nature. Stax provides a platform to measure the quality of AI-generated content, ensuring it aligns with your unique standards and criteria. With Stax, you can: ⚡ Experiment Rapidly: Use the "Quick Compare" feature to test different prompts across various models. 🌐 Evaluate at Scale: Utilize "Projects & Datasets" to run extensive evaluations on large sets of data. ✅ Define Your Own Criteria: Create custom evaluators tailored to your specific use case or use pre-built system evaluators for common criteria like fluency, groundedness, and safety. 📈 Analyze Performance: Visualize evaluation results and performance metrics on the Analytics page to gain insights into model behavior. Go try it now: https://lnkd.in/e_YvmrHC

54

2 Comments

This article is the BEST introduction to Deep Learning 👇 Transformers are foundational for any AI/ML engineer today. They power ChatGPT, GPT-4, BERT, and every major language model. Brandon Rohrer's "Transformers from Scratch" is the perfect starting point: • Starts with basic math, builds to GPT architecture • One-hot encoding → embeddings → attention → multi-head attention → full transformer • Uses simple 3-word vocabularies, then scales to actual transformers In the article, he shows you exactly: → Why attention beats RNNs → How ChatGPT generates text → Multi-head attention mechanics → Skip connections and normalization This isn't a quick tutorial. It's a proper deep dive that actually explains how transformers work. Worth every minute. Link to the article: https://lnkd.in/g22bUWmv -- ⭐ Bonus for non-native English engineers: Fluently - AI English speaking coach → https://GetFluently.app/ -- ♻️ Find it useful? Leave a reaction and share with your network

149

9 Comments

🚀 New blog post: How we're using multiple LLMs (Gemini, Claude, ChatGPT & more) to revolutionize software performance optimization at Fastcode ✨ Feeding academic papers + source code into AI models to uncover optimization opportunities that would take months to find manually 📈 Results: ~67% of AI recommendations lead to measurable performance gains 🔗 https://bit.ly/45WGw6u #AI #PerformanceEngineering #LLMs #SoftwareOptimization #Fastcode

65

3 Comments