Sachin Kumar Yadav

Heading to PyTorch India (Bangalore) — excited about vLLM × NVIDIA inference Large-scale LLM inference has quietly become a systems problem, not a model problem. As PyTorch India approaches, I’m particularly excited about discussions around: • vLLM internals (PagedAttention, continuous batching) • GPU-aware inference scheduling • Memory bandwidth + KV-cache pressure • Where PyTorch abstractions end — and hardware realities begin With sponsors like NVIDIA, IBM, and Red Hat, this event sits right at the intersection of: PyTorch → vLLM → GPU kernels → production reliability. Questions I’m carrying into the event: 1.What breaks first in real production — scheduler, KV cache, or memory bandwidth? 2.Where do PyTorch users *overestimate* batching gains? 3. How far can GPU-native optimizations take us before architectural changes are unavoidable? Looking forward to learning, debating, and connecting with folks building inference at scale.

18

Having trouble with Retrieval systems ("R" in RAG Systems), mining GBs of text for the correct information chunk for output generation? You are not new! Given millions of chunks, finding the top 5 is a challenging problem (with ~99.995% ranking accuracy). Would an LLM check all the chunks and find the best ones, spending days doing so? The solution lies in building an efficient "Engineering solution" driven by an Agentic AI Multi-hierarchical Retrieval System !! Typical Solution would be -- Millions of chunks --> Organized by key information (Hierarchical Knowledge Graphs, etc.) --> Identifying key top 100-1000 relevant chunks --> Using Agentic AI to generate outcome. Benefits of this approach-- 1. Bring down the accuracy requirements for retrieval systems by better organizing data 2. Leverage LLMs to share some grunt work Costs of this approach-- 1. ~5-30 mins of processing time 2. ~$10-$100 of the cost of generation Is it worth it? Have you experienced this problem yet?

21

5 Comments

📌 Epoch.AI just raised the bar: FrontierMath OpenProblems Instead of stopping at FrontierMath Tier 5, Epoch.AI went all in and released FrontierMath OpenProblems — a benchmark where models are asked to solve open research problems in mathematics. These are not contest puzzles. These are problems that professional mathematicians have worked on — and failed to solve so far. To make evaluation feasible, the benchmark only includes problems where a proposed solution can be automatically verified (which heavily limits the pool). For now, there are 14 problems, but submissions are open, so the set may grow — hopefully mathematicians will contribute. What kind of math? The problems span multiple areas, but the pilot set leans toward: combinatorics number theory These fields contain more statements that can be checked automatically. Human difficulty estimates Contributing mathematicians were asked: how many researchers have worked on the problem how long it might take humans to solve Some are estimated at 3–10 years of effort 😅 Current state of AI So far, today’s frontier models solve none of the tasks. Good luck to OpenAI, Google, and everyone else — if a breakthrough happens here, it will be a real milestone. #AI #Math #FrontierMath #EpochAI #LLM #Research #Benchmark #AutomatedReasoning #NumberTheory #Combinatorics #AGI

1

Apparently, when you google RBF, the top result is Resting Bitch Face 💀 But in data science land — and especially for those of us working with Bayesian or kernel-based models — RBF stands for Radial Basis Function. And it’s way more charming than its acronym suggests. At its core, an RBF measures similarity between two points: if two points are close, it tends to 1, if they’re far apart, it drops toward 0. Classic example — Gaussian. That distance-based smoothness is what makes RBFs so useful for capturing subtle, non-linear patterns, you can even stack them. RBFs actually power a lot of what we do: 🔔Gaussian Processes and Bayesian modeling (check out PyMC for a great implementation) ➗SVMs where they’re the classic kernel (remember that kernel trick when you “lift” data into a high-dimensional space and get the dot product there for free?) 🕸️Even neural networks, where RBFs can act as hidden activations So next time your data refuses to behave linearly — give it an RBF. This little appreciation post is inspired by Wesley Boelrijk's Bayesian modeling talk at PyData Amsterdam. #DataScience #BayesianModeling #MachineLearning #PyMC #Halloween

267

12 Comments

Build Self-Learning Agents — No Fine-Tuning Required Fine-tuning isn’t always the answer. In our latest newsletter, Hari Ohm Prasath Rajagopal breaks down how you can build self-improving LLM agents without touching model training — using two simple but powerful patterns: 🔹 Dynamic Tooling: Let agents create and load new tools on the fly when existing ones aren’t enough. 🔹 Dynamic System Prompts: Allow prompts to evolve automatically based on real user or reviewer feedback. The key idea: Instead of expensive fine-tuning and slow human feedback loops, let agents learn from usage by evolving their tools and instructions in real time. He also walks through how this works in practice using Strands (AWS’s agentic framework), with minimal code and tight AWS integration. If you’re a software engineer building agentic systems and want: - Better domain accuracy - Faster iteration - Less ML complexity This one’s for you. 👉 Read the full article here: https://lnkd.in/dW3jtJ2W

16

1 Comment

I have built Kimi-K2-HyDe-RAG-Basics-of-Imagelogy, a retrieval-augmented generation pipeline that combines Moonshot AI's Kimi-k2 LLM, a 1 Trillion parameter model with Hypothetical Document Embeddings (HyDE) for accurate question answering from the official DVET Maharashtra textbook Basics of Imagelogy (Standard XI – Radiology Technician training). Current implementation highlights: Full offline pipeline: PDF extraction (PyPDF2), semantic chunking with overlap, 768-dim embeddings using BAAI/bge-base-en-v1.5 Clean output files: embeddings (.npy), metadata (.pkl with page/chunk info), summary report CPU-friendly — no GPU needed for embedding generation In progress: HyDE step: generate hypothetical answers with Kimi-k2 to boost retrieval quality Final RAG generation with retrieved textbook chunks + source citations The system is designed purely for educational and research use in paramedical/vocational training, covering radiation physics, X-ray interactions, radiation protection, USG, CT, MRI, nuclear medicine, and interventional radiology. Repository coming to GitHub soon. If you're working in medical education tech, vocational AI, RAG systems, or paramedical curriculum support, feel free to connect or discuss potential extensions. #RetrievalAugmentedGeneration #RAG #HyDE #MedicalEducation #VocationalTraining #Radiology #AIinHealthcare #Paramedical #HuggingFace #OpenSourceAI #KimiK2 #LLMs #LargeLanguageModels #MoonshotAI #HypotheticalDocumentEmbeddings #Embeddings #BGEEmbeddings #SentenceTransformers #VocationalAI #RadiologyTech #ParamedicalTraining #DVET #BasicsOfImagelogy #AIForEducation #OpenSourceProject

3

🎥 𝗙𝗿𝗼𝗺 𝘁𝗵𝗲 𝗢𝗔𝗜 𝗔𝗿𝗰𝗵𝗶𝘃𝗲𝘀 If you're building or scaling large models, understanding how GPUs actually work together is non-negotiable. In this clip, Suraj Subramanian breaks down the mental models behind distributed training, clearly and practically 🔹 Data Parallelism – Splitting data across GPUs to scale training efficiently 🔹 Model Parallelism – Distributing model components across devices 🔹 Pipeline Parallelism – Structuring execution like an assembly line for inference Most importantly, he explains that these paradigms are orthogonal i.e. they don’t replace each other. They work together. This is the kind of foundational clarity that separates surface-level AI knowledge from production-ready ML engineering. 👉 Register for OAI 2026: https://lnkd.in/dh4mkdje 👉 Volunteer: https://lnkd.in/ewRTnz5U 👉 Join our Discord: https://lnkd.in/eDqBHamU #DistributedTraining #MachineLearning #LLMTraining #MLOps #OptimizedAIConference #ProductionAI

2

***** Algorithm of Thought (AoT) ***** Most of us have tried clever prompts to get better answers from large language models. But there is a new way of thinking called Algorithm of Thought (AoT). Instead of only giving instructions and hoping the model figures it out, AoT teaches the model how to explore a problem space. It does not settle for the first answer; it searches, tests, and refines, similar to how a person reasons through a tough question. Why this matters: More accurate and reliable results Handles complex, multi-step problems better Makes LLMs feel smarter and more useful in real work This feels like the shift from guessing to thinking. The prompting may not be about writing the perfect question but about guiding the model’s reasoning itself. #AI #PromptEngineering #LLM #AlgorithmOfThought #MachineLearning

28

8 Comments

Wrote a blog explaining "LoRA Fine-tuning Efficiency Under Different Loss Functions." LoRA made fine-tuning cheap, but not always efficient. This post explores how loss functions like Cross-Entropy, Label Smoothing, and Focal Loss change convergence, stability, and generalization in lightweight LLMs. Read here: https://lnkd.in/gh_vm2kG

2