Rajkumar D

After going through endless posts on linkedin about RAG and reading about in different tech blogs. I thought about actually implementing it locally for a not so traditional use case. Every year about 11 to 13 lakh people appear for Preliminary exam of UPSC civil services in india, which has a general studies paper of about 100 questions, out of which as much as 50% questions are based on current affairs or current events based on static portion of subjects like polity, economics, geography, sci/tech, environment. There are so many coachings producing tonnes of materials trying to cover all the current events that can be asked in the exam. I tried to test if UPSC 2025 prelims paper was solveable using one of those through a RAG pipeline. Some of the text after this is LLM generated because I am too lazy to type and format. The Problem Statement 🛑 Every year, UPSC aspirants read ~1,800 pages of "Current Affairs" monthly magazines (Vision, Forum, Vajiram). The burning question has always been: How many actual Prelims questions does this material truly cover? Instead of guessing, I built a data pipeline to find out. The Solution: Local RAG Pipeline 🛠️ I engineered a Retrieval-Augmented Generation (RAG) system to perform a semantic audit of 12 months of PDFs against the 47 current affairs questions from the 2025 Prelims. Ingestion: Parsed & chunked 1,500+ pages using PyMuPDF & LangChain, preserving metadata (page numbers) for citations. Vector Search: Used ChromaDB with BAAI/bge-m3 embeddings for semantic retrieval (catching concepts, not just keywords). Precision Reranking: Implemented a Cross-Encoder (ms-marco-MiniLM) to score relevance (0-10) and filter out noise. The Verdict (N=47 Questions) 📉 I tested three major institutes. Here is how their material held up against the actual exam: 🏛️ ForumIAS • Broad Match: 53.2% (25/47) • High Confidence: 31.9% (15/47) 🏛️ Vajiram • Broad Match: 48.9% (23/47) • High Confidence: 29.8% (14/47) 🏛️ VisionIAS • Broad Match: 40.4% (19/47) • High Confidence: 25.5% (12/47) The analysis document generated has each question of prelims given a match score , the relevant document name mentioned, the page number in document, and the text chunk it has significant match with it. Needless to say this wont be true test whether someone can actually solve the question if the person had read than chunk. (Disclaimer: This is a personal "toy project" built for educational purposes to test AI retrieval pipelines. The data presented is the result of an automated algorithm and may contain errors or inaccuracies. It is not a definitive scientific study or a commercial review of these institutes.) Attaching the github link if anyone wants to test it out on a different corpus (or make it better by attaching a smart LLM judge that actually solves the question) : https://lnkd.in/gKnsg83U #RAG #NLP #upsc

35

3 Comments

Why Cross-Validation Matters in Machine Learning (Explained with a Simple Analogy) In machine learning, hyperparameter tuning plays a crucial role in building accurate models. For example: Random Forest has ~7 hyperparameters XGBoost has ~12 hyperparameters RPART has just 1 hyperparameter In tree-based models, these hyperparameters include things like max depth, number of trees (n_estimators), max leaf nodes, etc. Once you start combining different values, you quickly end up with hundreds of possible combinations. 👉 The big question: How do we know which combination is the best? The answer is simple in principle: We build a model for every hyperparameter combination and test its performance. The combination that gives the highest accuracy becomes our final choice. But there’s a catch. Accuracy from a single train–test split can be misleading due to sample bias. That’s where cross-validation comes in. What is Cross-Validation (CV = 4)? The dataset is split into 4 parts: Train on parts 1, 2, 3 → test on part 4 Train on parts 2, 3, 4 → test on part 1 Train on parts 3, 4, 1 → test on part 2 Train on parts 4, 1, 2 → test on part 3 So each hyperparameter combination gets 4 chances to prove itself. The combination that performs best overall is selected. A Simple Analogy: UPSC IAS Exam 🎓 Think of: Each hyperparameter combination as a UPSC candidate Cross-validation as multiple exam attempts Giving only one attempt may be unfair—someone might just have a bad day. But after 4 attempts, the candidate who still comes out on top is truly the best. Similarly, if a hyperparameter combination performs consistently well across all CV folds—and no other combination beats it—even after multiple chances, that combination is genuinely superior. That’s why cross-validation helps us build robust, reliable machine learning models. 📘 If you enjoy learning ML & AI through simple stories and analogies, my upcoming book “AI Vidya” is scheduled to hit bookstores in 2026. Stay tuned! 🚀 #AIVidya #ML #AI #hyperparametertuning

35

2 Comments

I agree 👍 RAG is becoming a core requirement of AI engineering skills because every organization has a large amount of data distributed across different formats and systems. And they really want to build AI agents that can securely retrieve and use that information to speed up everyday workflows and decision making tasks.

1

The real argument isn’t data vs humans. It’s truth vs convenience. A dashboard that glows with 94 percent success is comforting, but comfort isn’t competence. When the human experience contradicts the metric, the metric is the one on trial. Data without lived reality becomes a decorative illusion: ✔️ Engagement numbers that mask frustration ✔️ Completion rates that hide empty compliance ✔️ Efficiency stats that ignore the unpaid labor patching system flaws If leaders celebrate the dashboard while teams quietly crumble, that isn’t transformation. That’s self-deception dressed in analytics. A successful system is one where the quantitative pulse and the qualitative heartbeat move in the same rhythm. Until they align, the dashboard is only reporting activity, not impact. Note: “Quantitative pulse” refers to the numbers: metrics, completion rates, dashboards, KPIs. It’s the measurable rhythm of a system that shows activity, output, and performance. “Qualitative heartbeat” refers to the human experience behind those numbers: how people feel, what challenges they face, whether the work makes sense, and whether the system actually helps or harms them.

7

Saying goodbye to my weekends for two months straight was a small price to pay for the "OG" of math/stats - Indian Statistical Institute(ISI), Kolkata. Balancing a lead role with weekend classes at India’s premier stats institute has been a trip. We covered the full gauntlet from Calculus foundations to GNNs, Causal ML, and ViTs. Moving from "it just works" to the first-principles math behind LLMs is already adding a new dimension to my projects. Turns out, the view is much better when you actually understand the engine! #DeepLearning #ISIKolkata #CausalML #DataScience #Maths #AI #WeekendGrind

30

6 Comments

Feature launch: What I look at beyond usage numbers Having spent 14 years in analytics focusing on product, I’ve often observed that when a feature is launched, we celebrate success by looking at usage and this is often the success metric for the launch. Having led many feature launches and post-launch analysis, I have realised that usage alone usually doesn’t give a lot of insight. True feature success is a combination of multiple things—usage being just one of them. Consider usage as a starting point. Not the signal. Usage only tells you that users have touched something. It doesn’t tell you whether it solved their problem. So what matters more? Whether user behaviour actually changes after they use it. To figure out if a feature actually delivers value, I look at usage alongside: Who is using it? Is it new users, power/key users, or accidental clicks? What changes after they use it? Do they come back more often? Do they complete a core workflow faster? Do they drop off less? What happens if they don’t use it? Comparing users who adopted the feature vs those who didn’t. Example : I once worked on a feature with 60% adoption conveying great usage numbers. But when we looked deeper, users who adopted it had the same retention and conversion rates as those who didn’t. The feature was discoverable but not valuable. Without looking beyond usage, we would have kept investing in the wrong thing. The metrics that matter: ∙ Usage (to understand discovery and adoption) ∙ Cohort retention post-adoption ∙ Funnel completion before vs after usage ∙ Time-to-value and repeat behavior Usage answers Is it discoverable? Outcomes answer Is it valuable? *For some features such as social sharing, data exports, keyboard shortcuts or accessibility tools, usage itself is the outcome. But for features meant to drive core value or retention like onboarding flows, recommendation engines, or collaboration tools, behaviour change is what actually tells you if it’s working. The next time you’re evaluating a feature launch, track both. But let outcomes drive your product decisions.

18

Why LLMs are unreliable arithmetic engines A recurring assumption I encounter among decision-makers is that one can "provide the data to the model and let it derive the measures." It is an understandable position — but it conflates two distinct capabilities: linguistic fluency and computational correctness. A large language model is a next-token predictor. Its training objective is to generate the most probable continuation of a sequence, not to execute a mathematical operation. Arithmetic, under this objective, is imitated from patterns observed in training data rather than computed. Two mechanisms explain the resulting fragility: 1. Tokenization. Numbers are not encoded as numbers. A value such as 47,829 is segmented into sub-word tokens, and that segmentation is frequently inconsistent across inputs. The model therefore lacks a stable, digit-aligned representation to operate on. 2. Absence of an algorithm. There is no internal carry-and-place procedure. The model matches against previously observed sums rather than computing them, and accuracy degrades systematically as operands grow and as reasoning chains lengthen — the regions where training coverage is sparsest. This compositional limitation is documented in the literature (Dziri et al., "Faith and Fate," 2023), and subsequent work on number representation and length generalization corroborates it. For many applications, this approximation is tolerable. In banking and financial services, it is not. A marginal error in an exposure, a balance, or a risk feature can propagate into reconciliation breaks, regulatory findings, and audit exceptions. The difficulty is compounded by calibration: an incorrect result is presented with the same confidence as a correct one. The appropriate response is not to abandon LLMs but to bound their role. Material quantities should be computed in deterministic code (SQL, pandas, Spark); calculation should be delegated to tool-calling or code execution; and the model should be reserved for interpreting intent, routing tasks, and explaining outputs — not for deriving metrics internally. LLMs are exceptional language models and unreliable arithmetic engines. Sound system design begins with that distinction. I would be interested to hear how others are formalizing this boundary between model-mediated reasoning and deterministic computation in their pipelines. #AIEngineering #DataScience #BankingAI #LLMs #MLOps Jagadeesh Jagarlamudi

17

1 Comment

If you can’t ship end to end without permission slips, you’re not a data scientist. You’re a meeting attendee with a laptop. Harsh, but accurate. Most “data science” roles today are: • writing tickets • waiting for access • waiting for approvals • waiting for infra • waiting for sign-off • presenting slides about work you didn’t ship Meanwhile, someone else is quietly: • pulling the data • building the pipeline • training the model • wrapping it in an API • monitoring it • shipping it • iterating That person becomes indispensable. The title doesn’t matter. Output does. If your week is blocked by dependency hell, stop calling it “collaboration”. It’s bureaucracy. And it will be automated or outsourced the second budgets tighten. Real data scientists ship. End to end. No permission slips.

59

11 Comments

So I built a small tool. It fetches real-time NAV data across AMCs and turns it into clean, analyzable datasets. Current version: • Tracks 600+ mutual funds • Calculates 1Y, 3Y, 5Y and since-inception returns • Exports structured data for analysis Stack: HTML • JavaScript • JSON APIs Data source: NAV feeds + MF APIs+Factsheet. Next version: • Fund rankings • Category filters • Risk metrics • A mutual fund recommendation engine based on investor surplus and allocation. This is Version 0. Building tools forces you to understand markets. Data → Insight → Better investing. #FinTech #MutualFunds #BuildInPublic #DataTools #Investing https://lnkd.in/dAmjzngq (click on fetch all funds or go for specific fund house)

79

5 Comments