Nikita Katyal

Why LLMs are unreliable arithmetic engines A recurring assumption I encounter among decision-makers is that one can "provide the data to the model and let it derive the measures." It is an understandable position — but it conflates two distinct capabilities: linguistic fluency and computational correctness. A large language model is a next-token predictor. Its training objective is to generate the most probable continuation of a sequence, not to execute a mathematical operation. Arithmetic, under this objective, is imitated from patterns observed in training data rather than computed. Two mechanisms explain the resulting fragility: 1. Tokenization. Numbers are not encoded as numbers. A value such as 47,829 is segmented into sub-word tokens, and that segmentation is frequently inconsistent across inputs. The model therefore lacks a stable, digit-aligned representation to operate on. 2. Absence of an algorithm. There is no internal carry-and-place procedure. The model matches against previously observed sums rather than computing them, and accuracy degrades systematically as operands grow and as reasoning chains lengthen — the regions where training coverage is sparsest. This compositional limitation is documented in the literature (Dziri et al., "Faith and Fate," 2023), and subsequent work on number representation and length generalization corroborates it. For many applications, this approximation is tolerable. In banking and financial services, it is not. A marginal error in an exposure, a balance, or a risk feature can propagate into reconciliation breaks, regulatory findings, and audit exceptions. The difficulty is compounded by calibration: an incorrect result is presented with the same confidence as a correct one. The appropriate response is not to abandon LLMs but to bound their role. Material quantities should be computed in deterministic code (SQL, pandas, Spark); calculation should be delegated to tool-calling or code execution; and the model should be reserved for interpreting intent, routing tasks, and explaining outputs — not for deriving metrics internally. LLMs are exceptional language models and unreliable arithmetic engines. Sound system design begins with that distinction. I would be interested to hear how others are formalizing this boundary between model-mediated reasoning and deterministic computation in their pipelines. #AIEngineering #DataScience #BankingAI #LLMs #MLOps Jagadeesh Jagarlamudi

17

1 ความคิดเห็น

DX-002 — Where Exponential Disruption Begins In DX-001, we explored why Digital Transformation happens. Now let’s examine how it begins. Exponential disruption does not emerge from strategy offsites. It begins with Physics. The first force is Moore’s Law — compute doubles, cost halves. In the 1990s, serious computing power was reserved for governments and large enterprises. Today, a small team can access infrastructure once available only to superpowers. Cheap compute → Cheap intelligence. Cheap intelligence → New business models. New business models → Industry restructuring. When intelligence becomes abundant, incumbency stops being protection. Adaptability becomes the moat. This is the first structural driver of exponential change. Two more physics laws next. #DX002 #DigitalTransformation #AI #ExponentialChange #DXSeries #DXByNilesh

33

1 ความคิดเห็น

🚀 AI Economics TODAY (2025) — India Cost Snapshot (₹90 = 1 USD) 1) GPU & Compute Costs (2025) (Based on realistic spot + on-demand cloud pricing) • ⚡ Spot / discounted GPU hour: ₹180 – ₹360 • ⚡ On-demand / dedicated GPU hour: ₹360 – ₹720 • 🏭 Small GPU cluster (4–8 GPUs): ₹30L – ₹1.8Cr • 🔌 Train a 7B–13B model: ₹15L – ₹1.5Cr • 🔥 Train a 70B+ model: ₹5Cr – ₹20Cr 2) AI API & SaaS Costs (2025) • 🧠 LLM API (1M tokens): ₹90 – ₹270 • 📈 SaaS AI tools: ₹1,000 – ₹2,500 per user/month • 👁️ Vision Embeddings: Rs 2 – ₹15 per 1,000 requests (depending on model + platform) • Vision (OCR, Image Classification, Read API): ₹90 – ₹180 per 1,000 requests • 🔎 Search + RAG queries: ₹9 – ₹36 per 1,000 queries 3) Infrastructure, Data & Ops (2025) • 📦 Storage + pipelines + backups: ₹20k – ₹3L per month • ⚙️ Enterprise data & compliance overhead: ₹1L – ₹10L per month • 🌐 Cloud egress, networking & redundancy: ₹20k – ₹2L per month 4) Talent & Development Costs (2025) • 👨‍💻 AI Engineer salary: ₹18L – ₹63L per year • 📊 Data Scientist: ₹10L – ₹45L per year • 🧠 LLM/AI Architect: ₹36L – ₹90L+ per year • �� AI tool development (simple): ₹7.2L – ₹22.5L • 🏢 SMB AI systems: ₹31.5L – ₹1.08Cr 5) What This Means for India (2025) • ⭐ API-first AI is affordable for MSMEs • ⭐ Spot GPUs make experimentation cheap • ❗ Production-grade AI infra still expensive • ❗ Training large models requires capital + optimization • 🚀 India benefits from lower talent cost + rising compute availability AI Today = High Infrastructure Cost, Low Experimentation Cost. This is the perfect window for India’s builders to scale prototypes, test agents, and prepare for a 2030 world where AI becomes dramatically cheaper. #AI #AICosts #AIEconomics #AIIndia #TechStrategy #CloudComputing #GPU #DigitalTransformation #AIAgents #MachineLearning #iDataTool

Happy to share that our paper, "NEUTAG: Graph Transformer for Attributed Graphs," has been published in Transactions on Machine Learning Research (TMLR) under the guidance of Prof. Srikanta Bedathur and Prof. Sayan Ranu. This marks the last publication of my thesis. The central idea behind NEUTAG is simple: instead of treating node features merely as attributes, we promote them to first-class entities in the graph as new nodes. This facilitates a new theoretical construct where information can flow through both structural and feature-induced connections, enabling transformer-like long-range interactions without explicit all-pair attention. And since most real-world systems can be modeled as graphs, NEUTAG is our attempt to rethink the graph itself. For more details: https://lnkd.in/gxBj6Qby

113

10 ความคิดเห็น

PART 3: 🤖 Introducing SLMs: The Missing Piece in NLP + Agentic AI Systems As enterprises begin adopting NLP-powered agents — from analytics agents to CX automation — one trend is becoming clear: Not every task needs a giant LLM. Sometimes, smaller, specialized language models (SLMs) deliver faster, cheaper, and more controlled performance. This hybrid model allows your analytics agents to be: 📝​Intelligent (LLM): Capable of complex, human-like reasoning and generating articulate final reports. 🦾​Efficient (SLM): Capable of rapid, cost-effective, and highly accurate execution of numerous specialized, high-volume tasks. And this is where the next evolution of agent systems begins. --- ⚡ What Are SLMs? SLMs (Small Language Models) are compact, efficient models trained for domain-specific tasks or optimized for edge, on-prem, or cost-sensitive environments. Examples include sentiment classification, intent detection, text clustering, entity extraction, etc. They are the “micro-engines” inside your broader agent ecosystem. --- 🔗 How SLMs Supercharge NLP Agents # 1. Cost-Efficient Reasoning Large LLMs handle complex reasoning, but SLMs perfect for high-frequency tasks like scanning thousands of customer messages daily. # 2. Speed for Real-Time Decisions # 3. Hybrid Agent Architecture Modern agents combine: LLM → for deep reasoning, planning, long-form generation & SLMs → for fast classification, routing, filtering This creates a two-layer intelligence system: SLMs filter → LLM reasons → Agent acts More accuracy, less cost. # 4. Better Data Privacy SLMs can be deployed on-premise or inside VPCs, making them ideal for where sensitive text data can’t leave the organization. # 5. Fine-Tuning Friendly SLMs can be fine-tuned on small datasets, this makes them sharper than giant LLMs for narrow tasks. --- 🔍 Example: Analytics Agent Powered by SLM + LLM Step 1 — SLM Filters Data * Classifies 50k feedback messages * Tags sentiment, urgency, theme clusters * Flags anomalies Step 2 — LLM Performs Deep Reasoning * Finds root causes * Summaries across segments * Correlates themes with metrics * Generates leadership insights Step 3 — Agent Executes Actions * Updates dashboards * Creates ops tickets * Sends alerts * Documents findings in memory --- ### Note: SLMs do the heavy lifting. LLMs do the heavy thinking. Agents do the execution. In short: 🧠LLM as the "Brain" (Controller) 🔧SLMs as the "Tools" (Specialists) #slm #agents #aiagents #trending #nlp #datascience #llm #hybrid #hybridslmllmagents

15

Role: AI Engineer CTC: 40+ LPA Question #12 / 24 ✅ In continuation of the 24-Part Series: DS/ML/AI Interview Questions I faced this year. Question: Your production model's performance has degraded over time. Why might immediately retraining the model be a bad idea? What would you investigate before deciding to retrain? CONCEPT: Imagine you trained a model 6 months ago. Initially -> Accuracy = 92% Today -> Accuracy = 81% The natural reaction is: Performance dropped → Retrain on newer data. But retraining assumes that the problem is caused by the model itself. What if the real issue lies somewhere else? Possible Causes: 1️⃣ Data Drift The incoming data distribution has changed. Example: The model was trained on customers from Region A. Now, most customers are coming from Region B. The model is seeing data it wasn't originally trained on. 2️⃣ Concept Drift The relationship between inputs and outputs has changed. Example: Customer behavior before and after a major economic event. The same features may no longer predict the same outcome. 3️⃣ Data Quality Issues A pipeline broke. A feature stopped populating correctly. Missing values increased. Retraining on bad data could actually make things worse. 4️⃣ Data Leakage Removal Sometimes a model performs worse because a previously leaky feature has been removed or corrected. The model appears worse, but is actually becoming more realistic. Retraining doesn't solve the underlying issue. 5️⃣ Label Issues The target labels themselves may have changed. If labels are delayed, incorrect, or inconsistently generated, retraining may simply learn from noisy labels. 🎯 Key Insight: Before deciding on retraining, we should first identify why the degradation occurred. Retraining can sometimes hide the problem instead of solving it. A useful framework: Performance Drop -> Investigate Drift -> Investigate Data Quality -> Investigate Labels -> Identify Root Cause -> Then Decide Whether Retraining Is Needed 💡 Interview Insight: This was a typical ML model-in-production question, with different possibilities for it failing. Please think more about production and deployment rather than just training and testing the model. Question #12 / 24 ✅ Next Question: AI Engineer role...Tomorrow, same time #AIEngineer #MachineLearning #MLOps #DataScience #ModelMonitoring #LLM #InterviewPreparation

80

2 ความคิดเห็น

Today I explored different chunking strategies for RAG systems, and it’s clear that choosing the right approach directly impacts retrieval quality. Here are the key strategies I understand, along with their trade-offs: 1. Character-based splitting ✔️ Pros: Simple to implement Works consistently across all text types Predictable chunk sizes ❌ Cons: Breaks context mid-sentence Poor semantic understanding Lower retrieval accuracy 2. Recursive character splitting ✔️ Pros: Preserves structure better than fixed splitting Flexible fallback mechanism (paragraph → sentence → word) Balanced chunk quality ❌ Cons: Still not fully semantic Requires tuning (chunk size, overlap) 3. Document-specific splitting ✔️ Pros: Leverages document structure (headings, sections, tables) Better contextual grouping Ideal for PDFs, markdown, structured docs ❌ Cons: Format-dependent Requires custom parsers per document type 4. Semantic chunking ✔️ Pros: Context-aware splitting Higher relevance in retrieval Improves embedding quality ❌ Cons: Computationally expensive Requires embeddings during preprocessing Slower pipeline 5. Agentic / intelligent chunking ✔️ Pros: Dynamic and context-driven Can adapt per use case Potentially best retrieval performance ❌ Cons: Complex to design Higher cost and latency Harder to debug One approach I’m currently exploring as well and that is Unstructured.io. It looks promising for handling multi-format documents (PDF, HTML, DOCX, etc.) and extracting structured elements before chunking. I am Curious about what chunking strategy has worked best in your RAG pipelines and why ? #AI #RAG #Backend #DataEngineering #LLM #SystemDesign

5

Quantization in Large Language Models (LLMs) As LLMs scale, intelligence is no longer the bottleneck — memory, latency, and cost are. Quantization turns out to be one of the most powerful (and misunderstood) techniques enabling real-world deployment of large models. In this blog, I break down: • Why LLMs are surprisingly quantization-friendly • INT8 vs INT4 vs NF4 — what actually works in practice • How modern architectures absorb low-bit noise • Where quantization fails and how to design around it Quantization is no longer an optimization step — it’s foundational infrastructure for production LLM systems. Link: https://lnkd.in/gqGje75G #LLMs #AIEngineering #Quantization #MachineLearning #GenAI #MLOps

9

2 ความคิดเห็น