Gautam Shende

Wide & Deep Network, DeepFM, Deep & Cross Network, Transformer, MaskNet, Multi-task Learning, User Sequence Modeling ~ What are all these architectures that have revolutionized the ranking system for modern-day recommendations? How has the ranking layer in recommendation systems at Pinterest evolved over time? Why have deep learning-based models outperformed tree-based models for ranking tasks in recommendation systems? 🎥 Check out the latest video where we cover all these questions. Video Link: youtu.be/WQj8QtjC3gA   Channel Link: youtube.com/@datatrek #datatrek #datascience #machinelearning #statistics #deeplearning #ai

6

Brian Armstrong, Coinbase CEO, explains why he fired engineers who didn’t try to use ai. i couldn’t agree more. last week i spoke to a product manager at one of india’s fastest growing ai startups. he told me about a problem his team had. it was so trivial, yet so annoying. they had no internal tool for it. instead of waiting months, he did this: asked chatgpt to draft a business requirement document (brd) got it approved used vibe coding to build a minimal mvp got help from engineers only when bugs appeared result? 16+ hours of work saved every single week. here’s the thing most people don’t get about ai: it doesn’t make you less talented. it helps you do the hard things faster. imagine you’re a non-tech person. you have an idea for a chatbot to help clients. traditionally: → convince the tech team → wait weeks for a “maybe” → hope the leadership buys it with ai: → turn your idea into a brd → pick the right vibe coding tool → build a prototype in days → test, iterate, prove it works ai isn’t cheating. it’s the fastest way to execute ideas. it’s about getting your hands dirty, not waiting for permission. let’s stop looking down on people who use ai. because the future belongs to those who do. #ai #productmanagement #vibecoding #innovation #execution

21

4 Comments

[Hugging Face Machine Learning Club India] The first episode is out! Here Lewis Tunstall takes us through the entire QED-Nano project. > Data curation > Rubrics creation > Reasoning cache as a scaffold > RL infra needed The last 5 mins is the heart (according to me) of the episode, where he opens excalidraw and creates diagrams to explain the meaning of "scaffolds".

276

9 Comments

For two decades, I have been wanting to read and understand “Primes is in P” paper (testing if an integer is prime can be done in polynomial time in the number of input bits) by Agrawal, Kayal, and Saxena, and finally, I managed to finish doing it. Let me share my understanding. The basic idea is that they wanted a test for primality that does not rely on randomness or unproven assumptions. Many earlier tests like Miller–Rabin were fast but still probabilistic. Their insight was to connect primality with a statement about polynomial identities. Specifically, they showed that an integer “n” is prime if and only if a certain congruence involving polynomials holds, namely that (x + a)^n is congruent to x^n + a modulo n for many small values of “a.” At first this looks like just a cute algebra trick, but it becomes the cornerstone of the whole algorithm. What they really did was take Fermat’s Little Theorem, which checks “n” by raising numbers to the “n" power, and extend it into the realm of polynomials, where structure is much richer and easier to test deterministically. The clever part is how they turned this characterization into something efficient. Just because a condition is equivalent to primality does not mean you can check it quickly. If you naïvely expanded (x + a)^n, the number of terms would explode. So they introduced a modulus not just by “n”, but also by a carefully chosen polynomial of low degree. This cut down the explosion and allowed the congruence to be checked within polynomial time. The trick is to pick the degree “r" of the polynomial modulus in such a way that it balances correctness and efficiency. They proved that such an “r” can also be found in polynomial time, and then the verification of the congruence for a limited range of a becomes manageable. The condition is that for all integers “a” up to a certain bound, the congruence still holds. This combination of reducing the polynomial and bounding the values of “a” makes the whole test run in deterministic polynomial time, which is the breakthrough. They showed that if “n” is prime, then the congruence always holds, which is the easier direction. The harder part is showing that if “n" is composite, then the congruence must fail for at least one small “a”, provided you have chosen “r" correctly. That proof mixes number theory and algebra, including the use of properties of “cyclotomic polynomials” and multiplicative orders. They essentially proved that if “n” is composite, then the polynomial identity cannot hide the factors forever. Once you see how they cleverly pieced it together, it feels like they pulled a very natural thread: they started from Fermat, lifted it to polynomials, contained the growth by reducing modulo another polynomial, and then proved that this containment cannot be faked by a composite number. The end result is a clean deterministic test that runs in polynomial time and settles one of the oldest open problems in computational number theory.

62

3 Comments

Sarvam AI just dropped two very interesting open-weight reasoning models. While everyone is waiting for DeepSeek V4, India just introduced two serious contenders: Sarvam 30B and Sarvam 105B. As a Data Scientist working on building AI agents, I’m less interested in hype and more interested in the engineering decisions behind these models. What stands out here isn’t just scaling - it’s thoughtful architectural optimization The Architectural Highlights 1️⃣ Multi-Head Latent Attention (MLA): The 105B model adopts MLA (popularized by DeepSeek-V2), which dramatically reduces the KV-cache bottleneck. Despite being nearly 4× larger than the 30B model, it maintains a surprisingly efficient memory footprint 2️⃣ Tokenizer Built for India: Sarvam trained a tokenizer from scratch, optimized for Indian languages, achieving ~4× higher token efficiency. In production systems, this directly translates to lower latency and lower API cost 3️⃣ Kernel-Level Optimizations: Beyond architecture, Sarvam implemented low-level kernel and code optimizations, reportedly delivering 20–40% higher throughput compared to comparable models like Qwen-3 Built for Agents: While many models focus on coding benchmarks, Sarvam performs surprisingly well on agentic reasoning benchmarks like Tau2, suggesting these models may be particularly useful for AI agents and task-completion systems, not just chat Deep Dive References: For the fellow nerds wanting to see the "why" behind the "what," check out these foundational papers: 1️⃣ DeepSeek-V2 (MLA Architecture): The blueprint for the 105B’s memory efficiency. (https://lnkd.in/g6txGckZ) 2️⃣ The GQA Paper: The logic behind the 30B’s balanced performance. (https://lnkd.in/g_XTk9FZ) 3️⃣ The Tau Bench: Why "agentic reasoning" is the new benchmark that actually matters. (https://lnkd.in/g627Qc5H) India is no longer just a consumer of AI - we are architecting its future #GenerativeAI #LLM #SarvamAI #AIArchitecture #DeepLearning #TechInnovation #IndiaAI

11

₹1,000 crore just moved to quantum startups. Most cities chase headcount. Hyderabad is building a new stack. Different bet. Different outcomes. Telangana isn't playing the same game as everyone else. Deputy CM Bhatti Vikramarka just declared Hyderabad India's quantum hub. The "Young India" startup fund backs it up with real money. This isn't typical government theater. It's systematic: • First state with a dedicated quantum strategy (TQS) • Complete value chain — from lab research to real applications • Aligned with NITI Aayog's quantum roadmap • Target: $3 trillion economy by 2047 As someone studying emerging tech, this hits different. Quantum computing will redefine how we solve problems in computing, communications, and cybersecurity. The academia-industry-government triangle creates real opportunities. Students and professionals can engage with cutting-edge research while building practical applications. Hyderabad already has digital infrastructure and skilled talent. Adding quantum-focused funding and policy support positions it as India's quantum research and commercialization center. For our generation entering tech, quantum represents the next frontier. Understanding its fundamentals now could define career trajectories in the coming decade. While other cities compete for headcount, Hyderabad is building the future stack. #QuantumComputing #TechCareers #Innovation 𝐒𝐨𝐮𝐫𝐜𝐞: https://lnkd.in/g-qmDFFb

4

1 Comment

Wrote an article on Byte Pair Encoding (subword Tokenization) and how it works which was potentially used in the the earlier versions of GPT / traditional Language Learning model paradigms. Took a sample of Hindi Language corpus text data to showcase the same. Article below - https://lnkd.in/gCu7Bgpf

33

Excited to share — we built GPT-OSS and Nano-GPT-OSS from scratch and opened the code. 🚀 One month ago I (Aman Murari Singh), Raj Abhijit Dandekar Sir and Naman Dwivedi set out to pre-train a truly open-source GPT-OSS — not just open weights, but the entire pipeline from data → tokenizer → architecture → pretraining → inference. Today we’re releasing two public repos and a full walkthrough video. What we released: (1) Nano-GPT-OSS — https://lnkd.in/gWg6RpnU • 500M parameters, keeps the core architectural innovations of GPT-OSS. • Trains in ~20 hours on a single A40 (≈ $0.4/hr) — reproducible for < $10. • Truly-Open-GPT-OSS — https://lnkd.in/gvsvXxQC (2) 20B parameter model pre-trained fully from scratch. • Pretraining used 5 × H200 GPUs. Estimated budget: $100–150. Full behind-the-scenes (3-hour) walkthrough: https://lnkd.in/gVvw72cX The video covers eight practical steps: (1) Tiny Stories — data preprocessing (2) GPT-OSS Harmony tokenizer (3) token embeddings, RMSNorm, RoPE (4) sliding attention & Grouped Query Attention (GQA) (5) attention bias & attention sinks (6) SwiGLU Mixture of Experts (MoE) (7) GPT-OSS pretraining loop (8) GPT-OSS inference Both repositories are actively maintained — contributions, issues, and pull requests are very welcome. If you’re curious about training small-to-large transformer models end-to-end, this is a hands-on resource. If you try it or contribute, please tag me — would love to see what you build! #OpenSource #NLP #LLM #GPTOSS #Attention #GenAI #LargeLanguageModel #GenerativeAI #ArtificialIntelligence #MachineLearning #DeepLearning #TransformerModels #AIResearch #ModelDeployment #OSS #opentoconnect

203

10 Comments

𝗦𝗽𝗲𝗰𝗶𝗳𝘆𝗶𝗻𝗴 𝗖𝗼𝗺𝗽𝘂𝘁𝗲 𝗕𝘂𝗱𝗴𝗲𝘁𝘀 𝗳𝗼𝗿 𝗟𝗟𝗠 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 ➡️In discussions around large language models, compute is the true scaling constraint. While parameter count and dataset size are often highlighted, serious planning and comparison of training runs rely almost entirely on FLOPs-based metrics. There are three foundational concepts around specifying and estimating compute budget for training : 𝗙𝗟𝗢𝗣𝘀, 𝗣𝗙𝗟𝗢𝗣/𝘀-𝗱𝗮𝘆𝘀, 𝗮𝗻𝗱 𝘁𝗵𝗲 𝗴𝗮𝗽 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝘁𝗵𝗲𝗼𝗿𝗲𝘁𝗶𝗰𝗮𝗹 𝗮𝗻𝗱 𝗮𝗰𝗵𝗶𝗲𝘃𝗲𝗱 𝗰𝗼𝗺𝗽𝘂𝘁𝗲. ✨ 𝗙𝗟𝗢𝗣𝘀: 𝗧𝗵𝗲 𝗙𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹 𝗨𝗻𝗶𝘁 𝗼𝗳 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗖𝗼𝘀𝘁 ➡️Training compute is measured in floating-point operations (FLOPs). FLOPs quantify the total arithmetic work performed during training and are independent of hardware choice. ➡️For dense transformer models(not MoE), total theoretical training compute is commonly approximated as: Training FLOPs ≈ 6 × N × T where • N is the number of trainable parameters • T is the total number of tokens processed ➡️This estimate accounts for forward passes, backward passes, and parameter updates. Importantly, it represents a 𝗹𝗼𝘄𝗲𝗿 𝗯𝗼𝘂𝗻𝗱: the minimum compute required under ideal execution. ➡️FLOPs are widely used because they allow model-agnostic comparison and correlate well with observed scaling behavior. ✨ 𝗣𝗙𝗟𝗢𝗣/𝘀-𝗗𝗮𝘆𝘀: 𝗠𝗮𝗸𝗶𝗻𝗴 𝗖𝗼𝗺𝗽𝘂𝘁𝗲 𝗕𝘂𝗱𝗴𝗲𝘁𝘀 𝗜𝗻𝘁𝗲𝗿𝗽𝗿𝗲𝘁𝗮𝗯𝗹𝗲 ➡️Because raw FLOP counts are enormous, compute budgets are often expressed in petaFLOP/s-days. ➡️𝟭 𝗣𝗙𝗟𝗢𝗣/𝘀-𝗱𝗮𝘆 corresponds to sustaining 10¹⁵ floating-point operations per second for 24 hours, or roughly 𝟴.𝟲𝟰 × 𝟭𝟬^𝟭𝟵 𝗙𝗟𝗢𝗣𝘀 in total. ➡️This unit is useful because it 𝘁𝗿𝗮𝗻𝘀𝗹𝗮𝘁𝗲𝘀 𝗮𝗯𝘀𝘁𝗿𝗮𝗰𝘁 𝗰𝗼𝗺𝗽𝘂𝘁𝗲 𝗶𝗻𝘁𝗼 how much 𝗰𝗹𝘂𝘀𝘁𝗲𝗿 𝗰𝗮𝗽𝗮𝗰𝗶𝘁𝘆 is consumed over time and 𝘁𝗵𝗲𝗿𝗲𝗯𝘆 𝗱𝗼𝗹𝗹𝗮𝗿𝘀 ➡️ PFLOP/s-days measure total work done, not achieved throughput. Two training runs can consume the same PFLOP/s-day budget while differing significantly in wall-clock time and cost. ✨𝗧𝗵𝗲𝗼𝗿𝗲𝘁𝗶𝗰𝗮𝗹 𝘃𝘀 𝗔𝗰𝗵𝗶𝗲𝘃𝗲𝗱 𝗖𝗼𝗺𝗽𝘂𝘁𝗲 ➡️Theoretical compute budgets assume that all available hardware capacity is used perfectly. In practice, this is rarely the case. ➡️Actual utilization is typically evaluated using metrics such as: • 𝗔𝗰𝗵𝗶𝗲𝘃𝗲𝗱 𝗙𝗟𝗢𝗣𝘀/𝘀 — the effective arithmetic throughput observed during training • 𝗠𝗼𝗱𝗲𝗹 𝗙𝗟𝗢𝗣 𝗨𝘁𝗶𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 (𝗠𝗙𝗨) — the ratio of achieved FLOPs/s to the hardware’s specification of peak theoretical FLOPs/s ➡️The difference between theoretical and achieved compute is because: • Gradient synchronization and pipeline parallelism introduce idle time. • Padding and shape inefficiencies and variable sequence lengths. • Recomputation because of activation checkpointing ➡️As a result, actual FLOPs executed and wall-clock cost almost always exceed the theoretical minimum. #ComputeBudget #Scalinglaws

14