In system design interviews, you’ve probably noticed that Apache Flink is frequently recommended for multiple real-time data stream use cases. But how do you actually build a Flink pipeline? And what is Apache Beam, and how does it fit into the picture? Check out my new Pluralsight course to get hands-on with real-time data stream processing using Apache Flink and Apache Beam, and learn how to put these powerful frameworks into practice. https://lnkd.in/gQAWGTkH Happy Learning!

[ANN] Graph Algorithms new version released! We added several new algorithms (Greedy Matching, DFS, Floyd Warshall, shortest path for ABCD, and Edmons Karp); we also improve some of the existing ones and did some other fixes. The full list of changes is available in the GitHub repo. https://lnkd.in/efmvSCtw

🎥 Here’s a quick 2-minute Loom video demo of my latest fun project https://lnkd.in/d56HGjUK Over the past few days, I’ve been working on a side project to deepen my systems programming skills. Instead of just using Redis, I decided to re-create some of its core features from scratch — and I learned a ton about how databases work under the hood. Right now it supports: - A sharded in-memory key-value store (using DashMap) - TTL with a background sweeper - An async TCP server (Tokio) that speaks a Redis-style protocol (SET, GET, DEL, EXPIRE, TTL, PING) - Append-only file persistence (so data survives restarts) This wasn’t about reinventing Redis — it was about learning how things like sharding, persistence, and async servers actually work under the hood. Redis is insanely optimized, but writing my own small version gave me a much deeper appreciation for the engineering behind it. Here’s the repo if you’re curious: [https://lnkd.in/d99uUJdQ] If you were to add one more feature to this “mini-Redis,” what would it be? #rust #backend #redis #buildinpublic #learning

In this post, we’ll dive deep into the technical differences between withColumn and withColumns, explore how Spark’s optimizer handles each, and share best practices for writing efficient, production-grade code.

Repository for everything Lakehouse! I started this repo to serve as a central hub for content (blogs, paper, code) that I create and insights that I share from different sources about the Lakehouse architecture. So glad to see it gaining traction (96 stars). Here’s what you will find in the repo: ⭐️ Blogs (how table formats such as Apache Iceberg, Apache Hudi, Delta Lake are being implemented for specific use cases) ⭐️ Research Papers summary (subjects: Distributed systems, Databases, Storage optimization, Compute engines) ⭐️ Code/Notebooks (How-To, applied use cases) ⭐️ Crisp Social posts (quick bites summary) If you are looking to do some reading or try out a few stuff hands-on, this resource should hopefully be beneficial. New stuff to be added soon. Share if you find it useful. Link in comments! 👇 #dataengineering #softwareengineering

If you want to write high-performance code, you can go surprisingly far by avoiding bad Big O complexities. Some examples: • O(1): hash map lookup • O(log n): binary search • O(n): single array iteration • O(n log n): merge/heap sort • O(n^2): generating all pairs of elements • O(n^3): triply nested loops • O(2^n): recursive (and naive) Fibonacci • O(n!): generating a complete set of permutations

Learn how to quickly install Kibana using Docker. Follow our step-by-step guide to set up and run Kibana in a Docker container.

🔥 Fresh from the #FabConEurope keynote – here are my top picks from Amir Netz’s announcement-packed session: • Fabric CLI is now Open Source • Fabric MCP for developers (PP) • Fabric Extensibility Toolkit (PP) • Variable library in GA • Variable support in Shortcuts • Full CI/CD support for all items • Multitasking experience in the UI (tabs) • Object Explorer view • Fabric Runtime 2.0 coming soon • Incremental refresh in materialized views • Python notebooks in GA • New Graph model item • Eventhouse Endpoint • Real-time Anomaly Detector • Shortcut transformation enhancements • New OneLake Table API • OneLake storage diagnostics • Web Modelling in GA • Snowflake bi-directional integration • New Maps item 🚀 What a lineup! For the full overview of everything announced, check out the official Fabric blog: 👉 https://lnkd.in/dBscCpZg

🚀 FastAPI Best Practices in a Nutshell Key takeaways: ✅ Organize code by domain, not file type ✅ Use async for I/O, sync for heavy/blocking work (with threadpool) ✅ Let Pydantic handle validation everywhere ✅ Manage dependencies smartly (chain & cache them) ✅ Handle sync libs carefully — don’t block the event loop ✅ Use proper migrations, naming conventions & error handling In short: Write APIs that are clean, modular, and production-ready.

This is really interesting: https://lnkd.in/dvHxpfC3 ✅ Key takeaways: Trade-offs between reliability/scalability/maintainability, B-tree vs LSM-trees, CAP theorem, and event-driven architecture. 💡 Learn why Cassandra uses LSM-trees for writes, how Kafka’s logs are a distributed system’s heartbeat, and the cost of eventual consistency. 🔥 The book isn’t just theory—it bridges gaps between concepts and real-world systems (like how replication works in PostgreSQL vs. Dynamo). 🚫 Minor gripes: Outdated examples (Kafka, cloud trends), heavy on theory, and breadth-over-depth. But the foundational principles are timeless. 🎯 Who should read it? Mid-career engineers/architects. New devs might struggle without prior context. 🧠 The biggest takeaway? Design systems for failure—assume everything breaks. Use retries, replication, and graceful degradation. 📊 Bonus: A cheat sheet of design rules (consistency vs latency, schema evolution, etc.) to use in code reviews. 📌 Final verdict: A must-read for anyone building scalable, reliable data systems. Don’t skim it—read it twice. #SoftwareEngineering #DistributedSystems #DataArchitecture #BookReview #TechLeadership