Salvador Corts Sánchez

How much does a Kubernetes engineer earn in Q3 2025? Is Platform Engineering really eating DevOps' lunch? We analyzed 509 Kubernetes job descriptions and discovered: 💰 North American salaries average $177,983 (€92,113 in Europe) 🚀 Platform Engineer roles jumped to 9% of positions (vs 4-7% last year) 👨‍💻 43% of jobs are for Software Engineers, but DevOps roles offer the best remote flexibility (56%) 🏠 Remote work paradox: 67% allow remote, but only 0.29% are truly location-independent Dive into the complete State of Kubernetes Job Market Q3 2025 report: https://lnkd.in/gfmuQWhe ⭐️ This report is brought to you by LearnKube — get started on your Kubernetes journey through comprehensive online, in-person, or remote training. https://lnkd.in/ge4aYVZq

141

🤔 New to creating event-driven Apache Airflow DAGs? Marc Lamberti shows you how to use the Apache Kafka asset watcher to make your first event-driven data pipelines. Watch the full video on our YouTube and find the link to the GitHub repo in the comments: https://bit.ly/3H8fHDQ

121

3 comentarios

Eric Thomas took an excellent lakehouse tutorial built with Airflow and rebuilt it with Dagster. The stack is the same: MinIO, Trino, Iceberg, and dbt. The orchestrator is different. The results were striking: → Event-driven sensors replaced time-based scheduling. Pipelines run when data arrives, not on a clock. → Smart partitioning enabled backfills and selective reruns. No more all-or-nothing processing. → Asset checks created multi-layered quality validation. Data quality became programmatic, not just hoped for. → Pure SQL patterns eliminated Python bottlenecks. Trino handles the heavy lifting. The Lakehouse provides the foundation, but the orchestration layer determines how effectively teams can actually use it. The original tutorial teaches lakehouse fundamentals beautifully. This comparison shows how much orchestration choice matters for production readiness. Check out the full blog today! Link in the comments

70

4 comentarios

Your teams shouldn’t have to choose between saving money and delivering great performance. 🌟 In this edition of This Month in Datadog, we spotlight Datadog Kubernetes Autoscaling, which offers workload scaling recommendations and makes it easy to optimize clusters based on resource usage. We also dive into: ➡️ Surfacing anomalies and aligning teams with our new Cloud Cost Management features ➡️ Monitoring and optimizing LLM-powered applications with our LiteLLM integration ➡️ Gaining deeper visibility into your Kubernetes infrastructure with Datadog Workload Protection Plus, learn how you can connect with your peers, grow your skills, and explore the latest innovations at an upcoming Datadog Summit. Watch the full episode ⤵️

95

1 comentario

LLMs often waste compute by reprocessing repeated inputs. LMCache fixes that. By caching token-level key-value pairs for repeated text, LMCache skips redundant computation. Redis stores and serves those cached chunks in real time, making interference faster and more efficient. Together, LMCache and Redis deliver cheaper, production-ready GenAI: https://lnkd.in/gwRECCBD

82

You can build a lightning-fast ticker plant that ingests 2.6 billion rows of tick data per minute. All in Python and kdb/q: DefconQ shows how in this tutorial, using #Databento and kola, a modern Python interface for #kdb built on top of Rust and Polars. ⬇️ Link to the tutorial in the comments below. #python #polars #rust #quant #algotrading

54

2 comentarios

MinIO Brings Iceberg Tables to AIStor: https://lnkd.in/g6QsFmjR MinIO is introducing Iceberg Tables, bringing native Apache Iceberg support directly into AIStor to unify enterprise data for AI. By integrating the Iceberg Catalog API directly into MinIO AIStor, MinIO is helping enterprises unlock more value from AI on their own terms with an AI data store that brings together structured and unstructured data under a single, open standard. Apache Iceberg | #DataStore #Iceberg #AIStor #AI

92

2 comentarios

𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭’𝐬 #𝐑𝐮𝐬𝐭 𝐁𝐞𝐭: 𝐅𝐫𝐨𝐦 𝐁𝐥𝐮𝐞 𝐒𝐜𝐫𝐞𝐞𝐧𝐬 𝐭𝐨 𝐒𝐚𝐟𝐞𝐫 𝐂𝐨𝐝𝐞 At RustConf, Mark Russinovich shared how deeply Microsoft is leaning into Rust to replace unsafe C/C++ in critical systems. This isn’t just about developer preference - it’s about making the world’s most widely used software safer and more reliable. Over the past few years, Rust has gone from being a "promising new language" to a serious player in modern software engineering. What makes this shift so interesting for professionals is not just hype, but the technical depth driving adoption across industries. - 𝗠𝗲𝗺𝗼𝗿𝘆 𝗦𝗮𝗳𝗲𝘁𝘆 𝗪𝗶𝘁𝗵𝗼𝘂𝘁 𝗚𝗮𝗿𝗯𝗮𝗴𝗲 𝗖𝗼𝗹𝗹𝗲𝗰𝘁𝗶𝗼𝗻: Rust's ownership model eliminates entire classes of runtime errors (null pointers, data races) while keeping performance close to C/C++. This isn't just a productivity boost - it's an engineering paradigm shift. - 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗧𝗵𝗮𝘁 𝗦𝗰𝗮𝗹𝗲𝘀: From cloud infrastructure to embedded devices, Rust consistently delivers low-level control with high-level ergonomics. Companies migrating critical workloads report reduced latency, lower resource usage, and stronger reliability. - 𝗚𝗿𝗼𝘄𝗶𝗻𝗴 𝗘𝗰𝗼𝘀𝘆𝘀𝘁𝗲𝗺: Crates io now provides robust libraries for networking, async programming, cryptography, and machine learning. What was once a "systems-only" language is now powering full-stack solutions, from APIs to blockchain. - 𝗔𝗱𝗼𝗽𝘁𝗶𝗼𝗻 𝗯𝘆 𝗜𝗻𝗱𝘂𝘀𝘁𝗿𝘆 𝗟𝗲𝗮𝗱𝗲𝗿𝘀: Amazon, Microsoft, and Cloudflare are investing in Rust at scale. The language is now part of Linux kernel development - a milestone that speaks volumes about long-term viability. - 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿 𝗘𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲: Tooling (Cargo, Clippy, Rust Analyzer) sets a standard that many seasoned developers wish they had in other languages. Strong documentation and a culture of mentorship also make Rust a welcoming ecosystem. As professionals, we're entering a new stage: - Rust is no longer the "language of enthusiasts". - It is becoming the language of critical systems. - It's a strategic choice for long-term resilience. For Rust engineers, this is both validation and opportunity. Whether you're building secure backend services, optimizing cloud infrastructure, or contributing to open source, the language's trajectory suggests that deep Rust expertise will remain in high demand for years ahead. The full release link in the comments 👇

67

2 comentarios

🎉 #VLDB2025 has published the paper “Disaggregated State Management in Apache Flink® 2.0”! 🎉 🤝 Who made it happen A true community effort—co-authored by the #ApacheFlink community, Alibaba Cloud, Boston University, and KTH Royal Institute of Technology. The work decouples state and compute, slashing snapshot cost and speeding recovery—a major leap toward cloud-native, high-scale stream processing. 📜 Why it matters • Exactly 10 years after Flink’s first VLDB paper defined consistent streaming state, this new work charts the next leap: disaggregated, cloud-native state. • Shows continued academic trust in Flink and Alibaba’s decade-long contribution to open innovation. • Kicks off a new chapter—Generic Incremental Compute: ForSt + batch push-down promise lower latency AND lower cost, bringing near-real-time analytics to everyone. 🗓️ VLDB 2025 is LIVE NOW in London! Join us at the 51st International Conference on Very Large Databases (VLDB) as we explore the future of distributed systems. 📅 When: September 1–5, 2025 📍 Where: London, UK Don’t miss Industry Session 1 on Tuesday, September 2nd (9/2)—Disaggregated State Management in Apache Flink® 2.0. This session is a must-attend for data engineers building next-gen architectures! 🔗 Learn More: Download the Paper: https://lnkd.in/gqcd9BYW Read the Technical Deep Dive: https://lnkd.in/gnwB48kn Watch the full recording on Youtube: https://lnkd.in/gV-n45sq #VLDB2025 #ApacheFlink #DataEngineering #StreamingSystems #DistributedComputing Yuan Mei

62

1 comentario

Q. What’s faster than a distributed database built for resilience? A. One that buffers its writes. Buffered writes in CockroachDB significantly reduce network round trips – cutting SQL complexity and workload latency. That’s not just a nice-to-have: This is production-level high performance, optimized for better user experiences. 🧠 Learn how buffered writes help teams build faster, more scalable apps: 🔗 https://lnkd.in/eP3qC8RY

41

Async Rust’s disk I/O is not always ideal, largely because fully asynchronous file I/O APIs are relatively new on most mainstream operating systems. #P99CONF speaker Tzu Gwo explains how they developed Fusio to provide a minimal cost abstraction layer that delivers unified asynchronous data access across various platforms. https://lnkd.in/eDi-gpX7 #ScyllaDB

32

2 comentarios

🚀 By processing queries in a streaming manner — instead of loading all samples at once — Mimir Query Engine boosts both speed and memory efficiency in Grafana Mimir. It’s a foundational feature for the upcoming Mimir 3.0 release. See how it works 👇 https://lnkd.in/g9TJXPTA

48

The most effective architectural changes are often subtractive - removing entire layers of cost and complexity. Behind the scenes, OpsHelm, Inc.'s streaming backbone was migrated from a costly MSK and unreliable NATS setup to a single diskless Kafka architecture. The results were immediate: - Annual spend dropped from >$50,000 to <$10,000. - Data loss incidents went from recurring to zero. - Manual scaling and cross-cloud networking fees were eliminated. Read the full story of their migration and 78% cost reduction: https://lnkd.in/dS6UK9-7 #disklesskafka #datastreaming #Aiven

30

Blend full-text and semantic search with custom weights using a linear retriever in Elasticsearch. Deliver more accurate, context-aware results for your complex queries. Watch the Elasticsearch Coding Session highlight from Ugo Sangiorgi to see smarter search in action and learn how to build your own linear model: https://go.es.io/3V1yN1X

132

3 comentarios

O'Reilly Media has just released Apache Polaris: The Definitive Guide—the first-ever book dedicated to the open-source, Iceberg-native catalog powering modern lakehouses. Authored by Dremio’s Alex Merced, this comprehensive guide offers hands-on strategies for building scalable, cloud-native data architectures using Apache Polaris. It covers everything from metadata management and governance to integrating with tools like Apache Spark, Flink, and Dremio. Whether you're a data engineer, architect, or analyst, this book is your roadmap to mastering the open lakehouse ecosystem. https://lnkd.in/givFEUgV

27

The Kubeflow Trainer project has been integrated into the PyTorch Ecosystem! This integration ensures that Kubeflow Trainer aligns with PyTorch’s standards and practices, giving developers a reliable, scalable, and community-backed solution to run PyTorch on Kubernetes. Kubeflow Trainer is a Kubernetes-native project enabling scalable, distributed training of AI models and purpose-built for fine-tuning large language models (LLMs). It simplifies the scale-out of training workloads on multiple nodes, managing large datasets efficiently and ensuring fault-tolerance. ➡️ Read more on in our blog: https://hubs.la/Q03zbthT0

391

5 comentarios

You might have a great machine learning model that performs well in notebooks. But unless it's deployed to production, helping people solve real problems, it's not nearly as valuable. In this detailed tutorial, Kuriko helps you build and deploy an ML system on serverless architecture. https://lnkd.in/gKkz3Hs3

874

3 comentarios

Big news for the Apache Kafka community! 🎉 Kafka 4.1 is out. This release includes 18 KIPs from the broader community. Kafka 4.1 hardens the 4.0 foundation and introduces new features for OAuth, operations, Streams, Connect, Queues, KIP-848, and more. Explore what’s new in the blog ➡️ https://cnfl.io/461BTbe

397

1 comentario