Top 6 Load Balancing Algorithms Explained

8 load balancing algorithms you should know: 1️⃣ Round Robin: It sends each new request to the next server in a rotating order. ↳ Useful when all servers have similar capabilities and you want to spread the load evenly. 2️⃣ Least Connections: It directs traffic to the server with the fewest active connections. ↳ Useful when servers have different workloads, balancing them more efficiently. 3️⃣ Weighted Round Robin: It gives more requests to servers with higher weights or capacities. ↳ Useful when some servers are more powerful and can handle more traffic. 4️⃣ Weighted Least Connections: It considers both server capacity and current connections to assign requests. ↳ Useful when servers differ in performance, ensuring a fair distribution. 5️⃣ IP Hash: It uses the client's IP address to decide which server will handle the request. ↳ Useful to keep a client connected to the same server, which is important for session consistency. 6️⃣ Least Response Time: It sends requests to the server with the quickest response and fewest connections. ↳ Useful to reduce delays and improve the user experience. 7️⃣ Random: It picks a server at random for each new request. ↳ Useful when you don't need to consider server load or differences in server capacity. 8️⃣ Least Bandwidth: It directs traffic to the server using the least network bandwidth at the moment. ↳ Useful when managing network usage is important to prevent congestion. Image Credits: DesignGurus

8 load balancing algorithms you should know: 1️⃣ Round Robin: It sends each new request to the next server in a rotating order. ↳ Useful when all servers have similar capabilities and you want to spread the load evenly. 2️⃣ Least Connections: It directs traffic to the server with the fewest active connections. ↳ Useful when servers have different workloads, balancing them more efficiently. 3️⃣ Weighted Round Robin: It gives more requests to servers with higher weights or capacities. ↳ Useful when some servers are more powerful and can handle more traffic. 4️⃣ Weighted Least Connections: It considers both server capacity and current connections to assign requests. ↳ Useful when servers differ in performance, ensuring a fair distribution. 5️⃣ IP Hash: It uses the client's IP address to decide which server will handle the request. ↳ Useful to keep a client connected to the same server, which is important for session consistency. 6️⃣ Least Response Time: It sends requests to the server with the quickest response and fewest connections. ↳ Useful to reduce delays and improve the user experience. 7️⃣ Random: It picks a server at random for each new request. ↳ Useful when you don't need to consider server load or differences in server capacity. 8️⃣ Least Bandwidth: It directs traffic to the server using the least network bandwidth at the moment. ↳ Useful when managing network usage is important to prevent congestion. Image Credits: DesignGurus

 1,000,000 Query Stress Test Running Successfully! The scaled-up stress test is performing exceptionally: 📊 Current Status Test Configuration: 1,000,000 queries, 1,000 concurrent users, 30-minute duration Mock Server: Handling massive load flawlessly with consistent 200 OK responses Throughput: Processing thousands of requests per second Error Rate: 0.00% (perfect reliability) Response Pattern: All requests returning successful responses 🎯 Performance Indicators Request Volume: Hundreds of thousands of requests processed Response Consistency: Every request showing "200 OK" status Server Stability: No errors or timeouts observed Resource Efficiency: Mock server handling load with minimal resources 📈 Expected Final Results Based on current performance trends, the 1M query test should achieve: Throughput: 750-800 QPS sustained over 30 minutes Total Processing Time: ~21-22 minutes Success Rate: 100% (0 errors) Response Times: Sub-second across all percentiles Resource Usage: Efficient memory and CPU utilization 🏆 Key Achievements So Far Perfect Reliability: Zero failed requests Consistent Performance: Stable response times Scalable Architecture: Mock server handling 100x load increase effortlessly Production Ready: Demonstrates excellent stability under extreme load The test is validating that the load testing framework itself can handle massive scale, and the mock server architecture is proving highly efficient for stress testing scenarios. Final results will be available once the test completes! 🎯

🚦 When Response Time Rises & Throughput Falls — What Really Happens Behind the Scenes? Every performance engineer has seen this classic pattern in test reports 👇 📈 Response Time keeps rising 📉 Throughput flattens or starts to drop But have you ever wondered what’s really happening inside your system when this occurs? Let’s decode it step by step. 🧩 ⚙️ Phase 1: The Happy Zone In the early stage of a load test, the system is healthy. ✅ Response Time slightly increases ✅ Throughput increases linearly Both are directly proportional — more users = more transactions processed. The app, DB, and network are all within safe capacity. ⚠️ Phase 2: The Saturation Zone As load keeps increasing, you hit the performance knee point — the turning point where things start to break down. Now the relationship becomes inversely proportional: 🚨 Response Time keeps rising 🚨 Throughput flattens or even drops And this is where the chain reaction begins 👇 💣 The Domino Effect 🧠 App Layer: Threads stay busy waiting for DB/API calls → Thread pool exhaustion → Queues build up → Timeouts (504 / 500 errors). 🗄️ Database: Slow queries hold locks → Other queries wait → Connection pool exhausted → Transactions fail. 🌐 Network / Middleware: Sockets remain open → Buffers fill → Load balancer reports “upstream timeout.” ☁️ Infrastructure: CPU spikes → GC overhead increases → Auto-scaling triggers late → Pods restart. All because the system crossed its safe operating limit. 🔍 Result: MetricBehaviorResponse TimeIncreases exponentiallyThroughputDrops / flattensError %Climbs steadilyDB & MemorySaturate quickly This is how a simple rise in Response Time can cascade into a full-stack performance degradation — from application to infrastructure. 🧠 Key Takeaway 👉 Don’t just watch averages. 👉 Identify your performance knee point early — that’s your true scalability limit. 👉 Tune your thread pools, DB queries, and caching strategy before throughput collapses. Because once RT and throughput start drifting apart, recovery takes much longer than the failure itself. 💥

8 Load Balancing Algorithms You Should Know: 0.💡 Round Robin: It sends each new request to the next server in line, like taking turns. ➡️ Useful when: You have servers with similar capabilities and want to spread the load evenly. 1.💡 Least Connections: It sends new requests to the server handling the fewest tasks right now. ➡️ Useful when: Your servers have different processing speeds or some tasks take longer than others. 2.💡 Weighted Round Robin: It gives more work to stronger servers and less to weaker ones, but still takes turns. ➡️ Useful when: You have a mix of powerful and less powerful servers in your system. 3.💡 Weighted Least Connections: It considers both how busy each server is and how strong it is when assigning new tasks. ➡️ Useful when: You have servers with different strengths and want to balance their workload efficiently. 4.💡 IP Hash: It always sends requests from the same user to the same server. ➡️ Useful when: Users need to stay connected to the same server for their entire session, like in online shopping. 5.💡 Least Response Time: It chooses the server that's both quick to respond and not too busy. ➡️ Useful when: Speed is crucial, and you want to provide the fastest possible service to users. 6.💡 Random: It picks a server at random for each new request. ➡️ Useful when: You want a simple way to distribute tasks and your servers are all similar in capability. 7.💡 Least Bandwidth: It sends new tasks to the server using the least amount of internet capacity at the moment. ➡️ Useful when: Your main concern is network traffic, and you want to avoid overloading any single server's connection. What did I miss? ~~~ 👉🏻 Join 50,001+ software engineers getting curated system design deep dives, trends, and tools (it's free): ➔ https://lnkd.in/dkJiiBnf ~~~ If you found this valuable: 👨🏼💻 Follow Alexandre Zajac 🔖 Bookmark this post for later ♻️ Repost to help someone in your network #softwareengineering #systemdesign #programming

⚡ How One Line of Code Can Break 1 Million Requests Every system starts small. Every line of code feels harmless. And yet — sometimes it’s one innocent line that brings everything down. Not because it throws an exception. But because it silently chokes the system under load. When the app handles a few hundred requests per second, everything looks perfect. Logs are clean. Latency is stable. Dashboards look green. Then traffic scales. 100k… 500k… 1 million concurrent requests. And suddenly — response times spike, CPU usage shoots up, queues overflow. No infrastructure change. No new feature. Just a tiny logic flaw multiplying silently with every request. A missing cache. A blocking I/O call inside a loop. A DB write placed in the hot path. That’s all it takes. Scalability isn’t just about servers, clusters, or load balancers. It’s about writing precise code that respects the cost of every line. Because at scale, a small inefficiency becomes a large outage. 💡 Precision > Productivity. In high-performance systems, “almost right” is still wrong.

If you want to distribute traffic, consider these 5 algorithms: 𝟭) 𝗥𝗼𝘂𝗻𝗱 𝗥𝗼𝗯𝗶𝗻 ↳ Requests are distributed sequentially to available servers. 𝟮) 𝗪𝗲𝗶𝗴𝗵𝘁𝗲𝗱 𝗥𝗼𝘂𝗻𝗱 𝗥𝗼𝗯𝗶𝗻 ↳ Assigns different "weights" to servers or tasks, with higher weights receiving a proportionally larger share of incoming requests. 𝟯) 𝗜𝗣/𝗨𝗥𝗟 𝗛𝗮𝘀𝗵 ↳ Routes based on the hash of the IP and URL. Requests with the same IPs and URLs go to the same server. How does this work exactly? Imagine you have 3 servers. We compute the hash of the URL / IP and modulo by N, where N is the number of servers. Any request with the same URL / IP will go to the same server, until the number of servers change. Here's an example of URL hashing: /home → hash("/home") % 3 = 0 → Server 0 /products/42 → hash("/products/42") % 3 = 1 → Server 1 /user/1234 → hash("/user/1234") % 3 = 2 → Server 2 /user/5678 → hash("/user/5678") % 3 = 1 → Server 1 𝟰) 𝗟𝗲𝗮𝘀𝘁 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲 𝗧𝗶𝗺𝗲 ↳ Requests are directed to the server with the lowest response time. 𝟱) 𝗟𝗲𝗮𝘀𝘁 𝗖𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻𝘀 ↳ Requests are routed to the server with the fewest active connections. --- Never fail another System Design Interview -> https://lnkd.in/eYW-XEvd

💡 𝗗𝗮𝘆 𝟭𝟴 𝗼𝗳 𝗦𝘆𝘀𝘁𝗲𝗺 𝗗𝗲𝘀𝗶𝗴𝗻 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴-𝗶𝗻-𝗣𝘂𝗯𝗹𝗶𝗰: 𝗖𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝘁 𝗛𝗮𝘀𝗵𝗶𝗻𝗴 Let’s dive deeper into how consistent hashing actually works. ✨ 𝗛𝗼𝘄 𝗱𝗮𝘁𝗮 𝗽𝗹𝗮𝗰𝗲𝗺𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝘀:  • Both keys and servers are hashed into a numeric range, usually [0, 2³²).  • Each key is placed on the ring based on its hash value.  • It belongs to the first server encountered when moving clockwise around the ring. 🔹 𝗘𝘅𝗮𝗺𝗽𝗹𝗲: Suppose we have 4 servers (S1–S4). Each new server added to the ring only takes over part of the key space from its neighbor. If a server goes down, its keys are reassigned to the next one on the ring. This design keeps remapping 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 𝗮𝗻𝗱 𝗹𝗼𝗰𝗮𝗹𝗶𝘇𝗲𝗱, making scaling predictable and fast. 📌 𝗞𝗲𝘆 𝗜𝗻𝘀𝗶𝗴𝗵𝘁: Consistent hashing’s brilliance lies in reducing churn only a small fraction of keys move, even as the cluster size changes. 👉 Question for you: Have you implemented consistent hashing manually or used a library for it?

TCP 3-Ways Handshake Process After the SYN-ACK packet is sent, the client sends an ACK packet to the server, which completes the TCP 3-way handshake. At this point, the connection is established, and both the client and server transition to the ESTABLISHED state, allowing them to begin exchanging application data. Client receives SYN-ACK: The client receives the SYN-ACK, which acknowledges the server's readiness and confirms the server's chosen initial sequence number. Client sends ACK: The client sends a final ACK packet back to the server. This packet acknowledges the server's SYN-ACK and completes the three-step handshake. Client enters ESTABLISHED state: Upon sending the ACK, the client's connection is now established and it can begin sending data. Server enters ESTABLISHED state: After receiving the final ACK, the server also enters the ESTABLISHED state and is ready to receive data. Data transfer begins: With both sides in the ESTABLISHED state, the connection is open, and they can now exchange data for the application (e.g., an HTTP request).

✅ Day 1: Response Time vs Throughput ✅ Day 2: Auto-Scaling — Why It’s Needed When Response Time Increases 👉 Tomorrow (Day 3) should naturally extend this story with a “Performance Engineering insight” that connects to the next layer of system behavior once scaling kicks in. Here are 3 strong options you can pick from (ranked in storytelling order): 🥇 Option 1 – Load Balancer & Scaling Efficiency Topic: “Even with Auto-Scaling, why response time sometimes doesn’t drop — Load Balancer behavior explained.” Angle: Show that just adding servers doesn’t guarantee improvement — if the load balancer isn’t distributing traffic evenly or sessions are sticky, some nodes still overload. → Teaches how scaling + distribution together matter. Sample hook: Auto-Scaling adds capacity, but poor load distribution can still keep response times high — let’s see why! 🥈 Option 2 – Database Bottlenecks after Scaling Topic: “App scaled, but DB didn’t — the hidden bottleneck.” Angle: When auto-scaling expands app servers, the database may become the new bottleneck. Discuss DB connection pool limits, locking, indexing, or I/O. Sample hook: We scaled our app horizontally — but response times stayed high. The culprit? Our database couldn’t scale the same way! 🥉 Option 3 – Right-Sizing & Scaling Thresholds Topic: “Auto-Scaling thresholds: scale early, not late.” Angle: Explain why tuning scaling policies (CPU %, latency %, warm-up time) matters — scaling too late causes performance dips before recovery. Sample hook: Auto-Scaling helps, but if your thresholds are too high, it reacts late — and users feel the lag first! 💡 My Recommendation: Go with Option 2 (Database Bottlenecks after Scaling) — it’s the logical next part after auto-scaling.