What Does a Load Balancer Actually Do in a Web Hosting Stack?

A load balancer sits in front of your web servers and decides which backend receives each incoming request. Its core job is to spread traffic across multiple servers so that no single machine becomes a bottleneck, and to stop sending traffic to a server that has failed. In a typical hosting stack it is the first layer that terminates client connections, then forwards requests to web servers that may themselves sit behind a cache and speak HTTP/2 or HTTP/3 to the outside world.

This article explains where that layer fits, how it chooses a backend, and when adding one is actually worth it.

The position of a load balancer in the stack

A common request path looks like this:

1. Client connects over TLS (often HTTP/2 or HTTP/3).
2. Load balancer terminates the connection, inspects the request, and picks a backend.
3. Web server (for example LiteSpeed, Nginx, or Apache) handles the request.
4. Cache layer may serve the response directly, or the web server generates it.
5. Application and database sit behind the web server.

Two design choices matter here:

• TLS termination point. If the load balancer terminates TLS, it holds the certificates and does the QUIC/HTTP/3 handshake. Backend traffic can then be plain HTTP inside a trusted network, or re-encrypted. Terminating at the edge centralises certificate management but means the balancer must handle the CPU cost of the handshake.
• Where caching lives. If the cache sits on each web server, the balancer must be aware that a request for a cached object could be served by any node. If the cache is a shared tier behind the balancer, cache hits are consistent regardless of which backend is chosen.

How a load balancer picks a backend

The selection method is called an algorithm. The common ones, at a conceptual level:

No algorithm is universally best. Round robin is simple but ignores that one request may take 10 ms and another 2 seconds. Least connections adapts better to uneven work. Weighted variants let you drain a smaller machine gradually rather than cutting it out abruptly.

Health checks

A load balancer only helps if it stops routing to broken servers. Health checks come in two broad forms:

• Passive: watch real traffic. If a backend returns repeated errors or times out, mark it unhealthy.
• Active: send a synthetic probe on a schedule (a TCP connect, an HTTP request to a known path, or a check of a status endpoint).

Active checks catch a dead server before a user does. Passive checks catch a server that answers probes but fails real requests. Most production setups use both. Tune the interval and failure threshold so a brief blip does not eject a healthy server, but a genuine outage is detected within seconds.

When you actually need one

A load balancer adds a hop, a configuration surface, and a potential single point of failure if deployed alone. It earns its place in these situations:

• Traffic exceeds one server. When a single web server is CPU- or connection-saturated at peak, horizontal scaling needs something to distribute requests.
• Redundancy is required. With two or more backends, one can fail without taking the site down.
• Maintenance windows. You can drain a server, patch it, and return it to the pool without downtime.
• TLS/QUIC offloading. Centralising HTTP/3 and certificate handling at the edge can simplify backend configuration.
• Uneven or bursty load. Autoscaling groups need a stable entry point that adds and removes instances.

You probably do not need one if a single well-tuned server with a cache handles your traffic comfortably. Adding a balancer in front of one backend gives you a failure point without the redundancy benefit.

Interaction with caching and HTTP/2 / HTTP/3

These layers affect each other in ways that are easy to miss.

Caching. A shared cache behind the balancer produces consistent hit rates. A per-server cache means the same object may be cached on node A but not node B, so a user hitting different nodes sees inconsistent cache behaviour. If you rely on per-server caching, consider consistent hashing so the same URL tends to land on the same node.

Session persistence. If your application stores sessions locally on each web server, a user bouncing between backends loses their session. Options are: sticky sessions (route a client to the same backend), or better, move session state to a shared store so any backend can serve any user. Sticky sessions reduce the balancing benefit and complicate failover, so shared state is usually the cleaner fix.

HTTP/2 and HTTP/3. The client-facing protocol is negotiated at the balancer. If the balancer speaks HTTP/3 to clients but HTTP/1.1 to backends, you still gain the client-side benefits (faster handshakes, multiplexing over lossy networks) while keeping backend configuration simple. Check that your balancer supports the protocol versions you intend to advertise, and that health checks use a compatible path.

Connection reuse. A balancer that keeps persistent connections to backends avoids a TCP and TLS handshake per request. This matters more as request volume grows.

A practical checklist before adding one

1. Confirm a single server is genuinely the limit — measure CPU, connections, and cache hit rate first.
2. Decide where TLS terminates and whether you need HTTP/3 at the edge.
3. Choose an algorithm that matches your request profile; start with least connections if unsure.
4. Configure both active and passive health checks, and test failover deliberately.
5. Decide how sessions are stored; prefer shared state over sticky routing.
6. Plan for the balancer itself: run more than one instance, or use a managed service, so it is not a single point of failure.
7. Verify that your cache tier behaves consistently no matter which backend is selected.

Summary

A load balancer distributes requests, detects failed backends, and gives you a stable entry point for scaling and maintenance. It sits at the edge, typically terminating TLS and HTTP/2 or HTTP/3, and forwards to web servers that may sit behind a cache. The algorithm and health-check design matter more than the brand. Add one when a single server can no longer carry the load or when redundancy and zero-downtime maintenance are requirements — and make sure the balancer itself is not the only thing standing between users and your site.