What Does a Web Server Actually Do, and Where Do Load Balancers and Caches Fit?

A web server is the piece of software that accepts network connections from clients, interprets HTTP requests, and returns responses—usually files, generated pages, or API data. A load balancer sits in front of one or more web servers and decides which one receives each request. A cache stores copies of responses so future requests can be served without asking the web server to do the work again. They are three separate jobs, and mixing them up is one of the most common reasons people tune the wrong layer when a site feels slow.

The core job of a web server

At its simplest, a web server does four things:

1. Listens on a port (typically 443 for HTTPS) for incoming connections.
2. Parses the HTTP request: method, path, headers, and body.
3. Decides what to return—a static file from disk, a response from an application process, or an error.
4. Sends the response back with a status code, headers, and body.

Everything else a web server does is an optimisation or a convenience built around that loop. Static file serving, compression, TLS termination, HTTP/2 and QUIC support, connection keep-alive, and access logging all exist to make that loop faster, safer, or more observable.

The request path, step by step

When a browser requests https://example.com/page:

• DNS resolution maps the hostname to an IP address.
• TCP and TLS handshake establish an encrypted connection. If the server supports QUIC (HTTP/3), this may happen over UDP instead, reducing round trips.
• Protocol negotiation decides whether the conversation uses HTTP/1.1, HTTP/2, or HTTP/3. HTTP/2 and HTTP/3 allow multiple requests over one connection, which matters most on pages with many assets.
• The web server processes the request. For a static file it reads from disk or memory. For a dynamic page it forwards the request to an application runtime (PHP, Node, Python, and so on) and waits for a response.
• The response is returned, possibly compressed, and the connection is reused or closed.

TLS termination is worth calling out. The web server (or a proxy in front of it) decrypts incoming traffic and re-encrypts outgoing traffic. This is CPU-intensive, which is why modern servers support session resumption, OCSP stapling, and hardware acceleration.

Where a load balancer fits

A load balancer's job is distribution, not content. It receives client connections and forwards them to one of several backend servers according to a policy: round robin, least connections, IP hash, or weighted variants.

Key distinctions:

• A load balancer usually does not generate content. It routes.
• It often performs health checks, removing unhealthy backends from rotation.
• It may terminate TLS itself, or pass encrypted traffic through to the backend.
• It enables horizontal scaling: you add servers rather than making one server bigger.

If your site runs on a single server, you probably do not need a load balancer yet. If you have multiple application servers, or you need zero-downtime deploys, you do.

Where a cache fits

A cache's job is reuse. Instead of recomputing a response, it stores a copy and serves that copy until it expires or is invalidated.

Caches appear at several layers:

The critical concept is cacheability. A response is only reusable if its headers say so (Cache-Control, Expires, ETag) and if it does not depend on per-user state. A logged-in dashboard is usually not cacheable; a marketing homepage usually is.

How the three interact

A typical deployment looks like this:

Client → CDN/edge cache → load balancer → web server(s) → application → database

Each layer can absorb load before it reaches the next:

• The edge cache handles repeat requests for static or cacheable content.
• The load balancer spreads remaining traffic across servers.
• The web server handles connections, TLS, protocol negotiation, and static files.
• The application does the dynamic work.

Which layer is usually the bottleneck?

In practice, the bottleneck is rarely the web server's raw connection handling. It is more often:

• The application or database, when dynamic requests are slow.
• The absence of caching, when identical responses are regenerated constantly.
• TLS handshakes, on very high-traffic sites without session resumption.
• A single server, when traffic exceeds what one machine can serve.

A well-configured web server can handle thousands of concurrent connections. If your site is slow, the web server is often the messenger, not the cause.

Practical signals for choosing what to fix

Use these signals to decide where to act:

• High CPU on the web server, low application CPU → look at TLS, compression, or static file handling.
• Slow time-to-first-byte, fast static assets → the application or database is the problem, not the server.
• Repeated identical requests hitting the application → add a cache layer.
• One server maxed out, others idle → you need a load balancer, or your routing is wrong.
• High traffic from many regions → a CDN or edge cache will help more than server tuning.
• Frequent deploys causing downtime → a load balancer with health checks enables rolling updates.

A quick decision guide

• Single server, mostly static content: tune the web server (compression, HTTP/2 or HTTP/3, caching headers).
• Single server, dynamic content, repeated requests: add a reverse-proxy or application cache.
• Multiple servers or need for high availability: introduce a load balancer.
• Global audience: put a CDN in front of everything.

Common misconceptions

• "The web server is the cache." Some servers include caching modules, but caching is a distinct function. Enabling a cache module does not mean your content is cacheable.
• "A load balancer improves performance." It improves scalability and availability. It does not make a single slow server faster.
• "HTTP/2 and QUIC are just speed features." They change how connections are used. HTTP/2 multiplexes requests over one connection; QUIC reduces handshake latency and handles packet loss better. Both require server support and, for QUIC, UDP reachability.
• "More layers always help." Each layer adds configuration surface and a potential point of failure. Add a layer when you can name the problem it solves.

Putting it together

Think of the stack as a pipeline with distinct responsibilities: the web server speaks HTTP and returns responses, the load balancer distributes connections, and the cache avoids repeated work. Diagnose performance by measuring each stage—connection time, TLS time, time to first byte, and content transfer—rather than assuming the web server is at fault. Once you know which stage is slow, the fix usually becomes obvious: tune the server, add a cache, or scale out behind a load balancer.