Latency vs Bandwidth

The problem

Fahim has a cousin in Canada who tried Adda and gave a blunt review: "It's nice, but it feels kind of laggy." This stings, because from Fahim's phone in Dhaka — a few kilometers from the laptop under Ria's desk — Adda feels instant.

Ria's first move is the obvious one: check the internet connection. It's fine. Her line is fast; she even upgrades her home plan to a fatter one, expecting the cousin's experience to snap into shape. It feels exactly the same to him. The bug reports keep the same shape: "the feed takes forever to load in Toronto." She throws more bandwidth at it and nothing changes. The reason is that "faster internet" quietly means one thing in her head, when it is really two very different things — and she just optimized the wrong one.

A first attempt

The instinct is: slow = not enough bandwidth, so buy a bigger pipe.

Sometimes that is right. If the cousin were downloading a 4 GB video from Adda, a fatter pipe would genuinely finish sooner. But loading a feed is not one giant transfer — it is many small requests, each waiting on a round trip between Toronto and Dhaka. Widening the pipe does nothing for the wait before data starts flowing. It is like widening a highway to fix a problem that is actually the distance between two cities: more lanes, same drive time. Throwing bandwidth at a latency problem is money set on fire.

The insight

Speed is two independent numbers, and confusing them is the root of most "why is it slow?" mysteries — including the cousin's.

• Latency — the delay before data starts arriving. Measured in milliseconds. Set mostly by distance and number of round trips. This is the when.
• Bandwidth — the rate data flows once it is moving. Measured in Mbps/Gbps. This is the how much per second.

Think of a pipe carrying water. Latency is how long until the first drop reaches you. Bandwidth is how wide the pipe is. A wide pipe does not make the first drop arrive sooner — and for small, chatty requests like loading Adda's feed, the first drop is almost the whole story. The cousin isn't short on bandwidth; he's paying a distance tax on every round trip to Dhaka.

How it works

The highway picture:

LATENCY = distance / trip time         BANDWIDTH = lanes / throughput

  City A ─────────────────── City B      ═══════════  narrow: 1 car/sec
        one lap = fixed minutes          ═══════════
                                         ═══════════  wide:  8 cars/sec
  More lanes do NOT shorten the lap.     ═══════════
  Only a shorter road (or fewer laps)    ═══════════
  reduces latency.                       more lanes ≠ shorter trip

The numbers

Latency to hold in your head (the "numbers every engineer should know"):

• L1 / L2 cache: ~1–7 ns
• Main memory (RAM): ~100 ns
• SSD random read: ~100 µs (100,000 ns)
• Same-datacenter round trip: ~0.5 ms
• Same-region network round trip: ~1–10 ms
• US coast-to-coast round trip: ~60–80 ms
• New York ↔ London round trip: ~55–80 ms (speed-of-light floor ~55 ms)
• New York ↔ Sydney round trip: ~150–250 ms

And the arithmetic that trapped Ria:

• Uploading a 10 MB video over 100 Mbps takes ~0.8 s of transfer — bandwidth-bound, so a bigger pipe helps.
• The cousin's feed making 20 sequential API calls over an 80 ms Toronto↔Dhaka link costs 1.6 s of pure round-trip waiting — latency-bound, so a bigger pipe helps not at all. Batching them into 2 calls cuts it to 160 ms.

Practice

Recap

• Latency is the delay before data starts (ms, driven by distance and round trips); bandwidth is the rate once it flows (Mbps/Gbps).
• Small, chatty requests like Adda's feed are latency-bound; large transfers are bandwidth-bound — diagnose which before you spend.
• Beat latency by cutting round trips and distance (batching, connection reuse, CDNs); beat bandwidth by shrinking bytes (compression, pagination).