Design a Video Platform
Adda adds video — and uploading, encoding, storing, and streaming to millions pulls together nearly every idea in this book. The team's final build.
The problem
For Adda's last big feature of the year, Ria wants video — creators posting clips, everyone watching. It's the design that pulls in nearly everything the team has learned, so they take their time at the whiteboard. Picture the two ends of it: a creator uploads a 4 K, 2 GB, ten-minute video from a laptop on hotel Wi-Fi. Minutes later, someone on a subway with one bar of signal taps play on their phone — and it starts almost instantly, at a resolution that quietly adjusts as their signal fades. Between those two moments, that single file has to be received, transformed into a dozen versions, stored durably, and pushed to servers near every viewer on Earth.
The naive version — "store the uploaded file, and when someone watches, stream that file to them" — fails on every axis. The upload dies halfway on flaky Wi-Fi. The 4 K file is unwatchable on a phone with weak signal. And streaming one stored copy from one datacenter to a global audience is both slow and ruinously expensive. A video platform, the team realizes, is really four systems stitched together: upload, encode, store, deliver.
Requirements
Functional
- Upload a video (resumable, large files).
- Transcode into multiple resolutions/bitrates.
- Stream on demand with smooth playback that adapts to bandwidth.
- Metadata: titles, thumbnails, view counts.
Non-functional
- Playback starts fast (< 2 s) and rarely buffers.
- Durable storage — an upload must never be lost.
- Scales to millions of concurrent viewers globally and petabytes of storage.
A scale estimate
Shuvo runs the firehose numbers:
- 500 hours of video uploaded per minute (YouTube-scale) → a firehose of ingestion and encoding work.
- Each source video is transcoded into ~6 renditions (240p → 4 K) × multiple codecs → encoding is CPU-heavy and slow (often longer than the video's runtime).
- Storage: raw + all renditions → petabytes, growing constantly.
- Viewing dwarfs uploading; delivery bandwidth is the dominant cost, which is why the CDN exists.
The insight
Uploading and watching are wildly different workloads that must be decoupled — the same instinct that split Adda's feed writes from its reads. The upload path is write-heavy, bursty, and slow (encoding). The watch path is read-heavy, latency-sensitive, and global. Between them sits an asynchronous pipeline: the user's upload finishes quickly, then a background system chews through encoding on its own schedule, and the watch path serves the finished renditions from the edge. Nothing on the fast paths waits for the slow work.
How it works
Upload resumably
The client uploads directly to object storage (S3-style) in chunks via pre-signed URLs, so a dropped connection resumes from the last chunk instead of restarting 2 GB. The app server just hands out URLs and records metadata — it never proxies the bytes.
Kick off encoding asynchronously
When the upload completes, drop a job onto the message queue — the same Part 3 queue that already carries Adda's fan-out work. The user's request returns "processing…" immediately. A fleet of encoding workers pulls jobs off the queue.
Transcode and segment
Workers split the video into small segments (2–10 s each) and encode each segment into every rendition — 240p, 480p, 720p, 1080p, 4 K — plus generate thumbnails. Segmenting lets workers process one video in parallel across many machines, and it's what makes adaptive streaming possible.
Store and distribute to the edge
Write all segments and a manifest to object storage, then push the hot ones to a CDN — the same edge network Tanvir set up in Part 1. Mark the video ready and update its metadata.
Stream with adaptive bitrate
The player fetches a manifest listing every rendition, then requests segments one at a time from the nearest CDN edge. Each segment fetch, it measures bandwidth and picks the next segment's quality — dropping to 480p when the subway signal fades, climbing back to 1080p when it recovers. This is HLS/DASH adaptive bitrate streaming.
Key decisions and trade-offs
Segmenting is the keystone
Cutting the video into short segments buys you three things at once: parallel encoding (many workers on one video), adaptive streaming (switch quality per segment), and cheap CDN caching (small, immutable files). One decision unlocks the entire watch experience.
Storage explodes — and most of it is cold
Six renditions plus the raw master multiply storage several-fold, and the vast majority of videos are watched rarely after their first week. Keep hot content on fast storage and at the edge; move cold content to cheap archival tiers. Don't pay SSD prices to store a video nobody's watched in a year.
Metadata (titles, view counts, the videoId → manifest URL mapping) is a separate, small, read-heavy service — much like the URL shortener drill the team ran — and is cached hard.
Bottlenecks and how to scale
- Encoding throughput: the queue absorbs upload bursts; scale workers horizontally to drain it. Prioritize lower resolutions first so a video becomes watchable before the 4 K rendition finishes.
- Delivery bandwidth: the CDN is non-negotiable — serving petabytes from the edge is what keeps playback fast and origin traffic (and cost) low. Cache hit rate at the edge is the metric to watch.
- Popular-video spikes: a viral video is a hot object; the CDN naturally absorbs it because segments are immutable and cacheable, so the origin barely notices.
- Upload reliability: resumable chunked uploads plus durable object storage (replicated across zones, echoing Mou's failover playbook from Part 4) mean a flaky network or a failed node never loses an upload.
Practice
Recap
- A video platform is four decoupled systems: resumable upload, async encoding, durable storage, edge delivery.
- Segmenting enables parallel encoding, adaptive bitrate streaming, and cacheable CDN files — one idea, three wins.
- Push encoding onto a queue, serve playback from a CDN, tier cold storage, and keep metadata a small cached key-value service.
CDNs
How the edge serves petabytes of video close to every viewer.
Message Queues
Decoupling the fast upload from the slow encoding pipeline.
Design a URL Shortener
The read-heavy key-value pattern the metadata service reuses.
In an interview
Resist diving into codecs. Frame the whole thing as upload → encode → store → deliver, and make "decouple the slow encoding from the fast paths with a queue" your spine. If you land two ideas — asynchronous encoding via a queue, and segmented adaptive streaming from a CDN — you've covered what matters. Bring up storage tiering and pre-signed uploads to show depth.
How is this guide?
Last updated on