Blog 17
Media Streaming and Recommendations
A photo is a file. A video is a broadcast — one that has to survive a train tunnel, a 4K TV, and a cracked phone on 3G, all playing the same title at the same second.
The bytes are the easy part.
Bonus D just placed every user at the nearest healthy edge — the prologue before the request. Today we hand that edge the heaviest cargo in computing.
Video is not "bigger images." It is a different category of problem. The numbers alone break every intuition Section 2 built: a single popular title, streamed globally, moves more data in an hour than the URL shortener would move in its entire lifetime. Streaming platforms account for the majority of internet traffic — not a share, the majority — and the architecture that makes that survivable looks almost nothing like the request-response systems we started with.
This is Section 3's payload-heavy peak. Same walkthrough template. A completely different physics.
Why Streaming Is a Different Category of Problem
The payload dwarfs everything. Part 8 warned that unstructured data is larger than everything else combined; video is the extreme of that extreme. One two-hour movie, in multiple qualities, is tens of gigabytes before a single viewer arrives. The design's center of gravity is bytes-in-motion, not requests-per-second.
The network is the enemy, and it's always changing. A URL redirect either works or doesn't. A video stream must survive a viewer walking from Wi-Fi into an elevator into cellular, mid-playback, without stopping. The system designs for a hostile, fluctuating channel, not around a reliable one.
Reads are absolute, writes are rare, and precompute is everything. A title is uploaded once and watched a billion times. That asymmetry (Part 13's insight at monstrous scale) means the expensive work happens before anyone watches — the opposite of a system that computes at request time.
And "the video" isn't one thing. It's dozens of files — every quality, every format, split into thousands of tiny chunks. What the user calls "a video" is an index pointing at an army of segments. Understanding that is understanding streaming.
A good architect does not ask only, "How do we store this video?" A good architect asks, "How do we make this billion-times-read, network-hostile, egress-dominated problem affordable?"
Step 1 — Clarify: The Questions Before the Boxes
"Design a video streaming platform" spans YouTube, Netflix, a live-sports app, and a corporate training portal — four different systems. The separating questions: on-demand or live — pre-recorded (encode ahead) or live (encode in real time), utterly different pipelines? Who uploads — a curated catalog (Netflix) or user-generated at scale (YouTube), changing the ingest problem by orders of magnitude? Device range — phones, TVs, browsers, low-end and high-end, how wide is the matrix? Personalized recommendations, or a simple catalog? DRM / content protection — licensed content or open? Scale — regional service or global billion-user platform?
Our assumptions, stated (Bonus A's rule):
Type: On-demand (VOD) — with a note on where live diverges
Uploads: Curated + creator uploads (both ingest paths matter)
Devices: Everything — phone, TV, browser, low-end to 4K
Recommends: Yes — personalized "what to watch next"
Protection: Yes — signed delivery; DRM noted, not deep-dived
Scale: Global, hundreds of millions — the numbers stop being polite
That first answer matters most: VOD lets us precompute everything. Live streaming removes that luxury, and we'll mark exactly where the design forks.
Step 2 — Requirements: The Building Code
Functional: upload/ingest video; transcode into multiple qualities and formats; store and distribute globally; stream with quality that adapts to the network; recommend what to watch next; track views and progress.
Non-functional — the numbers that shape the boxes:
Startup latency < 2s to first frame (every extra second sheds viewers)
Rebuffering near-zero — the ONLY quality metric users actually feel
Availability 99.9%+ for playback (a down player is a churn event)
Read/write ratio overwhelmingly read — upload once, stream billions
Global reach nearest-edge delivery everywhere (Bonus D placed them)
Cost egress-dominated — the bill IS the architecture (Part 8)
Recommendation freshness minutes-to-hours; approximate is fine
Content protection signed, expiring delivery; licensed content DRM-gated
Read the rebuffering line twice. It is the streaming NFR — the one metric that overrides the others. A viewer forgives a slightly soft picture; they abandon a spinning wheel. Every major design decision below serves that single number.
Step 3 — Estimate: The Numbers That Force the Design
Bonus A's method, and video's numbers are a different species:
Assume: 100M DAU, ~1 hour watched each per day
BANDWIDTH (the number that dominates everything)
Avg stream ~5 Mbps (mixed qualities)
100M concurrent-equivalent hours → peak concurrency maybe ~10M streams
10M × 5 Mbps = 50,000,000 Mbps = 50 Tbps at peak
↑ This is why the CDN is not optional — origin could never serve this
STORAGE (per title, before a single viewer)
One 2-hour movie, encoded into ~6 qualities × multiple formats
≈ 15–30 GB per title (vs the source ~5 GB)
→ encoding MULTIPLIES storage 3–6× — every title, every format
ENCODING COMPUTE (the hidden write cost)
1 hour of source → many hours of transcoding CPU/GPU across renditions
A creator-upload platform: thousands of hours uploaded per minute
→ the encoding farm is its own massive compute system (Part 9)
EGRESS COST (the bill that shapes the design)
50 Tbps of mostly-repeated content
→ CDN hit rate isn't a nicety, it's the difference between
a viable business and bankruptcy (Part 8's egress, Bonus A's math)
The verdict: storage and encoding are the write-side cost, bandwidth and egress are the read-side cost, and both are enormous. Unlike every prior system, there is no "light path" here. The design's entire job is making a fundamentally expensive problem affordable — and the CDN, plus precomputation, is how.
Step 4 — Design: The High-Level Architecture
Two pipelines that barely touch: the ingest/encode pipeline (rare, expensive, write-side) and the delivery pipeline (constant, optimized, read-side). Precomputation is the wall between them.
INGEST / ENCODE PIPELINE (write — happens ONCE per title)
────────────────────────────────────────────────────────
Creator/studio upload
→ object storage: raw master (Part 8, signed upload)
→ TRANSCODING FARM (Part 9 compute, queue-driven Part 7):
• decode master
• encode into ~6 quality renditions (240p … 4K)
• SEGMENT each rendition into ~2–10s chunks
• package into streaming formats (HLS / DASH)
• generate the MANIFEST (the index of all chunks/qualities)
• create thumbnails, extract metadata
→ store all outputs in object storage
→ write catalog metadata to DB (source of truth, Bonus H)
→ publish "title ready" event (Part 7)
DELIVERY PIPELINE (read — happens a BILLION times)
──────────────────────────────────────────────────
Viewer presses play
→ app fetches MANIFEST (tiny — lists qualities + chunk URLs)
→ player requests chunks from NEAREST CDN EDGE (Bonus D placed it)
• cache HIT (popular title) → served from edge, origin untouched
• cache MISS → edge fetches from object storage origin, caches, serves
→ player runs ABR: picks next chunk's quality by measured bandwidth
→ progress + view events → async pipeline (Part 7) → analytics + recommendations
Three inherited decisions, named. The store map (Bonus H): catalog DB is the source of truth; the encoded chunks live in object storage (Part 8); the CDN holds derived, evictable copies; view-progress and recommendations are derived. Nothing on the delivery path writes to truth. Precompute over compute-at-request (Part 13's asymmetry, maximized): every expensive operation — transcoding, segmenting, packaging — happens once, at ingest, so the billion reads are pure cache-friendly file serving. The CDN does the heaviest lifting in the series (Parts 4, 6): 50 Tbps served from edges, origin shielded to a trickle. The hit rate is the business model.
Deep Dive 1: Adaptive Bitrate — Surviving the Hostile Network
This is the mechanism that makes streaming feel reliable on an unreliable channel, and the single most important idea in the article.
The Problem It Solves
A viewer's bandwidth is never constant. Wi-Fi to elevator to cellular, congestion, a roommate starting a download — the available throughput swings second to second. Serve one fixed quality and you lose either way: too high, it buffers; too low, it looks terrible on a good connection.
The Mechanism: Many Qualities, Tiny Chunks, Per-Chunk Choice
The precompute pipeline already did the hard part — it encoded the video at multiple quality levels and sliced each into short segments (say, 2–10 seconds each). Delivery then becomes a series of small, independent decisions:
The manifest offers each ~4-second chunk at every quality:
chunk_047 available in: 240p, 360p, 480p, 720p, 1080p, 4K
The PLAYER decides, chunk by chunk:
measured bandwidth healthy? → request chunk_048 at 1080p
bandwidth just dropped? → request chunk_049 at 480p
buffer running low? → drop quality to avoid a stall
Two design truths hide in that loop. The client is the decision-maker — ABR logic lives in the player, not the server, because only the client can measure its own throughput and buffer health in real time; the server's job was to make every option available in advance. And quality degrades gracefully instead of stalling — the whole trade is deliberate (Bonus C, and Part 10's "partial answer beats a 500"): a viewer never notices a drop from 1080p to 720p mid-scene, they always notice a spinning wheel. ABR spends picture quality to buy the one thing users actually feel — continuous playback.
That is the rebuffering NFR, mechanized: adapt quality so the stream never stops. Short chunks make adaptation responsive; precomputed renditions make it instant; the client makes it accurate.
Why Chunks Change Everything
Chunking is the quiet hero. Because the video is thousands of independent segments, any chunk can be served by any edge, cached individually (Part 6). A quality switch is just "request the next chunk from a different rendition" — seamless. Seeking jumps to a chunk instead of streaming from the start. A failed chunk is retried alone, not the whole video (Bonus E's retry, scoped tiny).
"A video" was never a file. It's a manifest pointing at an army of cacheable chunks, and that structure is what makes global streaming possible at all.
Deep Dive 2: The Encoding Pipeline and CDN Economics
Encoding: The Expensive Write Nobody Sees
Transcoding is where the money and time go on the write side. One master becomes a matrix: qualities × formats × codecs, every one segmented and packaged. It's massively parallel (each rendition, even each chunk, is independent work), so it's a textbook queue-and-worker system (Part 7 plus Part 9): uploads publish jobs, a farm of encoding workers drains them, scaling on queue depth (Part 9's lesson, not CPU).
The trade-off worth naming (Bonus C): encode more renditions and formats and you get better device coverage and smoother ABR, but more compute and 3–6× more storage per title. A curated catalog can afford lavish encoding; a platform ingesting thousands of hours per minute must be ruthless about which renditions each title actually earns.
CDN Economics: The Hit Rate Is the Business
Return to the 50 Tbps. If the CDN serves it from edges at a 95%+ hit rate, the origin (object storage) sees a trickle, and the egress bill (Part 8's classic killer) stays survivable. Drop the hit rate and the origin melts and the bill explodes simultaneously.
So streaming platforms obsess over cache efficiency. Popular titles stay hot at every edge — the head of the catalog is cached everywhere, always. The long tail is the hard part — millions of rarely-watched titles can't all live at every edge; they're fetched from origin on demand and cached briefly. This is Bonus G's distribution problem: a few titles are watched constantly (easy to cache), a vast tail is watched rarely (expensive to serve). Cache economics are a power-law fight. Chunk-level caching means even a partially-watched long-tail title only pulls the chunks actually viewed. And pre-warming pushes an anticipated hit (a big premiere) to edges before the flood (Part 6's cache warming, at planetary scale).
The one-line truth: in streaming, the CDN hit rate is not a performance metric — it's the P&L.
Deep Dive 3: Recommendations — What Plays Next
Recommendations aren't decoration here; on a large platform, most watch-time originates from them. But this is a system design article, not an ML one, so the point is the shape, not the model.
The recommendation system is a classic precompute-vs-realtime split, the same tension as Blog 16's feed ranking, in a new costume:
OFFLINE (batch, hours-fresh):
crunch viewing history, similarities, trends
→ precompute candidate recommendations per user / per segment
→ store them ready to serve (derived, Bonus H)
ONLINE (request time, milliseconds):
fetch precomputed candidates
→ light real-time re-rank (just-watched, time of day, device)
→ return the row the user sees
Why the split (Bonus A's latency budget): scoring a giant catalog against a user at request time can't fit a fast response. So the heavy lifting happens offline, hours-fresh, which the NFR explicitly permits ("approximate is fine"), and only a cheap re-rank happens live. It's Blog 16's staged ranking again: precompute the expensive part, do the cheap part on the hot path.
The data feeding it is the view/progress event stream (Part 7) flowing to analytics, the same async pipeline that never sits on the playback path. Recommendations are downstream of playback, never in front of it.
Where Live Streaming Diverges
We chose VOD to keep precompute. Live removes it, so mark the fork. Encoding happens in real time, as the event unfolds — no ingest-ahead luxury, a real-time transcode pipeline with tight latency. Latency becomes a hard product metric — "live" means seconds behind reality, not minutes, the glass-to-glass delay is the whole game. The thundering herd is guaranteed — everyone watches the same moment simultaneously (the goal in cricket, the World Cup final), so there's no long tail to smooth the load, just one colossal synchronized spike. Chunks and CDN still apply — live is delivered as a rolling window of chunks, so ABR and edge caching still work, the manifest just keeps growing at the live edge.
The through-line: VOD trades storage for cheap delivery (precompute everything); live trades latency for immediacy (compute in the moment). Bonus C's physics, one problem, two settings.
Failure Analysis: Where This Design Bends and Breaks
A popular title isn't cached at an edge (cold cache). The edge fetches from origin, one origin read, then it's hot for everyone after. The danger is a synchronized miss (a new release at midnight): request coalescing at the edge so one origin fetch serves the thundering herd, plus pre-warming for anticipated hits (Part 6, at CDN scale).
A viewer's bandwidth collapses mid-stream. ABR's entire reason to exist: the player drops to a lower rendition chunk-by-chunk; playback continues, softer, without a stall. The failure is designed into the mechanism, not handled after it.
The encoding farm falls behind. Upload backlog grows; new titles take longer to become available (Part 7's consumer lag, watched). Playback of existing titles is completely unaffected, the pipelines are walled apart. Scale encoding workers on queue depth (Part 9).
An origin (object storage) region degrades. Edges keep serving cached (hot) content happily; only cold-cache misses for that origin suffer. Multi-region origin replication (Part 8) plus edge caching means most viewers never notice.
A live event draws 10× the expected audience. No long tail to help, a pure synchronized spike. Edge capacity, aggressive chunk caching (every viewer wants the same current chunk, a perfect cache hit, ironically), and the CDN's absorption carry it. The hot "key" here is the live edge chunk, and it's maximally cacheable.
DRM/license server is slow. Playback authorization stalls before the first frame. The license check is on the startup path, so it gets the same treatment as any hot dependency — caching of entitlements, tight timeouts, graceful messaging — because startup latency is an NFR.
Every failure: named, bounded, answered in advance, and the biggest recurring risk (the synchronized cold-cache herd) is answered by coalescing and pre-warming, chosen before the premiere, not during it.
Security and Abuse: The Part Everyone Skips
Streaming's security surface is dominated by one thing most engineers underestimate: content is valuable, and delivery is the theft vector. Design defensively (Part 11): signed, expiring delivery URLs (Parts 8, 11) — chunk and manifest URLs are time-limited and tokenized, so a leaked link dies quickly and can't be hotlinked into someone else's player. DRM for licensed content — encryption plus license-server-gated keys, so possessing the chunks isn't possessing the video; deep DRM mechanics are their own field, the system point is that the license check sits on the startup path and must be fast. Token-authenticated playback sessions — entitlement checked at start (is this user allowed this title, in this region?), not merely at catalog browse. Geo and licensing restrictions — content rights are per-region, enforcement rides Bonus D's geo-routing plus entitlement checks (a licensing requirement, not just a technical one). Anti-scraping and rate limits (Part 10) — bulk-download bots that try to vacuum the catalog get throttled per account/IP. Upload safety (creator platforms) — uploaded content needs scanning and moderation hooks before wide distribution, the ingest pipeline is an attack surface, not just a convenience. No sensitive data in delivery logs (Part 12) — viewing history is sensitive, who-watched-what stays protected.
The Observability Plan
Part 12's table, tuned to the metrics viewers actually feel:
SLO: 99.9% of playback starts < 2s; rebuffer ratio < 0.5%
Playback QoE: startup latency, REBUFFER RATIO (the metric that matters most),
average bitrate served, quality-switch frequency
CDN: hit rate per region (the P&L metric), origin offload %,
edge latency, origin fetch rate
Encoding: queue depth, jobs/hour, time-to-availability, failure rate
Delivery: chunk error rate, manifest fetch latency, ABR downshift rate
Origin: object storage read load, egress volume + COST, region health
Recommends: candidate freshness, click-through, watch-time attribution
Business: concurrent streams, watch hours, completion rate, churn signals
Synthetic: robot players in multiple regions starting + streaming test titles
Two metrics get the highlight. Rebuffer ratio is the north star — it's the NFR made visible, the number that predicts churn. And CDN hit rate per region is watched like a financial instrument, because in this system it literally is one: a few points of hit-rate drop is a budget line moving by a visible fraction.
What Interviewers Actually Probe
"How does the video adapt to changing network conditions?" — ABR: precomputed renditions, short chunks, client-side per-chunk quality choice; naming that the player decides, and why (only it can measure its own throughput), is the differentiator. "Why not just store one high-quality file?" — because a fixed bitrate buffers on bad networks and wastes bandwidth on good ones, and because devices differ; the renditions matrix is the answer. "What is a video, exactly, in storage?" — not a file: a manifest indexing thousands of chunks across qualities and formats; candidates who describe this understand streaming. "How do you serve 50 Tbps affordably?" — CDN hit rate as the business model, origin shielded to a trickle, the long-tail caching problem named honestly. "Live vs on-demand — what changes?" — precompute disappears, real-time encoding, latency as a hard metric, guaranteed synchronized herd. "Where do recommendations run?" — offline precompute plus light online re-rank, downstream of playback, never on the hot path. "What breaks first at 10×?" — origin egress and cold-cache herds, then encoding backlog, each with its metric.
The meta-move: precompute versus deliver. Nearly every streaming decision — encoding, chunking, ABR renditions, recommendations — is "do the expensive work once, ahead of time, so the billion reads are cheap." Naming that pattern shows structural understanding, not memorized parts.
What We Deliberately Didn't Build
Codec and encoding internals — H.264 vs. AV1, bitrate ladders, per-title encoding optimization: a deep specialty; we designed the pipeline shape, not the compression math. The recommendation ML — models, embeddings, training: Blog 16's boundary again; we designed the serving system, not the algorithm. Full DRM implementation — license servers and key exchange are a security specialty; we placed the check on the startup path and moved on. Video calling / real-time communication — that's WebRTC and Part 15's territory; interactive real-time media is a different discipline from broadcast streaming. Deep live-streaming build-out — we marked where it forks; the full low-latency-live design is its own article. Content moderation systems — safety-critical and largely separate; ingest provides the hooks.
Common Mistakes Engineers Make with This Problem
Treating a video as a single file — it's a manifest over thousands of chunks, missing this misses streaming entirely. Serving one fixed quality — buffers on weak networks, wastes bytes on strong ones, ignores device diversity, ABR exists for a reason. Putting ABR logic on the server — only the client can measure its own bandwidth and buffer, server-side adaptation is blind. Streaming from origin instead of the CDN — the 50 Tbps that bankrupts you and melts the origin simultaneously. Ignoring the long tail — assuming everything caches perfectly, the power-law tail is the real cache-economics fight (Bonus G). Encoding at request time — the expensive work belongs at ingest, once, not on the hot path a billion times (Part 13's asymmetry, violated). Synchronous encoding blocking upload response — ingest publishes a job and returns, the farm drains async (Part 7). No pre-warming for known spikes — the midnight premiere that cold-misses at every edge simultaneously. Ignoring egress cost — the architecture that works technically and bankrupts the business (Part 8, Bonus A). Recommendations on the playback hot path — heavy scoring blocking the first frame, it belongs offline plus light re-rank. No rebuffer-ratio monitoring — flying blind on the one metric users actually feel and churn on. Unsigned delivery URLs — the content leak that becomes someone else's free streaming service.
From My Journey: An Architectural Lesson
Video streaming looks, at first, like the simplest possible idea: store a video file, send it to whoever presses play. Many engineers picture exactly that — a file on a server, streamed down a pipe — and it even works for one viewer on a good connection.
Then reality arrives in three waves. The network wave: viewers on trains, in elevators, on throttled cellular, all mid-playback. The scale wave: not one viewer but ten million, at the same instant, for the same premiere. The cost wave: the bandwidth bill for serving raw video globally is large enough to end the company before the product finds an audience.
The realisation that reorganizes everything: the video was never the system. The delivery is the system. The actual engineering isn't storing bytes — it's precomputing every quality ahead of time, slicing everything into cacheable chunks, letting the client adapt in real time, and leaning on a global edge network so hard that the origin barely works. The file is trivial; making that file survive a hostile network, a synchronized crowd, and a finite budget is the entire discipline.
And every part of it is the same trade relocated. Precompute-vs-compute decides encoding. Quality-vs-continuity decides ABR. Storage-vs-delivery-cost decides the rendition matrix. Cache-hit-vs-egress decides whether the business lives. There was never a design that made video cheap, only a design honest about paying the cost once, up front, so the billion reads could be cheap.
The lesson that stuck: a streaming platform does not deliver video. It delivers the illusion of one continuous stream, assembled from thousands of small, cached, adaptively-chosen pieces.
Key Takeaways
- Video is a different category: payload-dominated, network-hostile, read-absolute — the design's job is making an expensive problem affordable.
- "A video" is not a file — it's a manifest indexing thousands of chunks across multiple qualities and formats.
- Precompute everything at ingest (transcode, segment, package) so the billion reads are cheap, cache-friendly file serving — Part 13's asymmetry at monstrous scale.
- Adaptive bitrate is the core mechanism: precomputed renditions plus short chunks plus client-side per-chunk quality choice equals a stream that degrades gracefully instead of stalling.
- The client owns ABR, because only it can measure its own bandwidth and buffer in real time.
- The CDN does the heaviest lifting in the series; its hit rate isn't a performance metric — it's the P&L (Part 8's egress made existential).
- The long tail is the real cache fight: a few titles watched constantly, a vast tail rarely — a power-law distribution problem (Bonus G).
- Recommendations are an offline-precompute plus online-light-rerank split, downstream of playback, never on the hot path.
- Live streaming trades precompute for real-time encoding and turns the synchronized herd into the central challenge.
- Rebuffer ratio is the north-star metric — the one users actually feel, and the one that predicts churn.
Interview Lens
Streaming rewards structural thinking over recalled facts, so lead with the two ideas that organize everything.
Open with "a video isn't a file." The moment you explain that a video is a manifest over thousands of chunks encoded at multiple qualities, every other answer — ABR, CDN caching, seeking, quality switching — follows naturally, because they're all consequences of that one structure.
Then name the precompute-vs-deliver axis. Encoding, chunking, renditions, recommendations, all "expensive work done once so reads are cheap." That single framing turns a sprawling problem into one organizing principle.
Let the numbers force the CDN (50 Tbps can't come from origin), let the hostile-network requirement force ABR, and volunteer the failure walk — cold-cache herd, bandwidth collapse, encoding backlog — with the mechanism attached to each.
The meta-skill: recognizing that the file is trivial and the delivery is the system. Candidates who obsess over storing the video miss the problem; the ones who obsess over serving it affordably to a hostile, global, synchronized audience have understood it.
Real-World Engineering Lens
Find the "encode once, serve many" pattern in your own systems. Any expensive computation feeding many reads — reports, thumbnails, search indexes, derived views — is the streaming lesson in miniature: precompute it, cache it, serve it cheap.
Watch your egress like a P&L line. Anywhere you serve large payloads (files, media, exports), the CDN hit rate is a financial metric, instrument it as one (Part 8, Bonus A).
Chunk large deliverables. Serving anything big — files, datasets, media — as independent, cacheable, resumable pieces beats one monolithic stream: better caching, better retries, better seeking (Bonus E's scoped retry).
Measure what users feel, not what servers report. Rebuffer ratio over average bitrate; startup latency over CPU. The QoE metric is the one that predicts churn (Part 12's business-metrics lesson).
Pre-warm for known spikes. Premieres, launches, campaigns — push content to the edge before the flood, never during it.
What's Next
We've delivered the heaviest reads on the internet. But we've been generous about one thing: streaming tolerates eventual consistency and approximate counts. A view counted twice, a recommendation an hour stale — nobody's harmed.
The next category removes that mercy entirely. When money moves, "approximately once" is a lawsuit, and "eventually consistent" is a missing payment.
Blog 18 — E-Commerce and Marketplaces: Catalog, Cart, Orders, and Payments. Where correctness stops being negotiable — inventory that can't oversell, carts that survive, orders that happen exactly once, and the consistency battles the whole series has been building toward.
Question for Readers
Find the most expensive-to-compute thing your system serves on a hot path — a report, a search result, a rendered view.
Are you computing it per request, or precomputing it once and serving cheap copies?
Streaming's entire architecture is that one question, answered at planetary scale. Your system is probably asking it too — quietly, on some endpoint, right now.