URL Shortener

Twelve articles of components. Four articles of method. One deceptively simple problem to spend them on.

Everything converges here.

Bonus A estimated this exact system's numbers. Bonus B wrote its requirements. Bonus C classified its edge-caching trade-off. Bonus H drew its store map. And one question has been parked since the estimation article, waiting for today: how do four writes per second get unique short codes without collisions?

Welcome to Section 2. Time to draw the boxes — properly.

Why the URL Shortener Is the Gateway Problem

Every system design education starts here, and it's worth understanding why.

It's small enough to hold in your head. Two features. One table. You will never lose the plot.

It's complete enough to use everything. Look at what this "simple" problem genuinely requires: an API contract (Part 10), a key-generation strategy, a source-of-truth store (Blog 5, Bonus H), a cache with real TTL decisions (Part 6), a load-balanced stateless fleet (Parts 9, Bonus E), async analytics (Part 7), replication questions (Bonus I), a security surface (Part 11), and an observability plan (Part 12). Nothing is missing. Nothing is decoration.

And it's deceptively simple — which is the actual test. Weak designs of this system work perfectly in the demo. The difference between a working shortener and a designed one lives entirely in the questions most people skip: what happens at peak, what happens on failure, what must never be wrong.

So we won't skip them. We'll walk C.R.E.D in full — the first complete lap of the framework since Part 3 promised it.

A good architect does not ask only, "Does it work?" A good architect asks, "What must never be wrong, what bends under peak, and which failures were designed for before they happened?"

Step 1 — Clarify: The Questions Before the Boxes

The problem statement is one sentence: "Design a service that turns long URLs into short ones and redirects users."

An engineer starts drawing. An architect starts asking: custom aliases — can users request short.ly/diwali-sale, or are all codes generated? Expiry — do links live forever, or can they expire? Analytics — do we count clicks, and how fresh must counts be? Who creates links — anonymous users, authenticated users, API clients? Can links be updated or deleted after creation? Scale ambitions — a team tool or a public service?

Our chosen answers, stated as assumptions (Bonus A's rule: wrong-but-clear beats hidden):

Custom aliases:   YES, optional (creates a collision surface — noted)
Expiry:           OPTIONAL per link; default = no expiry
Analytics:        YES — click counts; freshness in hours is fine
Creators:         API clients + authenticated users; redirects are public
Update/delete:    Deactivate only — a mapping, once created, never CHANGES
Scale:            Public service at Bonus A's assumed volumes

That fifth answer looks minor. It's the quiet star of this design — hold it.

Step 2 — Requirements: The Building Code

Inherited from Bonus B, now official. Functional: create a short URL (generated or custom alias); redirect a short URL to its original; record clicks.

Non-functional (the ones that will actually shape the boxes):

Redirect p99          < 100 ms
Availability          99.9% for redirects (reads outrank writes here)
Read/write ratio      100:1
Retention             3 years
Consistency           Redirects must NEVER serve a wrong URL;
                      analytics may lag by hours;
                      a just-created link should work for its creator immediately
Privacy               No sensitive query parameters in logs

Read that consistency line closely — it contains three different consistency requirements for three different data behaviors (Bonus F's per-data discipline, in the wild).

Step 3 — Estimate: The Numbers (Inherited and Owned)

Bonus A did the arithmetic; the design must now answer to it:

Writes:    ~4/sec average            ← almost nothing
Reads:     ~385 QPS avg, ~4,000 peak ← the actual problem
Storage:   360M records × 500 B ≈ 180 GB raw, ~½ TB replicated
Hot set:   ~18 GB — fits one cache node comfortably

The numbers issue their verdict before any diagram exists: this is a read-serving problem with a trivially light write path. Storage is a non-issue. The design's center of gravity is the redirect at peak, and the one genuinely interesting write question is key generation.

Which means we know where to spend our complexity budget, and where to refuse to.

Step 4 — Design: The High-Level Architecture

The shape, both paths on one diagram:

WRITE PATH (~4/sec)                    READ PATH (~4,000 QPS peak)
──────────────────                     ────────────────────────────
POST /shorten                          GET /aX9k2Bc
     │                                      │
     ▼                                      ▼
Load Balancer (Bonus E)                Load Balancer
     │                                      │
     ▼                                      ▼
API Service (stateless, Part 9)        Redirect Service (stateless)
     │                                      │
     ▼                                      ▼
Key Generation ──▶ DB write            Cache lookup (Part 6)
(the interesting part)  │                hit (90%+) ──▶ 302 redirect
                        │                miss ──▶ DB read ──▶ fill cache ──▶ 302
                        ▼                       │
                 DB = SOURCE OF TRUTH           ▼
                        │              Click event ──▶ queue (Part 7, async)
                        ▼                       │
                 outbox event                   ▼
                 (Bonus H)              Analytics store (hours-fresh)

Bonus H's store map, realized: one relational database as source of truth. One cache, derived and evictable. One event stream. One analytics store. Four stores, each with a requirement's name on it — and a search index deliberately left on the shelf.

Now the two paths, properly.

The Read Path: Spending a 100 ms Budget

The redirect is the product. Walk its budget (Bonus A's latency-table discipline):

LB + routing            ~5 ms
Redirect service        ~5 ms
Cache lookup            ~1–2 ms      (the 90%+ case)
Response                ~5 ms
                        ─────
Cache-hit total         ~15–20 ms    — comfortably under 100
Cache-miss total        +10–100 ms   (DB read) — still inside budget, barely

Two design consequences fall straight out.

Cache-aside, exactly as Part 6 taught it — with one gorgeous twist. Remember Clarify's fifth answer: a mapping, once created, never changes. Immutable data makes staleness nearly harmless. The usual caching nightmare — "what if the cached copy is wrong?" — mostly evaporates: a cached mapping can't drift from truth, because truth doesn't move. TTLs can be long; invalidation is only needed for the rare deactivation. This is why architects hunt for immutability: it's the cheapest consistency guarantee that exists.

Nothing synchronous rides the redirect except the redirect. The click event is fired to the queue after the response heads out (Part 7). Analytics has no business in this budget — Bonus B's requirement said hours-fresh, and hours-fresh it shall be.

The 301 vs 302 Decision

The redirect itself carries a design decision most candidates never notice.

HTTP 301 — Moved Permanently. Browsers and intermediaries may cache the redirect itself, aggressively. Subsequent clicks never reach your servers. Blisteringly fast — and you've just lost click analytics and the ability to ever deactivate or repoint the link for that user.

HTTP 302 — Found (temporary). Every click comes back to you. You keep analytics, expiry enforcement, and deactivation power — and you pay for it in traffic, which your cache absorbs.

Notice the type of decision (Bonus C): there's no physics here, both work. It's a business trade-off — is click data and link control worth serving every request? Our requirements say analytics matter, so: 302. A shortener with no analytics requirement should argue for 301 — and saying that sentence out loud is worth more in an interview than either choice.

The Write Path: Key Generation

The parked question, at last. Four writes per second need unique 7-character codes. First, the arithmetic that frames everything:

Base62 = [a–z, A–Z, 0–9] — 62 characters
62⁶ ≈ 56 billion combinations
62⁷ ≈ 3.5 trillion combinations
We need 360 million over 3 years
→ 7 characters uses ~0.01% of the space. Room for decades.

Now the strategies, with their honest bills.

Option 1: Hash the URL. hash(long_url) → take the first 7 characters. Gift: the same URL always yields the same code — free deduplication. Bill: truncated hashes collide — two different URLs, same 7 characters — so every write needs a collision check and a salted retry. And if two different users shorten the same URL, dedup means they share a link (and its analytics) — is that even wanted?

Option 2: Random generation. Generate 7 random base62 characters; check the DB; retry on the rare collision. Gift: simple, stateless, unpredictable (privacy-friendly — codes can't be enumerated by counting). Bill: every write pays a uniqueness check; collisions grow (slowly) as the space fills — at 0.01% occupancy, retries are vanishingly rare.

Option 3: Counter + base62 encoding. Keep a counter; encode each new ID into base62 (125,000,001 → "8m0Kx"). Gift: zero collisions, by construction. No check needed. Bill 1: a single global counter is a write hotspot and a SPOF — Bonus G would like a word. The fix: range allocation. Each API instance grabs a block (say, 100,000 IDs) from the coordinator, then issues locally from memory — the "counter" is consulted once per hundred-thousand writes, and instances stay effectively stateless (Part 9: a lost block just wastes numbers, the space forgives). Bill 2: sequential codes are enumerable — anyone can walk aX9k2Bc, aX9k2Bd, aX9k2Be… and discover every link ever created. For private-ish links, that's a real privacy leak. The mitigation: scramble the bits or mix in a random component before encoding, so consecutive IDs don't yield consecutive codes.

The Verdict

For our requirements: range-allocated counter + base62, with scrambling — collision-free writes, no per-write uniqueness check, hotspot avoided, enumeration blunted. And an honest footnote: random-with-check is equally defensible at this scale; the retry cost is negligible. This is a Bonus C maturity-flavored choice more than a physics one — pick the machinery your team can operate, and say why.

Custom aliases ride the same table with a uniqueness constraint — first come, first served, a clear error on conflict, and reserved-word filtering (/admin, /api) so a user can't claim your own routes.

The Data Model

The whole system, one table plus a stream:

url_mappings                              (source of truth — Bonus H)
─────────────────────────────────────────
short_code    CHAR(7)    PRIMARY KEY      ← lookup key AND shard key someday
long_url      TEXT       NOT NULL
creator_id    BIGINT     NULL
created_at    TIMESTAMP  NOT NULL
expires_at    TIMESTAMP  NULL
is_active     BOOLEAN    DEFAULT true     ← deactivation without deletion

click_events                              (stream → analytics, Part 7 / Bonus H)
─────────────────────────────────────────
short_code, timestamp, country, referrer, user_agent_class
— NO raw query parameters, NO PII        ← Bonus B's privacy NFR, enforced at the schema

Note what's absent: no click_count column on the mappings table. Incrementing a counter on every redirect would put a write on the read path — 4,000 write QPS at peak on a table designed for 4 — and hand Bonus G a hot row for every viral link. Counts live in the analytics store, computed from events, hours-fresh, exactly as required.

Scaling the Design

Walk the layers against the numbers.

Compute — both services are stateless (Part 9); the fleet scales horizontally behind Bonus E's balancer with health checks and draining. At ~4,000 peak QPS of cache-hits, a small fleet is bored.

Cache — the 18 GB hot set fits one node; add a replica for availability. At 90% hit rate, the database sees ~400 QPS at peak.

Database — ~400 QPS of primary-key lookups is a quiet afternoon for one well-indexed instance. Add read replicas for availability and read headroom (Bonus I) — and note the immutability gift again: replica lag on existing mappings is harmless, because they don't change.

Sharding — not yet, and say so proudly (Bonus H's discipline: starting simple is not junior). At ½ TB over three years, one database with replicas carries this design. If growth demands it someday: shard by hash of short_code — random-ish codes distribute beautifully (Bonus G) — and never, ever by creation date, or every day's traffic melts one shard.

Growth trigger, written down (Bonus C's default-plus-trigger): revisit sharding when the working set outgrows cache economics or write volume grows 100×.

Failure Analysis: Where This Design Bends and Breaks

A design isn't finished when the happy path works. Walk the failures deliberately.

The cache dies (or restarts cold). Suddenly 4,000 QPS of misses land on a database sized for 400 — Part 6's stampede, verbatim. Defenses, already in the toolkit: request coalescing on misses, jittered TTLs, cache warming after deploys, and the honest capacity question — can the DB survive a cold cache at peak? If not, the cache is a load-bearing dependency and gets designed like one (Part 6's rule).

A brand-new link 404s for its creator. The one place immutability doesn't save us: the link didn't exist two seconds ago, and a lagging replica hasn't heard yet (Bonus I). The fix is surgical, not global: read-your-writes for the creator — their confirmation view reads the primary (or the fresh cache entry written at creation). Everyone else discovers the link seconds later anyway.

A celebrity link goes viral. One code, a million clicks an hour — Bonus G's hot key. And here's why our design shrugs: it's a read hotspot on immutable data, the single most cacheable object in computing. The cache absorbs it; a 1-second TTL would absorb it even if it were mutable (Part 6's cricket score). This hotspot is the acceptable kind — known, bounded, covered (Bonus G's taxonomy).

Key generation collides or exhausts. Random path: retry-on-collision, monitored — a rising retry rate is the early-warning metric. Counter path: collisions are impossible; exhaustion at 62⁷ is a next-generation problem. A lost ID block wastes 100,000 numbers out of 3.5 trillion — the space absorbs carelessness by design.

The analytics pipeline fails. Events queue up; consumer lag grows; the DLQ catches poison events (Part 7's full kit). And nothing user-facing notices — the redirect path never depended on it. That isolation was purchased in the architecture, not hoped for.

A redirect instance turns zombie. Slow-but-alive, ruining every Nth redirect — Bonus E's outlier detection ejects it on per-instance p99, not health checks.

Every failure above has the same shape: named, bounded, and answered by a mechanism chosen in advance.

A system is not designed when it works. It is designed when its failures are understood.

Security and Abuse: The Part Everyone Skips

A public URL shortener is, bluntly, a redirection service anyone can weaponize — it launders scary URLs into innocent-looking ones. Design for it (defensively, per Part 11's discipline): malicious destination handling (validate URLs at creation — scheme allow-list, no redirect loops back into ourselves — check destinations against reputation/blocklist services, and support rapid deactivation, which is exactly why we chose 302 and is_active over 301's permanence); rate-limit creation (per API key and per user, Parts 10, 11 — an unthrottled write API is a spam cannon with your domain's reputation attached); blunt enumeration (the scrambled key generation from earlier — sequential codes are a directory of every link ever made); expiry as a safety tool (optional per-link expiry limits the lifetime of anything that slips through); privacy in logs (destination URLs can carry tokens and personal data in query strings — the schema strips them from events, and Part 12's logs follow the same rule); and auth on the write side (creation and deactivation require identity, Part 11 — redirects stay public by design).

None of this is exotic. All of it is the difference between a demo and a service someone else's security team will tolerate.

The Observability Plan

Part 12's table, applied — what this system emits from day one:

SLO:        99.9% of redirects succeed in < 100 ms   ← the promise, measured
Redirect:   p95/p99 latency, 404 rate, 302 volume
Cache:      hit rate (alert < 85%), evictions, hot keys
Database:   query p99, replica lag, connection pool usage
Pipeline:   consumer lag, DLQ size (alert > 0), event throughput
Writes:     creation success rate, key-collision retry rate
Business:   clicks per hour, creations per day, top links
Synthetic:  a robot creating + clicking a test link every minute, per region

Correlation IDs stamped at the balancer, carried through the queue. One runbook per alert. The 2 AM story assembles from one search — because we designed it to.

What Interviewers Actually Probe

The follow-ups this problem exists to generate, and the shape of strong answers. "301 or 302?" — name both, name the trade (permanence and speed vs. analytics and control), tie to the stated requirements; the trade-off is the answer. "How do you generate keys without collisions?" — the three options with bills attached; picking one and defending the runner-up signals real judgment. "What if two users shorten the same URL?" — depends on the dedup decision from the hash discussion; the strong move is noticing it's a product question wearing a technical costume. "One link gets a million hits an hour." — immutable read hotspot, the cache's favorite meal; walk the cold-cache case to show you know where the real risk hides. "How would you shard?" — hash of short_code, never date, and open with "not yet, and here's the trigger." "Where do click counts live?" — not on the mappings table; walk the write-on-read-path trap. "What breaks first at 100×?" — cache economics, then DB read capacity, then the analytics pipeline, in that order, with the metric that announces each.

What We Deliberately Didn't Build

Bonus C's discipline, applied out loud — the shelf of tools not taken. Custom domains (links.yourbrand.com) — real feature, pure product scope; the architecture barely changes. Full user account management — Part 11 covered the machinery; this design just consumes it. Multi-region active-active — 99.9% doesn't demand it, the cost and failover-drill muscles aren't justified; when global latency matters, the first step is edge caching, and how the world's users reach the nearest edge is DNS and Anycast territory (Bonus D, when it lands). ML-based abuse detection — blocklists and rate limits first, models when the abuse volume earns them. QR codes, link-in-bio pages, browser extensions — product surface, not system design.

Every omission has a reason and a trigger. That's not laziness — that's the budget being spent where the requirements are.

Common Mistakes Engineers Make with This Problem

Jumping to boxes before questions — the C in C.R.E.D exists, use it out loud. Designing for imaginary scale — sharding a half-terabyte dataset on day one, Bonus A's expressway again. A click_count column on the mappings table — a write on the read path, plus a hot row per viral link. Choosing 301 for "performance" with an analytics requirement on the table — the trade-off wasn't read. Sequential, enumerable codes — every link ever created, discoverable by counting. Synchronous analytics in the redirect — the 100 ms budget, donated to a dashboard. No collision story — "hashes are basically unique" is a retry loop you haven't written yet. Ignoring the cold-cache case — the design that works only while the cache is warm (Part 6's dependency rule). No abuse surface thinking — a public redirector with no rate limits is someone else's phishing infrastructure. No observability plan — a design review that ends before Part 12's table gets filled in isn't finished.

From My Journey: An Architectural Lesson

The URL shortener has a reputation as a beginner's problem — and many engineers, encountering it early, walk away with a working solution and a mild sense of anticlimax. A table, a hash function, a redirect. Done.

The anticlimax is the trap.

Because the distance between that working solution and a designed system turns out to contain nearly everything this series has covered. The working version answers "can it shorten URLs?" The designed version answers the questions production actually asks: what happens at 4,000 QPS when the cache restarts cold? Why does a just-created link 404 for its own creator? What does one viral link do to the storage layer? Which HTTP status code quietly decides whether analytics can exist at all? What does an abuse team need from this system on day one?

The realisation this problem exists to teach: simple features do not mean simple systems. The feature list fits in a sentence; the guarantees, failure modes, and trade-offs fill an article — and knowing which parts must never be wrong is the entire discipline.

That's why it's the gateway problem. Not because it's easy — because it's honest about what design actually is at a size where you can see all of it at once.

A system is not designed when it works. It is designed when its failures are understood.

Key Takeaways

• The URL shortener is the gateway problem because it's small enough to hold and complete enough to demand every building block.
• Clarify before boxes: aliases, expiry, analytics, mutability — the answers reshape the design, and immutability was the quiet star.
• Immutable data is the cheapest consistency guarantee in existence: long TTLs, harmless replica lag, a cache's favorite meal.
• 301 vs 302 is a business trade-off wearing an HTTP costume: permanence and speed vs. analytics and control.
• Key generation is the write path's real question: hash (dedup, collisions), random (simple, check-per-write), or range-allocated counter + base62 (collision-free, scramble against enumeration).
• No click_count on the mappings table — never put a write on the read path.
• Scale honestly: cache + replicas now; sharding by hashed short_code only when a written trigger fires, never by date.
• Every failure named in advance: cold cache, creator's 404, viral link, pipeline lag — each answered by a mechanism, not hope.
• A public redirector is an abuse surface: validate, rate-limit, deactivate fast, strip PII from logs.
• The design ships with its SLO, its dashboard, and its synthetic robot — or it isn't finished.

Interview Lens

This problem is the interview — so this section is short and structural.

The winning shape is the article you just read: clarify out loud (two minutes, maximum), state estimates with assumptions, let the numbers declare the problem type ("read-serving, light writes — so my complexity budget goes to the redirect path and key generation"), design both paths, then volunteer the failure analysis before being asked.

That last move — walking into "what breaks?" uninvited — is the single strongest signal in the whole interview. Most candidates wait to be pushed there. The ones who go there themselves are the ones the debrief calls senior.

And the meta-skill: every probe in this problem is a trade-off wearing a question mark. 301/302, hash/counter, shard/don't — none has a right answer; all have a defended answer. Bonus C gave you the taxonomy; this problem is where you perform it.

Real-World Engineering Lens

Use this article as a design-review template. The section headers — Clarify, Requirements, Estimate, Design, Failure Analysis, Security, Observability, Deliberately Didn't Build — are a review agenda for any system, not just this one.

Steal the immutability hunt. In every design, ask: which data, once written, never changes? Each answer is free consistency — long TTLs, lag immunity, cache confidence.

Steal the "didn't build" section especially. Design docs that list rejected scope with triggers age better than docs that list only what was built.

And run the failure walk before the review, not during the incident. Cold cache at peak, fresh-write lag, hot key, dead pipeline — four questions, fifteen minutes, every system.

What's Next

One system down. The pattern established: clarify, estimate, design, break it on paper, secure it, instrument it.

Now the problems get livelier — because the next system's data moves.

Blog 14 — Notification Service: Pub/Sub, Retry, and Delivery Guarantees. Part 7's machinery — queues, fan-out, retries, DLQs, effectively-once — promoted from concepts to the whole product. Where "the user must get exactly one email" meets a distributed system's sense of humor.

Question for Readers

Take the system you know best and run today's failure walk on it: cold cache at peak, a write that hasn't propagated, one key gone viral, the async pipeline down for an hour.

Four questions. How many have answers chosen in advance — and how many are currently scheduled to be answered by the incident?