System Design Diaries #4: How the Internet Handles Fame

Hey Readers.

Spotted: a startup celebrating one million users.

The founders are thrilled.

The engineers? Slightly terrified.

Because every successful application eventually runs into the same problem:

What happens when everyone shows up at once?

The features that worked perfectly for a thousand users suddenly struggle under the weight of a million. Requests pile up. Servers become overwhelmed. Databases start sweating. And if the architecture isn't prepared, success can quickly turn into downtime.

Today we're exploring the technologies that help systems survive popularity: proxy servers, load balancers, databases, ACID transactions, and scaling strategies.

Because on the internet, handling success is often harder than achieving it.

Proxy Servers: The Gatekeepers of the Internet

Imagine trying to meet a celebrity.

You probably wouldn't contact them directly. Instead, you'd go through an assistant, manager, or publicist.

That's essentially what a proxy server does.

A proxy sits between two parties and forwards requests from one side to another.

Instead of communicating directly, communication passes through an intermediary.

But not all proxies serve the same purpose.

Forward Proxy: Protecting the Client

A forward proxy stands between users and the internet.

Client
↓
Proxy
↓
Internet

When a user sends a request, the destination website sees the proxy instead of the actual user.

This creates a layer of separation that provides privacy and control.

Common examples include:

• VPNs

• Corporate internet gateways

• School and university networks

Organizations often use forward proxies to monitor internet usage, block specific websites, or enforce security policies.

VPNs use a similar concept by masking a user's real IP address.

From the website's perspective, the request appears to come from the proxy server rather than the original client.

Reverse Proxy: Protecting the Server

Now let's flip the situation.

Instead of protecting users, a reverse proxy protects servers.

Client
↓
Reverse Proxy
↓
Application Servers

Users interact with the reverse proxy, which then forwards requests to backend servers.

Most users never communicate directly with production servers.

Instead, the reverse proxy acts as the front door.

Popular reverse proxy solutions include:

• Nginx

• HAProxy

Reverse proxies provide several advantages:

Security

Backend servers remain hidden from the public internet.

Attackers see the reverse proxy rather than the actual infrastructure.

SSL Termination

The reverse proxy handles HTTPS encryption and decryption, reducing the workload on application servers.

Caching

Frequently requested content can be served directly by the reverse proxy without contacting backend services.

Load Balancing

Traffic can be distributed across multiple servers automatically.

Key Takeaways

• Forward proxies protect clients.

• Reverse proxies protect servers.

• Reverse proxies often provide security, caching, SSL termination, and load balancing.

Load Balancers: Preventing Popularity From Becoming a Problem

Imagine a flash sale attracting hundreds of thousands of visitors.

If every request hits a single server, that server eventually crashes.

When that happens:

Users
↓
Server
↓
Crash
↓
Downtime

Nobody wins.

A load balancer solves this problem by distributing traffic across multiple servers.

Users
↓
Load Balancer
↓
Server 1
Server 2
Server 3

Instead of relying on a single machine, the workload is shared.

This improves both performance and reliability.

Why Load Balancers Matter

Load balancers help systems achieve:

• Better scalability

• Higher availability

• Improved fault tolerance

If one server fails, requests can simply be routed to healthy servers.

Users may never even notice the failure.

Common Load Balancing Algorithms

Round Robin

Requests are distributed sequentially.

S1 → S2 → S3 → S1 → S2

Simple and widely used.

Least Connections

Requests are sent to the server currently handling the fewest active connections.

This works particularly well when requests vary in duration.

IP Hashing

The same user is consistently routed to the same server.

Useful for applications that maintain session-specific state.

Weighted Round Robin

Some servers are more powerful than others.

A server with greater resources receives a larger share of incoming traffic.

For example:

Server A = Weight 3
Server B = Weight 1

Server A receives approximately three times as many requests.

Key Takeaways

• Load balancers distribute incoming traffic.

• They improve scalability and availability.

• Different algorithms are optimized for different workloads.

Databases: The System's Memory

Applications can restart.

Servers can fail.

Caches can expire.

But databases are expected to remember everything.

They are the source of truth for a system.

Broadly speaking, modern databases fall into two categories.

SQL Databases

SQL databases store structured data using tables, rows, and relationships.

Popular examples include:

• PostgreSQL

• MySQL

Characteristics:

• Fixed schema

• Strong consistency

• ACID transactions

• Support for joins

These databases excel in situations where data accuracy is critical.

Examples include:

• Banking systems

• Payment processing

• Financial applications

NoSQL Databases

NoSQL databases prioritize flexibility and scalability.

Popular examples include:

• MongoDB

• Cassandra

Characteristics:

• Flexible schema

• Easy horizontal scaling

• High availability

They are commonly used for:

• Social media platforms

• Real-time applications

• Large-scale web systems

Key Takeaways

SQL emphasizes consistency and structure.

NoSQL emphasizes flexibility and scalability.

The right choice depends on the problem being solved.

ACID: Why We Trust Databases With Money

Imagine transferring ₹100 from one account to another.

The system deducts the money from the first account.

Then the server crashes.

What happens next?

Without proper guarantees, the money could disappear.

ACID properties prevent situations like this.

Atomicity

A transaction succeeds completely or fails completely.

There are no half-finished transactions.

Consistency

The database always moves from one valid state to another.

Rules and constraints remain intact.

Isolation

Transactions should not interfere with one another.

Users should never see incomplete results.

Durability

Once a transaction is committed, the data survives crashes and restarts.

Key Takeaways

• Atomicity = All or Nothing

• Consistency = Valid State

• Isolation = No Interference

• Durability = Permanent After Commit

Scaling Databases

Every successful application eventually faces the same challenge:

How do we handle more traffic?

There are two common approaches.

Vertical Scaling

Scale up.

Increase the resources of a single machine.

Examples:

• More CPU

• More RAM

• More Storage

Advantages:

• Easy implementation

• Minimal architectural changes

Disadvantages:

• Hardware limits exist

• Costs rise quickly

Eventually, the server cannot get any bigger.

Horizontal Scaling

Scale out.

Add more machines.

DB1
DB2
DB3
DB4

Advantages:

• Massive scalability

• Better fault tolerance

Disadvantages:

• Increased complexity

• Data distribution challenges

This is the approach used by most internet-scale systems.

Key Takeaways

• Vertical Scaling = Bigger Server

• Horizontal Scaling = More Servers

• Large-scale systems eventually adopt horizontal scaling because hardware limits are unavoidable.

Final Thoughts

Today's lesson revealed an uncomfortable truth.

Building a successful product is only half the battle.

The real challenge begins when that success arrives.

Proxy servers protect infrastructure.

Load balancers distribute traffic.

Databases preserve critical information.

ACID properties ensure reliability.

Scaling strategies prepare systems for growth.

Because on the internet, popularity isn't the finish line.

It's the moment the real test begins.

XOXO.