This educational course grows a simple application one bottleneck at a time. It explains when to add load balancing, stateless sessions, connection pooling, read replicas, caching, background jobs, and database sharding. The recurring lesson is to adopt each tool only after a real constraint appears, with explicit attention to latency, correctness, reliability, and operating cost.
flowchart LR
U[Users] --> LB[Load balancer]
LB --> A[Stateless app servers]
A <--> R[(Redis<br/>sessions and cache)]
A --> P[Connection pool]
P --> DB[(Primary database)]
P --> RR[(Read replicas)]
DB -. replication .-> RR
A --> Q[Job queue]
Q --> W[Workers]
DB -. when one machine<br/>cannot hold the data .-> S[(Shards)]
Topics
- Introduction
- Starting system
- Load balancing
- Stateless servers
- Connection pooling and read replicas
- Caching
- Queues and background jobs
- Sharding
- Closing
Key takeaways
- One server and one database can already serve thousands of users. Complexity should follow measured pressure, not anticipation.
1:40 - A typical application can handle hundreds of requests per second, or millions per day, on a $20-per-month server. Vertical scaling is therefore usually the first move.
4:06 - Horizontal scaling requires interchangeable app instances. Moving session state to a shared store makes any server able to handle the next request, at the cost of an extra lookup and another dependency.
10:0011:39 - Read replicas remove much of the database's read load, but replication lag makes some reads briefly stale. Systems should choose where that inconsistency is acceptable and route sensitive read-after-write paths to the primary.
17:5719:02 - Caching deliberately trades correctness for speed. A follower count may tolerate 30 seconds of staleness, while a bank balance should not be cached this way.
23:1723:51 - Queues shorten user-facing requests and isolate them from slow external services, but "done" may then mean "accepted for later work." Reliable systems retry failed jobs and retain repeatedly failing jobs for inspection.
28:4229:11 - Sharding solves the case where the dataset no longer fits on one machine, but cross-shard queries and later migration make it the last tool to reach for.
34:0134:57 - System-design skill is not naming boxes. It is choosing their order, knowing which ones the product needs, and accepting each tradeoff deliberately.
36:2237:31