Where the course stands

  • By this point in the semester students already have all the ingredients for a single-node DBMS like SQLite, MySQL or Postgres, built bottom-up through the projects: storage/buffer pool (P1), B+Tree index (P2), query execution & planning (P3), concurrency control (P4), plus logging & recovery 1:00 1:38.
  • Guest lecturer stands in because Andy Pavlo is in Chicago; these three distributed lectures are explicitly a crash course — the goal is vocabulary and design trade-offs, not building one 3:02 3:23.

The central theme: coordination

  • Everything single-node still applies; what is new is that nodes must exchange messages — at the query planner, at the disk/buffer-pool layer, or anywhere between 2:26.
  • The recurring question becomes: should a query pull data from other nodes, or push work to a central location? 2:48

Parallel vs. distributed databases

Parallel DBDistributed DB
Node localitySame data center / same rack 3:38Geographically far apart (east coast vs. west coast, Asia) 4:07
NetworkHigh-speed LAN 3:38Public network / internet 4:07
Communication costNanoseconds to microseconds, assumed small 3:38Large — the speed of light starts to matter across continents 4:30
ReliabilityMessages assumed to arrive 3:38Messages can be lost, duplicated, delayed, or reordered; TCP/IP retries only go so far and undersea cables get cut 4:40

Why bother going distributed at all?

  • Availability zones / disaster tolerance. A student asked whether it is just about cost; the answer was survivability: a west-coast-only parallel deployment goes dark when a cyclone cuts the region off, whereas an east+west deployment keeps serving 5:08 5:30.

What breaks per component

  • Query optimization/execution — the cost model must now account for where data lives and the cost of moving it between nodes 6:31.
  • Concurrency control — the hardest part: every node must agree on (1) what happened and (2) in what order; even agreeing on the current time is hard 6:56.
  • Logging & recovery — a single node is simply up or down; with 20 nodes and 2 failures, "is the system up?" has no obvious answer 6:56.

Agenda for the lecture

  1. System architectures 2. Design issues 3. Partitioning schemes 4. Fast preview of distributed concurrency control 5. DataStax flash talk 7:37 7:57.
  • Next lectures (with Andy back): distributed OLTP, then distributed OLAP 6:18.