A detailed analysis of TiDB Hackathon 2022’s Application Track, focusing on the top three award-winning projects—DataDance, Yunji, and Mirror. This write-up summarizes their architectures, design trade-offs, and engineering challenges from academic, system-engineering, and participant perspectives.
Competition: TiDB Hackathon 2022
Track: Application Group
Theme: Possibility at Scale
Official page: https://tidb.net/events/hackathon2022
TiDB Hackathon is the flagship annual event organized by PingCAP and the TiDB open-source community.
It is widely regarded as one of the most influential hackathons in the database and distributed-systems ecosystem.
The 2022 edition brought together:
The Application Track challenges participants to build real-world, large-scale distributed applications powered by TiDB’s HTAP capabilities, TiFlash, TiKV, and related ecosystem tools.
The hackathon requires teams to:
Evaluation criteria:
This write-up examines three award-winning projects:
https://github.com/datadance-fun/DataDance
https://github.com/VelocityLight/yunji
https://github.com/mirror-data/mirror
Goal:
A real-time data transformation platform (dbt-like + incremental ETL) built on TiDB.
User SQL → Parser → DAG Builder → Task Scheduler → Executors ↓ ↑ Metadata (TiDB)
https://github.com/VelocityLight/yunji
Goal:
A multi-tenant, cloud-native user analytics platform for real-time metrics & behavioral analysis.
Event Stream → Collector → Normalizer → TiDB ↓ Analytics API ↓ Dashboard / UI
https://github.com/mirror-data/mirror
Goal:
A developer-friendly visualization and introspection tool for TiDB clusters.
Frontend (React) → API Gateway → TiDB Information Schema / Stats ↓ Profiling & Insights Engine
A researcher may focus on:
These projects represent modern directions in distributed SQL systems.
Things a participant would care about:
Across DataDance, Yunji, and Mirror, common patterns emerge:
Used for: schemas, pipeline state, configs, offsets, profiles.
Used for: aggregations, segmentation, statistics, dashboards.
(DataDance and Yunji)
Yunji & Mirror show UI matters.
Most teams used Docker to simplify demo.
https://github.com/datadance-fun/DataDance
https://github.com/VelocityLight/yunji
https://github.com/mirror-data/mirror
These repositories serve as excellent references for:
TiDB Hackathon 2022 Application Track brings together innovation in:
From a research angle, these projects demonstrate practical prototypes of active research areas such as incremental view maintenance, hybrid streaming workloads, and HTAP optimization.
From a systems-engineering angle, they illustrate clean architecture boundaries, correct use of TiDB/TiFlash, and realistic distributed workload patterns.
From a contestant angle, the lesson is clear:
The strongest projects do not reinvent TiDB—they extend it in impactful, meaningful, and highly usable ways.
TiDB Hackathon remains a unique environment where distributed systems innovation meets real-world engineering constraints, producing ideas and prototypes that often evolve into community contributions.
]]>How to Copy Memory? Coordinated Asynchronous Copy as a First-Class OS Service
Jingkai He, Yunpeng Dong, Dong Du (Shanghai Jiao Tong University);
Mo Zou, Zhitai Yu, Yuxin Ren, Ning Jia (Huawei Technologies);
Yubin Xia, Haibo Chen (Shanghai Jiao Tong University)
SOSP 2025
This review follows an 8-section structure tailored for distributed-systems and OS researchers, engineers, and students seeking to extract high-level ideas and design insights without reading the entire 20-page SOSP paper.
In modern operating systems, memory copy remains a significant and persistent performance bottleneck.
It directly impacts:
recv, send)Even though memcpy looks trivial, data movement dominates both CPU cycles and memory bandwidth—often more than computation itself.
Existing solutions have limitations:
Memory copy today is:
As the authors summarize:
“Copying is everywhere, but coordinated nowhere.”
The paper argues that memcpy must be elevated into a first-class, OS-managed asynchronous service, enabling:
This leads to the core contribution: Copier.
The central idea:
Turn memory copy into an OS-level asynchronous service, similar to async I/O,
exposing a unified interface for both applications and kernel subsystems.
This is the fundamental conceptual advancement.
Copier’s architecture combines abstractions, dependency tracking, heterogeneous hardware scheduling, and OS-level management.
Below are the essential components.
csync()Breaks large copies into segments, enabling:
csync()csync() allows a client to say: “I am about to use this data; promote required segments now.”
This solves:
Two types of dependencies must be preserved:
Using syscall trap / return events to reconcile order between user-mode and kernel-mode tasks.
Ensures correctness under reordering, e.g., when csync() accelerates segments.
Copier uses task piggybacking:
This improves:
Copier can eliminate intermediate copies such as:
Kernel Buffer → Intermediate Buffer → Destination
becoming:
Kernel Buffer → Destination
It achieves this via:
This is one of the biggest performance wins.
Extends cgroup to track copy length as the resource.
A CFS-like scheduler ensures fair bandwidth across clients.
Lock virtual → physical mappings before issuing copy, ensuring DMA never faults inside Copier context.
recv() / send()The evaluation shows the design translates directly into real performance gains across multiple workloads.
The authors position Copier relative to prior approaches:
Similar to:
Most high-performance systems are bottlenecked not by compute, but by copy.
Databases (LSM-tree compaction, snapshotting, RPC serialization) can all benefit from this.
You can adopt a similar design:
Avoid exposing DMA/AVX details to application logic.
Redundant copies should be eliminated across system layers.
A recommended route:
You should be able to answer:
A1: No. Copier runs as kernel background worker threads using shared queues.
There are no extra context switches.
A2:
Copier batches multiple copy tasks and executes them continuously,
avoiding frequent AVX state save/restore.
Thus, it amortizes the cost and remains safe.
A3:
Copier uses polling threads to minimize submission overhead.
This improves throughput but increases energy usage.
The authors optimized polling via a NAPI-like mechanism to reduce overhead.
Paper: CortenMM: Efficient Memory Management with Strong Correctness Guarantees
Conference: SOSP 2025
Authors: Shanghai Jiao Tong University, Huawei
CortenMM proposes a new OS memory management architecture that replaces Linux’s traditional VMA (Virtual Memory Area) abstraction with a lightweight, per-page entry model that simplifies correctness, avoids locking bottlenecks, and significantly improves multi-core scalability.
This review follows the same format as the previous Copier analysis and focuses on:
Modern OSes (Linux, BSD, etc.) use a dual-layer memory management model:
This split causes several issues.
Even when threads access distinct regions, they might:
This becomes a bottleneck for:
Every mmap/munmap / CoW / mprotect / page fault must:
The two-layer design adds complexity and latency.
Correctness invariants must be maintained across:
CortenMM’s insight:
What if we eliminate the VMA layer entirely and make per-PTE metadata the only authoritative source of memory mappings?
CortenMM introduces a unified memory management model based entirely on:
Instead of storing mapping properties in large VMAs, CortenMM adds:
Each PTE holds:
This collapses VMA and page table into a single layer, improving both simplicity and performance.
Since VMAs are gone, the system still needs to support range-based operations such as:
mmapmunmapmprotectCortenMM uses a Concurrent Interval Skiplist, enabling:
The new model simplifies reasoning:
The paper includes a formal model ensuring correctness for:
CortenMM eliminates the classic mmap_lock bottleneck.
Measured against Linux:
| Workload | Improvement |
|---|---|
| mmap-heavy multithread apps | 3.3× – 11.6× faster |
| page-fault intensive workloads | 2× – 5× faster |
| memory allocators (e.g., tcmalloc) | 20% – 60% faster |
| JIT workloads | major latency reductions |
| general MMLRU / VM subsystem | significantly more parallel |
Memory overhead:
A1: No.
Metadata is allocated only for mapped pages.
A2: Logically yes, physically irrelevant.
The metadata array aligns with page table pages for easy indexing.
Each entry is only 8 bytes, so the physical requirement poses minimal overhead.
A3: Because Linux still relies on the global VMA tree, protected by mmap_lock.
Even when threads operate on unrelated memory areas, they may need to:
This unnecessary synchronization is the main source of poor scalability.
CortenMM shows that coarse-grained VMAs introduce:
Eliminating VMAs simplifies both design and verification.
A single-layer model:
It is “closer to the metal.”
The paper follows a common theme seen in modern OS research:
Remove legacy abstractions → Make fine-grained state the first-class entity.
Like Dune, Arrakis, FlexSC, and now Copier (previous review), CortenMM rethinks long-standing OS interfaces.
CortenMM resembles the shift toward:
The design philosophy could influence:
CortenMM re-architects OS memory management by:
It is one of the most impactful VM subsystem papers in recent years and suggests a future where OSes abandon coarse, process-wide abstractions in favor of per-page, concurrent, verifiable designs.
A concise analysis of the 5th Alibaba Middleware Performance Challenge, covering the adaptive load-balancing task in the preliminary round and the in-process message storage engine in the final round.
This article highlights core algorithms, system design trade-offs, and the practical engineering lessons learned from solving real-world middleware problems.
Competition: Fifth Alibaba Middleware Performance Challenge
Organizer: Alibaba Cloud / Tianchi Platform
Official Page: https://tianchi.aliyun.com/competition/entrance/231714
The Alibaba Middleware Performance Challenge is a hands-on, engineering-intensive competition designed to simulate the realities of large-scale distributed middleware systems.
Instead of algorithms or puzzles, this competition evaluates:
Participants compete in two stages:
Both reflect real internal challenges faced inside Alibaba’s infrastructure.
Challenge Page:
https://code.aliyun.com/middlewarerace2019/adaptive-loadbalance
Task: Implement a self-adaptive load balancing algorithm that automatically adapts to fluctuating provider capacity, minimizes response latency, and maximizes throughput.
Providers’ service capability fluctuates dynamically.
The system must avoid overload, starvation, and collapse.
If incoming request rate > global capacity,
the Gateway must proactively reject requests (global backpressure).
PTS (Performance Testing Service) sends HTTP requests: PTS → Gateway → Provider → Response
Provider capacity changes across:
Performance Metrics:
This round is a true systems problem involving stability, noisy feedback, and dynamic latency behavior.
We analyze it from three perspectives.
This task is mathematically a feedback control problem where:
Key principles required:
The problem structurally mirrors:
Production-grade load balancers must handle:
The competition intentionally simulates these real-world issues.
Key engineering behaviors:
Latency increases before throughput drops → ideal for early detection.
Immediate switching → thrashing → collapse.
This mirrors circuit breaking in Envoy, Nginx, Kafka, etc.
Reject requests if overloaded to preserve SLA.
Common Wrong Strategies:
Winning strategies:
The following is the distilled architecture used by top-ranking teams and widely adopted in real-world distributed systems.
Each provider is assigned a real-time score: score_i = EWMA(latency_i) + penalty_i + failure_rate_factor
Lower score = better provider.
Real-time capacity predicted as: estimated_capacity_i = k / EWMA(latency_i)
This approximates each provider’s μ(t).
Lower latency → more remaining capacity.
Before routing a new request: if incoming_rate > Σ estimated_capacity_i: shed requests
This prevents catastrophic collapse
and reflects real-world backpressure mechanisms.
Providers are selected probabilistically:
Penalties gradually decay: penalty_i = penalty_i * 0.9
Recovered providers should return gradually to rotation.
These two excellent resources help deepen understanding:
https://tianchi.aliyun.com/notebook/66591
https://tianchi.aliyun.com/notebook/60036
They cover routing instability, latency behavior, workload patterns, and tuning strategies.
Challenge Page:
https://code.aliyun.com/middlewarerace2019/mqrace2019
Task: Implement a persistent in-process message store supporting:
a)This problem is a scaled-down version of RocketMQ / Kafka log storage.
{a: int, t: timestamp}(t1, t2)This is essentially a small time-series database problem.
Conceptually involves:
Key design components:
[header][entries][metadata]
https://tianchi.aliyun.com/notebook/77159
Highlights:
https://tianchi.aliyun.com/forum/post/77949
Highlights:
https://tianchi.aliyun.com/notebook/77274
Highlights:
| Aspect | Champion | Runner-up | Third Place |
|---|---|---|---|
| Write Path | Most optimized | Stable batch | Straightforward |
| Query Engine | Prefix sums + binary search | Binary search | Linear scan |
| Indexing | Multi-level | Minimal | Lightweight |
| Durability | Tuned | Balanced | Simple |
| Complexity | High | Medium | Low |
This competition is an outstanding demonstration of real systems engineering, covering:
The competition simulates real challenges encountered in:
It trains participants to:
For anyone pursuing:
this competition provides exceptional preparation and demonstrable technical depth.
Adaptive Load Balancer Repo
https://code.aliyun.com/middlewarerace2019/adaptive-loadbalance
Message Storage Engine Repo
https://code.aliyun.com/middlewarerace2019/mqrace2019
Preliminary Round Notes
https://tianchi.aliyun.com/notebook/66591
https://tianchi.aliyun.com/notebook/60036
Final Round Winner Write-ups
Champion: https://tianchi.aliyun.com/notebook/77159
Runner-up: https://tianchi.aliyun.com/forum/post/77949
Third Place: https://tianchi.aliyun.com/notebook/77274
This article analyzes the OceanBase Database competition, a hands-on challenge that requires participants to build a miniature relational database from scratch. Unlike typical coding contests, this competition focuses on core DBMS architecture — including SQL parsing, execution, storage, and indexing.
Competition: OceanBase Database Competition (3rd Edition)
Official page: https://open.oceanbase.com/competition
The OceanBase Database Competition is part of the National Collegiate Computer System Ability Challenge, a nationwide contest for university students in China.
It is jointly organized by the System Capability Training Expert Group, the National Computer Education Research Association, and leading universities, and hosted by OceanBase.
The competition aims to promote technical excellence in data-intensive systems and to bridge the gap between academic concepts and real-world distributed database engineering. Students compete in teams of 1–3 members, and previous top-20 finalists cannot re-enter to ensure fairness.
The competition consists of two stages:
The preliminary stage is built on MiniOB, a lightweight relational database engine designed for teaching DBMS fundamentals. It exposes essential components—SQL parsing, planning, execution, storage, and indexing—allowing contestants to extend real database internals.
The organizers provide a list of MiniOB tasks here:
https://oceanbase.github.io/miniob/game/miniob_topics/
Topics include:
These tasks gradually guide contestants from zero experience to implementing DBMS kernel modules.
Reference implementation: https://github.com/S-1-T/miniob
A SQL query flows through:
MiniOB mirrors real RDBMS architecture while remaining small enough for students to extend.
MiniOB uses a staged event-driven architecture (SEDA):
done() signals event completionThis teaches non-blocking concurrency and modular performance design.
Distinguishing syntax errors from semantic errors, performing metadata validation, and returning correct error codes is essential.
Operations like DROP TABLE reveal the importance of catalog consistency and error propagation.
Example: Adding a native DATE type requires modifying lexing, grammar, type checking, predicate evaluation, and storage.
B+ Tree split logic, parent pointer updates, and page invariants must be correct to avoid silent corruption.
Common pitfalls include:
break statements%s/std::stringdone()Debugging MiniOB teaches students how subsystems interact and how bugs propagate across the engine.
For readers who want a more narrative, step-by-step view of working with MiniOB, I highly recommend a four-part development diary written by Zheng Jinghong (郑璟泓). The series follows one student’s journey through the 2021 OceanBase Database Competition preliminary round and is very close to what a real beginner-to-intermediate MiniOB path looks like.
https://deepz.cc/2021/10/miniob-1/ Covers setting up the development environment, cloning the official MiniOB repository, using VS Code with Remote SSH, and getting a high-level understanding of the MiniOB architecture and the “must-do” preliminary problems.
https://deepz.cc/2021/10/miniob-2/
Explains how MiniOB uses a staged event-driven architecture (SEDA): requests are split into stages, each with its own thread pool and event queue, and each event must call done() when a stage finishes processing it. This is a great way to learn how MiniOB structures its request pipeline beyond a simple thread-per-connection model.
https://deepz.cc/2021/10/miniod-3/
Focuses on implementing DROP TABLE correctly, and on adding metadata validation so that querying a dropped or non-existent table returns a proper error instead of silently succeeding. The diary emphasizes that a DBMS must not only run valid queries, but also reject invalid ones with clear failure modes.
DATE TypeDATE type: modifying the lexer and parser, storing YYYY-MM-DD, validating date strings, handling WHERE conditions on dates, and debugging subtle issues (such as using %s with std::string and a missing break in a switch). This part captures the “real feel” of database kernel development.These diaries complement this write-up: while this article focuses on the system-level structure of MiniOB, the diary shows what it is like to work through the competition tasks in practice—debugging, reading code, and gradually building confidence in database internals.
The final round transitions from MiniOB to OceanBase, a distributed SQL database used in production at large financial institutions.
Contestants must design and implement a bypass import (direct path load) pipeline.
The final round transitions from the teaching-oriented MiniOB system to the enterprise-grade distributed SQL database OceanBase.
Instead of extending a small relational engine, contestants work with real storage components and system-level APIs to implement a bypass import (direct path load) mechanism.
OceanBase’s existing data import path relies on converting input text into large batches of INSERT statements. These statements must pass through:
This results in a long execution path and limited import throughput.
Traditional DBMSs (e.g., Oracle, DB2) provide a “direct path load,” bypassing SQL and transactions and writing directly into SSTable storage files. OceanBase does not yet have this feature.
Contestants must design and implement a high-performance bypass load pipeline that:
This requires deep understanding of OceanBase internals, distributed storage principles, and high-throughput data ingestion.
References:
The final round shifts from “implementing components” to “designing an end-to-end high-performance system.”
Below is the analysis structured similarly to the preliminary round.
A bypass-import pipeline typically follows this flow:
This architecture mimics ingestion pipelines used in distributed warehouses such as ClickHouse, BigQuery, TiDB Lightning, and HDFS bulk loaders.
From a professor’s perspective, this stage demonstrates systems-level design in a distributed database kernel.
A large cluster may have dozens or hundreds of partitions.
A high-performance loader must:
Key insight:
Parallelism is not “one big thread pool,” but per-partition concurrency to minimize lock contention and network hops.
OceanBase stores rows in tightly packed, serialized structures.
Building these correctly requires:
This is a direct test of “reading real-world database code” ability.
Constructing valid SSTable files requires:
A single wrong byte may corrupt the block or crash compaction.
This is where system engineers and professors focus heavily on storage invariants.
Key performance bottlenecks include:
Top teams achieved massive speedups by:
Even though bypass-load skips SQL and transactions, correctness still matters:
This shows deep understanding of distributed consistency and storage durability.
Contestants frequently encounter real system-level bugs:
The debugging mindset shifts from “fix my code” to “validate invariants across multiple subsystems.”
This is exactly the type of experience that PhD advisors value.
The final round teaches a very different set of system skills than the MiniOB stage:
Overall, contestants move from “DBMS component-level understanding” to real distributed database engineering.
These write-ups provide high-quality insights into the final round challenge:
2022 Final Round Experience (very detailed analysis):
https://zhuanlan.zhihu.com/p/617520132
2023 Final Round Solution Breakdown:
https://zhuanlan.zhihu.com/p/677020265
Official OceanBase Final Round Blog:
https://open.oceanbase.com/blog/2325423616
Together, they illustrate design trade-offs, bottlenecks, and real implementation strategies used by top-performing teams.
Participating in both stages of the competition—from MiniOB to the full OceanBase kernel—provides a holistic view of database systems engineering.
MiniOB teaches component-level DBMS architecture.
OceanBase requires designing system-level ingestion pipelines.
Both stages emphasize metadata consistency, structured storage layout, and predictable error handling.
Contestants learn how real systems fail, how to debug multi-module issues, and how to reason about performance and correctness simultaneously.
This competition bridges academic foundations with industry-grade distributed systems.
It provides a rigorous, hands-on understanding of database internals—from SQL to parser, storage, distributed ingestion, and performance engineering.
The roadmap is divided into three levels:
Before diving into resources, it’s helpful to clarify some overloaded terms.
MySQL is one specific relational database system.
Other relational engines include PostgreSQL, Oracle, SQL Server, etc.
NoSQL is not a “replacement” for SQL — it is a complementary model.
Choose based on workload and consistency requirements.
Both are relational databases:
Either is fine for learning.
A complete database learning path spans multiple systems:
Goal:
Learn SQL basics, relational databases, NoSQL overview, and get hands-on experience.
Resources
Recommended practice after completing basics:
Install MySQL locally and practice queries.
Choose one:
Example download (one edition):
https://drive.google.com/file/d/1QJv4JF4DWJdO7V_ZxWLzhis4yDuDxvno/view
Topics covered:
Workflow: read → type every query → run it.
Example link:
https://drive.google.com/file/d/12I-7TYkq8rGxjyeCOV_NX2r0sR3LxGht/view
Uses PostgreSQL; highly visual, very beginner-friendly.
This is MongoDB’s official training platform.
Take MongoDB for Developers (7-week pace):
You will learn:
Optional deeper reading:
MongoDB: The Definitive Guide
https://drive.google.com/file/d/1CgQL2LjAOvAM8g6aVoY4_d5pWx9A5XL3/view
Read selected chapters (not cover-to-cover) from:
Database System Concepts
https://drive.google.com/file/d/11rIdRiVYrFChEiqmYCuVpN6A4wgTErMw/view
Focus on:
Pick one:
Redis Beginner’s Guide
https://drive.google.com/file/d/1aavGDWBzlZwbwDciAe8VLBisW5yJ9eG4/view
Redis Handbook
(overview of data types, operations, usage patterns)
Optional video course:
Redis 6 from Beginner to Advanced
https://www.bilibili.com/video/BV1Rv41177Af/
Distributed Systems for Fun and Profit
https://book.mixu.net/distsys/
A short and accessible overview of distributed thinking — essential context for distributed databases.
HBase: The Definitive Guide
https://drive.google.com/file/d/1pHaX4c0iPsL24OKhg46ZcSjz8ccwlJIm/view
Goal:
Understand SQL engine internals, storage engines, transactions, NoSQL internals, and official docs.
Covers:
Assignments are based on BusTub, a real DBMS implementation.
How to read:
MySQL Technical Insider: The InnoDB Storage Engine
https://drive.google.com/file/d/1kSQsS9-_QujtdpMVVV2Lnvy0Olv63bXa/view
Recommended chapters: 2, 4, 5.
Best source for modeling, aggregation, indexes, deployments.
Redis Design and Implementation
https://drive.google.com/file/d/1PyPTLFGywqbHEmKvHQrV0rjJD4THPjcJ/view
Paired with source code reading → excellent for understanding systems internals.
MIT 6.824 Distributed Systems
Implement Raft, MapReduce, replicated key-value stores.
Deep, well-structured, and great for understanding relational engine design.
Goal:
Performance tuning, indexing, internals, distributed storage, and research papers.
Database Index Design and Optimization
https://drive.google.com/file/d/10SrTxPtZVrI4f93m7AmAXcutWJvYGZKe/view
Learn:
High Performance MySQL
https://drive.google.com/file/d/1nwhSrOXYufqbzIMC99cHQNjL7IZRIWhq/view
Covers:
Redis Development and Operations
https://drive.google.com/file/d/1G2tCdsy5ph10n_eq6Gzv8LiqWLBUIBtg/view
Redis Deep Dive: Core Principles and Practical Applications
https://drive.google.com/file/d/1VyLppzqXyxseaIWjFKGODg1_zESlFkJO/view
Graduate-level seminar focusing on:
Example links:
https://drive.google.com/file/d/1Gm6ICrMofFzRUd9B4GBvVLY3tZH-z9xX/view
https://drive.google.com/file/d/1wzaVK5PStnudeWA_PTvE5uL-VEw9OaWz/view
https://drive.google.com/file/d/1nhbid-TlM73bCKQ4G3DwNRxTInAlo8fR/view
Readings in Databases
https://github.com/rxin/db-readings
Pick a theme → read 2–3 papers → write notes.
One recommended progression:
Most importantly: alternate theory and hands-on practice.
Databases become intuitive only when you build, debug, measure, and reason about real systems.
For the purposes of learning, we can roughly break the space into four areas:
A solid learning path touches all four, with progressively more depth.
Before diving in:
You don’t need to be an expert; you will reinforce these along the way.
Architecture types
System families
You don’t need to master all of these at once. The rest of this roadmap gives a sequence of concrete resources to follow.
Goal:
Build core intuition for replication, consistency, and failure while implementing small systems.
Distributed Systems for Fun and Profit
Why this course
👉 If you’re new to Go:
A solid undergraduate-level course.
If you want a smoother ramp-up, one option is:
15-440 → 6.824 → 15-712 (later at intermediate/advanced).
Taught by Martin Kleppmann (author of Designing Data-Intensive Applications).
Great for developing a clean conceptual model of concurrency and communication.
CMU 15-445 Intro to Database Systems
Covers:
Labs use bustub in C++.
If you are not a C++ user, you can still follow lectures and read the code at a higher level.
Designing Data-Intensive Applications by Martin Kleppmann
What you get:
Distributed Systems: Concepts and Design (Coulouris et al.)
Contains a traditional textbook treatment of distributed systems.
Good as a reference to clarify terms and models.
Short, practical notes on what makes distributed systems hard in real life.
Goal:
Connect coursework to theoretical models and real research papers.
This is a paper roadmap — it lists key results (CAP, FLP, Paxos, etc.) and tells you why they matter for engineers.
A graduate-level seminar focusing on:
Discussion-based, centered around classic and modern distributed systems papers.
Good for training your paper reading + presentation skills.
Nine core chapters:
Comes with figures and some Python examples.
It’s a good “second textbook” once you already know the basics.
Short and practical discussion of:
Mathematically rigorous and challenging.
Best approached once you already have some intuition about consensus and failures.
Goal:
Read and implement real systems; understand research problems and trade-offs at scale.
These four papers are foundational:
Example links / collections:
PingCAP TiDB Talent Plan
Guided labs to implement components inspired by TiDB/TiKV using Go or Rust:
Perfect follow-up after MIT 6.824.
Some good collections:
Pick a small set of papers at a time and write notes after reading.
Examples worth reading (at least in summary):
For PhD applications, writing paper notes on a subset of these is excellent evidence of depth.
You don’t need to understand every line of code. Start by tracing:
Storage:
Consensus & coordination:
Computation:
As you go deeper, it’s normal to circle back to:
Distributed systems sit on top of all of these.
| Level | Focus | Main Goal |
|---|---|---|
| Beginner | Labs + core concepts (6.824, 15-445, DDIA) | Build intuition and implementation skills |
| Intermediate | Paper roadmaps and advanced courses | Connect theory with real systems |
| Advanced | Classic papers, open-source, and research topics | Think and work like a systems researcher/engineer |
This roadmap is not meant to be followed rigidly.
Pick a starting point that fits your background, alternate between theory and implementation, and write down what you learn — that’s how the knowledge becomes your own.