diff --git a/README.md b/README.md index 8c289a1f..d0c2f2ef 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,11 @@ # The TinyKV Course -This is a series of projects on a key-value storage system built with the Raft consensus algorithm. These projects are inspired by the famous [MIT 6.824](http://nil.csail.mit.edu/6.824/2018/index.html) course, but aim to be closer to industry implementations. The whole course is pruned from [TiKV](https://github.com/tikv/tikv) and re-written in Go. After completing this course, you will have the knowledge to implement a basic key-value storage fault-tolerance service with distributed transaction support and a better understanding of TiKV implementation. +This is a series of projects on a key-value storage system built with the Raft consensus algorithm. These projects are inspired by the famous [MIT 6.824](http://nil.csail.mit.edu/6.824/2018/index.html) course, but aim to be closer to industry implementations. The whole course is pruned from [TiKV](https://github.com/tikv/tikv) and re-written in Go. After completing this course, you will have the knowledge to implement a horizontal scalable, high available, key-value storage service with distributed transaction support and a better understanding of TiKV implementation. The whole project is a skeleton code for a kv server and a scheduler server at initial, and you need to finish the core logic step by step: - Project1: build a standalone key-value server -- Project2: build a fault tolerant key-value server with Raft +- Project2: build a high available key-value server with Raft - Project3: support multi Raft group and balance scheduling on top of Project2 - Project4: support distributed transaction on top of Project3 @@ -13,7 +13,9 @@ The whole project is a skeleton code for a kv server and a scheduler server at i ## Course -Before starting, you’d better read the overview design of TiKV and PD to get a general impression on what you will build: +Here is a [reading list](doc/reading_list.md) for the knowledge of distributed storage system. Though not all of them are highly related with this course, it can help you construct the knowledge system in this field. + +Also, you’d better read the overview design of TiKV and PD to get a general impression on what you will build: - TiKV - (Chinese Version) @@ -50,10 +52,10 @@ The whole project is organized into the following directories: Please follow the course material to learn the background knowledge and finish code step by step. -- [Project1 - StandaloneKV](courses/project1-StandaloneKV.md) -- [Project2 - RaftKV](courses/project2-RaftKV.md) -- [Project3 - MultiRaftKV](courses/project3-MultiRaftKV.md) -- [Project4 - Transaction](courses/project4-Transaction.md) +- [Project1 - StandaloneKV](doc/project1-StandaloneKV.md) +- [Project2 - RaftKV](doc/project2-RaftKV.md) +- [Project3 - MultiRaftKV](doc/project3-MultiRaftKV.md) +- [Project4 - Transaction](doc/project4-Transaction.md) ## Deploy a cluster diff --git a/courses/imgs/balance1.png b/doc/imgs/balance1.png similarity index 100% rename from courses/imgs/balance1.png rename to doc/imgs/balance1.png diff --git a/courses/imgs/balance2.png b/doc/imgs/balance2.png similarity index 100% rename from courses/imgs/balance2.png rename to doc/imgs/balance2.png diff --git a/courses/imgs/keyspace.png b/doc/imgs/keyspace.png similarity index 100% rename from courses/imgs/keyspace.png rename to doc/imgs/keyspace.png diff --git a/courses/imgs/multiraft.png b/doc/imgs/multiraft.png similarity index 100% rename from courses/imgs/multiraft.png rename to doc/imgs/multiraft.png diff --git a/courses/imgs/overview.png b/doc/imgs/overview.png similarity index 100% rename from courses/imgs/overview.png rename to doc/imgs/overview.png diff --git a/courses/imgs/region.png b/doc/imgs/region.png similarity index 100% rename from courses/imgs/region.png rename to doc/imgs/region.png diff --git a/courses/project1-StandaloneKV.md b/doc/project1-StandaloneKV.md similarity index 100% rename from courses/project1-StandaloneKV.md rename to doc/project1-StandaloneKV.md diff --git a/courses/project2-RaftKV.md b/doc/project2-RaftKV.md similarity index 100% rename from courses/project2-RaftKV.md rename to doc/project2-RaftKV.md diff --git a/courses/project3-MultiRaftKV.md b/doc/project3-MultiRaftKV.md similarity index 100% rename from courses/project3-MultiRaftKV.md rename to doc/project3-MultiRaftKV.md diff --git a/courses/project4-Transaction.md b/doc/project4-Transaction.md similarity index 100% rename from courses/project4-Transaction.md rename to doc/project4-Transaction.md diff --git a/doc/reading_list.md b/doc/reading_list.md new file mode 100644 index 00000000..2800a5a9 --- /dev/null +++ b/doc/reading_list.md @@ -0,0 +1,121 @@ +- Storage Engine + - Disk I/O + - Flavors of I/O + - + - + - Amplification and RUM + - + - + - + ure-read-write-space-and.html + - B-Tree (Optional) + - Reading Materials + - Database Internals: 2. B-Tree Basics + - Database Internals: 3. File Formats + - Database Internals: 4. Implementing B-Trees + - Reference + - + ms.html + - + - LSM-Tree + - Reading Materials + - Database Internals: 7. Log-Structured Storage + - Reference + - + - + - + - B-Tree vs LSM-tree + - + - LSM-Tree Evolution + - Wisckey + - + - HashKV (Optional) + - + - Monkey (Optional) + - + - Reference + - + - +- Serializing & RPC + - Protocol Buffer + - Reading Materials + - + - Reference + - + - + - gRPC + - Reading Materials + - + - Reference + - +- Data Partitioning + - Range vs Hash + - + - Partitioning of DynamoDB (Optional) + - + - + - Partitioning of TiKV + - +- Replication & Consistency + - Reading Materials + - Database Internals: 11. Replication and Consistency + - CAP vs PACELE + - + - Clock and Time + - + - Consistency + - +- Consensus + - Quorum + - + - Paxos (Optional) + - Reading Materials + - Database Internals: 14. Consensus @ Paxos + - Reference + - + - Raft + - Reading Materials + - Database Internals: 14. Consensus @ Raft + - Reference + - + - +- Scale & Balance + - Multi-Raft + - + - Split & Merge + - + - Balance + - +- Distributed Transactions + - Reading Materials + - Database Internals: 5. Transaction Processing and Recovery + - Database Internals: 13. Distributed Transactions + - ACID + - + - Isolation (Optional) + - + - (Chinese) + - Spanner (Optional) + - + - (Chinese) + - Percolator + - + - +- Coprocessor (Optional) + - + - (Chinese) +- Benchmark & Testing + - sysbench (Optional) + - + - + - (Chinese) + - go-tpc (Optional) + - + - go-ycsb + - + - + - (Chinese) + - Chaos + - + - + -