Case · Go backend
Raft KV: a key-value store on Raft
A from-scratch teaching implementation of Raft on the pure stdlib: leader election, log replication, quorum commit, a log and term/vote on disk with fsync, snapshots with compaction, linearizable reads (read-index) and follower-to-leader redirects. It is tested with a deterministic network and time simulator that checks linearizability; the dashboard can kill the leader and cut the network (demo).
- Go
- Raft
- Distributed systems
- Consensus
- stdlib only
- HTTP/JSON
- SSE
- WAL + fsync
- Snapshots
- Linearizability
- Deterministic simulation testing
- Vanilla JS
- Docker
- GitHub Actions
01 · How it looks
Screenshots
02 · About
What it is and why
Raft KV is a cluster of three nodes that agree on the order of writes with the Raft algorithm. The algorithm is written from scratch on the pure Go standard library, with no raft libraries or other dependencies, following the paper by Ongaro and Ousterhout, "In Search of an Understandable Consensus Algorithm".
Next to it is a vanilla JS dashboard: node roles, term, commit and the log tail in real time, and in demo mode you can kill the leader or cut the network and watch the cluster elect a new one. This is a teaching implementation: it has not been through Jepsen and is not meant for production.
Limitations
The cluster is fixed at exactly three nodes, with no dynamic membership or joint consensus. There is no pre-vote or check-quorum: a partitioned follower inflates its term, and a partitioned leader believes it is the leader until it sees a higher term (it does not serve linearizable reads meanwhile).
There are no client sessions or exactly-once. A 504 means "outcome unknown": the command may have committed later. Repeating a PUT is safe because it is idempotent.
The verification is home-made, not Jepsen. The simulator has a perfect disk, a snapshot goes out as one message and is held in memory whole, there is no TLS or authentication, and every PUT is a separate fsync.
One hand-made mutation is not caught by the tests: committing entries of earlier terms (figure 8 of the paper) has no dedicated scenario. The -race tests run only in CI (there is no gcc locally), and the Docker image has not been built.
What is inside
- Elections with a random timeout, a vote only for a candidate whose log is not behind; term and votedFor are written to disk before the requests go out
- Log replication with a ConflictIndex hint: the leader backs up straight to the start of the conflicting term, not one step at a time
- Quorum commit only for entries of the current term (section 5.4.2 of the paper); a new leader appends a no-op right away
- Persistence: a WAL with CRC32 and fsync, state.json through a temp file and rename, a torn log tail is truncated on load
- Snapshots and compaction by file "generations": a crash at any step leaves either the whole old or the whole new state; a lagging follower receives a snapshot
- Linearizable reads through read-index with no clocks or leases; a read with ?stale=1 is local and may lag
- A raft.Node core with no goroutines or clocks: time arrives through Tick and messages through Step, so the same code runs in the runtime and in the simulator
- A deterministic network and time simulator (loss, delay, partitions, crashes) that checks invariants and the linearizability of the history (Wing and Gong)
- An SSE dashboard: role, term, commit/applied, the log tail; the failure buttons and node stop work only with SEED_DEMO=1
03 · How to run
Run it locally
git clone <repository on request> raft-kv
cd raft-kv
SEED_DEMO=1 go run ./cmd/raft-kv # dashboard http://localhost:8098, nodes on 8095-8097
curl -L -X PUT localhost:8096/kv/city -d Berlin # a follower redirects to the leader04 · Code
The repository is private
The source lives in a private repository. I will grant access on request: write to me and I will send an invitation.
Request the code