transaction-correctness

Source excerpt starting at line 1.

1---2name: transaction-correctness3description: How WAL mechanics, checkpointing, concurrency rules, recovery work in tursodb4---5# Transaction Correctness Guide6 7Turso uses WAL (Write-Ahead Logging) mode exclusively.8 9Files: `.db`, `.db-wal` (no `.db-shm` - Turso uses in-memory WAL index)10 11## WAL Mechanics12 13### Write Path141. Writer appends frames (page data) to WAL file (sequential I/O)152. COMMIT = frame with non-zero db_size in header (marks transaction end)163. Original DB unchanged until checkpoint17 18### Read Path191. Reader acquires read mark (mxFrame = last valid commit frame)202. For each page: check WAL up to mxFrame, fall back to main DB213. Reader sees consistent snapshot at its read mark22 23### Checkpointing24Transfers WAL content back to main DB.25 26```27WAL grows → checkpoint triggered (default: 1000 pages) → pages copied to DB → WAL reused28```29 30Checkpoint types:31- **PASSIVE**: Non-blocking, stops at pages needed by active readers32- **FULL**: Waits for readers, checkpoints everything33- **RESTART**: Like FULL, also resets WAL to beginning34- **TRUNCATE**: Like RESTART, also truncates WAL file to zero length35 36### WAL-Index37SQLite uses a shared memory file (`-shm`) for WAL index. **Turso does not** - it uses in-memory data structures (`frame_cache` hashmap, atomic read marks) since multi-process access is not supported.38 39## Concurrency Rules40 41- One writer at a time42- Readers don't block writer, writer doesn't block readers43- Checkpoint must stop at pages needed by active readers44 45## Recovery46 47On crash:481. First connection acquires exclusive lock492. Replays valid commits from WAL503. Releases lock, normal operation resumes51 52## Turso Implementation53 54Key files:55- [WAL implementation](../../../core/storage/wal.rs) - WAL implementation56- [Page management, transactions](../../../core/storage/pager.rs)57 58### Connection-Private vs Shared59 60**Per-Connection (private):**61- `Pager` - page cache, dirty pages, savepoints, commit state62- `WalFile` - connection's snapshot view:63  - `max_frame` / `min_frame` - frame range for this connection's snapshot64  - `max_frame_read_lock_index` - which read lock slot this connection holds65  - `last_checksum` - rolling checksum state66 67**Shared across connections:**68- `WalFileShared` - global WAL state:69  - `frame_cache` - page-to-frame index (replaces `.shm` file)70  - `max_frame` / `nbackfills` - global WAL progress71  - `read_locks[5]` - read mark slots (TursoRwLock with embedded frame values)72  - `write_lock` - exclusive writer lock73  - `checkpoint_lock` - checkpoint serialization74  - `file` - WAL file handle75- `DatabaseStorage` - main `.db` file76- `BufferPool` - shared memory allocation77 78## Correctness Invariants79 801. **Durability**: COMMIT record must be fsynced before returning success812. **Atomicity**: Partial transactions never visible to readers823. **Isolation**: Each reader sees consistent snapshot834. **No lost updates**: Checkpoint can't overwrite uncommitted changes84 85## References86 87- [SQLite WAL](https://sqlite.org/wal.html)88- [WAL File Format](https://sqlite.org/walformat.html)89