Building SQLite with a small swarm

Source: Hacker News

tl;dr

I tasked Claude, Codex, and Gemini to build a SQLite-like engine in Rust.

• 19k lines of code.
• Parser, planner, volcano executor, pager, B+‑trees, WAL, recovery, joins, aggregates, indexing, transaction semantics, grouped aggregates, and stats‑aware planning all implemented.
• 282 unit tests, all passing.

background

Treat software engineering like distributed systems, and force coordination with: git, lock files, tests, and merge discipline.

harness

├── AGENT_PROMPT.md       // main agent task prompt
├── BOOTSTRAP_PROMPT.md   // bootstrap (initialization) prompt
├── COALESCE_PROMPT.md    // deduplication prompt for coalescer agent
├── launch_agents.sh      // launches all agents and sets up isolated workspaces
├── agent_loop.sh         // per‑agent loop/worker script
├── restart_agents.sh     // restarts agents
└── coalesce.sh           // invokes the coalescing script

workflow

bootstrap phase – one Claude run generates baseline docs, crate skeleton, and test harness.

├── Cargo.toml         // crate manifest
├── DESIGN.md          // architecture design notes
├── PROGRESS.md        // test & build progress
├── README.md          // project overview
├── agent_logs         // per‑agent log files
├── crates             // workspace subcrates
├── current_tasks      // lock files
├── notes              // inter‑agent notes
├── target             // build artifacts
└── test.sh            // test harness script

worker phase – six workers loop forever (2x Claude, 2x Codex, 2x Gemini).

loop

• Each agent pulls the latest main.
• Claims one scoped task.
• Implements + tests against sqlite3 as oracle.
• Updates shared progress/notes.
• Pushes changes.

analysis

coordination tax

• 84 / 154 commits (54.5 %) were lock/claim/stale‑lock/release coordination.
• Demonstrates parallel‑agent throughput depends heavily on lock hygiene and stale‑lock cleanup discipline.

what helped most

Two things looked decisive:

1. Oracle‑style validation + high test cadence (cargo test … and ./test.sh --fast/full runs captured in PROGRESS.md).
2. Strong module boundaries (parser → planner → executor ↔ storage) so agents could work on orthogonal slices with fewer merge collisions.

redundancy

I implemented a coalescer with Gemini to clean duplication/drift, since that is the largest problem with parallel agents. It only ran once at the end of the project, so it was never actually used during the run itself. A cron job runs it daily, but Gemini couldn’t complete the entire de‑duplication when I ran it during the experiment; it stopped mid‑way.

takeaways

• Parallelism is great, but only with strict task boundaries.
• Shared‑state docs (PROGRESS.md, design notes) are part of the runtime, not “documentation.”
• Tests are the anti‑entropy force.
• Give agents a narrow interface, a common truth source, and fast feedback, and you get compounding throughput on real systems code.

replication

git clone git@github.com:kiankyars/parallel-ralph.git
mv parallel-ralph/sqlite .
chmod 700 sqlite/*.sh
./sqlite/launch_agents.sh

Restart agents:

./sqlite/restart_agents.sh claude/codex/gemini

Coalesce agent:

./sqlite/coalesce.sh

Assumes you have the relevant CLIs installed (claude, codex, gemini), plus screen, git, the Rust toolchain, and sqlite3.

limitations

• The documentation in the repo became enormous; PROGRESS.md grew to 490 lines, and the sheer amount of notes shows that the coalesce agent must be run as often as the other agents.
• There isn’t a great way to record token usage since each platform uses a different format, so I don’t have a grasp on which agent pulled the most weight.

future work

• Track “substantive run rate”, since many runs are rate‑limited or produce no activity.
• Only Claude adds itself as a co‑author to each commit; I need to add appropriate commit messages for Gemini and Codex.
• Add stricter observability because many errors were likely due to rate limits being hit mid‑work, resulting in half‑finished work being pushed.

inspiration

appendix

code size snapshot

154 commits between 2026‑02‑10 and 2026‑02‑12.

usage

Gemini does not offer a way to monitor usage with its CLI. It’s measured on a 24‑hour basis. For Codex, I used 100 % of the Pro Plan weekly usage (currently on a 2× promotion). I used 70 % of the Claude Pro weekly usage.

codex

claude

disclaimer

• Codex wrote the first draft for this post.

citation

@misc{kyars2026sqlite,
  author = {Kian Kyars},
  title = {Building SQLite With a Small Swarm},
  year = {2026},
  month = feb,
  day = {12},
  howpublished = {\url{https://kiankyars.github.io/machine_learning/2026/02/16/sqlite.html}},
  note = {Blog post}
}