SNKV- Key value store based on sqlite b-tree engine

Source: Dev.to

Overview

There are many key‑value stores available, such as RocksDB and LevelDB.
Even though SQLite is a SQL database, people often use it as a storage engine. This led to the creation of SNKV:

• Repository:

Architecture

The architecture of SNKV is very simple:

kvstore layer → b‑tree layer → pager → OS

The lower layers are already battle‑tested and production‑ready.

Using SNKV in C/C++ Projects

1. Generate the single‑header file:

   make snkv.h

   Or download the ZIP from the 0.4.0 release.

2. Include the header in your .c or .cpp files and define the implementation macro before the first inclusion:

   #define SNKV_IMPLEMENTATION
   #include "snkv.h"

   This ensures the implementation is compiled into your executable.

3. Review the API specification for details:

Python Support

Python developers can install SNKV via pip:

pip install snkv

Documentation for the Python APIs:

Data Layout

The entire SQL layer is bypassed. Data is stored in the format:

key_len | key | value

This layout allows fetching values based on key prefixes quickly in O(log n) time.
Each table maintains a cached read cursor, improving the performance of get, exists, etc.

Transaction Model

• Read transactions start automatically after kvstore_open, reducing the time needed to find mxFrame.
• Write operations follow this sequence:
  1. The current read transaction is committed.
  2. A write transaction runs and commits.
  3. A new read transaction starts again.

Examples

• Python examples:
• General examples (C/C++):

Crash‑Safety Test

A crash test was implemented that:

1. Writes deterministic key‑value pairs into a 10 GB WAL‑mode database.
2. Forcefully kills the writer using SIGKILL during active writes.
3. Verifies on restart that:
   • Every committed transaction exists with byte‑exact values.
   • No partial transactions are visible.
   • The database shows zero corruption.

Ecosystem Support

Since SNKV uses the same storage engine as SQLite, tools that rely on SQLite’s lower layers work directly with SNKV. Verified tools include:

• LiteFs
• WAL‑based backup tools
• Rollback journal tools

Contributing

If you encounter any issues or have suggestions, please open an issue:

Feedback and thoughts are welcome.