How I built a key-value database in a single C++ header

If you've ever needed persistent storage in a C++ project but didn't want to add a dependency, this is for you. I built fluxen, a key-value database that lives in one .hpp file. Drop it in, include it, done.

#include "fluxen.hpp"

fluxen::DB db("app.db");
db.put("city", "Stockholm");
db.put("score", int32_t{100});

auto city  = db.get("city");           // std::optional<std::string>
auto score = db.get<int32_t>("score"); // std::optional<int32_t>

No CMake. No vcpkg. No linker flags.

Why I built it

I kept hitting the same wall on small projects. Need to save some state between runs, nothing fancy, just a handful of keys and values. Every option I looked at either required linking against a library, a build step, or had way more API surface than I needed. SQLite is great but it felt like overkill for storing a dozen keys.

So I built something simpler. This post is a walkthrough of how it works internally, including the parts that took the most thought to get right.

Append-only log with a hash index

Nothing is ever modified in place. Every write, including overwrites and deletes, appends to the end of the file. That one constraint is what makes crash recovery almost free.

The file layout looks like this:

┌─────────────────────────────────┐
│  Magic header: "FLUXEN01"       │  8 bytes
├─────────────────────────────────┤
│  Entry                          │
│    flags   : 1 byte             │  0x00 = live, 0x01 = deleted
│    key_len : 1 byte             │  max 255
│    val_len : 4 bytes            │  little-endian uint32
│    key     : key_len bytes      │
│    value   : val_len bytes      │
├─────────────────────────────────┤
│  Entry                          │
│  ...                            │
└─────────────────────────────────┘

In memory, each key maps to a small struct that records where its value lives in the file:

struct IndexEntry {
    size_t   val_offset;  // byte offset into the file
    uint32_t val_len;
};

std::unordered_map<std::string, IndexEntry> index_;

On startup, fluxen scans the log once and builds this map. Last write wins, tombstones (entries marked with the delete flag) remove entries. After that, every read is just a hash lookup and a pointer dereference with no disk access needed.

This is a Bitcask-style design. The main tradeoff is that all keys live in RAM, which is fine for embedded use cases but worth knowing upfront.

Memory-mapped I/O

The file gets mmap'd into the process's address space rather than read through read() syscalls. That means there's no kernel-to-userspace copy on the read path. The OS just pages in whatever parts of the file are needed.

How much copying actually happens depends on which method you call. The each() and prefix() iteration methods are zero-copy: the callback gets a std::span<const std::byte> pointing directly into the mapped region. get<T>() for structs and scalars does one memcpy into the return value. get<std::string>() does one allocation and copy into the string buffer.

Keeping the mapping coherent after writes took some thought. Writes go through write(), not through the map, so the mapping goes stale. I track this with an atomic dirty flag and a double-checked remap pattern:

void ensure_mapped() const {
    if (!file_.is_dirty()) return;  // fast path: no lock needed

    std::unique_lock sync_lock(sync_mutex_);
    if (file_.is_dirty()) {         // still dirty? we remap
        file_.remap();              // subsequent readers skip this
    }
}

The first reader after a write does the remap. Other readers waiting on sync_mutex_ check the flag again once they acquire the lock, see it's already clear, and skip the remap.

Thread safety

The main lock is a std::shared_mutex. Read methods (get, has, each, prefix) take a shared lock, so any number of threads can read simultaneously. Write methods (put, remove, transaction, compact) take an exclusive lock.

The secondary sync_mutex_ used for remapping is only ever acquired while already holding a shared lock on the main mutex. Writers can only set the dirty flag while holding an exclusive lock, so by the time any reader observes a dirty mapping, no writer is running. The two mutexes don't interact in a way that can deadlock.

Transactions

Individual put() calls append to the file but skip fsync. Fast, but a crash could lose the last write. If you need durability guarantees, transaction() batches everything into one buffer, writes it in one syscall, and calls fsync once:

db.transaction([](fluxen::Tx& tx) {
    tx.put("balance",  int32_t{500});
    tx.put("currency", std::string("USD"));
    tx.remove("old_session");
    return fluxen::commit;  // or fluxen::rollback to discard
});

The in-memory index is updated only after the fsync succeeds. If fsync fails, the partial tail gets truncated before throwing, leaving the database unchanged. If truncation also fails, the DB enters a poisoned state where every subsequent call throws, because at that point consistency can't be guaranteed.

Crash recovery

Since nothing is ever modified in place, a crash mid-write leaves a partial entry at the tail of the file. On the next open, load_index() walks the log and stops when it finds an entry whose claimed size doesn't fit:

if (pos + hdr.key_len + hdr.val_len > file_.size()) break;

The partial bytes get truncated automatically. Everything before the crash is intact.

compact() reclaims space from overwritten and deleted values. It writes live entries to a .tmp file, fsyncs it, then renames it over the original. On POSIX, rename() is atomic, so a crash during compaction leaves one of the two files fully intact.

Storing typed values without a serializer

C++ lets you memcpy any trivially copyable type, so fluxen uses a concept constraint to route between strings and raw bytes automatically:

template <typename T>
    requires(std::is_trivially_copyable_v<T> &&
             !std::is_convertible_v<T, std::string_view>)
void put(std::string_view key, const T& value);

The !is_convertible_v<T, std::string_view> part makes sure string literals and std::string still go through the string overload. On read, if the stored size doesn't match sizeof(T), get<T>() returns nullopt rather than doing a bad memcpy:

struct Config { int port; bool debug; };

db.put("cfg",   Config{8080, false});
db.put("score", int32_t{42});

auto cfg   = db.get<Config>("cfg");    // std::optional<Config>
auto score = db.get<int32_t>("score"); // std::optional<int32_t>

Limitations worth knowing

• No range queries. The index is a hash map, so there's no sorted order.
• One process per file. No shared access across processes.
• No SQL or secondary indexes.
• All keys live in RAM.

If any of those are dealbreakers, SQLite might be the right tool.

Source and full API docs at github.com/dvuvud/fluxen.

How do you usually handle small-scale persistence in your C++ tools? I'm curious to hear what other "no-dependency" solutions you've used!