Everyone tells you not to build your own distributed storage system. "Just use S3," they'll say. "Use Ceph. Use MinIO. Real engineers don't reinvent wheels." They're wrong, real engineers build whatever interests them, with expectations in reality. So let'
I've been building a storage system on CXL-attached persistent memory, and one component that took more iteration than expected was the NUMA-aware allocator. The concept is simple--bind memory to specific NUMA nodes--but the implementation has some sharp edges that aren't obvious until you're debugging
I've been knee-deep in building a distributed storage system and wanted to talk about a component that doesn't get much love in the literature: The address space manager. It's one of those pieces that sits quietly in the middle of everything, just doing its
A write-ahead log (WAL) is one of those database concepts that sounds deceptively simple. You write a record to disk before applying it to your in-memory state. If you crash, you replay the log and recover. Done. Except your disk is lying to you. PostgreSQL, SQLite, RocksDB, Cassandra... every production
When you're thinking about code generation at scale, the question that keeps coming back up is, "where does it live?" If an AI coding agent generates a function, you write it to a file. Next function, different file. You organize with directories. You follow naming conventions.
NOTE: WILD now has a write-ahead log for durability, and the commentary below that states it's not durable no longer applies. It's safe to use if you can find a use case for it. Remember when we thought putting databases in RAM was revolutionary? Great performance,
I've been working on a complex multi-model database for a few weeks now, and recently I took time to simplify and test out an idea I had on a simple key-value database. I started with the basics: A hash table in memory, a simple append-only log for persistence
Have you ever looked at something and thought, "Yeah, that's the right way to do it"? Because that's how I felt when digging into the replication architecture of LibSQL when I was adding replication to Kevo. If you don't care about the
So I've been thinking a lot about database internals the past few weeks. For instance, how do these things actually work under the hood? I mean, we're all writing code that hammers these poor databases with concurrent requests, and somehow they don't explode into
Database architecture might seem like an an arcane art, but at its core, it's about solving a fundamental challenge: how to retrieve and store data efficiently. While you've likely heard of relational databases using B-trees (like PostgreSQL or MySQL), there is another architectural approach that excels