Last post I said the most important decision in Orbit is splitting cluster state into two planes, and then I made you wait for it. Here it is. If you only read one post in this series, read this one, because almost everything else is downstream of this single cut.
Everyone tells you not to build your own cluster manager. They're mostly right. Kubernetes exists, it works, it used all over the place...better job prospects if you use K8s. Building another one is the kind of decision that gets you eyerolls in conversation. So let me undermine
foldb has one source of truth: an append-only, Raft-replicated log. Everything else, the storage layer, the indexes, the materialized query results, is just a fold(log). A pure function applied to a sequence of committed entries in order. A transaction commits when its entry is durable in the
Out of the gate, what is TrueTime? It's an algorithm used in Google Spanner, giving it the ability to do something no distributed database had done before...globally consistent transactions without a centralized timestamp authority. Not only is it a breakthrough algorithm for the time, it's
Every software product pretends to be built for its users. But most are actually built on them... extracting value in the form of data, attention, or compliance. We see this everywhere: * CRMs that sales people hate but must use * Social platforms that drain users to sell to advertisers * Productivity tools
Everyone tells you not to build your own distributed storage system. "Just use S3," they'll say. "Use Ceph. Use MinIO. Real engineers don't reinvent wheels." They're wrong, real engineers build whatever interests them, with expectations in reality. So let'
I've been building a storage system on CXL-attached persistent memory, and one component that took more iteration than expected was the NUMA-aware allocator. The concept is simple--bind memory to specific NUMA nodes--but the implementation has some sharp edges that aren't obvious until
I've been knee-deep in building a distributed storage system and wanted to talk about a component that doesn't get much love in the literature: The address space manager. It's one of those pieces that sits quietly in the middle of everything, just doing
A write-ahead log (WAL) is one of those database concepts that sounds deceptively simple. You write a record to disk before applying it to your in-memory state. If you crash, you replay the log and recover. Done. Except your disk is lying to you. PostgreSQL, SQLite, RocksDB, Cassandra.
When you're thinking about code generation at scale, the question that keeps coming back up is, "where does it live?" If an AI coding agent generates a function, you write it to a file. Next function, different file. You organize with directories. You follow naming conventions.
NOTE: WILD now has a write-ahead log for durability, and the commentary below that states it's not durable no longer applies. It's safe to use if you can find a use case for it. Remember when we thought putting databases in RAM was revolutionary? Great
I've been working on a complex multi-model database for a few weeks now, and recently I took time to simplify and test out an idea I had on a simple key-value database. I started with the basics: A hash table in memory, a simple append-only