This is the post where "improve on Borg today" stops being a slogan. Borg was designed for a world of fungible CPU and memory, and that world is gone for an enormous and growing slice of what data centers actually do, which is train large models. Almost every
Networking is the one place Orbit refuses to take Borg's advice. Borg's own retrospective says to give every unit its own IP, and Kubernetes did exactly that, an IP per pod, and then spent a decade building CNI plugins, overlay networks, IPAM to hand out the
The reclamation post ended on a debt: the whole utilization story depends on the reservation being a good estimate of real usage, and nobody said where that estimate comes from. The answer is a control loop, and the deeper answer is a position I'll defend for the rest
Here's a number that should bother you: most clusters run at something like a third of their capacity, and the operators know it, and they pay for the other two thirds anyway. The reason is that everyone sizes their workloads for the worst case. You ask for the
Everything I've written about so far is the control plane: the ledger, the schedulers, the airlock. Now go stand next to an actual machine in an actual rack. On that machine runs the Satellite, the per-node agent, and it has one property that matters more than all
Every system needs a front door, and the front door is where you decide what "allowed in" means before anything inside has to care. In Orbit that's the Airlock. Nothing enters a Constellation without passing it: the client proves who they are, the mission proves it&
A scheduler is where an orchestrator earns its keep, and it's also where Borg admitted it had painted itself into a corner. The scheduling policy lived in one place, it grew over years to serve every workload Google had, and the paper is candid that the single scheduler
The telemetry plane is the easy half of the state split to describe and the easy half to get quietly wrong. It carries everything the cluster observes about itself: usage, health, pressure, liveness, GPU temperature. It's high-volume, it's allowed to be stale, and the entire
The Ephemeris is the part of Orbit I was most nervous about, because it's the part that has to be correct. Telemetry can drop a sample and shrug. The ledger cannot drop a placement, can't double-book a node, and can't disagree with itself
Before Orbit can place a workload or reclaim a core, it has to know how to say what a workload is. That sounds like the boring part. It isn't. The shape of the object model is where Borg and Kubernetes both left scars, in opposite directions, and getting
Last post I said the most important decision in Orbit is splitting cluster state into two planes, and then I made you wait for it. Here it is. If you only read one post in this series, read this one, because almost everything else is downstream of this single cut.
Everyone tells you not to build your own cluster manager. They're mostly right. Kubernetes exists, it works, it used all over the place...better job prospects if you use K8s. Building another one is the kind of decision that gets you eyerolls in conversation. So let me undermine