# 2025 in review: two thousand vCPUs and one quiet pager > A year-ender written in an empty Tokyo office: the numbers that held, the two things I shipped that mattered, the failure I did not see coming, and what has been pulling at my curiosity all year. Author: Karan Vijayakumar Published: 2025-12-30 Tags: year-in-review, sre, observability, career Reading time: 7 min Canonical URL: https://karanvk.me/blog/2025-in-review --- Tokyo between the holidays is my favorite version of this city. The trains have room on them. Half the shutters on my street are down with a printed note taped to the glass about reopening on the fourth. The office is empty except for me and one person from the data team who I think may live here. Everything smells faintly of cleaning products because of the year-end 大掃除 ritual, where the whole building gets scrubbed on the way out of December. It's the only week of the year when I can think about the year. ## The numbers Plainly, because they're better plain. We held 99.99% availability across the platform. That's the number on the wall and it was the number at the end. The estate ran at 98% utilization. Two thousand and change vCPUs, a hybrid of cloud and our own capacity, running that hot on purpose. Those two facts sitting next to each other is the thing I'm proudest of this year, because four nines is easy if you're willing to buy three times the hardware, and 98% utilization is easy if you don't mind falling over. Doing both means the system has to degrade correctly rather than have room to be sloppy in. Our observability stack ingested north of two terabytes an hour. Self-hosted, the whole LGTM set, running on our own infrastructure. And no Datadog invoice arrived. Not in January, not in any month. I'm aware that's a slightly cheap way to state a win, and I'm going to keep stating it that way, because when I priced the equivalent volume against a vendor at the start of the year the number was genuinely difficult to say out loud in a meeting. The pager was quiet. Not silent, and I'll get to a week where it wasn't, but quieter than 2024 by a wide margin, and the quiet was engineered rather than lucky. That's the whole story of the year in one sentence. ## What actually shipped Two things I'd point at. The first was teaching the deploy pipeline to read the SLO before it commits to anything, which I wrote up in August in [teaching deploys to read the SLO first](/blog/slo-gated-deploys). Short version: a deploy now asks the error budget whether it's welcome, watches its own effect on burn rate, and reverts itself when the answer turns out to be no, with nobody in the loop. It took most of a summer and the part that took longest wasn't the gating logic. It was getting the SLO definitions honest enough to be worth gating on, which meant admitting that a few of our published SLOs were describing things users could not perceive. That change is most of why the pager got quiet. Not because fewer things break, but because the thing that used to break at 6pm and page at 6:40 now gets reverted at 6:04 while everyone is still in the room. The second was the self-hosting write-up in November, [one year in](/blog/lgtm-one-year-ledger), a full ledger of running our own metrics and logs and traces for a year. Costs, hours, incidents, the things I'd do differently. I tried hard to make that one honest in both directions, because the genre is usually either vendor-sponsored or triumphalist, and the truth is that self-hosting at this volume is cheaper by a large multiple and also costs you real engineering weeks you would otherwise have spent elsewhere. Both halves are true and you have to decide which currency you're short on. ## The failure Here's the one that stuck with me. Getting the estate from the low eighties to 98% utilization was a technical success and a social failure, and I didn't notice the second part for about two months. I shipped the changes faster than anyone's mental model of the system could update. Better bin-packing, tighter requests, more aggressive scale-down, and it all worked, and the graphs were beautiful. What I hadn't accounted for is that people who own a service and see headroom disappear will do something about it, and what they will do is quietly protect themselves. Two teams added static overprovisioning to their own deployments. Nothing dramatic, no meeting, no ticket, just a floor on replica count and a request bump "for safety" in a PR that reviewed fine. Between them they ate a meaningful slice of the savings, and because it looked exactly like normal capacity from where I was standing, my dashboards told me nothing. I found out because someone mentioned it in passing at lunch, in the tone you'd use for something completely uncontroversial. Which it was, to them. They'd watched their headroom vanish and taken a reasonable precaution. The lesson isn't about autoscaling. It's that I optimized a number and not the trust around the number, and a platform change nobody believes in gets reverted by hand, in ways your instrumentation cannot see. The fix was not technical at all: a walkthrough of what the scheduler would actually do during a spike, per team, with their service and their real traffic, plus a dashboard they could look at themselves during an event instead of taking my word for it. The static floors came off within a month, voluntarily. I should have done the walkthrough first and the bin-packing second. I've started treating "who has to believe this for it to hold" as part of the design work, not the rollout comms. ## What kept pulling at me Two itches all year, and they turned out to be the same itch. Typed configuration. I've spent an embarrassing number of evenings reading about CUE and Nickel and, briefly and regrettably, arguing about Dhall on the internet. The reason is simple: nearly every bad night I've had in the last three years began with a YAML file that was syntactically perfect and semantically nonsense. A value in the wrong unit, a field that silently did nothing because it was misspelled, a combination of options that isn't legal but nothing checks. The idea that configuration could have constraints, and that an illegal combination could fail at render time in a PR instead of at 3am in a cluster, has stopped feeling like an aesthetic preference and started feeling like the actual answer. And the hardware itch. Running at 98% utilization is where you lose the ability to pretend hardware is an abstraction. You start caring which NUMA node a process landed on. You read the kernel's scheduler docs on a Sunday. You develop opinions about page cache and cgroup memory accounting and what your hypervisor is lying to you about. For most of my career the layer beneath the container was somebody else's problem, and this year it repeatedly became mine, and I found that I liked it much more than I expected to. Those are the same itch, I think. Both are about pushing the moment of truth earlier and closer to the metal, so the system can't quietly be wrong in the gap between what you declared and what actually runs. ## December I did Advent of Code, the way I do every year, and on day 15 I finished second globally. I wrote that one up separately in [performance sport](/blog/advent-of-code-leaderboard), including the part where I couldn't read my own code an hour later. Puzzles unlock at 2pm here, which is the best hobby scheduling anywhere on earth, and it makes December feel like a fifteen-year-old's winter break every single year. ## Next Next year the vCPUs become megawatts. I'll leave it there.