Every once in a while, the TechEmpower Framework Benchmarks surface something that makes everyone involved squint at the numbers and say, “Wait… what?”

Round 23 had one of those.

On the face of it, nothing particularly dramatic had changed about the idea of the benchmarks. Same philosophy as always:

What did change in R23 was that one set of tests, the spring-mongo implementations, looked much worse than anyone expected. Not “eh, that’s a little low” worse, but “this is probably pointing at a real problem” worse.

And because the internet is sometimes better than we deserve, the lead performance engineer at MongoDB noticed, reached out, and we ended up on a little adventure together.

MongoDB’s Java story in the benchmarks is represented by spring-mongo (a Spring-based implementation) as well as a few Node.js variants. In Round 22, the spring-mongo tests were not the fastest things on the planet, but they were in a reasonable ballpark.

By round 23, some of the spring-mongo results in our continuous benchmark environment (Citrine) had fallen off a cliff.

For one of the core tests (query) the numbers looked roughly like this (requests per second, spring-mongo only):

That’s not noise. That’s a smoking gun.

At the same time, other frameworks and other MongoDB-based implementations weren’t seeing that same kind of collapse. That’s usually a hint that something environmental or configuration-related is going wrong, rather than “MongoDB suddenly became slow” or “Spring suddenly can’t talk to the database.”

This is where Ger Hartnett from MongoDB enters the story. Ger is the lead performance engineer at MongoDB, and he did exactly what I wish more vendors would do: he treated the weird benchmark as a bug report.

He emailed me and, very politely, started asking all the questions you’d expect a performance engineer to ask:

That kicked off a back-and-forth where we coordinated on where and how their team could safely tweak the spring-mongo implementations.

Meanwhile, MongoDB pulled in Intel to help analyze what was happening on their own lab hardware, including runs with and without “performance mode” enabled at the platform level.

So we now had three axes of investigation:

The team ended up sending two pull requests:

Once both PRs were in and our continuous benchmarking pipeline had cranked through them, Ger sent a summary email with the numbers for spring-mongo:

The query test went from 583 to 14.2k in our environment. That’s roughly a 25x relative to the Round 23 baseline, and significantly better than the Round 22 result. The other tests also improved substantially, especially once the allocator and pool configuration were both in a sane place.

This was not just about our benchmark setup. MongoDB and Intel ran the same workload in their own environments, with and without CPU “performance mode,” and saw similar patterns: the combination fixes make a huge difference.

I like stories where everyone comes out looking better in the end. MongoDB has a much stronger showing in the benchmarks, the benchmark implementations themselves improve as reference examples, and if you’re out there picking a stack for your next project, you get numbers that are closer to what’s actually possible when people who care about performance have had a chance to sharpen the edges.