Tracking down changes in database performance is one of the hardest parts of engineering, especially when the change is buried somewhere in a long commit history.
To make this work easier and more repeatable, I put together a small but important tool:
This script does one thing well:
Given a commit hash, it checks out that commit, builds it cleanly, and packages it in a deterministic way so the Test Automation Framework (TAF) can run consistent performance tests.
Why this matters
- When you are bisecting or doing a manual binary search across hundreds of commits, you need reproducible builds.
- You need each commit packaged the same way so TAF can compare performance results fairly.
- You need a fast, reliable way to prepare builds when you are hunting for the commit that changed TPS, latency, or other performance metrics.
This script is built for exactly that workflow.
It pairs naturally with git bisect or manual commit selection and removes a lot of the friction that normally slows down change hunting. Instead of hand-building each commit, you feed the hash to the script and get a clean, packaged build ready for TAF.
There are plans for future TAF releases to automate the entire commit-tracking process end-to-end. Until then, this tool is available for anyone who wants to help identify which commits impact MariaDB performance.
Knowing which commit introduced a performance change shortens investigation, reduces back-and-forth, and leads to faster fixes when needed.
Acknowledgment
Special thanks to Amrendra Kumar for contributing an important patch that improved SQL file handling and ensured compatibility with traditional “source” and “.” usage in the MySQL client. This made the tool more robust and aligned with long-standing workflows.
Special thanks to Alex Padula for his contribution to the TAF ecosystem through his recent pull request, which improved integration with TidesDB and expanded TAF’s support for emerging storage engine work. His efforts strengthen the flexibility of the framework and help ensure that new engines, like TidesDB, can be tested,
validated, and compared consistently as they evolve.
Looking Ahead
These are busy and exciting times for MariaDB, and the MariaDB Foundation.
New storage engines are exploring joining the ecosystem, which means a surge of performance and functional testing across many areas of the server.
Tools like this help keep the process consistent and reproducible as the code-base evolves.
Stay tuned to the MariaDB Foundation for more updates as we continue strengthening our performance tooling and testing infrastructure.
The Foundation welcomes community feedback as TAF continues to mature.
About the Author
Jonathan (Jeb) Miller has experience in military leadership, Fire/EMS leadership, computer operations leadership, teaching, and more than 26 years of database performance engineering (PervasiveSQL, MySQL, MariaDB). TAF is the third benchmarking framework he has helped design and build.