Objective

One command → a full GPU/CPU benchmark & an interactive HTML report

You can now measure your consumer GPU and/or CPU performance on typical Artificial Intelligence and Machine Learning workloads in a controlled way, with some pre‑set reference results.

The reproducible benchmarks cover:

• Ollama LLMs (token latency & throughput on various 3B → 14B parameter models)
• XGBoost (training & inference on the HIGGS dataset, on 100k → 10M+ rows)

Everything is orchestrated by a single YAML file (ai_bench_suite.yaml) and a runner script (run_suite.py), so you can launch an entire set of tests with one command.

Results are visible:

• immediately at the end of the benchmark, in a notebook that is automatically produced, with comparison against a handful of reference systems;
• on a regularly updated Streamlit dashboard, to better interact with a growing number of results: https://ai-ml-gpu-bench.streamlit.app

Quick start

git clone https://github.com/albedan/ai-ml-gpu-bench
cd ai-ml-gpu-bench
uv run run_suite.py

For Ollama benchmarks, make sure Ollama is installed and running at http://localhost:11434. To automatically pull missing Ollama models during the full benchmark:

uv run run_suite.py --autopull

What happens during a run

1. A unique run_id is generated.
2. The benchmarks specified in the configuration YAML file are executed.
3. The results of each test are recorded in two separate CSVs for XGBoost and Ollama (if both are selected).
4. The Jupyter notebook is executed and exported to HTML; it opens automatically in the browser (the bars with a thick border are those from the just‑completed run).
5. If you’d like to help grow the reference result base, the two CSVs are encrypted (RSA 4096 bit) and uploaded to Filebin, submitting only technical data (opt-out available).
6. A daily ingestion process imports new results and publishes them to the Streamlit dashboard. More on the architecture underneath: https://allaboutdata.substack.com/p/benchmarking-ai-and-ml-on-local-cpugpus

What to expect: two examples

Multiple machines benchmarked on Deepseek-R1 14B via Ollama (Streamlit dashboard):

XGBoost tested on the full HIGGS dataset, both with GPU and CPU (Streamlit dashboard):

Official results are regularly updated and published on the Streamlit dashboard: https://ai-ml-gpu-bench.streamlit.app

For convenience, a quick Jupyter notebook is immediately shown at the end of the benchmark.

Get started!

Requirements

Make sure you have installed at least the must‑have components below

Environment setup

In a local folder, just clone this repository:

git clone https://github.com/albedan/ai-ml-gpu-bench

Python 3.13.* will be automatically installed (unless already present) via uv.

Configuration: ai_bench_suite.yaml

All benchmark parameters are in this YAML file.

• System defaults for your machine name and hardware details will be used, unless you manually override any of these three fields.

machine_info:
  machine: "" # Choose your preferred computer name (default: hostname)
  cpu: "" # Please specify your CPU
  gpu: "" # Please specify your GPU

• Commenting an LLM model entry in the ollama section will exclude it from the benchmark. If possible, leave them all as provided, to help collecting results on a standard set of LLMs.
• For the active LLMs (i.e., uncommented), you can:
  • Have the benchmark automatically check and if necessary pull them for you, by using the flag --autopull. Consider that it could take a few minutes depending on your connection.
  • Verify manually if they’re available with ollama list and install them with ollama pull [model_name].
• Every combination listed in rows × gpu (for XGBoost) and models × gpu (for Ollama) is run automatically during the benchmark.

Execution

A single command reads the configuration YAML file and orchestrates the test execution, logging and result visualization.

Simply run:

uv run run_suite.py

The first run may take a bit longer because uv will create the environment and install Python/package dependencies automatically. Ollama has to be already installed for LLM benchmarks. You can automatically pull the models specified in the YAML file just by adding --autopull.

--autopull downloads missing Ollama models, but it does not install or update Ollama itself.

At startup, the Ollama suite checks your installed Ollama version against the latest stable release and prints a warning if an update is recommended. The benchmark continues either way.

Consider also the option --fast for benchmarking only on a subset made of the fastest models.

Common commands

Privacy

Output

• CSV: result files xgb.csv and ollama.csv are written, one row per benchmark, with metrics and basic machine metadata.
• Notebook (bench_results_analysis_altair.ipynb): executed and opened automatically in the browser. It lets you explore the newly obtained results and compare them with reference benchmarks.

❓ Q&A

🔚 Thanks for testing!

If you find an issue or have an idea, open an Issue – or, even better, a Pull Request!
Whenever possible, please keep result sharing enabled to help grow the references! 🚀

Happy benchmarking and experimenting!