In this blog post, we will see how I built iamspeed.dev, a fast.com-style LLM API speed benchmark tool that measures Time to First Token (TTFT) and tokens-per-second throughput directly in your browser.

If you have ever stared at a spinning cursor waiting for an LLM response and wondered "is this slow, or is it just me?" this tool is for you.

The tool is designed for quick, lightweight benchmarking, modeled after the fast.com experience for internet speed tests. It uses an extensible provider adapter architecture, making it straightforward to add new providers such as Gemini or Groq. Planned additions include historical results, model comparison mode, and support for local models via Ollama.

iamspeed.dev is an open-source, browser-based benchmarking tool for LLM APIs that measures two key performance metrics: Time to First Token (TTFT) and tokens-per-second throughput. It supports OpenAI and Anthropic providers and stores API keys locally using AES-GCM encryption, with no backend or data transmission.

The Problem That Sparked This

I spend a lot of time benchmarking systems. Load testing APIs, profiling microservices, measuring throughput it is what I do at QAInsights and at my day job.

When LLMs started becoming part of production stacks, I noticed that most developers just eye-balled "it feels fast" or "it feels slow." There was no quick, browser-based tool you could open, configure your API key, and immediately get a concrete number.

Tools like artificial analysis do heavy-lifting comparisons across hundreds of models. But I wanted something lighter. Something you could open on a Tuesday afternoon and just run.

That is exactly how fast.com works for internet speed tests. You open it, it runs, you see a number. Done.

So I built the same thing for LLM APIs: iamspeed.dev.

---

What Is iamspeed.dev?

iamspeed.dev is an open-source, browser-based benchmarking tool for LLM APIs. It streams live tokens from supported providers OpenAI and Anthropic today and shows you real-time performance metrics as they happen.

No backend. No data collection. No surprises.

Your API key is stored locally in your browser using AES-GCM encryption, meaning it never leaves your machine.

The interface is deliberately minimal, as shown below just a logo, a metric display, a Run button, and a settings panel. Inspired directly by the fast.com aesthetic.

---

Key Metrics: What Gets Measured

If you work with LLMs in production, you already know that raw response time is a misleading number. The two metrics that actually matter are:

1. Time to First Token (TTFT)

This is the time between sending your request and receiving the very first token back from the model. It reflects how quickly the LLM starts generating a response.

TTFT is what users feel. A high TTFT means that awkward pause before anything appears on screen.

For interactive applications, keeping TTFT low is critical. Reasoning models (extended thinking, deep think modes) can inflate TTFT by 5x to 30x because of the additional compute happening before the first visible token arrives.

2. Tokens Per Second (Throughput)

This is the rate at which the model streams tokens to you after the first one arrives. It is the "output speed" metric.

High tokens per second means the text appears fast and fluid on screen. Low throughput feels choppy and slow even if the TTFT was acceptable.

Together, these two numbers give you the full picture of how an LLM API performs for your use case.

---

Features at a Glance

Here is a quick summary of what iamspeed.dev supports:

• Live streaming output with real-time metric updates
• TTFT measurement captured precisely at the moment the first token arrives
• Tokens/sec throughput tracking updated continuously during generation
• AES-GCM encrypted API key storage local only, never transmitted
• OpenAI provider support (GPT-4o, GPT-4.1, and compatible models)
• Anthropic provider support (Claude Sonnet, Claude Haiku, and more)
• Extensible provider architecture via a clean ProviderAdapter interface
• Responsive minimal UI inspired by fast.com

The key thing I want to highlight is the local encryption. I have seen too many tools that ask for your API key and quietly send it somewhere. iamspeed.dev does not do that. Your key is AES-GCM encrypted and stored only in your browser's local storage.

The provider architecture is clean and intentional. Each LLM provider is implemented as an adapter that satisfies the ProviderAdapter interface. This makes adding new providers straightforward and keeps the core benchmark logic provider-agnostic.

The project is hosted at iamspeed.dev and the full source is available on GitHub.

---

How to Run It Locally

Running iamspeed.dev locally takes under a minute. Here are the steps:

1. Clone the repository:

git clone https://github.com/QAInsights/iamspeed.dev.git
cd iamspeed.dev

2. Install dependencies:

npm install

3. Start the development server:

npm run dev

4. Head to http://localhost:4321 in your browser.
5. Click the gear icon (Settings) and enter your OpenAI or Anthropic API key.
6. Hit Run.

You will immediately see the tokens streaming in and the tokens/sec counter updating live, as shown below.

Here are all the available commands:

---

How to Add a New Provider

This is where the architecture really shines. If you want to add support for, say, Gemini or Groq, the process is clean:

1. Create a new adapter file in src/lib/providers/. Your adapter must implement the ProviderAdapter interface.
2. Register it in src/lib/providers/index.ts.
3. Add the provider metadata (name, models, etc.) to src/lib/config.ts.

That is it. No changes to the benchmark engine, no changes to the UI logic. The adapter pattern keeps concerns separated cleanly.

I am planning to add more providers over time. If you want to contribute one, pull requests are welcome.

---

Why This Matters for Performance Engineers

I want to speak directly to performance engineers here for a second.

We are used to measuring systems with JMeter, k6, Gatling. We understand throughput, latency percentiles, concurrency, think time. LLM APIs add a new dimension to all of this.

When you are building an AI-powered product, you are not just measuring HTTP response time anymore. You are dealing with:

• TTFT as a user-perceived latency metric (equivalent to time-to-interactive in web perf)
• Streaming throughput as a sustained delivery rate (not a one-shot measurement)
• Provider variability the same model can behave very differently across regions and time of day
• Reasoning overhead thinking models add invisible compute time before the first visible token

Tools like iamspeed.dev give you a quick sanity check. Before you design a full performance test suite for your LLM-powered API, run a quick benchmark here to understand your baseline numbers.

I have written extensively about LLM performance metrics on the QAInsights blog and built the jmeter-llm-sampler plugin for measuring TTFT and TTLT in JMeter test plans. iamspeed.dev is the browser-friendly companion to those deeper tools.

---

What's Next

A few things I want to add to iamspeed.dev:

• More providers: Gemini, Groq, Mistral, and local Ollama support
• Historical results: Run multiple benchmarks and compare them over time
• Model comparison mode: Run the same prompt across two models side by side
• Shareable result links: Generate a URL you can share with your team
• Prompt customization: Let you choose the input prompt length to simulate different workloads

If any of these sound useful to you, drop a star on the GitHub repo and let me know what you want to see first.

---

Try It Now

Head to iamspeed.dev, configure your API key in settings, and hit Run.

You will have your tokens-per-second number in about 10 seconds.

The source code is MIT licensed and available at github.com/QAInsights/iamspeed.dev. Contributions are open.

Happy Testing!

---

What LLM provider are you using in production today, and what TTFT are you seeing? Drop a comment below I would love to know how the numbers compare.

---