While recent breakthroughs in AI reasoning have largely been driven by massive scale, pouring in billions of parameters to cross complex cognitive thresholds—VibeThinker-3B is charting a completely different path.
Created by researchers from Sina Weibo Inc (China), this 3-billion-parameter model proves that efficiency can punch far above its weight class. Released under an open-source MIT license, VibeThinker-3B matches the performance of models hundreds of times its size on verifiable tasks like mathematics, coding, and STEM disciplines.
VibeThinker-3B is a compact dense model built on the Qwen2.5-Coder-3B base. It is post-trained, not pretrained from scratch. The research team applies supervised fine-tuning, reinforcement learning, and self-distillation on top.
The training framework continues the Spectrum-to-Signal Principle (SSP) from the earlier VibeThinker-1.5B. SFT (Supervised Fine-Tuning) builds a broad space of valid reasoning paths, the ‘Spectrum.’ RL then amplifies the correct paths, the ‘Signal.’
The model targets one job: reasoning where a verifier can confirm the answer. The research team recommends larger general models for open-domain knowledge tasks. VibeThinker-3B is a specialist by design.
It runs on standard stacks. The model weights require transformers>=4.54.0. For faster inference it recommends vLLM==0.10.1 or SGLang>=0.4.9.post6. The BF16 weights are roughly 6 GB, small enough for a single GPU.
On AIME26, VibeThinker-3B scores 94.3. According to the research paper, this is comparable to DeepSeek V3.2 (671B) and Kimi K2.5 (1T).
On LiveCodeBench v6, it reaches 80.2 Pass@1. On OJBench, another code benchmark, it scores 38.6, below the largest models. On HMMT25 it scores 89.3, and on BruMO25 it reaches 93.8. On IMO-AnswerBench, a 400-problem IMO-level set, it scores 76.4.
The table below compares it against much larger reasoning models. The ‘+CLR’ row uses test-time scaling. It stands for Claim-Level Reliability Assessment
| Model | Params | AIME26 | HMMT25 | IMO-Ans | LCBv6 | GPQA-D |
|---|---|---|---|---|---|---|
| VibeThinker-3B | 3B | 94.3 | 89.3 | 76.4 | 80.2 | 70.2 |
| VibeThinker-3B +CLR | 3B | 97.1 | 95.4 | 80.6 | — | 72.9 |
| GPT-OSS (high) | 120B | 93.2 | 90.0 | 75.6 | 81.9 | 80.1 |
| DeepSeek V3.2 | 671B | 94.2 | 90.2 | 78.3 | 80.8 | 82.4 |
| GLM-5 | 744B | 95.8 | 97.9 | 82.5 | 85.5 | 86.0 |
| Kimi K2.5 | 1T | 93.3 | 95.4 | 81.8 | 85.0 | 87.6 |
The pattern is consistent. On verifiable math and code, the 3B model sits near the top cluster. On GPQA-Diamond, a knowledge-heavy benchmark, the gap to large models stays visible.
The research team also ran an out-of-distribution coding test. It used recent LeetCode weekly and biweekly contests, from Apr 25 to May 31, 2026. The model passed 123 of 128 first-attempt Python submissions. That is a 96.1% acceptance rate on unseen problems.
The post-training pipeline runs in four stages. Each one targets a different weakness of small reasoning models.
First comes curriculum-based two-stage SFT. Stage 1 covers math, code, STEM, dialogue, and instruction following broadly. Stage 2 shifts to harder, longer-horizon samples filtered by reasoning length and difficulty. Diversity-Exploring Distillation preserves multiple valid solution paths through both stages.
Second comes multi-domain Reasoning RL. The research team reuses MaxEnt-Guided Policy Optimization (MGPO). MGPO weights prompts near the model’s current capability boundary, where correct and incorrect rollouts coexist. Training runs sequentially across Math, Code, and STEM.
A notable detail: VibeThinker-3B drops progressive context expansion. The research team found high-truncation warm-up hurt long reasoning at this scale. So RL uses a single 64K long-context window throughout.
Math RL adds a Long2Short stage. It redistributes reward among correct trajectories by length. Shorter correct answers get higher reward, longer ones lower, with the group mean unchanged. The goal is fewer redundant tokens without losing accuracy.
Third, Offline Self-Distillation merges the RL checkpoints back into one student model. Fourth, Instruct RL improves instruction adherence. That stage explains the 93.4 IFEval and 74.5 IFBench scores. Both show reasoning tuning did not break controllability.
Claim-Level Reliability Assessment (CLR) is the report’s test-time scaling method. It runs on answer-verifiable tasks and adds no parameters.
The procedure has two steps. The model first generates K = 32 trajectories per problem. From each, it extracts M = 5 decision-relevant claims plus a final answer.
The model then acts as its own verifier. It validates or falsifies each claim, producing binary verdicts. CLR maps these into a nonlinear trajectory reliability score, where one weak claim sharply lowers the weight.
Answers are clustered by equivalence, and the highest reliability-weighted answer wins. The full flow runs 8 times, and the averaged Pass@1 is reported. CLR lifts AIME26 to 97.1 and BruMO25 to 99.2.
The interactive demo below lets you flip claims and watch the score collapse. It also lets you switch benchmarks and compare against larger models.
The research team frames VibeThinker-3B as a specialist, so use cases follow the verifiable-reasoning boundary.
Serving with vLLM exposes an OpenAI-compatible endpoint:
pip install vllm
vllm serve "WeiboAI/VibeThinker-3B"
curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "WeiboAI/VibeThinker-3B",
"messages": [{"role":"user","content":"Prove there are infinitely many primes."}],
"temperature": 1.0, "top_p": 0.95
}'Direct Transformers usage mirrors the official card:
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("WeiboAI/VibeThinker-3B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
"WeiboAI/VibeThinker-3B", torch_dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "Your prompt"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=102400,
do_sample=True, temperature=1.0, top_p=0.95)
print(tok.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))The high max_new_tokens matters. The model produces long reasoning traces, so short caps can truncate answers.
Check out the Paper, Model weight and Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。