A Chinese language model with just three billion parameters sometimes matches models a hundred times larger on math and coding tasks. The researchers behind it have developed a hypothesis about how AI capabilities are structured.
Weibo's parent company Sina has released a small language model that competes with today's top models on hard math and coding tasks. According to a technical report, VibeThinker-3B performs on par with DeepSeek V3.2 and Kimi K2.5 on competitive benchmarks like AIME26. Both of those models have 200 to 333 times more parameters.
Sina positions the model as an experiment in figuring out how much compute a model actually needs to compete at the top. Its predecessor, VibeThinker-1.5B, launched in November 2025. The new version pushes further, asking whether a small model can hit genuine top-tier performance, not just be "good for its size."
The results tell two different stories. On structured tasks with clearly verifiable solutions, like math olympiads or programming challenges, VibeThinker-3B matches models like GLM-5 or Gemini 3 Pro. On LiveCodeBench, it beats every other model under 20 billion parameters.
Factual knowledge is a different story. On the knowledge-heavy GPQA-Diamond benchmark, the model falls well behind its much larger competitors.
To rule out data contamination, the team had the model compete in LeetCode contests held between late April and late May 2026, after training wrapped up. VibeThinker-3B solved 123 out of 128 problems on the first try. That puts it ahead of GPT-5.2, Qwen3-Max, Kimi K2.5, and Claude Opus 4.6. It trails only GPT-5.3-Codex, Gemini 3.1 Pro, and Gemini 3 Flash, but not by much.
VibeThinker-3B builds on Alibaba's Qwen2.5-Coder-3B. Sina's contribution is the post-training, everything that happens after generic pre-training on large data sets. According to the report, that's what brings a 3B model close to the top performers.
Post-training happens in stages. First, the model learns a broad range of tasks through supervised fine-tuning, covering math, coding, and general dialogue. Then it gets tailored for hard, multi-step reasoning problems.
Reinforcement learning follows, applied sequentially for math, programming, and STEM. Self-distillation then consolidates the skills from each phase into a single model. A final step makes sure the model better follows instructions.
During fine-tuning, the team deliberately builds up a wide variety of solution paths. Reinforcement learning then strengthens the ones that work. The argument is that performance comes from training methods, data quality, and reliable validation signals rather than from more parameters.
Based on these results, the authors propose what they call the "Parametric Compression-Coverage Hypothesis." Different AI capabilities have different structures and need different numbers of parameters.
Logical reasoning, like solving a math problem step by step, relies on a few recurring patterns. Searching, checking conditions, correcting errors, combining intermediate results. That kind of skill can be packed into a compact core. World knowledge works differently. Answering open-ended questions across many topics requires broad coverage, meaning lots of parameters storing lots of facts.
This reframes what small models are for, the researchers say. They're not just cheap, lightweight versions built for budget inference but an independent research path running parallel to traditional scaling logic. Where tasks are verifiable and have clear solution structures, parameter count isn't the bottleneck anymore.
VibeThinker-3B is openly available on Hugging Face and GitHub.
Small models catching up to much larger systems on narrow tasks is becoming a pattern. In April, Alibaba's Qwen3.6-27B outperformed its predecessor, which was 15 times larger, across all coding benchmarks. Falcon H1R 7B from Abu Dhabi hit the performance level of models two to seven times its size, according to its makers. Earlier studies on logical gaps in small models suggested they generally hit a wall on multi-step reasoning. The VibeThinker results on verifiable tasks challenge exactly that assumption.
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。