惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

The GitHub Blog
The GitHub Blog
IT之家
IT之家
B
Blog RSS Feed
罗磊的独立博客
GbyAI
GbyAI
博客园 - Franky
Y
Y Combinator Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Google DeepMind News
Google DeepMind News
博客园 - 聂微东
N
Netflix TechBlog - Medium
博客园 - 三生石上(FineUI控件)
人人都是产品经理
人人都是产品经理
U
Unit 42
博客园 - 叶小钗
Jina AI
Jina AI
MyScale Blog
MyScale Blog
雷峰网
雷峰网
B
Blog
Hugging Face - Blog
Hugging Face - Blog
Blog — PlanetScale
Blog — PlanetScale
Recent Announcements
Recent Announcements
腾讯CDC
酷 壳 – CoolShell
酷 壳 – CoolShell

Hacker News: Best

madhadron - The seven programming ur-languages GitHub - smol-machines/smolvm: Tool to build & run portable, lightweight, self-contained virtual machines. I Measured Claude 4.7's New Tokenizer. Here's What It Costs You. Introducing Claude Design by Anthropic Labs It Is Time to Ban the Sale of Precise Geolocation The creative software industry has declared war on Adobe Isaac Asimov: The Last Question Newly unsealed records reveal Amazon’s price-fixing tactics, California attorney general claims Clojure - Documentary Android CLI and skills: Build Android apps 3x faster using any agent Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7 Codex:全能型助手 Introducing Claude Opus 4.7 Qwen Studio The Future of Everything is Lies, I Guess: Where Do We Go From Here? Virginia Bans Sale of Geolocation Data YouTube now lets you turn off Shorts Burgers | マクドナルド公式 ChatGPT for Excel Ask HN: Who is using OpenClaw? Live Nation illegally monopolized ticketing market, jury finds Google Broke Its Promise to Me. Now ICE Has My Data. Open Source Isn't Dead. The Future of Everything is Lies, I Guess: New Jobs Unexpected €54k billing spike in 13 hours: Firebase browser key without API restrictions used for Gemini requests IPv6 – Google Your Backpack Got Worse On Purpose Good sleep, good learning, good life Fixing a 20-year-old bug in Enlightenment E16. Does Gas Town 'steal' usage from users' LLM credits & paid services to improve itself?
Kimi Linear: An Expressive, Efficient Attention Architecture
Kimi Team, Yu Zhang, Zongyu Lin, Xingcheng Yao, Jiaxi Hu, Fanqin · 2025-10-31 · via Hacker News: Best

We introduce Kimi Linear, a hybrid linear attention architecture that, for the first time, outperforms full attention under fair comparisons across various scenarios -- including short-context, long-context, and reinforcement learning (RL) scaling regimes. At its core lies Kimi Delta Attention (KDA), an expressive linear attention module that extends Gated DeltaNet with a finer-grained gating mechanism, enabling more effective use of limited finite-state RNN memory. Our bespoke chunkwise algorithm achieves high hardware efficiency through a specialized variant of the Diagonal-Plus-Low-Rank (DPLR) transition matrices, which substantially reduces computation compared to the general DPLR formulation while remaining more consistent with the classical delta rule. We pretrain a Kimi Linear model with 3B activated parameters and 48B total parameters, based on a layerwise hybrid of KDA and Multi-Head Latent Attention (MLA). Our experiments show that with an identical training recipe, Kimi Linear outperforms full MLA with a sizeable margin across all evaluated tasks, while reducing KV cache usage by up to 75% and achieving up to 6 times decoding throughput for a 1M context. These results demonstrate that Kimi Linear can be a drop-in replacement for full attention architectures with superior performance and efficiency, including tasks with longer input and output lengths. To support further research, we open-source the KDA kernel and vLLM implementations, and release the pre-trained and instruction-tuned model checkpoints.