惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

F
Fortinet All Blogs
MyScale Blog
MyScale Blog
Microsoft Security Blog
Microsoft Security Blog
量子位
B
Blog
aimingoo的专栏
aimingoo的专栏
Apple Machine Learning Research
Apple Machine Learning Research
阮一峰的网络日志
阮一峰的网络日志
The GitHub Blog
The GitHub Blog
T
The Exploit Database - CXSecurity.com
N
News | PayPal Newsroom
Cloudbric
Cloudbric
A
About on SuperTechFans
AI
AI
Hacker News: Ask HN
Hacker News: Ask HN
S
Schneier on Security
Recent Commits to openclaw:main
Recent Commits to openclaw:main
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
C
Cyber Attacks, Cyber Crime and Cyber Security
L
LINUX DO - 最新话题
T
The Blog of Author Tim Ferriss
Simon Willison's Weblog
Simon Willison's Weblog
有赞技术团队
有赞技术团队
H
Heimdal Security Blog
J
Java Code Geeks
大猫的无限游戏
大猫的无限游戏
D
Docker
Security Archives - TechRepublic
Security Archives - TechRepublic
N
News and Events Feed by Topic
IT之家
IT之家
Know Your Adversary
Know Your Adversary
N
Netflix TechBlog - Medium
T
Tailwind CSS Blog
B
Blog RSS Feed
C
Cybersecurity and Infrastructure Security Agency CISA
C
Cisco Blogs
博客园 - 叶小钗
美团技术团队
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
H
Hackread – Cybersecurity News, Data Breaches, AI and More
L
LangChain Blog
The Hacker News
The Hacker News
Y
Y Combinator Blog
I
Intezer
The Register - Security
The Register - Security
F
Full Disclosure
V
V2EX
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Last Week in AI
Last Week in AI
Martin Fowler
Martin Fowler

Hacker News: Best

madhadron - The seven programming ur-languages GitHub - smol-machines/smolvm: Tool to build & run portable, lightweight, self-contained virtual machines. I Measured Claude 4.7's New Tokenizer. Here's What It Costs You. Introducing Claude Design by Anthropic Labs It Is Time to Ban the Sale of Precise Geolocation The creative software industry has declared war on Adobe Isaac Asimov: The Last Question Newly unsealed records reveal Amazon’s price-fixing tactics, California attorney general claims Clojure - Documentary Android CLI and skills: Build Android apps 3x faster using any agent Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7 Codex for almost everything Introducing Claude Opus 4.7 Qwen Studio The Future of Everything is Lies, I Guess: Where Do We Go From Here? Virginia Bans Sale of Geolocation Data YouTube now lets you turn off Shorts Burgers | マクドナルド公式 ChatGPT for Excel Ask HN: Who is using OpenClaw? Live Nation illegally monopolized ticketing market, jury finds Google Broke Its Promise to Me. Now ICE Has My Data. Open Source Isn't Dead. The Future of Everything is Lies, I Guess: New Jobs Unexpected €54k billing spike in 13 hours: Firebase browser key without API restrictions used for Gemini requests IPv6 – Google Your Backpack Got Worse On Purpose Good sleep, good learning, good life Fixing a 20-year-old bug in Enlightenment E16. Does Gas Town 'steal' usage from users' LLM credits & paid services to improve itself? Tell HN: Fiverr left customer files public and searchable Cybersecurity Looks Like Proof of Work Now Getting the Flock out Release OpenSSL 4.0.0 · openssl/openssl Internet será irrespirable los días de fútbol y otros deportes. Telefónica extiende los bloqueos a Champions, tenis y golf. Automate work with routines - Claude Code Docs The Future of Everything is Lies, I Guess: Work Thousands of rare concert recordings are landing on the Internet Archive — listen now What is jj and why should I care? Backblaze has quietly stopped backing up your data Cal.com Goes Closed Source: Why AI Security Is Forcing Our Decision | Cal.com - Scheduling Software for Online Bookings Codex Hacked a Samsung TV The Future of Everything is Lies, I Guess: Safety GitHub - sterlingcrispin/nothing-ever-happens: Polymarket bot that buys "No" on all non-sports markets. For entertainment only, mostly a meme. Make tmux Pretty and Usable - Ham Vocke Microsoft isn't removing Copilot from Windows 11, it's just renaming it Servo is now available on crates.io - Servo aims to empower developers with a lightweight, high-performance alternative for embedding web technologies in applications. We May Be Living Through the Most Consequential Hundred Days in Cyber History, and Almost Nobody Has Noticed All elementary functions from a single binary operator 奈拜提耶市 Seven countries now generate 100% of their electricity from renewable energy Pro Max 5x Quota Exhausted in 1.5 Hours Despite Moderate Usage Tell HN: docker pull fails in spain due to football cloudflare block Bring Back Idiomatic Design @adlrocha - How the "AI Loser" may end up winning Apple update turns Czech mate for locked-out iPhone user Cache TTL silently regressed from 1h to 5m around early March 2026, causing quota and cost inflation The peril of laziness lost AI Will Be Met With Violence, and Nothing Good Will Come of It Center for Responsible, Decentralized Intelligence at Berkeley The disturbing white paper Red Hat is trying to erase from the internet – OSnews The Future of Everything is Lies, I Guess: Annoyances 447 Terabytes per Square Centimetre at Zero Retention Energy: Non-Volatile Memory at the Atomic Scale on Fluorographane Show HN: Pardonned.com – A searchable database of US Pardons 20 Years on AWS and Never Not My Job Artemis II crew splashes down near San Diego after historic moon mission Molotov Cocktail Is Hurled at Home of Sam Altman, OpenAI’s CEO France to ditch Windows for Linux to reduce reliance on US tech On filing the corners off my MacBooks Installing every* Firefox extension Chimpanzees in Uganda locked in vicious 'civil war', say researchers linux/Documentation/process/coding-assistants.rst at master · torvalds/linux GitHub - callumlocke/json-formatter: Makes JSON easy to read. A compelling title that is cryptic enough to get you to take action on it GitHub - Keychron/Keychron-Keyboards-Hardware-Design: Industrial design files for Keychron keyboards and mice. 100+ models with CAD assets in STEP, DXF, DWG, and PDF. Source-available, with commercial use allowed for original compatible accessories within the license terms. [ANNOUNCE] WireGuardNT v0.11 and WireGuard for Windows v0.6 Released 1D-Chess Helium Is Hard to Replace FBI used iPhone notification data to retrieve deleted Signal messages Microsoft suspends dev accounts for high-profile open source projects Why you can’t trust Privacy & Security Serenity Forge (@serenityforge.com) A new trick brings stability to quantum operations OpenAI Backs Bill That Would Limit Liability for AI-Enabled Mass Deaths or Financial Disasters Netflix Prices Went Up Again – I Bought a DVD Player Instead DOJ Wants to Scrap Watergate-Era Rule That Makes Presidential Records Public EFF is Leaving X How NASA built Artemis II’s fault-tolerant computer Meta removes ads for social media addiction litigation How Pizza Tycoon simulated traffic on a 25 MHz CPU Claude mixes up who said what, and that's not OK Reallocating $100/Month Claude Code spend to Zed and OpenRouter Help Keep Thunderbird Alive! Why Are Flock Employees Watching Our Children? The Pentagon Threatened Pope Leo XIV’s Ambassador With the Avignon Papacy Fragments: April 2 Native Instant Space Switching on MacOS Bitcoin miners are losing $19,000 on every BTC produced as difficulty drops 7.8% God sleeps in the minerals Apple Silicon and Virtual Machines: Beating the 2 VM Limit
Do Transformers Need Three Projections? Systematic Study of QKV Variants
Ali Kayyam, Anusha Madan Gopal, M Anthony Lewis · 2026-06-02 · via Hacker News: Best

Transformers have become the standard solution for various AI tasks, with the query, key, and value (QKV) attention formulation playing a central role. However, the individual contribution of these three projections and the impact of omitting some remain poorly understood. We systematically evaluate three projection sharing constraints: a) Q-K=V (shared key-value), b) Q=K-V (shared query-key), and c) Q=K=V (single projection). The last two variants produce symmetric attention maps; to address this, we also explore asymmetric attention via 2D positional encodings. Through experiments spanning synthetic tasks, vision (MNIST, CIFAR, TinyImageNet, anomaly), and language modeling (300M and 1.2B parameter models on 10B tokens), we discovered that our transformers perform on par or occasionally better than the QKV transformer. In language modeling, Q-K=V projection sharing achieves 50% KV cache reduction with only 3.1% perplexity degradation. Crucially, projection sharing is complementary to head sharing (GQA/MQA): combining Q-K=V with GQA-4 yields 87.5% cache reduction, while Q-K=V + MQA achieves 96.9%, enabling practical on-device inference. We show that Q-K=V preserves quality because keys and values can occupy similar representational spaces and attention operates in a low-rank regime, whereas Q=K-V breaks attention directionality. Our results systematically characterize projection sharing as an underexplored instance of weight tying in attention, with direct, quantifiable inference memory benefits, particularly valuable for edge deployment. The code is publicly available at https://github.com/Brainchip-Inc/Do-Transformers-Need-3-Projections