惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

F
Full Disclosure
博客园 - 三生石上(FineUI控件)
MyScale Blog
MyScale Blog
Apple Machine Learning Research
Apple Machine Learning Research
L
LINUX DO - 最新话题
T
The Blog of Author Tim Ferriss
P
Proofpoint News Feed
宝玉的分享
宝玉的分享
小众软件
小众软件
Hugging Face - Blog
Hugging Face - Blog
GbyAI
GbyAI
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
V
Visual Studio Blog
爱范儿
爱范儿
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
博客园_首页
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
月光博客
月光博客
博客园 - 叶小钗
D
Docker
H
Hackread – Cybersecurity News, Data Breaches, AI and More
T
Tailwind CSS Blog
D
DataBreaches.Net
酷 壳 – CoolShell
酷 壳 – CoolShell
B
Blog RSS Feed
量子位
美团技术团队
Vercel News
Vercel News
Y
Y Combinator Blog
IT之家
IT之家
Martin Fowler
Martin Fowler
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
S
SegmentFault 最新的问题
腾讯CDC
Recent Announcements
Recent Announcements
Google DeepMind News
Google DeepMind News
罗磊的独立博客
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
G
Google Developers Blog
Microsoft Azure Blog
Microsoft Azure Blog
The Register - Security
The Register - Security
博客园 - 司徒正美
N
Netflix TechBlog - Medium
S
Schneier on Security
博客园 - 聂微东
U
Unit 42
D
Darknet – Hacking Tools, Hacker News & Cyber Security
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
雷峰网
雷峰网
Latest news
Latest news

Hacker News

Introducing Claude Opus 4.7 Qwen Studio The Future of Everything is Lies, I Guess: Where Do We Go From Here? GitHub - SeanFDZ/macmind: Single-layer transformer in HyperTalk for the classic Macintosh Show HN: Agent-cache – Multi-tier LLM/tool/session caching for Valkey and Redis Moving a large-scale metrics pipeline from StatsD to OpenTelemetry / Prometheus GitHub - Nightmare-Eclipse/RedSun: The Red Sun vulnerability repository GitHub - SethPyle376/hiraeth: Local AWS emulator focused on fast integration testing, with SQS support, SQLite-backed state, and a debug-friendly web UI. GitHub - macOS26/Agent: Any AI, replaces Claude Code, Cursor, OpenClaw. Over 18 LLM providers (Claude, OpenAI, Gemini, Ollama, Zai, HF, Qwen) wired into a native Mac app that writes code, builds Xcode projects, bumps versions, manages git, automates Safari, use AppleScript, JS or Accessibility, extend Agent! w/ MCP Servers, run tasks from your iPhone via Messages. YouTube now lets you turn off Shorts I Made a Terminal Pager Burgers | マクドナルド公式 Commands — HackerNews CLI documentation ChatGPT for Excel PiCore - Raspberry Pi Port of Tiny Core Linux Live Nation illegally monopolized ticketing market, jury finds Google Broke Its Promise to Me. Now ICE Has My Data. Founding Engineer at Adaptional | Y Combinator CRISPR takes important step toward silencing Down syndrome’s extra chromosome GitHub - saffron-health/libretto: The AI toolkit for building reliable browser automations US v. Heppner (S.D.N.Y. 2026) no attorney-client privilege for AI chats [pdf] Unexpected €54k billing spike in 13 hours: Firebase browser key without API restrictions used for Gemini requests Retrofitting JIT Compilers into C Interpreters IPv6 – Google The Accursèd Alphabetical Clock Cybersecurity Looks Like Proof of Work Now Fragments: April 14 Cal.com Goes Closed Source: Why AI Security Is Forcing Our Decision | Cal.com - Scheduling Software for Online Bookings Laravel raised money and now injects ads directly into your agent When moving fast, talking is the first thing to break Too much Discussion of the XOR swap trick – Heather Cafe Introduction to Spherical Harmonics for Graphics Programmers The Grand Line Building a Z-Machine in the worst possible language High-Level Rust: Getting 80% of the Benefits with 20% of the Pain GitHub - duguyue100/midnight-captain: Inspired by Midnight Commander, tailored to my taste. How to build a `git diff` driver · Jamie Tanna | Software Engineer Center for Responsible, Decentralized Intelligence at Berkeley The Local Universe’s Expansion Rate Is Clearer Than Ever, but Still Doesn’t Add Up - A new synthesis of astronomical measurements confirms a persistent mismatch that could point to physics beyond current models The air throughout our homes is infused with microplastics. But there are things you can do to breathe less of them The disturbing white paper Red Hat is trying to erase from the internet – OSnews The Future of Everything is Lies, I Guess: Annoyances ‘Abhorrent’: the inside story of the Polymarket gamblers betting millions on war Productive procrastination — Max van IJsselmuiden maps, territory and LMs 447 Terabytes per Square Centimetre at Zero Retention Energy: Non-Volatile Memory at the Atomic Scale on Fluorographane Show HN: Pardonned.com – A searchable database of US Pardons 20 Years on AWS and Never Not My Job The Seasons are Wrong Artemis II crew splashes down near San Diego after historic moon mission We gave an AI a 3 year retail lease in SF and asked it to make a profit | Andon Labs How a dancer with ALS used brainwaves to perform live On filing the corners off my MacBooks Installing every* Firefox extension OpenClaw’s memory is unreliable, and you don’t know when it will break Steve Blank Nowhere Is Safe Chimpanzees in Uganda locked in vicious 'civil war', say researchers watgo - a WebAssembly Toolkit for Go linux/Documentation/process/coding-assistants.rst at master · torvalds/linux GitHub - callumlocke/json-formatter: Makes JSON easy to read. Founding Product Engineer at Bild AI | Y Combinator A compelling title that is cryptic enough to get you to take action on it GitHub - Keychron/Keychron-Keyboards-Hardware-Design: Industrial design files for Keychron keyboards and mice. 100+ models with CAD assets in STEP, DXF, DWG, and PDF. Source-available, with commercial use allowed for original compatible accessories within the license terms. [ANNOUNCE] WireGuardNT v0.11 and WireGuard for Windows v0.6 Released 1D-Chess Helium Is Hard to Replace Cooperative Vectors Introduction | Evolve Keeping a Postgres queue healthy — PlanetScale Our response to the Axios developer tool compromise Do Americans read print books, e-books or audiobooks more? The Zettelkasten Method in Obsidian: A Practical Setup Guide Artemis II Is Competency Porn and We Are Starving For It WeakC4 Flight Viz — Cockpit View A Mexican surveillance giant you’ve never heard of is now watching the U.S. border Surelock: Deadlock-Free Mutexes for Rust RISC-V 101 – what is it and what does it mean for Canonical? | Ubuntu The Problem That Built an Industry How Much Linear Memory Access Is Enough? | Solidean Investigating Split Locks on x86-64 Simplest hash functions Sybilproof reputation mechanisms (2005) [pdf] What is a property? How Complex is my Code? Static code analysis in Kotlin — tools overview Toffoli gates are all you need PGLite evangelism dcmake: a new CMake debugger UI Clojure on Fennel part one: Persistent Data Structures Fragments: April 2 Python Release Python install manager 26.1 The Life and Death of the Book Review - Liberties Introducing Database Traffic Control — PlanetScale Bitcoin miners are losing $19,000 on every BTC produced as difficulty drops 7.8% God sleeps in the minerals Building slogbox Apple Silicon and Virtual Machines: Beating the 2 VM Limit Who was “Not Even Wrong” first? Pokemon Evolution Vs Darwinian Evolution The APL Programming Language Source Code
I Measured Claude 4.7's New Tokenizer. Here's What It Costs You.
aray07 · 2026-04-17 · via Hacker News

Anthropic's Claude Opus 4.7 migration guide says the new tokenizer uses "roughly 1.0 to 1.35x as many tokens" as 4.6. I measured 1.47x on technical docs. 1.45x on a real CLAUDE.md file. The top of Anthropic's range is where most Claude Code content actually sits, not the middle.

Same sticker price. Same quota. More tokens per prompt. Your Max window burns through faster. Your cached prefix costs more per turn. Your rate limit hits sooner.

So Anthropic must be trading this for something. What? And is it worth it?

I ran two experiments. The first measured the cost. The second measured what Anthropic claimed you'd get back. Here's where it nets out.

What does it cost?

To measure the cost, I used POST /v1/messages/count_tokens — Anthropic's free, no-inference token counter. Same content, both models, one number each per model. The difference is purely the tokenizer.

Two batches of samples.

First: seven samples of real content a Claude Code user actually sends — a CLAUDE.md file, a user prompt, a blog post, a git log, terminal output, a stack trace, a code diff.

Second: twelve synthetic samples spanning content types — English prose, code, structured data, CJK, emoji, math symbols — to see how the ratio varies by kind.

The core loop is three lines of Python:

from anthropic import Anthropic
client = Anthropic()

for model in ["claude-opus-4-6", "claude-opus-4-7"]:
    r = client.messages.count_tokens(
        model=model,
        messages=[{"role": "user", "content": sample_text}],
    )
    print(f"{model}: {r.input_tokens} tokens")

Real-world Claude Code content

Seven samples pulled from real files a Claude Code user actually sends:

Content type

chars

4.6 tokens

4.7 tokens

ratio

CLAUDE.md (real file, 5KB)

5,000

1,399

2,021

1.445

User prompt (typical Claude Code task)

4,405

1,122

1,541

1.373

Blog post excerpt (Markdown)

5,000

1,209

1,654

1.368

Git commit log

2,853

910

1,223

1.344

Terminal output (pytest run)

2,210

652

842

1.291

Python stack trace

5,255

1,736

2,170

1.250

Code diff

4,540

1,226

1,486

1.212

Weighted ratio across all seven: 1.325x (8,254 → 10,937 tokens).

Content-type baseline (12 synthetic samples)

For comparison across well-defined content types:

Content type

chars

4.6

4.7

ratio

Technical docs (English)

2,541

478

704

1.47

Shell script

2,632

1,033

1,436

1.39

TypeScript code

4,418

1,208

1,640

1.36

Spanish prose

2,529

733

986

1.35

Markdown with code blocks

2,378

604

812

1.34

Python code

3,182

864

1,112

1.29

English prose

2,202

508

611

1.20

JSON (dense)

48,067

13,939

15,706

1.13

Tool definitions (JSON Schema)

2,521

738

826

1.12

CSV (numeric)

9,546

5,044

5,414

1.07

Japanese prose

993

856

866

1.01

Chinese prose

750

779

789

1.01

English-and-code subset, weighted: 1.345x. CJK subset: 1.01x on both.

What changed in the tokenizer

Three patterns in the data:

  1. CJK, emoji, and symbol content moved 1.005–1.07x. A wholesale new vocabulary would shift these more uniformly. That didn't happen. Consistent with the non-Latin portions of the vocabulary changing less than the Latin. Token counts don't prove which specific slots were preserved.

  2. English and code moved 1.20–1.47x on natural content. Consistent with 4.7 using shorter or fewer sub-word merges for common English and code patterns than 4.6 did.

  3. Code is hit harder than unique prose (1.29–1.39x vs 1.20x). Code has more repeated high-frequency strings — keywords, imports, identifiers — exactly the patterns a Byte-Pair Encoding trained on code would collapse into long merges.

Chars-per-token on English dropped from 4.33 to 3.60. TypeScript dropped from 3.66 to 2.69. The vocabulary is representing the same text in smaller pieces.

That's a hypothesis, not a proof. Counting tokens doesn't tell you which specific entries in Anthropic's proprietary vocabulary changed.

Free Claude Code crash course

60-min video lesson + CLAUDE.md starter kit. Yours when you subscribe.

Why ship a tokenizer that uses more tokens

Anthropic's migration guide: "more literal instruction following, particularly at lower effort levels. The model will not silently generalize an instruction from one item to another."

Smaller tokens force attention over individual words. That's a documented mechanism for tighter instruction following, character-level tasks, and tool-call precision. Partner reports (Notion, Warp, Factory) describe fewer tool errors on long runs.

The tokenizer is one plausible contributor. Weights and post-training also changed. Token counts can't separate them.

Does 4.7 actually follow instructions better?

That's the cost, measured. Now the question: what did Anthropic trade for it?

Their pitch is "more literal instruction following." Plausible, but the token-count data doesn't prove it. I ran a direct test.

IFEval (Zhou et al., Google, 2023) is a benchmark of prompts with verifiable constraints. "Respond in exactly N words." "Include the word X twice." "No commas." "All uppercase." Each constraint has a Python grader. Binary pass/fail.

IFEval ships 541 prompts. I sampled 20 with a fixed seed, ran each through both models, and graded with IFEval's published checker.

The results:

Metric

4.6

4.7

Delta

Strict, prompt-level (all passed)

17/20 (85%)

18/20 (90%)

+5pp

Strict, instruction-level

25/29 (86%)

26/29 (90%)

+4pp

Loose, prompt-level

18/20 (90%)

18/20 (90%)

0

Loose, instruction-level

26/29 (90%)

26/29 (90%)

0

A small but directionally consistent improvement on strict instruction following. Loose evaluation is flat. Both models already follow the high-level instructions — the strict-mode gap comes down to 4.6 occasionally mishandling exact formatting where 4.7 doesn't.

Only one instruction type moved materially: change_case:english_capital (0/1 → 1/1). Everything else tied. The one prompt that actually separated the models was a four-constraint chain where 4.6 fumbled one and 4.7 got all four.

A few caveats worth naming:

  • N=20. IFEval has 541 prompts. A 20-prompt sample is enough to see direction, not enough to be confident about size. A +5pp delta at N=20 is consistent with anything from "no real difference" to "real +10pp improvement."

  • This measures the net effect of 4.6 → 4.7. Tokenizer, weights, and post-training all changed. I can't isolate which one drove the +5pp. The causal link between "smaller tokens" and "better instruction following" remains a hypothesis.

  • Single generation per prompt. Multiple runs per prompt would tighten the estimate.

So: 4.7 follows strict instructions a few points better than 4.6 on this subset. Small effect, small sample. Not the "dramatic improvement" framing Anthropic's partners used in launch quotes — at least not on this benchmark.

The extra tokens bought something measurable. +5pp on strict instruction-following. Small. Real. So: is that worth 1.3–1.45x more tokens per prompt? Here's the cost, session by session.

Dollar math for one Claude Code session

Imagine a long Claude Code session — 80 turns of back-and-forth on a bug fix or refactor.

The setup (what's in your context each turn):

  • Static prefix: 2K CLAUDE.md + 4K tool definitions = 6K tokens, same every turn

  • Conversation history: grows ~2K per turn (500-token user message + 1,500-token reply), reaches ~160K by turn 80

  • User input: ~500 fresh tokens per turn

  • Output: ~1,500 tokens per turn

  • Cache hit rate: ~95% (typical within the 5-minute TTL)

One thing to explain upfront: the average cached prefix across the 80 turns is ~86K tokens, not 6K. The static 6K is tiny; the average history across all turns (0 at turn 1, 160K at turn 80, average ~80K) dominates. Since most of the cache-read cost happens in late turns where the history is huge, that ~86K average is what actually gets billed per turn.

4.6 session cost

Line item

Math

Cost

Turn 1 cache-write

8K × $6.25/MTok

$0.05

Turns 2–80 cache reads

79 × 86K × $0.50/MTok

$3.40

Fresh user input

79 × 500 × $5/MTok

$0.20

Output

80 × 1,500 × $25/MTok

$3.00

Total

~$6.65

Cache reads dominate input cost. Output dominates overall.

4.7 session cost

Every token in the prefix scales by its content ratio:

  • CLAUDE.md: 1.445x → 2K becomes 2.9K

  • Tool defs: 1.12x → 4K becomes 4.5K

  • Conversation history (mostly English and code): 1.325x → 160K becomes 212K by turn 80, averaging ~106K across the session

  • User input: 1.325x → 500 becomes ~660

Average cached prefix on 4.7: ~115K tokens (up from 86K). Output tokens are a wildcard — roughly the same as 4.6, up to ~30% higher if Claude Code's new xhigh default produces more thinking tokens.

Line item

Math

Cost

Turn 1 cache-write

10K × $6.25/MTok

$0.06

Turns 2–80 cache reads

79 × 115K × $0.50/MTok

$4.54

Fresh user input

79 × 660 × $5/MTok

$0.26

Output

80 × 1,500–1,950 × $25/MTok

$3.00–$3.90

Total

~$7.86–$8.76

The delta

~$6.65 → ~$7.86–$8.76. Roughly 20–30% more per session.

The per-token price didn't change. The per-session cost did, because the same session packs more tokens.

For Max-plan users hitting rate limits instead of dollars: your 5-hour window ends sooner by roughly the same ratio on English-heavy work. A session that ran the full window on 4.6 probably doesn't on 4.7.

How this hits the prompt cache

Prompt caching is the architecture Claude Code runs on.

The 4.7 tokenizer change interacts with caching in three ways:

  1. First 4.7 session starts cold. Anthropic's prompt cache is partitioned per model — switching from 4.6 to 4.7 invalidates every cached prefix, the same way switching between Opus and Sonnet does. The tokenizer change doesn't cause this, but it makes the cold-start more expensive: the prefix you're writing to the new cache is 1.3–1.45x larger than the 4.6 equivalent.

  2. Cache volume grows by the token ratio. 1.445x more tokens in the CLAUDE.md portion means 1.445x more tokens paying cache-write once, and 1.445x more paying cache-read every turn after. The mechanism still works. There's just more of it to pay for.

  3. Same transcript, different count. Re-run a 4.6 session on 4.7 and your logs show a different number. If you baseline billing or observability off historical token counts, expect a step-change the day you flip the model ID.

Objections

"Input is mostly cache reads. The per-token cost barely changed."

Legitimate. In a session that stays within the 5-minute TTL, 96% of input is cache reads at $0.50/MTok — already 90% off nominal. A 1.325x ratio on the cached portion is a smaller dollar impact than on fresh input.

But Max plans count all tokens toward rate limits, not dollars. And several patterns hit uncached territory: first session after a TTL expiry, every cache-bust event (CLAUDE.md edits, tool-list changes, model switches), and every compaction event that rewrites the prefix. On those turns you pay the full ratio on the cache-write. The steady-state is a bright spot. The edges got noisier.

"Anthropic documented 1.0–1.35x as a range, not a hard ceiling."

Agreed. The real-world weighted ratio (1.325x) lands near the top of their range. Individual file types exceed it — CLAUDE.md at 1.445x, technical docs at 1.473x. That's the useful finding: the top of the documented range is where most Claude Code content sits, not the middle. Plan around the upper range, not the average.

So: tokens are 1.3–1.45x more expensive on English and code. Anthropic bought you +5pp on strict instruction following. The sticker price didn't change. The effective per-session cost did.

Is it worth it? That depends on what you send. You're paying ~20–30% more per session for a small but real improvement in how literally the model follows your prompt.