惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

酷 壳 – CoolShell
酷 壳 – CoolShell
Microsoft Security Blog
Microsoft Security Blog
Recent Announcements
Recent Announcements
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Last Week in AI
Last Week in AI
罗磊的独立博客
腾讯CDC
云风的 BLOG
云风的 BLOG
月光博客
月光博客
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
博客园 - 三生石上(FineUI控件)
宝玉的分享
宝玉的分享
U
Unit 42
I
InfoQ
D
DataBreaches.Net
Blog — PlanetScale
Blog — PlanetScale
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
V
V2EX
美团技术团队
IT之家
IT之家
Stack Overflow Blog
Stack Overflow Blog
F
Fortinet All Blogs
GbyAI
GbyAI
S
SegmentFault 最新的问题

How does Prompt Caching work?

How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work?
How does Prompt Caching work?
svelidanda · 2024-10-25 · via How does Prompt Caching work?

I am using gpt-4o-mini: Scenario 1: I have more than 5672 (includes system prompts, tools, user messages) tokens and upon sending same request multiple times finding 5432 tokens cached, I would expect all tokens get cached, why is it lesser number of tokens getting cached? Scenario 2: if I remove a word from token and sent the request subsequently, the cached tokens are reduced more than 1000 tokens compared to its previous request, can someone explain how exactly this prompt caching works ? ...