惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

有赞技术团队
有赞技术团队
Martin Fowler
Martin Fowler
N
Netflix TechBlog - Medium
WordPress大学
WordPress大学
罗磊的独立博客
H
Help Net Security
MongoDB | Blog
MongoDB | Blog
A
About on SuperTechFans
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
D
Docker
云风的 BLOG
云风的 BLOG
Microsoft Security Blog
Microsoft Security Blog
Blog — PlanetScale
Blog — PlanetScale
P
Proofpoint News Feed
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
I
InfoQ
J
Java Code Geeks
博客园 - 聂微东
大猫的无限游戏
大猫的无限游戏
Engineering at Meta
Engineering at Meta
美团技术团队
小众软件
小众软件
Stack Overflow Blog
Stack Overflow Blog
C
Check Point Blog

How does Prompt Caching work?

How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work? How does Prompt Caching work?
How does Prompt Caching work?
svelidanda · 2024-10-25 · via How does Prompt Caching work?

I am using gpt-4o-mini: Scenario 1: I have more than 5672 (includes system prompts, tools, user messages) tokens and upon sending same request multiple times finding 5432 tokens cached, I would expect all tokens get cached, why is it lesser number of tokens getting cached? Scenario 2: if I remove a word from token and sent the request subsequently, the cached tokens are reduced more than 1000 tokens compared to its previous request, can someone explain how exactly this prompt caching works ? ...