Prompt 缓存
高效 Prompt 缓存技术，降低成本和延迟
🌐 查看英文原文 | 源码 Notebook
使用 Claude API 进行 Prompt 缓存
Prompt 缓存允许你在 prompt 中存储和复用上下文，对于重复性任务可将延迟降低 2 倍以上，成本降低高达 90%。
有两种方式可以启用 prompt 缓存：
- 自动缓存（推荐）：在请求的顶层添加一个
cache_control字段。系统会自动为你管理缓存断点。 - 显式缓存断点：在各个内容块上设置
cache_control，以精确控制缓存的内容。
本教程演示了两种方法，从更简单的自动方法开始。
环境配置
%pip install --upgrade 'anthropic>=0.83.0' bs4 requests python-dotenv --quietNote: you may need to restart the kernel to use updated packages.import time
import anthropic
import requests
from bs4 import BeautifulSoup
from dotenv import load_dotenv
load_dotenv()
client = anthropic.Anthropic()
MODEL_NAME = "claude-sonnet-4-6"
# Unique prefix to ensure we don't hit a stale cache from a previous run
TIMESTAMP = int(time.time())让我们获取《傲慢与偏见》的全文（约 187k tokens）作为我们的大上下文。
def fetch_article_content(url):
response = requests.get(url, timeout=30)
soup = BeautifulSoup(response.content, "html.parser")
for script in soup(["script", "style"]):
script.decompose()
text = soup.get_text()
lines = (line.strip() for line in text.splitlines())
chunks = (phrase.strip() for line in lines for phrase in line.split(" "))
text = "\n".join(chunk for chunk in chunks if chunk)
return text
book_url = "https://www.gutenberg.org/cache/epub/1342/pg1342.txt"
book_content = fetch_article_content(book_url)
print(f"Fetched {len(book_content)} characters from the book.")
print("First 500 characters:")
print(book_content[:500])Fetched 737526 characters from the book.
First 500 characters:
The Project Gutenberg eBook of Pride and Prejudice
This ebook is for the use of anyone anywhere in the United States and
most other parts of the world at no cost and with almost no restrictions
whatsoever. You may copy it, give it away or re-use it under the terms
of the Project Gutenberg License included with this ebook or online
at www.gutenberg.org. If you are not located in the United States,
you will have to check the laws of the country where you are located
before using this eBook.
Title:我们还将定义一个小辅助函数来打印使用统计：
def print_usage(response, elapsed):
"""Print token usage and timing for an API response."""
usage = response.usage
cache_create = getattr(usage, "cache_creation_input_tokens", 0)
cache_read = getattr(usage, "cache_read_input_tokens", 0)
print(f" Time: {elapsed:.2f}s")
print(f" Input tokens: {usage.input_tokens}")
print(f" Output tokens: {usage.output_tokens}")
if cache_create:
print(f" Cache write tokens: {cache_create}")
if cache_read:
print(f" Cache read tokens: {cache_read}")示例 1：自动缓存（单轮）
自动缓存是最简单的入门方式。在 messages.create() 调用的顶层添加 cache_control={"type": "ephemeral"}，系统会处理其余工作 — 自动将缓存断点放在最后一个可缓存块上。
我们将比较三种场景： 1. 不使用缓存——基线 2. 首次缓存调用——创建缓存条目（耗时与基线类似） 3. 第二次缓存调用——从缓存读取（大幅提速）
基线：不使用缓存
start = time.time()
baseline_response = client.messages.create(
model=MODEL_NAME,
max_tokens=300,
messages=[
{
"role": "user",
"content": str(TIMESTAMP)
+ "<book>"
+ book_content
+ "</book>"
+ "\n\nWhat is the title of this book? Only output the title.",
}
],
)
baseline_time = time.time() - start
print(f"Response: {baseline_response.content[0].text}")
print_usage(baseline_response, baseline_time)Response: Pride and Prejudice
Time: 4.89s
Input tokens: 187364
Output tokens: 8首次使用自动缓存的调用（缓存写入）
唯一的改变是顶层的 cache_control 参数。首次调用会写入缓存，因此耗时与基线类似。
start = time.time()
write_response = client.messages.create(
model=MODEL_NAME,
max_tokens=300,
cache_control={"type": "ephemeral"}, # <-- one-line change
messages=[
{
"role": "user",
"content": str(TIMESTAMP)
+ "<book>"
+ book_content
+ "</book>"
+ "\n\nWhat is the title of this book? Only output the title.",
}
],
)
write_time = time.time() - start
print(f"Response: {write_response.content[0].text}")
print_usage(write_response, write_time)Response: Pride and Prejudice
Time: 4.28s
Input tokens: 3
Output tokens: 8
Cache write tokens: 187361第二次使用自动缓存的调用（缓存命中）
再次发送相同的请求。这次会复用缓存的前缀，因此你应该能看到显著的提速。
start = time.time()
hit_response = client.messages.create(
model=MODEL_NAME,
max_tokens=300,
cache_control={"type": "ephemeral"},
messages=[
{
"role": "user",
"content": str(TIMESTAMP)
+ "<book>"
+ book_content
+ "</book>"
+ "\n\nWhat is the title of this book? Only output the title.",
}
],
)
hit_time = time.time() - start
print(f"Response: {hit_response.content[0].text}")
print_usage(hit_response, hit_time)
print("\n" + "=" * 50)
print("COMPARISON")
print("=" * 50)
print(f"No caching: {baseline_time:.2f}s")
print(f"Cache write: {write_time:.2f}s")
print(f"Cache hit: {hit_time:.2f}s")
print(f"Speedup: {baseline_time / hit_time:.1f}x")Response: Pride and Prejudice
Time: 1.48s
Input tokens: 3
Output tokens: 8
Cache read tokens: 187361
==================================================
COMPARISON
==================================================
No caching: 4.89s
Cache write: 4.28s
Cache hit: 1.48s
Speedup: 3.3x示例 2：多轮对话中的自动缓存
自动缓存在多轮对话中真正大放异彩。随着对话的增长，缓存断点会自动向前移动 — 你不需要自己管理任何标记。
| 请求 | 缓存行为 |
|---|---|
| Request 1 | System + User:A cached (write) |
| Request 2 | System + User:A read from cache; Asst:B + User:C written to cache |
| Request 3 | System through User:C read from cache; Asst:D + User:E written to cache |
system_message = f"{TIMESTAMP} <file_contents> {book_content} </file_contents>"
questions = [
"What is the title of this novel?",
"Who are Mr. and Mrs. Bennet?",
"What is Netherfield Park?",
"What is the main theme of this novel?",
]
conversation = []
for i, question in enumerate(questions, 1):
print(f"\n{'=' * 50}")
print(f"Turn {i}: {question}")
print("=" * 50)
conversation.append({"role": "user", "content": question})
start = time.time()
response = client.messages.create(
model=MODEL_NAME,
max_tokens=300,
cache_control={"type": "ephemeral"}, # automatic caching
system=system_message,
messages=conversation,
)
elapsed = time.time() - start
assistant_reply = response.content[0].text
conversation.append({"role": "assistant", "content": assistant_reply})
print(f"\nAssistant: {assistant_reply[:200]}{'...' if len(assistant_reply) > 200 else ''}")
print()
print_usage(response, elapsed)
==================================================
Turn 1: What is the title of this novel?
==================================================
Assistant: The title of this novel is **Pride and Prejudice**, written by **Jane Austen**.
Time: 5.19s
Input tokens: 3
Output tokens: 24
Cache write tokens: 187361
==================================================
Turn 2: Who are Mr. and Mrs. Bennet?
==================================================
Assistant: Mr. and Mrs. Bennet are a married couple who are central characters in the novel. They live at **Longbourn** and are the parents of **five daughters**: Jane, Elizabeth, Mary, Catherine (Kitty), and Ly...
Time: 8.27s
Input tokens: 3
Output tokens: 272
Cache write tokens: 38
Cache read tokens: 187361
==================================================
Turn 3: What is Netherfield Park?
==================================================
Assistant: **Netherfield Park** is a large estate located near the village of **Longbourn** in Hertfordshire, where the Bennet family lives. It plays an important role in the novel as it is the home that is let ...
Time: 8.74s
Input tokens: 3
Output tokens: 300
Cache write tokens: 283
Cache read tokens: 187399
==================================================
Turn 4: What is the main theme of this novel?
==================================================
Assistant: **Pride and Prejudice** explores several important themes throughout the novel. Here are the main ones:
**1. Pride and Prejudice**
- The most obvious theme is reflected in the title itself. Mr. Darcy...
Time: 7.06s
Input tokens: 3
Output tokens: 300
Cache write tokens: 315
Cache read tokens: 187682在第一轮之后，后续每一轮几乎 100% 的输入 token 都从缓存中读取。对话代码只是一个普通的消息列表 — 各个块上不需要特殊的 cache_control 标记。
示例 3：显式缓存断点
如需更多控制，你可以直接在各个内容块上设置 cache_control。这在以下情况下很有用：
- 你想以不同的 TTL 缓存不同的部分
- 你需要独立于消息内容缓存 system prompt
- 你想精细控制缓存的内容
你还可以组合使用两种方法：为 system prompt 使用显式断点，同时让自动缓存处理对话。
下面，我们直接在书籍内容块上设置 cache_control，并在每一轮手动将断点前移。
class ConversationWithExplicitCaching:
"""Multi-turn conversation that manually places cache_control on the last user message."""
def __init__(self):
self.turns = []
def add_user(self, content):
self.turns.append({"role": "user", "content": [{"type": "text", "text": content}]})
def add_assistant(self, content):
self.turns.append({"role": "assistant", "content": [{"type": "text", "text": content}]})
def get_messages(self):
"""Return messages with cache_control on the last user message."""
result = []
last_user_idx = max(i for i, t in enumerate(self.turns) if t["role"] == "user")
for i, turn in enumerate(self.turns):
if i == last_user_idx:
result.append(
{
"role": "user",
"content": [
{
"type": "text",
"text": turn["content"][0]["text"],
"cache_control": {"type": "ephemeral"},
}
],
}
)
else:
result.append(turn)
return result
conv = ConversationWithExplicitCaching()
for i, question in enumerate(questions, 1):
print(f"\n{'=' * 50}")
print(f"Turn {i}: {question}")
print("=" * 50)
conv.add_user(question)
start = time.time()
response = client.messages.create(
model=MODEL_NAME,
max_tokens=300,
system=[
{
"type": "text",
"text": system_message,
"cache_control": {"type": "ephemeral"}, # explicit breakpoint on system
},
],
messages=conv.get_messages(),
)
elapsed = time.time() - start
assistant_reply = response.content[0].text
conv.add_assistant(assistant_reply)
print(f"\nAssistant: {assistant_reply[:200]}{'...' if len(assistant_reply) > 200 else ''}")
print()
print_usage(response, elapsed)
==================================================
Turn 1: What is the title of this novel?
==================================================
Assistant: The title of this novel is **Pride and Prejudice**, written by **Jane Austen**.
Time: 4.53s
Input tokens: 3
Output tokens: 24
Cache read tokens: 187361
==================================================
Turn 2: Who are Mr. and Mrs. Bennet?
==================================================
Assistant: Mr. and Mrs. Bennet are a married couple who are central characters in the novel. They live at **Longbourn** and are the parents of **five daughters**: Jane, Elizabeth (Lizzy), Mary, Catherine (Kitty)...
Time: 7.57s
Input tokens: 3
Output tokens: 283
Cache read tokens: 187399
==================================================
Turn 3: What is Netherfield Park?
==================================================
Assistant: **Netherfield Park** is a large estate located near the village of **Longbourn** in Hertfordshire, where the Bennet family lives. It plays an important role in the novel as it is the residence that se...
Time: 6.85s
Input tokens: 3
Output tokens: 300
Cache write tokens: 294
Cache read tokens: 187399
==================================================
Turn 4: What is the main theme of this novel?
==================================================
Assistant: **Pride and Prejudice** explores several interconnected themes throughout the novel. Here are the main ones:
**1. Pride and Prejudice**
The most central theme is reflected in the title itself. Both M...
Time: 7.00s
Input tokens: 3
Output tokens: 300
Cache write tokens: 315
Cache read tokens: 187693选择方案
| 自动缓存 | 显式断点 | |
|---|---|---|
| 易用性 | 一行代码即可 | 需手动放置和移动 cache_control 标记 |
| 多轮对话 | 断点自动前移 | 你自己管理断点位置 |
| 精细控制 | 否 | 最多 4 个独立断点 |
| 混合 TTL | 自动断点使用单一 TTL | 每个断点可使用不同 TTL |
| 可组合 | 是 — 自动 + 显式可同时使用 | 是 |
从自动缓存开始。 它以最小的努力覆盖了大多数用例。仅在你需要精细控制时才切换到显式断点。
关键细节
- 最小可缓存长度： Sonnet 为 1,024 tokens；Opus 和 Haiku 4.5 为 4,096 tokens
- 缓存 TTL： 默认 5 分钟（每次命中时刷新）。也提供 1 小时 TTL，价格为基础输入价格的 2 倍。
- 定价： 缓存写入费用为基础输入价格的 1.25 倍。缓存读取费用为基础输入价格的 0.1 倍。
- 断点限制： 每次请求最多 4 个显式断点。自动缓存使用一个槽位。
完整详情请参阅 prompt 缓存文档。
Tip
中文读者提示：本章节的代码和输出为英文原文。如需理解具体实现细节，可参考上方中文导读和代码注释。如有疑问，欢迎在 GitHub Issues 讨论。