解读图表与 PPT
让 Claude 读取和分析图表、图示及 PowerPoint
🌐 查看英文原文 | 源码 Notebook
处理图表、图形和幻灯片
Claude 在处理图表、图形和更广泛的幻灯片方面具有很强的能力。根据你的使用场景，有许多技巧和窍门可以利用。本教程将展示使用 Claude 处理这些材料的常见模式。
图表和图形
在大多数情况下，使用 Claude 处理图表和图形非常简单。让我们逐步了解如何摄取它们并将其传递给 Claude，以及一些改善结果的常见技巧。
摄取和调用 Claude API
将图表和图形传递给 Claude 的最佳方式是利用其视觉功能和 PDF 支持功能。也就是说，给 Claude 一个包含图表或图形的 PDF 文档，以及关于它的文本问题。
目前，只有 claude-sonnet-4-6 支持 PDF 功能。由于该功能仍处于测试阶段，你需要提供 pdfs-2024-09-25 beta 头部。
# Install and create the Anthropic client.
%pip install anthropicimport base64
from anthropic import Anthropic
# While PDF support is in beta, you must pass in the correct beta header
client = Anthropic(default_headers={"anthropic-beta": "pdfs-2024-09-25"})
# For now, only claude-sonnet-4-6 supports PDFs
MODEL_NAME = "claude-sonnet-4-6"# Make a useful helper function.
def get_completion(messages):
response = client.messages.create(
model=MODEL_NAME, max_tokens=8192, temperature=0, messages=messages
)
return response.content[0].text# To start, we'll need a PDF. We will be using the .pdf document located at cvna_2021_annual_report.pdf.
# Start by reading in the PDF and encoding it as base64.
with open("./documents/cvna_2021_annual_report.pdf", "rb") as pdf_file:
binary_data = pdf_file.read()
base_64_encoded_data = base64.b64encode(binary_data)
base64_string = base_64_encoded_data.decode("utf-8")让我们看看如何将此文档连同一个简单的问题一起传递给模型。
messages = [
{
"role": "user",
"content": [
{
"type": "document",
"source": {
"type": "base64",
"media_type": "application/pdf",
"data": base64_string,
},
},
{"type": "text", "text": "What's in this document? Answer in a single sentence."},
],
}
]
print(get_completion(messages))This is a page from Carvana's 2021 Annual Report showing four key metrics: retail units sold, total revenue, total markets at year end, and car vending machines, all displaying significant growth from 2014 to 2021.效果不错！现在让我们问一些更有用的问题。
questions = [
"What was CVNA revenue in 2020?",
"How many additional markets has Carvana added since 2014?",
"What was 2016 revenue per retail unit sold?",
]
for index, question in enumerate(questions):
messages = [
{
"role": "user",
"content": [
{
"type": "document",
"source": {
"type": "base64",
"media_type": "application/pdf",
"data": base64_string,
},
},
{"type": "text", "text": question},
],
}
]
print(f"\n----------Question {index + 1}----------")
print(get_completion(messages))
----------Question 1----------
According to the graph showing Total Revenue ($M), Carvana's revenue in 2020 was $5,587 million (or approximately $5.59 billion).
----------Question 2----------
According to the "TOTAL MARKETS AT YEAR END" graph, Carvana started with 4 markets in 2014 and grew to 311 markets by 2021. Therefore, Carvana added 307 markets since 2014 (311 - 4 = 307 additional markets).
----------Question 3----------
Let me calculate this for you:
2016 Revenue: $365 million
2016 Retail Units Sold: 18,761 units
$365 million ÷ 18,761 units = $19,455 per unit (rounded to nearest dollar)
So in 2016, Carvana's revenue per retail unit sold was approximately $19,455.如你所见，Claude 能够回答关于图表和图形的相当详细的问题。不过，有一些技巧和窍门可以帮助你充分利用它： - 有时 Claude 的算术能力会成为障碍。你会注意到，如果你对上面第三个问题进行多次测试，它偶尔会输出不正确的最终答案，因为它在算术上出了错。考虑为 Claude 提供一个计算器工具，以确保它不会犯这类错误。 - 对于非常复杂的图表和图形，我们可以要求 Claude “首先描述你在文档中看到的每个数据点”，以此来获得类似传统思维链的改进效果。 - Claude 偶尔会在处理依赖大量颜色来传达信息的图表时遇到困难，例如包含许多分组的分组柱状图。要求 Claude 首先使用 HEX 代码识别图形中的颜色可以提高其准确性。
幻灯片
既然我们知道 Claude 是图表和图形的高手，那么将其扩展到图表和图形的真正归宿——幻灯片——就是顺理成章的了！
幻灯片是许多领域（包括金融服务）的关键信息来源。虽然你可以使用 PyPDF 等包从幻灯片中提取文本，但它们以图表/图形为主的特性往往使这成为一个糟糕的选择，因为模型将难以获取它们实际需要的信息。
因此，PDF 支持功能可以作为一个很好的替代方案。它在处理 PDF 文档时同时使用提取的文本和视觉功能。在本节中，我们将介绍如何在 Claude 中使用 PDF 文档来审阅幻灯片，以及如何处理此方法的一些常见陷阱。
将典型幻灯片导入 Claude 的最佳方式是将其下载为 PDF 并直接提供给 Claude。
# Open the multi-page PDF document the same way we did earlier.
with open("./documents/twilio_q4_2023.pdf", "rb") as pdf_file:
binary_data = pdf_file.read()
base_64_encoded_data = base64.b64encode(binary_data)
base64_string = base_64_encoded_data.decode("utf-8")# Now let's pass the document directly to Claude. Note that Claude will process both the text and visual elements of the document.
question = "What was Twilio y/y revenue growth for fiscal year 2023?"
content = [
{
"type": "document",
"source": {"type": "base64", "media_type": "application/pdf", "data": base64_string},
},
{"type": "text", "text": question},
]
messages = [{"role": "user", "content": content}]
print(get_completion(messages))According to the financial results shown in the presentation, Twilio's year-over-year revenue growth for fiscal year 2023 was 9%. This can be found in the "Total Company Results Highlights" section, which shows FY 2023 revenue growth of 9%.这种方法是入门的好方法，对于某些用例提供了最佳性能。不过也存在一些限制： - 一次请求中所有提供的文档总页数不能超过 100 页（我们计划随时间推移增加此限制）。 - 如果你在 RAG 中使用幻灯片内容，将多模态 PDF 引入你的 embedding 可能会导致问题
幸运的是，我们可以利用 Claude 的视觉能力，获得比普通 PDF 文本提取更高质量的文本形式幻灯片表示。
我们发现最好的方法是要求 Claude 从头到尾依次叙述幻灯片，将当前幻灯片及其之前的叙述一起传递给它。让我们看看如何操作。
# Define a prompt for narrating our slide deck. We would adjut this prompt based on the nature of the deck, but keep the structure largely the same.
prompt = """
You are the Twilio CFO, narrating your Q4 2023 earnings presentation.
The entire earnings presentation document is provided to you.
Please narrate this presentation from Twilio's Q4 2023 Earnings as if you were the presenter. Do not talk about any things, especially acronyms, if you are not exactly sure you know what they mean.
Do not leave any details un-narrated as some of your viewers are vision-impaired, so if you don't narrate every number they won't know the number.
Structure your response like this:
<narration>
<page_narration id=1>
[Your narration for page 1]
</page_narration>
<page_narration id=2>
[Your narration for page 2]
</page_narration>
... and so on for each page
</narration>
Use excruciating detail for each page, ensuring you describe every visual element and number present. Show the full response in a single message.
"""
messages = [
{
"role": "user",
"content": [
{
"type": "document",
"source": {
"type": "base64",
"media_type": "application/pdf",
"data": base64_string,
},
},
{"type": "text", "text": prompt},
],
}
]
# Now we use our prompt to narrate the entire deck. Note that this may take a few minutes to run (often up to 10).
completion = get_completion(messages)import re
# Next we'll parse the response from Claude using regex
pattern = r"<narration>(.*?)</narration>"
match = re.search(pattern, completion.strip(), re.DOTALL)
if match:
narration = match.group(1)
else:
raise ValueError("No narration available. Likely due to the model response being truncated.")现在我们有了一个基于文本的叙述（虽然远非完美但相当不错），我们可以在任何纯文本工作流中使用这份幻灯片了。包括向量搜索！
作为最后的健全性检查，让我们向仅基于叙述的设置提几个问题！
questions = [
"What percentage of q4 total revenue was the Segment business line?",
"Has the rate of growth of quarterly revenue been increasing or decreasing? Give just an answer.",
"What was acquisition revenue for the year ended december 31, 2023 (including negative revenues)?",
]
for index, question in enumerate(questions):
prompt = f"""You are an expert financial analyst analyzing a transcript of Twilio's earnings call.
Here is the transcript:
<transcript>
{narration}
</transcript>
Please answer the following question:
<question>
{question}
</question>"""
messages = [{"role": "user", "content": [{"type": "text", "text": prompt}]}]
print(f"\n----------Question {index + 1}----------")
print(get_completion(messages))
----------Question 1----------
Let me calculate this:
Segment revenue in Q4 2023: $75 million
Total revenue in Q4 2023: $1,076 million
$75M ÷ $1,076M = 0.0697 or approximately 7%
Therefore, the Segment business line represented approximately 7% of Twilio's total Q4 2023 revenue.
----------Question 2----------
Decreasing. The transcript shows Q4 2023 revenue growth was 5% year-over-year, while for the full year 2023 revenue growth was 9% year-over-year, indicating a slowing growth rate. Additionally, the Q1 2024 guidance projects even lower growth of 2-3% year-over-year, confirming the declining trend.
----------Question 3----------
Let me help calculate the acquisition revenue for 2023.
From the transcript, we can see:
- Total revenue for 2023: $4,154 million
- Organic revenue for 2023: $4,146 million
Therefore, acquisition revenue would be:
Total Revenue - Organic Revenue = $4,154M - $4,146M = $8 million
So the acquisition revenue for the year ended December 31, 2023 was $8 million.
This can be verified by the fact that the difference between total revenue growth (9%) and organic revenue growth (10%) also suggests a small contribution from acquisitions.看起来不错！掌握了这些技术，你就可以开始将模型应用于图表和图形密集的内容（如幻灯片）了。
Tip
中文读者提示：本章节的代码和输出为英文原文。如需理解具体实现细节，可参考上方中文导读和代码注释。如有疑问，欢迎在 GitHub Issues 讨论。