维基百科搜索
将维基百科作为 Claude 的知识源
🌐 查看英文原文 | 源码 Notebook
使用 Claude 迭代搜索维基百科
[声明：此 notebook 是使用 Claude 2 模型创建的，被视为旧版内容。]
有些问题 Claude 无法仅凭记忆回答。也许它们是关于时事的。也许你有一个非常详细的问题，Claude 并没有记住答案。不用担心！通过一些 prompt 工程和架构设计，Claude 可以搜索网络来找到答案。在本教程中，我们将创建一个虚拟研究助手，它能够搜索维基百科来找到你问题的答案。同样的方法也可以用来让 Claude 搜索更广泛的网络，或你提供的一组文档。
这是什么方法？从广义上讲，它属于”工具使用”的范畴。我们创建一个搜索工具，告诉 Claude 它的存在，然后让它去工作。用伪代码来说：
- 用搜索工具的描述、最佳使用方式以及如何”调用”它（通过发出一个特殊字符串）来 prompt Claude。
- 告诉 Claude 你的问题。
- Claude 像正常一样生成 token。如果它生成了特殊字符串，则终止 token 生成流，并向搜索 API 发出查询。
- 构建一个新的 prompt，由第 1 步的 prompt、Claude 在搜索调用字符串之前生成的所有内容，以及 API 调用的结果组成。
- 重复直到 Claude 认为完成了。
让我们深入了解工具使用和检索的 prompt。
Prompt
# Tool Description Prompt
wikipedia_prompt = """You will be asked a question by a human user. You have access to the following tool to help answer the question. <tool_description> Search Engine Tool * The search engine will exclusively search over Wikipedia for pages similar to your query. It returns for each page its title and full page content. Use this tool if you want to get up-to-date and comprehensive information on a topic to help answer queries. Queries should be as atomic as possible -- they only need to address one part of the user's question. For example, if the user's query is "what is the color of a basketball?", your search query should be "basketball". Here's another example: if the user's question is "Who created the first neural network?", your first query should be "neural network". As you can see, these queries are quite short. Think keywords, not phrases. * At any time, you can make a call to the search engine using the following syntax: <search_query>query_word</search_query>. * You'll then get results back in <search_result> tags.</tool_description>"""
print(wikipedia_prompt)You will be asked a question by a human user. You have access to the following tool to help answer the question. <tool_description> Search Engine Tool * The search engine will exclusively search over Wikipedia for pages similar to your query. It returns for each page its title and full page content. Use this tool if you want to get up-to-date and comprehensive information on a topic to help answer queries. Queries should be as atomic as possible -- they only need to address one part of the user's question. For example, if the user's query is "what is the color of a basketball?", your search query should be "basketball". Here's another example: if the user's question is "Who created the first neural network?", your first query should be "neural network". As you can see, these queries are quite short. Think keywords, not phrases. * At any time, you can make a call to the search engine using the following syntax: <search_query>query_word</search_query>. * You'll then get results back in <search_result> tags.</tool_description>请注意，这个 prompt 中包含了很多关于如何正确搜索维基百科的建议。我们都习惯了在 Google 中随便输入一些内容就能得到不错的结果，因为查询解析逻辑非常强大。但维基百科搜索不是这样的。举个例子：考虑查询”What’s the best way to purchase potatoes in the United Arab Emirates”。这个查询在维基百科上的热门结果是美国奴隶制、1973 年石油危机、Wendy’s 和 Tim Horton’s（？？）。而 Google 则正确地将你直接带到 Carrefour UAE。
另一个区别是维基百科搜索返回的是整个页面。使用向量搜索时，你可能会得到更窄的片段，因此你可能需要请求更多结果、使用更具体的查询，或两者兼而有之。总体要点是，你的结果可能因这里的选择而有很大差异，所以要注意！
retrieval_prompt = """Before beginning to research the user's question, first think for a moment inside <scratchpad> tags about what information is necessary for a well-informed answer. If the user's question is complex, you may need to decompose the query into multiple subqueries and execute them individually. Sometimes the search engine will return empty search results, or the search results may not contain the information you need. In such cases, feel free to try again with a different query.
After each call to the Search Engine Tool, reflect briefly inside <search_quality></search_quality> tags about whether you now have enough information to answer, or whether more information is needed. If you have all the relevant information, write it in <information></information> tags, WITHOUT actually answering the question. Otherwise, issue a new search.
Here is the user's question: <question>{query}</question> Remind yourself to make short queries in your scratchpad as you plan out your strategy."""
print(retrieval_prompt)Before beginning to research the user's question, first think for a moment inside <scratchpad> tags about what information is necessary for a well-informed answer. If the user's question is complex, you may need to decompose the query into multiple subqueries and execute them individually. Sometimes the search engine will return empty search results, or the search results may not contain the information you need. In such cases, feel free to try again with a different query.
After each call to the Search Engine Tool, reflect briefly inside <search_quality></search_quality> tags about whether you now have enough information to answer, or whether more information is needed. If you have all the relevant information, write it in <information></information> tags, WITHOUT actually answering the question. Otherwise, issue a new search.
Here is the user's question: <question>{query}</question> Remind yourself to make short queries in your scratchpad as you plan out your strategy.我们在这里使用 scratchpad 是出于常规的思维链原因——它使 Claude 能够制定一个连贯的计划来回答问题。搜索质量反思用于促使 Claude 保持耐心，不要在收集所有相关信息之前就急于回答问题。但为什么我们要让 Claude 综合信息而不是立即回答呢？
answer_prompt = "Here is a user query: <query>{query}</query>. Here is some relevant information: <information>{information}</information>. Please answer the question using the relevant information."
print(answer_prompt)Here is a user query: <query>{query}</query>. Here is some relevant information: <information>{information}</information>. Please answer the question using the relevant information.通过提取信息并在新的查询中呈现给 Claude，我们让 Claude 将所有注意力集中在将信息综合成正确答案上。没有这一步，我们发现 Claude 有时会先预设一个答案，然后用搜索结果来”证明”它，而不是让结果来引导它。
下面是实现搜索 + 检索 + 重新 prompt 伪代码的一堆代码。
搜索实现
import re
from abc import abstractmethod
from dataclasses import dataclass
import wikipedia
from anthropic import AI_PROMPT, HUMAN_PROMPT, Anthropic
@dataclass
class SearchResult:
"""
A single search result.
"""
content: str
class SearchTool:
"""
A search tool that can run a query and return a formatted string of search results.
"""
def __init__():
pass
@abstractmethod
def raw_search(self, query: str, n_search_results_to_use: int) -> list[SearchResult]:
"""
Runs a query using the searcher, then returns the raw search results without formatting.
:param query: The query to run.
:param n_search_results_to_use: The number of results to return.
"""
raise NotImplementedError()
@abstractmethod
def process_raw_search_results(
self,
results: list[SearchResult],
) -> list[str]:
"""
Extracts the raw search content from the search results and returns a list of strings that can be passed to Claude.
:param results: The search results to extract.
"""
raise NotImplementedError()
def search_results_to_string(self, extracted: list[str]) -> str:
"""
Joins and formats the extracted search results as a string.
:param extracted: The extracted search results to format.
"""
result = "\n".join(
[
f'<item index="{i + 1}">\n<page_content>\n{r}\n</page_content>\n</item>'
for i, r in enumerate(extracted)
]
)
return result
def wrap_search_results(self, extracted: list[str]) -> str:
"""
Formats the extracted search results as a string, including the <search_results> tags.
:param extracted: The extracted search results to format.
"""
return f"\n<search_results>\n{self.search_results_to_string(extracted)}\n</search_results>"
def search(self, query: str, n_search_results_to_use: int) -> str:
raw_search_results = self.raw_search(query, n_search_results_to_use)
processed_search_results = self.process_raw_search_results(raw_search_results)
displayable_search_results = self.wrap_search_results(processed_search_results)
return displayable_search_results@dataclass
class WikipediaSearchResult(SearchResult):
title: str
class WikipediaSearchTool(SearchTool):
def __init__(self, truncate_to_n_tokens: int | None = 5000):
self.truncate_to_n_tokens = truncate_to_n_tokens
if truncate_to_n_tokens is not None:
self.tokenizer = Anthropic().get_tokenizer()
def raw_search(self, query: str, n_search_results_to_use: int) -> list[WikipediaSearchResult]:
search_results = self._search(query, n_search_results_to_use)
return search_results
def process_raw_search_results(self, results: list[WikipediaSearchResult]) -> list[str]:
processed_search_results = [
f"Page Title: {result.title.strip()}\nPage Content:\n{self.truncate_page_content(result.content)}"
for result in results
]
return processed_search_results
def truncate_page_content(self, page_content: str) -> str:
if self.truncate_to_n_tokens is None:
return page_content.strip()
else:
return self.tokenizer.decode(
self.tokenizer.encode(page_content).ids[: self.truncate_to_n_tokens]
).strip()
def _search(self, query: str, n_search_results_to_use: int) -> list[WikipediaSearchResult]:
results: list[str] = wikipedia.search(query)
search_results: list[WikipediaSearchResult] = []
for result in results:
if len(search_results) >= n_search_results_to_use:
break
try:
page = wikipedia.page(result)
print(page.url)
except wikipedia.exceptions.WikipediaException:
# The Wikipedia API is a little flaky, so we just skip over pages that fail to load
continue
content = page.content
title = page.title
search_results.append(WikipediaSearchResult(content=content, title=title))
return search_resultsdef extract_between_tags(tag: str, string: str, strip: bool = True) -> list[str]:
ext_list = re.findall(rf"<{tag}\s?>(.+?)</{tag}\s?>", string, re.DOTALL)
if strip:
ext_list = [e.strip() for e in ext_list]
return ext_list
class ClientWithRetrieval(Anthropic):
def __init__(self, search_tool: SearchTool, verbose: bool = True, *args, **kwargs):
super().__init__(*args, **kwargs)
self.search_tool = search_tool
self.verbose = verbose
# Helper methods
def _search_query_stop(
self, partial_completion: str, n_search_results_to_use: int
) -> tuple[list[SearchResult], str]:
search_query = extract_between_tags("search_query", partial_completion + "</search_query>")
if search_query is None:
raise Exception(
"Completion with retrieval failed as partial completion returned mismatched <search_query> tags."
)
print(f"Running search query against SearchTool: {search_query}")
search_results = self.search_tool.raw_search(search_query, n_search_results_to_use)
extracted_search_results = self.search_tool.process_raw_search_results(search_results)
formatted_search_results = self.search_tool.wrap_search_results(extracted_search_results)
return search_results, formatted_search_results
def retrieve(
self,
query: str,
model: str,
n_search_results_to_use: int = 3,
stop_sequences: list[str] = None,
max_tokens_to_sample: int = 1000,
max_searches_to_try: int = 5,
temperature: float = 1.0,
) -> tuple[list[SearchResult], str]:
if stop_sequences is None:
stop_sequences = [HUMAN_PROMPT]
prompt = (
f"{HUMAN_PROMPT} {wikipedia_prompt} {retrieval_prompt.format(query=query)}{AI_PROMPT}"
)
starting_prompt = prompt
print("Starting prompt:", starting_prompt)
token_budget = max_tokens_to_sample
all_raw_search_results: list[SearchResult] = []
for tries in range(max_searches_to_try):
partial_completion = self.completions.create(
prompt=prompt,
stop_sequences=stop_sequences + ["</search_query>"],
model=model,
max_tokens_to_sample=token_budget,
temperature=temperature,
)
partial_completion, stop_reason, stop_seq = (
partial_completion.completion,
partial_completion.stop_reason,
partial_completion.stop,
)
print(partial_completion)
token_budget -= self.count_tokens(partial_completion)
prompt += partial_completion
if stop_reason == "stop_sequence" and stop_seq == "</search_query>":
print(f"Attempting search number {tries}.")
raw_search_results, formatted_search_results = self._search_query_stop(
partial_completion, n_search_results_to_use
)
prompt += "</search_query>" + formatted_search_results
all_raw_search_results += raw_search_results
else:
break
final_model_response = prompt[len(starting_prompt) :]
return all_raw_search_results, final_model_response
# Main methods
def completion_with_retrieval(
self,
query: str,
model: str,
n_search_results_to_use: int = 3,
stop_sequences: list[str] = None,
max_tokens_to_sample: int = 1000,
max_searches_to_try: int = 5,
temperature: float = 1.0,
) -> str:
if stop_sequences is None:
stop_sequences = [HUMAN_PROMPT]
_, retrieval_response = self.retrieve(
query,
model=model,
n_search_results_to_use=n_search_results_to_use,
stop_sequences=stop_sequences,
max_tokens_to_sample=max_tokens_to_sample,
max_searches_to_try=max_searches_to_try,
temperature=temperature,
)
information = extract_between_tags("information", retrieval_response)[-1]
prompt = f"{HUMAN_PROMPT} {answer_prompt.format(query=query, information=information)}{AI_PROMPT}"
print("Summarizing:\n", prompt)
answer = self.completions.create(
prompt=prompt, model=model, temperature=temperature, max_tokens_to_sample=1000
).completion
return answer运行查询
我们准备好执行查询了！让我们选一个： - 较新的话题，这样不太可能在 Claude 的训练数据中 - 复合/复杂的，这样需要多次搜索
import os
# Create a searcher
wikipedia_search_tool = WikipediaSearchTool()
ANTHROPIC_SEARCH_MODEL = "claude-2"
client = ClientWithRetrieval(
api_key=os.environ["ANTHROPIC_API_KEY"], verbose=True, search_tool=wikipedia_search_tool
)
query = "Which movie came out first: Oppenheimer, or Are You There God It's Me Margaret?"
augmented_response = client.completion_with_retrieval(
query=query,
model=ANTHROPIC_SEARCH_MODEL,
n_search_results_to_use=1,
max_searches_to_try=5,
max_tokens_to_sample=1000,
temperature=0,
)
print(augmented_response)Starting prompt:
Human: You will be asked a question by a human user. You have access to the following tool to help answer the question. <tool_description> Search Engine Tool * The search engine will exclusively search over Wikipedia for pages similar to your query. It returns for each page its title and full page content. Use this tool if you want to get up-to-date and comprehensive information on a topic to help answer queries. Queries should be as atomic as possible -- they only need to address one part of the user's question. For example, if the user's query is "what is the color of a basketball?", your search query should be "basketball". Here's another example: if the user's question is "Who created the first neural network?", your first query should be "neural network". As you can see, these queries are quite short. Think keywords, not phrases. * At any time, you can make a call to the search engine using the following syntax: <search_query>query_word</search_query>. * You'll then get results back in <search_result> tags.</tool_description> Before beginning to research the user's question, first think for a moment inside <scratchpad> tags about what information is necessary for a well-informed answer. If the user's question is complex, you may need to decompose the query into multiple subqueries and execute them individually. Sometimes the search engine will return empty search results, or the search results may not contain the information you need. In such cases, feel free to try again with a different query.
After each call to the Search Engine Tool, reflect briefly inside <search_quality></search_quality> tags about whether you now have enough information to answer, or whether more information is needed. If you have all the relevant information, write it in <information></information> tags, WITHOUT actually answering the question. Otherwise, issue a new search.
Here is the user's question: <question>Which movie came out first: Oppenheimer, or Are You There God It's Me Margaret?</question> Remind yourself to make short queries in your scratchpad as you plan out your strategy.
Assistant:
<scratchpad>
To answer this question, I need to find the release dates for the two movies:
- Oppenheimer release date
- Are You There God It's Me Margaret release date
I can search for each movie title individually to get the release date.
</scratchpad>
<search_query>Oppenheimer movie
Attempting search number 0.
Running search query against SearchTool: ['Oppenheimer movie']
https://en.wikipedia.org/wiki/Oppenheimer_(film)
The search results indicate that Oppenheimer is scheduled for theatrical release on July 21, 2023. This provides the release date for Oppenheimer.
<search_quality>The search results directly provided the release date for Oppenheimer, so I now have enough information to answer this part of the question.</search_quality>
<search_query>Are You There God It's Me Margaret movie
Attempting search number 1.
Running search query against SearchTool: ["Are You There God It's Me Margaret movie"]
https://en.wikipedia.org/wiki/Are_You_There_God%3F_It%27s_Me,_Margaret.
The search results indicate that the film adaptation of Are You There God? It's Me, Margaret was released on April 28, 2023. This provides the release date for Are You There God? It's Me, Margaret.
<search_quality>The search results directly stated the release date for the Are You There God? It's Me, Margaret movie adaptation, so I now have enough information to fully answer the question.</search_quality>
<information>
- Oppenheimer was released on July 21, 2023
- Are You There God? It's Me, Margaret was released on April 28, 2023
</information>
Based on the release dates found through my searches, Oppenheimer came out first, being released on July 21, 2023, while Are You There God? It's Me, Margaret was released later on April 28, 2023.
Summarizing:
Human:Here is a user query: <query>Which movie came out first: Oppenheimer, or Are You There God It's Me Margaret?</query>. Here is some relevant information: <information>- Oppenheimer was released on July 21, 2023
- Are You There God? It's Me, Margaret was released on April 28, 2023</information>. Please answer the question using the relevant information.
Assistant:
Based on the information provided, Are You There God? It's Me, Margaret was released first, on April 28, 2023. Oppenheimer was released later, on July 21, 2023.很好，Claude 能够制定计划、执行查询，并将信息综合成准确的答案。注意：如果没有额外的信息提取步骤，Claude 有时会正确确定电影的上映日期，但在最终答案中搞错顺序。让我们再试一个。
augmented_response = client.completion_with_retrieval(
query="Who won the 2023 NBA championship? Who was that team's best player in the year 2009?",
model=ANTHROPIC_SEARCH_MODEL,
n_search_results_to_use=1,
max_searches_to_try=5,
max_tokens_to_sample=1000,
temperature=0,
)
print(augmented_response)Starting prompt:
Human: You will be asked a question by a human user. You have access to the following tool to help answer the question. <tool_description> Search Engine Tool * The search engine will exclusively search over Wikipedia for pages similar to your query. It returns for each page its title and full page content. Use this tool if you want to get up-to-date and comprehensive information on a topic to help answer queries. Queries should be as atomic as possible -- they only need to address one part of the user's question. For example, if the user's query is "what is the color of a basketball?", your search query should be "basketball". Here's another example: if the user's question is "Who created the first neural network?", your first query should be "neural network". As you can see, these queries are quite short. Think keywords, not phrases. * At any time, you can make a call to the search engine using the following syntax: <search_query>query_word</search_query>. * You'll then get results back in <search_result> tags.</tool_description> Before beginning to research the user's question, first think for a moment inside <scratchpad> tags about what information is necessary for a well-informed answer. If the user's question is complex, you may need to decompose the query into multiple subqueries and execute them individually. Sometimes the search engine will return empty search results, or the search results may not contain the information you need. In such cases, feel free to try again with a different query.
After each call to the Search Engine Tool, reflect briefly inside <search_quality></search_quality> tags about whether you now have enough information to answer, or whether more information is needed. If you have all the relevant information, write it in <information></information> tags, WITHOUT actually answering the question. Otherwise, issue a new search.
Here is the user's question: <question>Who won the 2023 NBA championship? Who was that team's best player in the year 2009?</question> Remind yourself to make short queries in your scratchpad as you plan out your strategy.
Assistant:
<scratchpad>
To answer this question, I need to find:
1. The team that won the 2023 NBA championship
2. The best player on that team in 2009
I can search for these things separately.
</scratchpad>
<search_query>2023 nba championship winner
Attempting search number 0.
Running search query against SearchTool: ['2023 nba championship winner']
https://en.wikipedia.org/wiki/List_of_NBA_champions
<search_quality>The search results contain the team that won the 2023 NBA championship, so I have the information I need to answer the first part of the question.</search_quality>
<information>
The Denver Nuggets won the 2023 NBA championship.
</information>
<search_query>denver nuggets best player 2009
Attempting search number 1.
Running search query against SearchTool: ['denver nuggets best player 2009']
https://en.wikipedia.org/wiki/2009%E2%80%9310_Denver_Nuggets_season
<search_quality>The search results indicate that Carmelo Anthony was the Nuggets' best player in 2009, so I now have all the information needed to fully answer the question.</search_quality>
<information>
- The Denver Nuggets won the 2023 NBA championship.
- Carmelo Anthony was the Nuggets' best player in 2009.
</information>
Summarizing:
Human:Here is a user query: <query>Who won the 2023 NBA championship? Who was that team's best player in the year 2009?</query>. Here is some relevant information: <information>- The Denver Nuggets won the 2023 NBA championship.
- Carmelo Anthony was the Nuggets' best player in 2009.</information>. Please answer the question using the relevant information.
Assistant:
<response>
Based on the provided information:
- The Denver Nuggets won the 2023 NBA championship.
- Carmelo Anthony was the Nuggets' best player in 2009.
</response>就是这样了！你可能注意到搜索工具代码非常抽象，只需少量修改就可以适配你选择的搜索 API。只需记住向 Claude 解释有效使用该工具所需的任何技巧。你甚至可以给 Claude 一些理想查询计划和查询结构的少样本示例来进一步提升性能。
Tip
中文读者提示：本章节的代码和输出为英文原文。如需理解具体实现细节，可参考上方中文导读和代码注释。如有疑问，欢迎在 GitHub Issues 讨论。