使用Braintrust构建LLM作为裁判检测幻觉的评估方法

本文介绍如何利用Braintrust平台和OpenAI模型构建LLM-as-a-judge评估器，用于检测生成式AI中的幻觉问题。通过CoQA数据集生成幻觉样本，对比数值评分、思维链推理和分类三种评估策略，最终分类器达到98%准确率。文章强调评估器设计的关键在于明确标准和分类而非模糊评分，并提供了完整的代码实现和优化建议。

技术背景

在生成式AI应用中，评估模型输出质量至关重要，尤其是检测幻觉（生成与事实不符的内容）。传统方法如字符串匹配（Levenshtein距离）无法理解语义，而人工评估成本高且不可扩展。LLM-as-a-judge利用大语言模型的语义理解能力，自动评估答案质量，适用于客服机器人、内容生成等场景。但该方法依赖模型判断，存在偏差风险，需通过精心设计提示词和评估框架来优化。

核心原理

LLM-as-a-judge的核心是利用LLM对答案进行语义比较，而非表面字符串匹配。通过设计提示词，让LLM根据给定标准（如事实一致性）对生成答案评分或分类。本文采用三种策略：数值评分（1-10分）、思维链推理（要求模型解释评分理由）、分类（将答案分为子集、超集、一致、冲突等类别）。分类方法通过明确标准，减少模型主观性，提高准确率。

架构分析

系统架构包括数据层（CoQA数据集通过DuckDB加载）、生成层（使用OpenAI模型生成幻觉答案）、评估层（Braintrust平台管理评估流程，支持并行调用和结果记录）。评估流程：数据准备→任务定义→评分函数→Braintrust Eval执行。关键组件：AsyncOpenAI客户端（并行请求）、braintrust.wrap_openai（记录调用）、@braintrust.traced（追踪函数）、Eval（运行评估）。

完整实战教程

假设你正在开发一个客户服务机器人，并试图评估其回复的质量。考虑这样一个问题：“你们的退货政策是什么？”如果正确答案是“您可以在购买后30天内退货”，但你的机器人生成的是“您可以在30天内退货”，你会如何评估这是否是一个好的回复？

像Levenshtein字符串距离这样的启发式方法会表明该回复不正确。然而，更好的方法是使用LLM-as-a-judge来评估回复的准确性。LLM-as-a-judge是一种利用LLM对答案质量进行评分的技术。LLM能够超越表面字符串比较，对语言进行推理，从而更准确地评估答案。

在本教程中，我们将逐步介绍如何使用Braintrust（一个与OpenAI模型兼容的第三方评估平台）构建一个能够检测幻觉的LLM-as-a-judge评分器。

安装依赖

让我们安装一些基本依赖。我们将使用CoQA数据集（通过DuckDB）、Braintrust进行评估，以及OpenAI的模型。请注意，Braintrust是第三方评估平台，在继续之前，您应查看其服务条款和隐私政策。

%pip install autoevals duckdb braintrust openai --quiet

接下来，让我们初始化 OpenAI 客户端。我们将使用 AsyncOpenAI 客户端，以便并行处理请求。braintrust.wrap_openai 函数包装了 OpenAI 客户端，以启用将 LLM 调用记录到 Braintrust。我们将使用 Braintrust 来促进下面的评估。在继续之前，您应该注册一个 Braintrust 账户，并在您的环境中设置 BRAINTRUST_API_KEY 为有效的 API 密钥。

import os

import braintrust
from openai import AsyncOpenAI

braintrust.login(api_key=os.environ["BRAINTRUST_API_KEY"])
client = braintrust.wrap_openai(AsyncOpenAI(api_key=os.environ["OPENAI_API_KEY"]))

探索数据集

我们将使用 CoQA 数据集，其中包含多样化的段落、问题和答案。由于 CoQA 相当大，我们只查看前几个段落。与任何公共数据集一样，底层 LLM 有可能记住了数据集的某些方面，因此在开发自己的评分器时，最好使用自己的私有数据进行测试。

import duckdb

# DuckDB has an easy wrapper for loading datasets from Hugging Face.
con = duckdb.connect(":memory:")
full_result = con.query("""
    SELECT * FROM 'hf://datasets/stanfordnlp/coqa/data/validation-00000-of-00001.parquet'
        LIMIT 40
""").fetchall()

single_result = full_result[10]

print("Passage:")
print(single_result[1])

print("\nQuestion:")
print(single_result[2][0])

print("\nAnswer:")
print(single_result[3]["input_text"][0])

数据包含一系列段落，每个段落都有若干问题和答案。让我们将其展平为(passage, question, answer)元组的列表。

from dataclasses import dataclass


@dataclass
class QuestionAnswer:
    passage: str
    question: str
    expected_answer: str
    generated_answer: str


qa_pairs = [
    QuestionAnswer(
        passage=r[1],
        question=question,
        generated_answer=r[3]["input_text"][i],
        expected_answer=r[3]["input_text"][i],
    )
    for r in full_result
    for (i, question) in enumerate(r[2])
]

print(len(qa_pairs))

添加幻觉

由于 Braintrust 的评分器旨在测试幻觉，我们可以利用问答对来生成已知的幻觉。我们将通过让一个 LLM 在不使用段落的情况下自信地生成每个问题的答案，从而创建幻觉答案。

import asyncio
import random

random.seed(42)


async def hallucinate_answer(qa):
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": """\
You are a helpful hallucinating assistant, who makes up fake answers to questions.

Answer the following question in 1 sentence. If you know the answer, then make up some fake
superfluous details that are not in the passage you have memorized.

Make sure to always answer it confidently, even if you don't know the answer. Do not use words
like "perhaps", "likely", "maybe", etc. or punctuation like "...".Do not admit that you cannot
or do not know the answer.""",
            },
            {"role": "user", "content": qa.question},
        ],
        temperature=1,
        max_tokens=100,
    )
    return response.choices[0].message.content


hallucinated_answers = await asyncio.gather(
    *[hallucinate_answer(qa) for qa in qa_pairs]
)


hallucinations = [
    QuestionAnswer(
        passage=qa.passage,
        question=qa.question,
        expected_answer=qa.expected_answer,
        generated_answer=hallucination,
    )
    for (qa, hallucination) in zip(qa_pairs, hallucinated_answers)
    # Exclude simple yes/no answers.
    if "yes" not in hallucination.lower() and "no" not in hallucination.lower()
]

print("Passage:")
print(hallucinations[0].passage)
print("\nQuestion:")
print(hallucinations[0].question)
print("\nExpected Answer:")
print(hallucinations[0].expected_answer)
print("\nGenerated Answer:")
print(hallucinations[0].generated_answer)

print("\n\nNumber of hallucinations:", len(hallucinations))

创建评估器

我们将考虑几种流行的 LLM 作为评判者的创建方法。对于每种方法，我们将创建一个评分器，然后对其进行“元评估”以查看其表现。

由于我们知道幻觉答案是不正确的，我们将通过测试评估器将幻觉答案评分为 0 的频率来评估其质量。

LLM 作为评判者 #1：数值评分器

创建 LLM 作为评判者时，一个常见的初步想法是让 LLM 以 1 到 5 的等级对答案进行评分。这种方法的好处是很容易将 LLM 的输出转换为数值分数。

我们将使用 Factuality 模板的修改版本，但要求 LLM 以 1 到 10 的等级对答案进行评分。

import json

PROMPT = """\
You are comparing a submitted answer to an expert answer on a given question. Here is the data:
[BEGIN DATA]
************
[Question]: {input}
************
[Expert]: {expected}
************
[Submission]: {output}
************
[END DATA]

Compare the factual content of the submitted answer with the expert answer. Ignore any differences in style, grammar, or punctuation.
Rate the submission on a scale of 1 to 10.
"""


@braintrust.traced
async def numeric_rater(input, output, expected):
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "user",
                "content": PROMPT.format(input=input, output=output, expected=expected),
            }
        ],
        temperature=0,
        tools=[
            {
                "type": "function",
                "function": {
                    "name": "rate",
                    "description": "Rate the submission on a scale of 1 to 10.",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "rating": {"type": "integer", "minimum": 1, "maximum": 10},
                        },
                        "required": ["rating"],
                    },
                },
            }
        ],
        tool_choice={"type": "function", "function": {"name": "rate"}},
    )
    arguments = json.loads(response.choices[0].message.tool_calls[0].function.arguments)
    return (arguments["rating"] - 1) / 9


print(qa_pairs[10].question, "On a correct answer:", qa_pairs[10].generated_answer)
print(
    await numeric_rater(
        qa_pairs[10].question,
        qa_pairs[10].generated_answer,
        qa_pairs[10].expected_answer,
    )
)

print(
    hallucinations[10].question,
    "On a hallucinated answer:",
    hallucinations[10].generated_answer,
)
print(
    await numeric_rater(
        hallucinations[10].question,
        hallucinations[10].generated_answer,
        hallucinations[10].expected_answer,
    )
)

这看起来很有希望！既然我们已经在单个示例上进行了合理性检查，现在让我们运行一个正式的评估，看看它在更广泛的数据集上的表现。评估由三个部分组成：

from dataclasses import asdict

from braintrust import Eval


def data():
    for pair in hallucinations:
        yield dict(
            input=dict(asdict(pair)), expected=0, metadata=dict(hallucination=True)
        )


async def task(input):
    return await numeric_rater(
        input=input["question"],
        output=input["generated_answer"],
        expected=input["expected_answer"],
    )


def normalized_diff(output, expected):
    return 1 - abs(output - expected)


await Eval(
    "LLM-as-a-judge",
    data=data,
    task=task,
    scores=[normalized_diff],
    experiment_name="Numeric rater",
    max_concurrency=10,
)

看起来数字评分器总分接近94%。这还不错，但如果6%的评估被错误判断，那可能会让你很难信任它们。让我们深入Braintrust界面，了解发生了什么。

Partial credit看起来许多错误答案的评分在1到10之间。然而，我们目前对模型为何给出这些分数没有任何洞察。接下来让我们看看能否解决这个问题。

LLM 作为评判者 #2：添加推理

让我们调整提示词，让 LLM 也对其评分进行推理。这种方法被称为思维链推理。除了可能提高分数外，它还能让我们了解模型为何给出这些分数。

@braintrust.traced
async def numeric_rater(input, output, expected):
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "user",
                "content": PROMPT.format(input=input, output=output, expected=expected),
            }
        ],
        temperature=0,
        tools=[
            {
                "type": "function",
                "function": {
                    "name": "rate",
                    "description": "Rate the submission on a scale of 1 to 10.",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "reasons": {
                                "description": "Write out in a step by step manner your reasoning to be sure that your conclusion is correct. Avoid simply stating the correct answer at the outset.",
                                "title": "Reasoning",
                                "type": "string",
                            },
                            "rating": {"type": "integer", "minimum": 1, "maximum": 10},
                        },
                        "required": ["rating"],
                    },
                },
            }
        ],
        tool_choice={"type": "function", "function": {"name": "rate"}},
    )
    arguments = json.loads(response.choices[0].message.tool_calls[0].function.arguments)
    return (arguments["rating"] - 1) / 9


print(qa_pairs[10].question, "On a correct answer:", qa_pairs[10].generated_answer)
print(
    await numeric_rater(
        qa_pairs[10].question,
        qa_pairs[10].generated_answer,
        qa_pairs[10].expected_answer,
    )
)

print(
    hallucinations[10].question,
    "On a hallucinated answer:",
    hallucinations[10].generated_answer,
)
print(
    await numeric_rater(
        hallucinations[10].question,
        hallucinations[10].generated_answer,
        hallucinations[10].expected_answer,
    )
)

await Eval(
    "LLM-as-a-judge",
    data=data,
    task=task,
    scores=[normalized_diff],
    experiment_name="Numeric rater with reasoning",
    max_concurrency=10,
)

看起来添加推理并没有帮助得分（实际上，它反而差了3%）。然而，如果我们看看其中一个失败案例，就能洞察模型在想什么。下面是一个幻觉回答的例子：

Output而分数及其推理过程：

Reasoning

看起来模型正在运用自己的判断来计算部分分数。这是数值评分中常见的问题——无论对模型还是对人类都是如此——通常可以通过更好的提示来解决。

LLM 作为评判者 #3：分类而非评分

接下来，我们将明确具体标准，并要求模型根据这些标准对答案进行分类。这种方法使我们能够更精确地引导模型关注我们正在测试的幻觉。直觉上，给模型具体的评分标准会得到更准确的分数。

PROMPT = """\
You are comparing a submitted answer to an expert answer on a given question. Here is the data:
[BEGIN DATA]
************
[Question]: {input}
************
[Expert]: {expected}
************
[Submission]: {output}
************
[END DATA]

Compare the factual content of the submitted answer with the expert answer. Ignore any differences in style, grammar, or punctuation.
The submitted answer may either be a subset or superset of the expert answer, or it may conflict with it. Determine which case applies. Answer the question by selecting one of the following options:
(A) The submitted answer is a subset of the expert answer and is fully consistent with it.
(B) The submitted answer is a superset of the expert answer and is fully consistent with it.
(C) The submitted answer contains all the same details as the expert answer.
(D) There is a disagreement between the submitted answer and the expert answer.
(E) The answers differ, but these differences don't matter from the perspective of factuality.

Answer the question by calling `select_choice` with your reasoning in a step-by-step matter to be
sure that your conclusion is correct. Avoid simply stating the correct answer at the outset. Select a
single choice by setting the `choice` parameter to a single choice from A, B, C, D, or E.
"""

# Since we're testing for hallucinations, penalize (B) as much as (D).
CHOICE_SCORES = {
    "A": 0.5,
    "B": 0,
    "C": 1,
    "D": 0,
    "E": 1,
}


@braintrust.traced
async def classifier(input, output, expected):
    response = await client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "user",
                "content": PROMPT.format(input=input, output=output, expected=expected),
            }
        ],
        temperature=0,
        tools=[
            {
                "type": "function",
                "function": {
                    "name": "rate",
                    "description": "Call this function to select a choice.",
                    "parameters": {
                        "properties": {
                            "reasons": {
                                "description": "Write out in a step by step manner your reasoning to be sure that your conclusion is correct. Avoid simply stating the correct answer at the outset.",
                                "type": "string",
                            },
                            "choice": {
                                "description": "The choice",
                                "type": "string",
                                "enum": ["A", "B", "C", "D", "E"],
                            },
                        },
                        "required": ["reasons", "choice"],
                        "type": "object",
                    },
                },
            }
        ],
        tool_choice={"type": "function", "function": {"name": "rate"}},
    )
    arguments = json.loads(response.choices[0].message.tool_calls[0].function.arguments)
    choice = arguments["choice"]
    return CHOICE_SCORES[choice] if choice in CHOICE_SCORES else None


print(qa_pairs[10].question, "On a correct answer:", qa_pairs[10].generated_answer)
print(
    await classifier(
        qa_pairs[10].question,
        qa_pairs[10].generated_answer,
        qa_pairs[10].expected_answer,
    )
)

print(
    hallucinations[10].question,
    "On a hallucinated answer:",
    hallucinations[10].generated_answer,
)
print(
    await classifier(
        hallucinations[10].question,
        hallucinations[10].generated_answer,
        hallucinations[10].expected_answer,
    )
)

async def task(input):
    return await classifier(
        input=input["question"],
        output=input["generated_answer"],
        expected=input["expected_answer"],
    )


await Eval(
    "LLM-as-a-judge",
    data=data,
    task=task,
    scores=[normalized_diff],
    experiment_name="Classifier",
    max_concurrency=10,
)

分类器的得分达到了98%，这是一个显著的提升！

将这一模式规范化

上述分类器可以简单地重写为：

PROMPT = """\
You are comparing a submitted answer to an expert answer on a given question. Here is the data:
[BEGIN DATA]
************
[Question]: {{input}}
************
[Expert]: {{expected}}
************
[Submission]: {{output}}
************
[END DATA]

Compare the factual content of the submitted answer with the expert answer. Ignore any differences in style, grammar, or punctuation.
The submitted answer may either be a subset or superset of the expert answer, or it may conflict with it. Determine which case applies. Answer the question by selecting one of the following options:
(A) The submitted answer is a subset of the expert answer and is fully consistent with it.
(B) The submitted answer is a superset of the expert answer and is fully consistent with it.
(C) The submitted answer contains all the same details as the expert answer.
(D) There is a disagreement between the submitted answer and the expert answer.
(E) The answers differ, but these differences don't matter from the perspective of factuality.

Answer the question by calling `select_choice` with your reasoning in a step-by-step matter to be
sure that your conclusion is correct. Avoid simply stating the correct answer at the outset. Select a
single choice by setting the `choice` parameter to a single choice from A, B, C, D, or E.
"""

Classifier = autoevals.LLMClassifier(
    name="Hallucination detector",
    prompt_template=PROMPT,
    choice_scores={"A": 0.5, "B": 0, "C": 1, "D": 0, "E": 1},
    use_cot=True,
)

后续步骤

作为下一步，您可以深入研究各个改进和回退，评估它们并考虑对提示词进行未来改进。您还可以在自己的数据上测试它，并再次确认结果在您的用例中是否成立。

您还可以衡量像 o1 这样的模型，尝试微调较小的模型，看看结果是否可复现，或者使用少样本提示来使模型与更主观的标准对齐。

在所有情况下，您都应该努力评估您的结果，以便严格评估每项更改的影响。

常见问题

常见错误：1) 未设置BRAINTRUST_API_KEY或OPENAI_API_KEY导致认证失败；2) 数据集加载失败（网络问题或格式错误）；3) 并行调用过多导致速率限制；4) 提示词设计不当导致评分偏差。排查方法：检查环境变量、使用小样本测试、查看Braintrust日志。优化方案：使用分类而非评分、添加思维链推理、调整提示词明确标准、使用少样本示例。

企业应用扩展

可扩展方向：1) 在私有数据上测试评估器，确保泛化性；2) 尝试不同模型（如o1）或微调小模型；3) 使用少样本提示对齐主观标准；4) 集成到CI/CD流水线，持续监控模型输出质量；5) 结合人工审核，建立反馈循环。

总结

本文介绍如何利用Braintrust平台和OpenAI模型构建LLM-as-a-judge评估器，用于检测生成式AI中的幻觉问题。通过CoQA数据集生成幻觉样本，对比数值评分、思维链推理和分类三种评估策略，最终分类器达到98%准确率。文章强调评估器设计的关键在于明确标准和分类而非模糊评分，并提供了完整的代码实现和优化建议。

内容来源与验证

继续系统学习

站内还有更多完整实战教程

加入会员，解锁全部教程、完整代码与持续更新内容。

加入会员
查看全部模型评估与微调教程