使用OpenAI gpt-oss-120B、Cerebras和Parallel构建自己的内容事实核查器

本教程指导开发者构建一个自动化事实核查系统，利用Cerebras的超低延迟推理和Parallel的搜索API，结合OpenAI的gpt-oss-120B模型，从文本或URL中提取声明、检索证据并给出真/假/不确定的判定。教程涵盖环境配置、LLM调用、网络搜索、声明提取、证据整理、单条声明核查及完整流程，并提供示例和注意事项。

技术背景

在信息爆炸的时代，虚假信息传播迅速，人工核查效率低下。本教程旨在通过自动化技术，帮助开发者快速验证文本中的事实性声明。该技术适用于新闻验证、社交媒体内容审核、学术研究辅助等场景。主要限制包括：依赖外部API的可用性和准确性，模型可能产生误判，且对提示注入等安全威胁缺乏防护。

核心原理

系统核心原理是“提取-检索-评估”流水线。首先，利用LLM（gpt-oss-120B）从输入文本中提取原子化的事实声明（如日期、数字、实体）。然后，通过Parallel的搜索API检索相关网络证据，并整理成结构化上下文。最后，再次调用LLM，结合证据对每个声明进行评判，输出真、假或不确定，并附上理由和引用来源。关键设计决策包括：使用one-shot搜索模式以平衡速度和质量，采用严格JSON输出格式以便解析，以及设置reasoning_effort参数控制推理深度。

架构分析

系统由四个主要组件构成：1) Cerebras客户端，负责与gpt-oss-120B模型交互，提供低延迟推理；2) Parallel客户端，用于执行网络搜索，返回带摘录的结果；3) 声明提取模块，利用LLM将文本分解为可核查的原子声明；4) 事实核查模块，结合证据和声明生成判定。数据流为：输入文本→声明提取→对每个声明进行网络搜索→构建证据上下文→LLM评估→输出结构化结果。设计上强调模块化，便于扩展和替换组件。

完整实战教程

使用OpenAI gpt-oss-120B、Cerebras和Parallel构建自己的内容事实核查器

你是否曾读过一篇文章，后来却发现其中一些“事实”是捏造的？随着信息变得越来越丰富，验证其准确性也变得越来越具有挑战性。

本指南提供了一种实用的自动化方法，用于大规模评估事实准确性。它从任何文本或URL中提取声明，检索真实世界的证据，并使用由Cerebras超低延迟推理驱动的gpt-oss-120B评估每个声明。

在此处查看演示：内容事实核查器。

本指南需要设置以下账户：

了解更多关于 gpt-oss-120B 的最佳实践，请参阅此处。

步骤 1：环境设置（Colab 或本地）

本指南支持本地 Jupyter 环境和 Google Colab。设置以下环境变量：

python3 -m pip install -U cerebras_cloud_sdk parallel-web

import os
from cerebras.cloud.sdk import Cerebras
from parallel import Parallel

# API keys: Colab userdata (if available) -> env vars fallback
try:
    from google.colab import userdata  # type: ignore
    CEREBRAS_API_KEY = userdata.get("CEREBRAS_API_KEY") or os.getenv("CEREBRAS_API_KEY")
    PARALLEL_API_KEY = userdata.get("PARALLEL_API_KEY") or os.getenv("PARALLEL_API_KEY")
except ImportError:
    CEREBRAS_API_KEY = os.getenv("CEREBRAS_API_KEY")
    PARALLEL_API_KEY = os.getenv("PARALLEL_API_KEY")

if not CEREBRAS_API_KEY or not PARALLEL_API_KEY:
    raise RuntimeError("Set CEREBRAS_API_KEY and PARALLEL_API_KEY as environment variables.")

cerebras_client = Cerebras(
    api_key=CEREBRAS_API_KEY,
    default_headers={
        "X-Cerebras-3rd-Party-Integration": "parallel-ai-workshop"
    }
)

parallel_client = Parallel(api_key=PARALLEL_API_KEY)

CEREBRAS_MODEL_NAME = "gpt-oss-120B"

print("Clients initialized, model:", CEREBRAS_MODEL_NAME)

步骤 2：设置 LLM

现在，环境已准备就绪，创建将调用 LLM 的函数。

def call_cerebras_chat(
    user_content: str,
    system_content: str | None = None,
    model: str = CEREBRAS_MODEL_NAME,
    temperature: float = 1.0,
    top_p= 1.0,
    max_tokens: int = 4096,
    reasoning_effort: str = "medium"
):
    """
    Calls the Cerebras chat completion API.

    Args:
        user_content (str): The user's message.
        system_content (str | None): Optional system message to guide the LLM.
        model (str): The Cerebras model to use.
        temperature (float): Controls the randomness of the output.
        max_tokens (int): The maximum number of tokens in the response.

    Returns:
        str: The content of the LLM's response.
    """
    messages = []
    # Add a system message to guide the model's behavior
    if system_content:
        messages.append({"role": "system", "content": system_content})

    messages.append({"role": "user", "content": user_content})

    # Make the API call to Cerebras chat completions
    resp = cerebras_client.chat.completions.create(
        model=model,
        messages=messages,
        temperature=temperature,
        top_p=top_p,
        max_tokens=max_tokens,
        reasoning_effort=reasoning_effort,
    )
    return resp.choices[0].message.content

步骤 3：将 LLM 连接到互联网

为了核实某个说法，模型需要在网上找到证据，而这一步将构建将 LLM 连接到互联网的函数。

注意几个字段：

def search_web(query: str, num: int = 5, mode: str = "one-shot"):
    """
    Search the web using Parallel's Search API.

    Returns a list of dicts with:
      - url
      - title
      - publish_date
      - excerpts (list of strings)
    """

    # Instruct the LLM find quality sources.
    objective = (
        f"Find high-quality, up-to-date sources that answer the question:\n\n{query}\n\n"
        "Prefer authoritative sites (e.g., .gov, .edu, major news, or official org websites)."
    )

    # Initiatiate the LLM for web search
    search = parallel_client.beta.search(
        objective=objective,
        search_queries=[query],
        mode=mode,
        max_results=num,
        excerpts={
            "max_chars_per_result": 8000,
        },
    )

    results = []
    # Process the search results and extract information like URL, title, and excerpts.
    for r in search.results:
        results.append(
            {
                "url": r.url,
                "title": getattr(r, "title", None),
                "publish_date": getattr(r, "publish_date", None),
                "excerpts": list(r.excerpts or []),
            }
        )
    return results

第4步 – 整理并总结网络结果

从网络获取信息后，将其整理成清晰、易读的格式。此步骤将搜索结果中的关键摘录汇编成简单的摘要，以供评估。

import textwrap
from typing import List, Dict, Any

def build_evidence_context(results: List[Dict[str, Any]], max_chars: int = 8000) -> str:

    blocks = []
    for idx, r in enumerate(results):
        excerpts_text = "\n\n".join(r["excerpts"][:2])
        block = textwrap.dedent(f"""
        [Source {idx+1}]
        Title: {r['title'] or r['url']}
        URL: {r['url']}
        Publish date: {r['publish_date']}

        Excerpts:
        {excerpts_text}
        """).strip()
        blocks.append(block)

    context = "\n\n".join(blocks)

    if len(context) > max_chars:
        context = context[:max_chars] + "\n\n[Context truncated for length]"
    return context

步骤 5 – 找出需要核实的声明

接下来，确定文本中需要核实的特定陈述。与其一次性分析整篇文章，LLM 应将其分解为多个清晰、独立的声明，以便逐字判断。

例如，对于这样一段简短的段落：“2024 年 3 月失业率降至 3.5%，同周 X 公司宣布了一项 100 亿美元的合并。”

LLM 应提取出独立的事实陈述，例如：

* “2024 年 3 月失业率降至 3.5%。”

* “X 公司宣布了一项 100 亿美元的合并。”

然后可以独立检查每项声明，这使得整个事实核查过程精确且可靠。

import json
import re
import time

def extract_claims_from_text(text: str, max_claims: int = 8) -> list[str]:
    """
    Use Cerebras LLM to extract atomic factual claims from text.
    Output format (strict JSON):
    {
      "claims": ["...", "."...]
    }
    """
    # Instruct the LLM to extract factual claims
    system_prompt_content = (
        "You are an information extraction assistant.\n"
        "From the user's text, extract up to {max_claims} atomic factual claims.\n"
        "Each claim should:\n"
        "- Be checkable against external sources (dates, numbers, named entities)\n"
        "- Be concrete and not an opinion.\n\n"
        "Return STRICT JSON:\n"
        "{{\n"
        '  "claims": ["...", "..."]\n'
        "}}\n"
    ).format(max_claims=max_claims)

    # Prompt the LLM for claim extraction
    user_prompt_content = f"Text:\n\n{text}\n\nExtract up to {max_claims} factual claims."

    messages = [
        {"role": "system", "content": system_prompt_content},
        {"role": "user", "content": user_prompt_content}
    ]

    start_time = time.time()
    # Call Cerebras LLM (gpt-oss-120B) for claim extraction
    resp = cerebras_client.chat.completions.create(
        model=CEREBRAS_MODEL_NAME,
        messages=messages,
        temperature=1.0,
        top_p=1.0,
        max_tokens=4096,
        reasoning_effort="medium",
    )
    raw = resp.choices[0].message.content.strip()
    end_time = time.time()
    print(f"Cerebras LLM claim extraction took {end_time - start_time:.2f} seconds")

    # Clean up the raw JSON output
    raw = re.sub(r"^\s*```(?:json)?\s*", "", raw, flags=re.IGNORECASE)
    raw = re.sub(r"\s*```\s*$", "", raw)

    try:
        data = json.loads(raw)
        claims = data.get("claims", [])
        claims = [c.strip() for c in claims if isinstance(c, str) and c.strip()]
        return claims[:max_claims]
    except Exception as e:
        print("Error parsing claims JSON:", e)
        print("Raw model output:\n", raw)
        return []

print("Claim extraction ready")

第6步 – 将声明与证据进行核对（真/假/不确定）

收集声明并将其提取为独立的事实性声明后，LLM现在可以评估每个声明并给出判定。该过程分为两个步骤：

1) 使用Parallel检索证据：

首先，使用Parallel查询与声明相关的权威来源。

2) 使用Cerebras评判声明：

然后，将证据和原始声明发送给Cerebras进行评估。在这里，Cerebras的超快速推理变得至关重要，LLM可以分析多条证据，权衡矛盾，并生成判定。

模型将返回三种结构化判定之一：

每个判定都附有解释和引用的URL，使模型的推理过程透明可见。

from typing import Dict, Any
import textwrap
import re
import time

def fact_check_single_claim(claim: str) -> Dict[str, Any]:
    """
    Fact-check a single claim using:
      - Parallel Search for evidence
      - Cerebras LLM for verdict

    Args:
        claim (str): The factual claim to be checked.

    Returns:
        Dict[str, Any]: A dictionary containing the claim, verdict, reason, and sources.
        {
          "claim": str,
          "verdict": "true" | "false" | "uncertain",
          "reason": str,
          "sources": [url, ...]
        }
    """
    print(f"\nFact-checking claim: {claim}")

    # Search the web for evidence relevant to the claim
    results = search_web(query=claim, num=6, mode="one-shot")
    print(f"Retrieved {len(results)} evidence sources")

    # Compile the search results into a clean, readable context for the LLM
    evidence_context = build_evidence_context(results)

    # Define the system prompt to instruct the Cerebras LLM (gpt-oss-120B) on how to evaluate each claim
    system_prompt_content = (
        "You are a careful, skeptical fact-checking assistant.\n"
        "You get a factual claim and web search excerpts.\n"
        "Decide if the evidence supports, contradicts, or does not clearly resolve the claim.\n\n"
        "Respond with STRICT JSON:\n"
        "{\n"
        '  "verdict": "true" | "false" | "uncertain",\n'
        '  "reason": "short explanation",\n'
        '  "top_sources": ["url1", "url2", ...]\n'
        "}\n"
        "Use 'true' only when the evidence strongly supports the claim.\n"
        "Use 'false' only when it clearly contradicts the claim.\n"
        "Otherwise use 'uncertain'."
    )

    # Construct the user prompt
    user_prompt_content = textwrap.dedent(f"""
    Claim:
    {claim}

    Evidence (web search excerpts):
    {evidence_context}
    """)

    messages = [
        {"role": "system", "content": system_prompt_content},
        {"role": "user", "content": user_prompt_content}
    ]

    start_time = time.time()
    # Call the Cerebras LLM (gpt-oss-120B) to get a structured verdict
    resp = cerebras_client.chat.completions.create(
        model=CEREBRAS_MODEL_NAME,
        messages=messages,
        temperature=1.0,
        top_p=1.0,
        max_tokens=4096,    
        reasoning_effort="medium"
    )
    raw = resp.choices[0].message.content.strip()
    end_time = time.time()
    print(f"Cerebras LLM judgment for this claim took {end_time - start_time:.2f} seconds")

    # Clean up the raw JSON output from the LLM
    raw = re.sub(r"^\s*```(?:json)?\s*", "", raw, flags=re.IGNORECASE)
    raw = re.sub(r"\s*```\s*$", "", raw)

    try:
        data = json.loads(raw)
    except Exception as e:
        print("Error parsing judgment JSON:", e)
        print("Raw model output:\n", raw)
        data = {
            "verdict": "uncertain",
            "reason": "Could not parse model output.",
            "top_sources": [],
        }

    # Extract and normalize the verdict (true, false, or uncertain)
    verdict = str(data.get("verdict", "uncertain")).lower()
    if verdict not in {"true", "false", "uncertain"}:
        verdict = "uncertain"

    # Extract and format the top sources cited by the LLM
    top_sources = data.get("top_sources") or []
    if not isinstance(top_sources, list):
        top_sources = [str(top_sources)]
    top_sources = [str(u) for u in top_sources][:5]

    # Consolidate all the fact-checking results into a single dictionary
    result = {
        "claim": claim,
        "verdict": verdict,
        "reason": data.get("reason", ""),
        "sources": top_sources,
    }

    # Print the detailed fact-checking result for clarity
    print("Verdict:", result["verdict"].upper())
    print("Reason:", result["reason"])
    if result["sources"]:
        print("Sources:")
        for s in result["sources"]:
            print("  •", s)

    return result

print("Single-claim fact-checker ready")

步骤 7 - 对整篇文本进行事实核查

这最后一步将所有内容整合在一起。在这里，选取任意一段文本，并让每一段都经过你构建的完整事实核查流程。

def fact_check_text(text: str, max_claims: int = 6):
    # First, extract factual claims from the input text
    claims = extract_claims_from_text(text, max_claims=max_claims)

    print(f"Extracted {len(claims)} claims:")
    for i, c in enumerate(claims, 1):
        print(f"  {i}. {c}")

    all_results = []
    # Iterate through each extracted claim and perform a single fact-check
    for i, claim in enumerate(claims):
        print(f"\n{'='*50}\nFact-checking Claim {i+1} of {len(claims)}: '{claim}'")
        single_claim_result = fact_check_single_claim(claim)
        all_results.append(single_claim_result)
        print(f"{'='*50}")

    # After all claims are checked, print a summary of all results
    print("\n\n--- Summary of All Fact-Checking Results ---\n")
    for result in all_results:
        print(f"Claim: {result['claim']}")
        print(f"Verdict: {result['verdict'].upper()}")
        print(f"Reason: {result['reason']}")
        if result['sources']:
            print("Sources:")
            for s in result['sources']:
                print(f"  • {s}")
        print("\n" + "-"*50 + "\n")

    return all_results

print("Full fact-checking pipeline ready")

步骤 8：直接从 URL 进行事实核查

最后，为了让事实核查器更加易用，添加一个直接接受 URL 的函数。

import requests
from bs4 import BeautifulSoup

def extract_claims_from_url(url: str, max_claims: int = 8) -> list[str]:
    """
    Extracts atomic factual claims from the main content of a given URL.
    Fetches content using requests/BeautifulSoup and uses Cerebras LLM for claim extraction.
    """
    print(f"Fetching content from URL: {url}")
    try:
        # Fetch the content of the URL
        response = requests.get(url, timeout=10)
        response.raise_for_status()
        soup = BeautifulSoup(response.text, 'html.parser')

        # Attempt to find the main content by looking for 'article' or 'main' tags
        main_content_div = soup.find('article') or soup.find('main')
        if main_content_div:
            main_text = ' '.join([p.get_text() for p in main_content_div.find_all('p')])
        else:
            main_text_elements = soup.find_all(['p', 'h1', 'h2', 'h3'])
            main_text = ' '.join([elem.get_text() for elem in main_text_elements])

        # Check if enough text was extracted
        if not main_text or len(main_text.strip()) < 100:
            print(f"Warning: Not enough main text found for URL: {url}")
            return []

        print(f"Extracted {len(main_text)} characters from the URL. Now extracting claims...")
        # Use the LLM to extract claims from the cleaned text
        claims = extract_claims_from_text(main_text, max_claims=max_claims)
        return claims
    except requests.exceptions.RequestException as e:
        print(f"Error fetching content from URL {url}: {e}")
        return []
    except Exception as e:
        print(f"Error processing URL {url}: {e}")
        return []

print("URL claim extraction function ready")

示例

先从一段简短的示例文本开始。

sample_text = """\nThe Earth is flat and the moon is made of cheese. Humans landed on Mars in 1969. Albert Einstein was born in Germany in 1879.\n"""

print("Fact-checking the following text:\n")
print(sample_text)

fact_check_results = fact_check_text(sample_text)

display(fact_check_results)

现在，粘贴一段400字的陈述，看看事实核查器会怎么说。

[注意：这是一个复合文本示例，旨在验证内容事实核查器。它包含看似合理但虚构的声明。]

long_sample_text = """
In recent months, a number of widely shared posts and articles have circulated online making bold claims about technology, science, and public health. One viral thread asserted that Apple released the world’s first smartphone in 1992, long before the launch of the iPhone. The post claimed the device had a touchscreen, mobile internet capabilities, and even early forms of voice control. In reality, Apple did not release a smartphone in 1992, and the first widely recognized smartphone, the IBM Simon, was introduced in 1994 with far more limited features. The iPhone, launched in 2007, is credited with defining the modern smartphone era.

Another widely repeated claim stated that Mount Everest has shrunk by more than 500 meters due to rapid climate change. Several posts argued that melting ice and tectonic shifts had dramatically reduced the mountain’s height, supposedly confirmed by new satellite imagery. Geologists and survey data contradict this, showing that Everest’s height has changed only minimally over time. Recent revisions to Everest’s official height reflect improved measurement technology—not catastrophic geological change or the environmental collapse suggested online.

A sensational article suggested that NASA announced Earth will experience 15 days of complete darkness in November 2025 because of a rare planetary alignment. This claim resurfaces every few years in slightly different forms, yet NASA has consistently debunked every version of it. Astronomers explain that no known configuration of planets could block sunlight from reaching Earth for even a single day, let alone two weeks.

Another persistent piece of misinformation claimed that COVID-19 vaccines contain microchips designed for government tracking. Public health organizations worldwide have addressed this rumor repeatedly, stating unequivocally that no such technology exists in vaccines and that microelectronics cannot function or survive in biological environments in the way conspiracy theories suggest. Despite extensive scientific communication, this claim continues to spread across certain corners of the internet.

More recently, a trending health blog claimed that drinking eight cups of coffee per day reduces the risk of heart disease by 70%. While moderate coffee consumption has been studied for potential health benefits, no reputable research supports the exaggerated 70% figure promoted in the article. Excessive caffeine intake can create health concerns for many individuals, including increased heart rate, anxiety, and disrupted sleep.

In the tech sector, several posts gained traction by asserting that electric vehicles routinely explode in temperatures above 80 degrees Fahrenheit. Critics use this claim to argue that EVs pose unique safety threats. However, investigations by fire departments, insurance groups, and automotive engineers show no evidence of spontaneous combustion linked to moderate ambient temperatures. Vehicle fires—when they do occur—typically result from accidents, mechanical failures, or battery punctures, not temperature alone.

Another claim circulating widely suggests that major tech companies are secretly restricting home Wi-Fi speeds to force consumers into new subscription tiers. Internet service providers and independent network analysts have found no support for this, noting that slowdowns are far more commonly caused by outdated hardware, overcrowded networks, or poor signal placement within the home.

"""

print("Fact-checking the following longer text:\n")
print(long_sample_text[:500] + ('...' if len(long_sample_text) > 500 else ''))

long_fact_check_results = fact_check_text(long_sample_text)

display(long_fact_check_results)

直接粘贴一个 URL 链接。

current_doc_url = "https://www.snopes.com/fact-check/drinking-at-disney-world/"
print(f"Extracting and fact-checking claims from: {current_doc_url}")

url_extracted_claims = extract_claims_from_url(current_doc_url)

if url_extracted_claims:
    print(f"\nSuccessfully extracted {len(url_extracted_claims)} claims from the URL. Now fact-checking them...")
    claims_text_for_fact_check = "\n".join(url_extracted_claims)
    url_fact_check_results = fact_check_text(claims_text_for_fact_check)
    display(url_fact_check_results)
else:
    print("Could not extract claims from the URL to fact-check.")

这里还有一个带 URL 的示例。

article_url = "https://theonion.com/shedeur-sanders-confident-he-can-deliver-everything-browns-fans-have-come-to-expect/"

print(f"Extracting and fact-checking claims from: {article_url}")

claims_from_url = extract_claims_from_url(article_url)

if claims_from_url:
    print(f"\nSuccessfully extracted {len(claims_from_url)} claims from the URL. Now fact-checking them...")

    claims_text_for_fact_check = "\n".join(claims_from_url)

    fact_check_results = fact_check_text(claims_text_for_fact_check)

    display(fact_check_results)
else:
    print("Could not extract claims from the URL to fact-check.")

就这样，你成功使用 gpt-oss-120B、Cerebras 和 Parallel 构建了一个事实核查器！

⚠️ 免责声明：

本指南仅作为教育性入门材料。为简化内容，此处代码省略了若干生产环境注意事项，如提示注入、输入清理及更严格的输出验证。若你决定将其用于实际应用，请务必添加这些防护措施。

贡献者

本指南由 OpenAI、Cerebras Systems 和 Parallel Web Systems 联合编写，感谢以下人员的宝贵反馈和支持。

常见问题

常见错误包括：API密钥未设置或无效，导致客户端初始化失败；网络请求超时或返回错误，需检查URL可访问性；LLM输出非JSON格式，导致解析失败，代码中已包含清理和异常处理；搜索结果为空或证据不足，导致判定为不确定。排查方法：检查环境变量、打印原始输出、增加重试机制。生产环境优化：添加输入清理和提示注入防护，使用异步调用提高吞吐量，缓存搜索结果，以及实现更严格的输出验证。

企业应用扩展

可扩展方向：1) 支持更多模型（如GPT-4、Claude）作为备选；2) 增加多语言支持；3) 集成数据库存储核查结果；4) 开发Web界面或API服务；5) 添加批量处理能力；6) 实现更复杂的证据聚合和矛盾检测算法；7) 引入用户反馈机制以持续改进判定准确性。

总结

本教程指导开发者构建一个自动化事实核查系统，利用Cerebras的超低延迟推理和Parallel的搜索API，结合OpenAI的gpt-oss-120B模型，从文本或URL中提取声明、检索证据并给出真/假/不确定的判定。教程涵盖环境配置、LLM调用、网络搜索、声明提取、证据整理、单条声明核查及完整流程，并提供示例和注意事项。

内容来源与验证

继续系统学习

站内还有更多完整实战教程

加入会员，解锁全部教程、完整代码与持续更新内容。

加入会员
查看全部RAG 与知识库教程