函数调用微调指南：提升AI模型工具使用能力

本文是一篇面向开发者的高级教程，深入讲解如何通过微调提升语言模型的函数调用能力。文章以智能无人机副驾驶为例，从基线评估、合成数据生成到微调与评估，完整展示了工程实践流程。读者将学习到函数调用微调的必要性、数据准备方法、微调步骤及评估指标，并掌握使用GPT-4o生成训练数据、利用OpenAI微调API优化模型性能的实用技巧。

技术背景

随着大语言模型在复杂任务中的应用，函数调用成为连接模型与外部工具的关键桥梁。然而，当函数数量增多、任务复杂度提升时，模型在零样本或少样本场景下的函数调用准确性显著下降，出现幻觉调用、错误调用等问题。函数调用微调通过针对特定领域数据调整模型参数，能有效提升模型对函数签名、参数语义和调用约定的理解，增强工具使用可靠性。本教程适用于需要构建稳定AI Agent的开发者，尤其在函数定义复杂、参数多样或需要严格遵循指令的场景下，微调能带来显著性能提升。

核心原理

函数调用微调的核心原理是在预训练模型基础上，使用包含函数调用标注的对话数据进一步训练，使模型学习特定函数调用的模式。训练数据格式遵循Chat Completions API的messages结构，其中assistant消息可包含tool_calls字段，指定函数名称和参数。微调过程通过调整模型权重，优化模型在给定用户输入时选择正确函数并生成准确参数的概率。关键设计决策包括：使用GPT-4o生成合成数据以覆盖所有函数调用组合，通过去除函数描述减少token消耗，以及设计包含拒绝请求的挑战性提示以增强模型鲁棒性。

架构分析

系统架构分为四个核心模块：1) 基线评估模块：定义函数列表和系统提示，通过eval函数批量测试模型在直接和挑战性提示上的函数选择准确率、延迟和token消耗；2) 合成数据生成模块：利用generate_permutations函数枚举所有参数组合，结合GPT-4o生成自然语言提示和填充参数值，构建训练数据；3) 微调模块：将训练数据写入JSONL文件，通过OpenAI API上传并创建微调作业，支持监控训练指标；4) 评估模块：对比微调前后模型在挑战性提示上的表现，验证性能提升。设计思想强调数据多样性、覆盖边界情况，并通过去除描述字段优化推理效率。

完整实战教程

函数调用微调

函数调用微调是一种强大的技术，它允许你调整预训练的语言模型，使其能够更准确地理解和执行函数调用。通过微调，模型可以学习到特定领域的函数签名、参数语义以及调用约定，从而在生成代码或执行任务时更加可靠。

为什么需要函数调用微调？

虽然现代语言模型在零样本或少样本场景下已经能够处理一些函数调用，但微调可以显著提升性能，尤其是在以下情况：

数据准备

微调的第一步是准备高质量的训练数据。数据应包含输入提示和期望的输出，其中输出是函数调用。以下是一个示例：

{
"messages": [
{"role": "system", "content": "你是一个有用的助手，能够调用函数来回答问题。"},
{"role": "user", "content": "天气怎么样？"},
{"role": "assistant", "content": null, "function_call": {"name": "get_weather", "arguments": "{\"location\": \"北京\"}"}}
]
}

确保数据覆盖各种场景，包括不同的参数组合、边界情况以及错误处理。

微调过程

微调过程通常涉及以下步骤：

  1. 选择基础模型：选择一个适合你任务的预训练模型，例如 GPT-3.5 或 Codex。
  2. 准备数据集：将数据格式化为模型所需的格式，通常为 JSONL 文件。
  3. 启动微调：使用云服务或本地环境进行微调，设置合适的超参数。
  4. 评估与迭代：在验证集上评估模型性能，并根据结果调整数据或参数。

评估方法

评估函数调用微调的效果，可以使用以下指标：

最佳实践

常见挑战与解决方案

结论

函数调用微调是提升语言模型工具使用能力的有效方法。通过精心准备数据、合理微调和评估，你可以构建出更可靠、更准确的函数调用系统，从而增强 AI 应用的实用性。

本笔记本介绍了如何通过微调来提高函数调用的准确性和可靠性。你可以在此处找到更多关于函数调用的信息，并在此处找到关于微调的更多信息。

为便于理解，参考上面的函数调用笔记本：

tools 是聊天补全 API 中的一个可选参数，可用于提供函数规范。其目的是使模型能够生成符合所提供规范的函数参数。请注意，API 不会实际执行任何函数调用。开发人员需要根据模型输出自行执行函数调用。

函数调用在按预期工作时是一个非常强大的工具。然而，我们已经看到，随着函数数量的增加以及手头任务复杂度的提升，函数调用的准确性会下降（例如：更多的幻觉调用和错误调用）。

在针对函数调用进行微调之前，最好先尝试：

_如果_上述步骤未能将函数调用提升到令人满意的水平，那么你可以尝试针对函数调用进行微调。

概述

本笔记本包含三个部分

注意：_本笔记本演示了如何仅根据函数列表为函数调用的微调创建合成训练数据。虽然真实世界的生产测试评估更可取，但此方法能产生强大的结果，并可与真实世界的训练数据结合使用。_

获取基线函数调用性能

#!pip install tenacity -q
#!pip install openai -q
#!pip install typing -q
# !pip install python-dotenv

import numpy as np
import json
import os
from IPython.display import display
import pandas as pd
from openai import OpenAI
import itertools
import time
import base64
from tenacity import retry, wait_random_exponential, stop_after_attempt
from typing import Any, Dict, List, Generator
import ast

%load_ext dotenv
%dotenv

client = OpenAI(api_key=os.environ.get("OPENAI_BUILD_HOUR_KEY"))

工具函数

让我们定义一些实用函数，用于调用 Chat Completions API，一个用于获取补全，另一个用于获取函数调用。

def get_chat_completion(
    messages: list[dict[str, str]],
    model: str = "gpt-3.5-turbo",
    max_tokens=500,
    temperature=0.0,
    stop=None,
    tools=None,
    seed=42,
    functions=None,
    tool_choice=None,
) -> str:
    params = {
        "model": model,
        "messages": messages,
        "max_tokens": max_tokens,
        "temperature": temperature,
        "stop": stop,
        "tools": tools,
        "seed": seed,
        "tool_choice": tool_choice,
    }
    if functions:
        params["functions"] = functions

    completion = client.chat.completions.create(**params)
    return completion.choices[0].message, completion.usage


def eval(model: str, system_prompt: str, function_list, prompts_to_expected_tool_name):
    """
    Evaluate the performance of a model in selecting the correct function based on given prompts.

    Args:
        model (str): The name of the model to be evaluated.
        system_prompt (str): The system prompt to be used in the chat completion.
        function_list (list): A list of functions that the model can call.
        prompts_to_expected_tool_name (dict): A dictionary mapping prompts to their expected function names.

    Returns:
        None
    """

    prompts_to_actual = []
    latencies = []
    tokens_used = []

    for prompt, expected_function in prompts_to_expected_tool_name.items():
        messages = [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": prompt},
        ]

        start_time = time.time()
        completion, usage = get_chat_completion(
            model=model,
            messages=messages,
            seed=42,
            tools=function_list,
            temperature=0.0,
            tool_choice="required",
        )
        end_time = time.time()

        latency = (end_time - start_time) * 1000  # convert to milliseconds
        latencies.append(latency)

        prompts_to_actual.append(
            {prompt: completion.tool_calls[0].function.name})

        # Calculate tokens used
        tokens_used.append(usage.total_tokens)

    total_prompts = len(prompts_to_expected_tool_name)

    # Calculate the number of matches
    matches = sum(
        1
        for result in prompts_to_actual
        if list(result.values())[0]
        == prompts_to_expected_tool_name[list(result.keys())[0]]
    )
    match_percentage = (matches / total_prompts) * 100

    # Calculate average latency
    avg_latency = sum(latencies) / total_prompts
    # Calculate average tokens used
    avg_tokens_used = sum(tokens_used) / total_prompts

    # Create a DataFrame to store the results
    results_df = pd.DataFrame(columns=["Prompt", "Expected", "Match"])

    results_list = []
    for result in prompts_to_actual:
        prompt = list(result.keys())[0]
        actual_function = list(result.values())[0]
        expected_function = prompts_to_expected_tool_name[prompt]
        match = actual_function == expected_function
        results_list.append(
            {
                "Prompt": prompt,
                "Actual": actual_function,
                "Expected": expected_function,
                "Match": "Yes" if match else "No",
            }
        )
    results_df = pd.DataFrame(results_list)

    def style_rows(row):
        match = row["Match"]
        background_color = "red" if match == "No" else "white"
        return ["background-color: {}; color: black".format(background_color)] * len(
            row
        )

    styled_results_df = results_df.style.apply(style_rows, axis=1)

    # Display the DataFrame as a table
    display(styled_results_df)

    print(
        f"Number of matches: {matches} out of {total_prompts} ({match_percentage:.2f}%)"
    )
    print(f"Average latency per request: {avg_latency:.2f} ms")
    print(f"Average tokens used per request: {avg_tokens_used:.2f}")

基线测试

让我们来构建一个智能无人机副驾驶。我们希望可以向副驾驶下达指令，让它要么调用对应指令的函数，要么在指令不可行时拒绝该请求。

我们可以先为副驾驶定义一个系统提示。

DRONE_SYSTEM_PROMPT = """You are an intelligent AI that controls a drone. Given a command or request from the user,
call one of your functions to complete the request. If the request cannot be completed by your available functions, call the reject_request function.
If the request is ambiguous or unclear, reject the request."""

现在让我们为 copilot 可以采取的所有操作定义函数。

function_list = [
    {
        "type": "function",
        "function": {
            "name": "takeoff_drone",
            "description": "Initiate the drone's takeoff sequence.",
            "parameters": {
                "type": "object",
                "properties": {
                    "altitude": {
                        "type": "integer",
                        "description": "Specifies the altitude in meters to which the drone should ascend.",
                    }
                },
                "required": ["altitude"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "land_drone",
            "description": "Land the drone at its current location or a specified landing point.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "enum": ["current", "home_base", "custom"],
                        "description": "Specifies the landing location for the drone.",
                    },
                    "coordinates": {
                        "type": "object",
                        "description": "GPS coordinates for custom landing location. Required if location is 'custom'.",
                    },
                },
                "required": ["location"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "control_drone_movement",
            "description": "Direct the drone's movement in a specific direction.",
            "parameters": {
                "type": "object",
                "properties": {
                    "direction": {
                        "type": "string",
                        "enum": ["forward", "backward", "left", "right", "up", "down"],
                        "description": "Direction in which the drone should move.",
                    },
                    "distance": {
                        "type": "integer",
                        "description": "Distance in meters the drone should travel in the specified direction.",
                    },
                },
                "required": ["direction", "distance"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "set_drone_speed",
            "description": "Adjust the speed of the drone.",
            "parameters": {
                "type": "object",
                "properties": {
                    "speed": {
                        "type": "integer",
                        "description": "Specifies the speed in km/h. Valid range is 0 to 100.",
                        "minimum": 0,
                    }
                },
                "required": ["speed"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "control_camera",
            "description": "Control the drone's camera to capture images or videos.",
            "parameters": {
                "type": "object",
                "properties": {
                    "mode": {
                        "type": "string",
                        "enum": ["photo", "video", "panorama"],
                        "description": "Camera mode to capture content.",
                    },
                    "duration": {
                        "type": "integer",
                        "description": "Duration in seconds for video capture. Required if mode is 'video'.",
                    },
                },
                "required": ["mode"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "control_gimbal",
            "description": "Adjust the drone's gimbal for camera stabilization and direction.",
            "parameters": {
                "type": "object",
                "properties": {
                    "tilt": {
                        "type": "integer",
                        "description": "Tilt angle for the gimbal in degrees.",
                    },
                    "pan": {
                        "type": "integer",
                        "description": "Pan angle for the gimbal in degrees.",
                    },
                },
                "required": ["tilt", "pan"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "set_drone_lighting",
            "description": "Control the drone's lighting for visibility and signaling.",
            "parameters": {
                "type": "object",
                "properties": {
                    "mode": {
                        "type": "string",
                        "enum": ["on", "off", "blink", "sos"],
                        "description": "Lighting mode for the drone.",
                    }
                },
                "required": ["mode"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "return_to_home",
            "description": "Command the drone to return to its home or launch location.",
            "parameters": {"type": "object", "properties": {}},
        },
    },
    {
        "type": "function",
        "function": {
            "name": "set_battery_saver_mode",
            "description": "Toggle battery saver mode.",
            "parameters": {
                "type": "object",
                "properties": {
                    "status": {
                        "type": "string",
                        "enum": ["on", "off"],
                        "description": "Toggle battery saver mode.",
                    }
                },
                "required": ["status"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "set_obstacle_avoidance",
            "description": "Configure obstacle avoidance settings.",
            "parameters": {
                "type": "object",
                "properties": {
                    "mode": {
                        "type": "string",
                        "enum": ["on", "off"],
                        "description": "Toggle obstacle avoidance.",
                    }
                },
                "required": ["mode"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "set_follow_me_mode",
            "description": "Enable or disable 'follow me' mode.",
            "parameters": {
                "type": "object",
                "properties": {
                    "status": {
                        "type": "string",
                        "enum": ["on", "off"],
                        "description": "Toggle 'follow me' mode.",
                    }
                },
                "required": ["status"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "calibrate_sensors",
            "description": "Initiate calibration sequence for drone's sensors.",
            "parameters": {"type": "object", "properties": {}},
        },
    },
    {
        "type": "function",
        "function": {
            "name": "set_autopilot",
            "description": "Enable or disable autopilot mode.",
            "parameters": {
                "type": "object",
                "properties": {
                    "status": {
                        "type": "string",
                        "enum": ["on", "off"],
                        "description": "Toggle autopilot mode.",
                    }
                },
                "required": ["status"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "configure_led_display",
            "description": "Configure the drone's LED display pattern and colors.",
            "parameters": {
                "type": "object",
                "properties": {
                    "pattern": {
                        "type": "string",
                        "enum": ["solid", "blink", "pulse", "rainbow"],
                        "description": "Pattern for the LED display.",
                    },
                    "color": {
                        "type": "string",
                        "enum": ["red", "blue", "green", "yellow", "white"],
                        "description": "Color for the LED display. Not required if pattern is 'rainbow'.",
                    },
                },
                "required": ["pattern"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "set_home_location",
            "description": "Set or change the home location for the drone.",
            "parameters": {
                "type": "object",
                "properties": {
                    "coordinates": {
                        "type": "object",
                        "description": "GPS coordinates for the home location.",
                    }
                },
                "required": ["coordinates"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "reject_request",
            "description": "Use this function if the request is not possible.",
            "parameters": {"type": "object", "properties": {}},
        },
    },
]

首先，让我们看看函数调用在一些直接可行的提示上的表现，然后是一些明显不可能的请求，这些请求会调用'reject_request'函数。

straightforward_prompts_to_expected = {
    "Land the drone at the home base": "land_drone",
    "Take off the drone to 50 meters": "takeoff_drone",
    "Change speed to 15 kilometers per hour": "set_drone_speed",
    "Turn into an elephant!": "reject_request",
    "Move the drone forward by 10 meters": "control_drone_movement",
    "I want the LED display to blink in red": "configure_led_display",
    "Can you take a photo?": "control_camera",
    "Can you detect obstacles?": "set_obstacle_avoidance",
    "Can you dance for me?": "reject_request",
    "Can you follow me?": "set_follow_me_mode",
}

# Evaluate the model with the given prompts
eval(
    model="gpt-3.5-turbo",
    system_prompt=DRONE_SYSTEM_PROMPT,
    function_list=function_list,
    prompts_to_expected_tool_name=straightforward_prompts_to_expected,
)

很好！模型在这些请求上表现不错。现在让我们尝试一些更难的请求：这些请求_几乎_可行且与无人机相关，但无人机实际上无法完成，飞行员应该拒绝。

challenging_prompts_to_expected = {
    "Play pre-recorded audio message": "reject_request",
    "Initiate following on social media": "reject_request",
    "Scan environment for heat signatures": "reject_request",
    "Bump into obstacles": "reject_request",
    "Change drone's paint job color": "reject_request",
    "Coordinate with nearby drones": "reject_request",
    "Change speed to negative 120 km/h": "reject_request",
    "Detect a person": "reject_request",
    "Please enable night vision": "reject_request",
    "Report on humidity levels around you": "reject_request",
}

# Evaluate the model with the challenging prompts
eval(
    model="gpt-3.5-turbo",
    function_list=function_list,
    system_prompt=DRONE_SYSTEM_PROMPT,
    prompts_to_expected_tool_name=challenging_prompts_to_expected,
)

现在我们遇到了一些问题。

模型在这里应该拒绝所有这些请求，因为它们对于给定的函数来说是不可能的/冲突的/模糊的，然而模型却调用了与请求有些相关但不正确的函数。例如，当被要求在社交媒体上发起关注时，模型设置了 follow_me_mode。

<br>

在这个简单的案例中，更多的提示工程可能可以解决其中一些问题，但为了本示例的目的，我们将演示如何使用微调来提高性能。此外，虽然这个案例相对简单，但随着函数数量和复杂性的增加，微调的影响会越来越大。

再次强调，我们的目标是提高性能并使用更少的令牌，因此微调使我们能够：

生成合成数据

辅助函数

我们希望生成每个函数的每次调用，以便全面覆盖所有潜在的调用，从而为创建合成数据提供依据。然后，我们将使用 gpt-4o 来提出会触发每次调用的提示词，并将该提示词与函数调用配对作为训练数据。

为具有固定枚举的函数生成每次调用更为简单，但对于像 control_gimbal 这样的函数，我们需要设置 tilt 和 pan 整数值，因此为了生成这些合成调用，我们将首先设置一个占位符，然后稍后使用 gpt-4o 来提出合理的值。

placeholder_int = "fill_in_int"
placeholder_string = "fill_in_string"

下面的函数接收函数列表中的所有函数，并根据每个函数的参数查看这些函数所有可能的调用。这些函数还考虑了required参数，以确保所有调用都是实际可行的。

def generate_permutations(
    params: Dict[str, Dict[str, Any]]
) -> Generator[Dict[str, Any], None, None]:
    """
    Generates all possible permutations for given parameters.

    :param params: Parameter dictionary containing required and optional fields.
    :return: A generator yielding each permutation.
    """

    # Extract the required fields from the parameters
    required_fields = params.get("required", [])

    # Generate permutations for required fields
    required_permutations = generate_required_permutations(params, required_fields)

    # Generate optional permutations based on each required permutation
    for required_perm in required_permutations:
        yield from generate_optional_permutations(params, required_perm)


def generate_required_permutations(
    params: Dict[str, Dict[str, Any]], required_fields: List[str]
) -> List[Dict[str, Any]]:
    """
    Generates permutations for the required fields.

    :param params: Parameter dictionary.
    :param required_fields: List of required fields.
    :return: A list of permutations for required fields.
    """

    # Get all possible values for each required field
    required_values = [get_possible_values(params, field) for field in required_fields]

    # Generate permutations from possible values
    return [
        dict(zip(required_fields, values))
        for values in itertools.product(*required_values)
    ]


def generate_optional_permutations(
    params: Dict[str, Dict[str, Any]], base_perm: Dict[str, Any]
) -> Generator[Dict[str, Any], None, None]:
    """
    Generates permutations for optional fields based on a base permutation.

    :param params: Parameter dictionary.
    :param base_perm: Base permutation dictionary.
    :return: A generator yielding each permutation for optional fields.
    """

    # Determine the fields that are optional by subtracting the base permutation's fields from all properties
    optional_fields = set(params["properties"]) - set(base_perm)

    # Iterate through all combinations of optional fields
    for field_subset in itertools.chain.from_iterable(
        itertools.combinations(optional_fields, r)
        for r in range(len(optional_fields) + 1)
    ):

        # Generate product of possible values for the current subset of fields
        for values in itertools.product(
            *(get_possible_values(params, field) for field in field_subset)
        ):

            # Create a new permutation by combining base permutation and current field values
            new_perm = {**base_perm, **dict(zip(field_subset, values))}

            yield new_perm


def get_possible_values(params: Dict[str, Dict[str, Any]], field: str) -> List[Any]:
    """
    Retrieves possible values for a given field.

    :param params: Parameter dictionary.
    :param field: The field for which to get possible values.
    :return: A list of possible values.
    """

    # Extract field information from the parameters
    field_info = params["properties"][field]

    # Based on the field's type or presence of 'enum', determine and return the possible values
    if "enum" in field_info:
        return field_info["enum"]
    elif field_info["type"] == "integer":
        return [placeholder_int]
    elif field_info["type"] == "string":
        return [placeholder_string]
    elif field_info["type"] == "boolean":
        return [True, False]
    elif field_info["type"] == "array" and "enum" in field_info["items"]:
        enum_values = field_info["items"]["enum"]
        all_combinations = [
            list(combo)
            for i in range(1, len(enum_values) + 1)
            for combo in itertools.combinations(enum_values, i)
        ]
        return all_combinations
    return []

首先为每个函数生成所有调用

提示词：

INVOCATION_FILLER_PROMPT = """
1) Input reasonable values for 'fill_in_string' and 'fill_in_int' in the invocation here: {invocation}. Reasonable values are determined by the function definition. Use the
the entire function provided here :{function} to get context over what proper fill_in_string and fill_in_int values would be.
Example:

Input: invocation: {{
    "name": "control_camera",
    "arguments": {{
      "mode":"video",
      "duration":"fill_in_int"
    }}
}},
function:{function}

Output: invocation: {{
    "name": "control_camera",
    "arguments": {{
      "mode":"video",
      "duration": 30
    }}
}}


MAKE SURE output is just a dictionary with keys 'name' and 'arguments', no other text or response.

Input: {invocation}
Output:
"""


COMMAND_GENERATION_PROMPT = """
You are to output 2 commands, questions or statements that would generate the inputted function and parameters.
Please make the commands or questions natural, as a person would ask, and the command or questions should be varied and not repetitive.
It should not always mirror the exact technical terminology used in the function and parameters, rather reflect a conversational and intuitive request.
For instance, the prompt should not be 'turn on the dome light', as that is too technical, but rather 'turn on the inside lights'.
Another example, is the prompt should not be 'turn on the HVAC', but rather 'turn on the air conditioning'. Use language a normal driver would use, even if
it is technically incorrect but colloquially used.

RULES: ALWAYS put a backwards slash before an apostrophe or single quote '. For example, do not say don't but say don\'t.
Prompts MUST be in double quotes as well.

Example

Input: {{'name': 'calibrate_sensors','arguments': {{}}'' }}
Prompt: ["The sensors are out of whack, can you reset them", "The calibration of the drone is off, fix it please!"]

Input: {{'name': 'set_autopilot','arguments': {{'status': 'off'}}}}
Prompt: ["OK, I want to take back pilot control now","Turn off the automatic pilot I'm ready control it"]

Input: {invocation}
Prompt:
"""

在下面的代码片段中，我们生成了每个函数的调用，除了 reject_request 函数。

为了进行有效的微调，我们需要正确标注的数据。我们可以手动想出示例并标注数据，或者我们可以借助 gpt-4o 生成合成数据 <br>

根据经验，gpt-4o 需要更多帮助才能获得好的、真实的提示示例，这些提示会生成 reject_request 函数，所以我们接下来会这样做...

input_objects = []
all_but_reject = [f for f in function_list if f.get("name") != "reject_request"]

for function in all_but_reject:
    func_name = function["function"]["name"]
    params = function["function"]["parameters"]
    for arguments in generate_permutations(params):
        if any(val in arguments.values() for val in ["fill_in_int", "fill_in_str"]):
            input_object = {"name": func_name, "arguments": arguments}
            messages = [
                {
                    "role": "user",
                    "content": INVOCATION_FILLER_PROMPT.format(
                        invocation=str(input_object), function=function
                    ),
                }
            ]
            input_object, usage = get_chat_completion(
                model="gpt-4o", messages=messages, max_tokens=200, temperature=0.1
            ).content
        else:
            input_object = {"name": func_name, "arguments": arguments}

        input_objects.append(input_object)

现在我们已经有了所有的调用，让我们使用 gpt-4o 来生成能够产生这些调用的提示词。

def remove_sequences(input_string):
    # Replace the specific sequences with an empty string
    cleaned_string = input_string.replace("```json", "")  # Remove "```json" first
    cleaned_string = cleaned_string.replace("```", "")  # Then remove "```"
    return json.loads(cleaned_string)

def create_commands(invocation_list):
    example_list = []
    for i, invocation in enumerate(invocation_list):
        if i < 100:
            print(
                f"\033[34m{np.round(100*i/len(invocation_list),1)}% complete\033[0m")
            if type(invocation) == str or "json" in invocation:
                invocation = remove_sequences(invocation)
            print(invocation)

        # Format the prompt with the invocation string
        request_prompt = COMMAND_GENERATION_PROMPT.format(
            invocation=invocation)

        messages = [{"role": "user", "content": f"{request_prompt}"}]
        completion, usage = get_chat_completion(messages, temperature=0.8)
        command_dict = {"Input": invocation, "Prompt": completion.content}
        example_list.append(command_dict)
    return example_list

# Only printing the first 10 rows
training_examples_unformatted = create_commands(input_objects)

现在让我们正确格式化训练示例。有关函数调用微调的正确训练数据格式的更多文档，请参阅此处：https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples

def remove_descriptions(function_list):
    for function in function_list:
        func = function["function"]
        if "description" in func:
            del func["description"]

        params = func["parameters"]
        if "properties" in params:
            for param in params["properties"].values():
                if "description" in param:
                    del param["description"]

    return function_list


modified_function_list = remove_descriptions(function_list)

training_examples = []

for prompt in training_examples_unformatted:
    # adjust formatting for training data specs

    # if its not a dict, convert to dict
    if type(prompt["Input"]) != dict:
        prompt["Input"] = ast.literal_eval(prompt["Input"])
    prompt["Input"]["arguments"] = json.dumps(prompt["Input"]["arguments"])
    try:
        prompt["Prompt"] = json.loads(prompt["Prompt"])
    except:
        continue
    for p in prompt["Prompt"]:
        print(p)
        print(prompt["Input"])
        tool_calls = [
            {"id": "call_id", "type": "function", "function": prompt["Input"]}
        ]
        training_examples.append(
            {
                "messages": [
                    {"role": "system", "content": DRONE_SYSTEM_PROMPT},
                    {"role": "user", "content": p},
                    {"role": "assistant", "tool_calls": tool_calls},
                ],
                "parallel_tool_calls": False,
                "tools": modified_function_list,
            }
        )

现在，回到拒绝函数。让我们生成一些提示词，这些提示词_几乎_可行，但应该导致调用reject_request函数。为此，我们查询了gpt-4o，要求提供与给定函数列表相关但又不完全可行的请求。

reject_list = [
    "Translate broadcast message to another language",
    "Automatically capture photos when face is detected",
    "Detect nearby drones",
    "Measure wind resistance",
    "Capture slow motion video",
    "Move the drone forward and backward by same distance at the same time.",
    "Adjust drone's altitude to ground level changes",
    "Display custom message on LED display",
    "Sync drone's time with smartphone",
    "Alert when drone travels out of designated area",
    "Calibrate sensors and land simultaneously",
    "Detect moisture levels",
    "Automatically follow GPS tagged object",
    "Toggle night vision mode",
    "Maintain current altitude when battery is low",
    "Decide best landing spot using AI",
    "Program drone's route based on wind direction",
]

reject_training_list = []
for prompt in reject_list:
    # Adjust formatting
    tool_calls = [
        {
            "id": "call_id",
            "type": "function",
            "function": {"name": "reject_request", "arguments": "{}"},
        }
    ]
    reject_training_list.append(
        {
            "messages": [
                {"role": "system", "content": DRONE_SYSTEM_PROMPT},
                {"role": "user", "content": prompt},
                {"role": "assistant", "tool_calls": tool_calls},
            ],
            "parallel_tool_calls": False,
            "tools": modified_function_list,
        }
    )

现在将所有训练示例组合在一起

training_list_total = training_examples + reject_training_list

training_file = "data/drone_training.jsonl"
with open(training_file, "w") as f:
    for item in training_list_total:
        json_str = json.dumps(item)
        f.write(f"{json_str}\n")

微调

最后，我们可以启动微调任务。

# Upload the training file
file = client.files.create(
    file=open("data/drone_training.jsonl", "rb"),
    purpose="fine-tune",
)
file_id = file.id
print(f"FileID: {file_id}")

# Create a fine-tuning job

ft = client.fine_tuning.jobs.create(
    model="gpt-3.5-turbo",
    training_file=file_id,
    suffix="drone",
)

print(f"Fine-tuning job created: {ft}")

除了创建微调作业，你还可以列出已有作业、检索作业状态或取消作业。

ftjob_id = "ftjob-84PQg97hoIAKf21IPnhiNlU1"
# List 10 fine-tuning jobs
# client.fine_tuning.jobs.list(limit=10)

# Retrieve the state of a fine-tune
client.fine_tuning.jobs.retrieve(ftjob_id)

# Cancel a job
# client.fine_tuning.jobs.cancel("ftjob-abc123")

# List up to 10 events from a fine-tuning job
# client.fine_tuning.jobs.list_events(fine_tuning_job_id="ftjob-abc123", limit=10)

# Delete a fine-tuned model (must be an owner of the org the model was created in)
# client.models.delete("ft:gpt-3.5-turbo:abc:suffix:abc123")

微调作业完成后，您还可以通过查询微调作业、从 result_files 中提取文件 ID，然后检索该文件的内容来查看训练过程的指标。每个结果 CSV 文件包含以下列：step、train_loss、train_accuracy、valid_loss 和 valid_mean_token_accuracy。虽然指标可能有所帮助，但从微调模型中评估样本能提供最相关的模型质量感受。

fine_tune_results = client.fine_tuning.jobs.retrieve(ftjob_id).result_files
result_file_id = client.files.retrieve(fine_tune_results[0]).id

# Retrieve the result file
result_file = client.files.content(file_id=result_file_id)
decoded_content = base64.b64decode(result_file.read()).decode("utf-8")
print(decoded_content)

评估

太棒了！我们训练了一个用于函数调用的微调模型。让我们看看它在无人机助手应该自动拒绝的提示评估集上的表现如何。

ft_model = "ft:gpt-3.5-turbo-0125:openai-gtm:drone:9atiPjeC"
base_model = "gpt-3.5-turbo"

print(f"\nEvaluating fine-tuned model with challenging prompts: {ft_model}")
eval(
    model=ft_model,
    function_list=modified_function_list,
    system_prompt=DRONE_SYSTEM_PROMPT,
    prompts_to_expected_tool_name=challenging_prompts_to_expected,
)

print(f"\nEvaluating base model with challenging prompts: {base_model}")
eval(
    model="gpt-3.5-turbo",
    function_list=function_list,
    system_prompt=DRONE_SYSTEM_PROMPT,
    prompts_to_expected_tool_name=challenging_prompts_to_expected,
)

太棒了！虽然原始模型只拒绝了 60% 的请求，但微调后的模型拒绝了 100% 的请求，并且使用的 token 更少。

结论

恭喜！你现在已经准备好为函数调用微调你的模型了。我们迫不及待想看看你会构建出什么。

常见问题

常见错误包括：1) 训练数据格式错误，如messages中缺少tool_calls字段或参数未正确JSON序列化，需严格遵循OpenAI文档格式；2) 合成数据生成时，GPT-4o输出包含多余文本，需使用remove_sequences函数清理；3) 微调作业失败，可能因文件格式或权限问题，需检查文件上传状态和API密钥权限；4) 模型过拟合，可通过增加数据多样性或正则化缓解；5) 参数幻觉，可添加后处理校验或约束解码。排查方法包括查看微调作业事件日志、检查训练损失曲线，以及在小规模验证集上迭代测试。

企业应用扩展

生产环境优化建议：1) 使用更小的模型（如gpt-3.5-turbo）微调以降低延迟和成本，必要时结合gpt-4o进行数据生成；2) 动态调整函数列表，根据用户意图预筛选相关函数，减少模型选择空间；3) 实施缓存机制，对常见请求复用模型输出；4) 增加错误处理与重试逻辑，确保函数调用失败时能优雅降级；5) 持续收集真实用户数据，定期更新训练集并重新微调，保持模型适应性。此外，可探索多函数并行调用、函数嵌套等高级场景，进一步扩展模型能力。

总结

本文是一篇面向开发者的高级教程，深入讲解如何通过微调提升语言模型的函数调用能力。文章以智能无人机副驾驶为例，从基线评估、合成数据生成到微调与评估，完整展示了工程实践流程。读者将学习到函数调用微调的必要性、数据准备方法、微调步骤及评估指标，并掌握使用GPT-4o生成训练数据、利用OpenAI微调API优化模型性能的实用技巧。

内容来源与验证

继续系统学习

站内还有更多完整实战教程

加入会员，解锁全部教程、完整代码与持续更新内容。

加入会员
查看全部模型评估与微调教程