駕駛艙智能體工程（2025）

行動：最後一公里任務

自2023年基礎模型安裝到車輛以來，駕駛艙AI助理在每個階段都承擔著不同的任務。到2025年，駕駛艙AI助理將專注於執行特定行動。這意味著“幫助用戶完成任務”，而不是“提供建議”，這標誌著從“助手”轉變為“智能體”的重要一步。

2025年駕駛艙AI助理的一個典型應用場景是在餐廳點餐。

2024年，如果用戶想點咖啡，駕駛艙AI助理只能在地圖上搜尋附近的咖啡店，用戶需要手動選擇和導航。用戶負責下單和付款，人工智慧助理不提供任何協助。

到 2025 年，當用戶訂購咖啡時，駕駛艙人工智慧助理將能夠確認用戶的意圖，並自動完成一系列操作，包括下單和付款。這將消除用戶的干擾，並優化用戶體驗。

整個過程涉及與長期記憶、工具調用和多智能體協作相關的技術。

案例 1：工具呼叫

2024 年初，OpenAI 的函數呼叫成為駕駛艙智能體呼叫工具的主流技術，實現了單一模型與單一工具之間的直接互動。

Anthropic 於 2024 年 11 月推出了模型上下文協定 (MCP)，解決了基於函數呼叫的 "多元件協作" 問題，並提升了函數呼叫的應用場景和效率。

2025年4月，Google提出了A2A（Agent2Agent）協議，旨在進一步規範不同智能體之間的通訊與協作。

例如，李祥同學的2025年智能體應用方案就採用了MCP/A2A技術框架（CUA也可作為獨立框架使用）。

MCP/A2A：IVI智能體作為多智能體系統（MAS）的領導者，將任務分配給第三方智能體，每個智能體完成各自的工作流程。

CUA（Cockpit Using Agent）：作業系統呼叫多模態基礎模型來理解、分解和規劃指令/任務，產生最終動作，然後呼叫小程式或應用程式來完成指令/任務。 例如，在支付場景中，經過一系列理解和規劃後，麗香同學呼叫API連接到支付寶的汽車助手，並透過支付寶生態系統使用相關小程式完成支付。

在訓練過程中，麗香同學團隊利用MCP管理工具服務優化智能體強化學習階段的獎勵模組。具體來說，我們使用MCP Hub提供與訓練任務和業務需求相對應的可呼叫工具資源目錄。

在下一階段，我們計劃增強多態性能力並實現動作鏈（COA）。這意味著同一個模型將持續推理如何呼叫外部工具、解決問題和執行動作，進一步加強不同模組在工具呼叫、推理和動作執行上的協作。

本報告研究分析了中國汽車產業，提供了有關座艙智能體現狀、工程階段和研發技術路線圖以及主要OEM廠商智能體特徵的資訊。

目錄

定義

第一章：座艙代理的現況與發展趨勢

• 座艙代理概述
• 座艙代理應用場景概述
• 座艙代理的現況
• 座艙代理的發展趨勢

第二章：OEM代理解決方案

• 座艙AI代理/AI助理概述（2025）
• 座艙AI代理/AI助理概述（2025）
• 力翔同學
• 蔚來汽車
• 小鵬汽車
• 吉利汽車
• 小米汽車
• 長城汽車電機
• 北汽
• 上汽
• 奇瑞
• 其他

第三章 供應商代理解決方案

• 華為
• 阿里巴巴雲
• 百度雲
• 騰訊雲
• 位元組跳動 & 火山引擎
• 商湯科技
• 智浦人工智慧
• 科大訊飛
• Thundersoft
• Kotei Agent
• 聯想
• TINNOVE

第四章 實用代理技術

• 意圖識別
• 知識圖譜與搜尋
• 情感識別
• 推理加速
• 推薦系統
• 工具調用
• 多智能體系統 (MAS)
• 圖形使用者介面 (GUI)智能體

第五章 智能體應用問題

• 問題 1：邊緣雲部署中運算能力的平衡點
• 問題 2：多智能體系統架構設計（1）
• 問題 2：多智能體系統架構設計（2）
• 問題 2：多智能體系統架構設計（3）- 基礎模型選擇
• 問題 3：商業模式設計
• 問題 4：場景應用有效性
• 問題 5：訓練偏差
• 問題 6：資料隱私

Cockpit Agent Engineering Research: Breakthrough from Digital AI to Physical AI

Cockpit Agent Engineering Research Report, 2025 starts with the status quo of cockpit agents, summarizes the technical roadmap of the R&D and engineering stages and the characteristics of agents from leading OEMs, and predicts the future trends and priorities of cockpit agent application.

Action: Last Mile Mission

Since foundation models were installed in vehicles in 2023, cockpit AI assistants have assumed different tasks at different stage. In 2025, cockpit AI assistants focus on action, which means they "help users get things done" instead of "giving suggestions", marking an important step in the transformation from "assistants" to "agents".

One typical scenario for cockpit AI assistants in 2025 is ordering food at restaurants:

In 2024, when a user wanted to order coffee, a cockpit AI assistant could only find nearby coffee shops on the map for the user to manually select and navigate to, but the ordering and payment were all fulfilled by the user himself/herself while the AI assistant cannot help at all.

By 2025, when a user orders coffee, the cockpit AI assistant will be able to confirm his/her intention and automatically complete a series of operations such as ordering and payment, without the user having to worry about it, thus optimizing the user experience.

The entire process involves technologies related to long-term memory, tool calling, and multi-agent collaboration.

1.Case 1: Tool Calling

In early 2024, OPEN AI's Function Calling was the mainstream technology used by cockpit agents when calling tools, facilitating the direct interaction between a single model and a single tool.

The Model Context Protocol (MCP) introduced by Anthropic in November 2024 addresses the issue of "multi-component collaboration" on basis of Function Calling and improves the application scenarios and efficiency of Function Calling.

In April 2025, Google proposed the A2A (Agent2Agent) protocol to further standardize the communication and collaboration between different agents.

For example, the agent application solution of Lixiang Tongxue in 2025 includes a MCP/A2A technical framework (another framework is CUA):

MCP/A2A: The IVI agent acts as the leader of the multi-agent system (MAS), assigning tasks to third-party agents, which then complete their respective workflows.

CUA (Cockpit Using Agent): The operating system calls a multimodal foundation model to understand instructions/tasks, decomposes and plans them, generates the final action, and calls applets and apps to complete the instructions/tasks. For example, in the payment scenario, after a series of understandings and plans, Lixiang Tongxue calls the API to connect to Alipay's automotive assistant, and uses the relevant applet through Alipay's ecosystem to complete the payment.

During the training process, the team of Lixiang Tongxue uses the MCP management tool service in the Reward module optimization of the agent reinforcement phase, such as using MCP Hub to provide a catalog of callable tool resources for training tasks and business requests.

In the next phase, Lixiang Tongxue plans to strengthen its multimodal capabilities and implement COA (Chain of Action), which means that the same model continuously thinks about how to call external tools to solve problems and take action, further improving the synergy between different modules for tool calling, reasoning and action.

2.Case 2: GUI Agent

A GUI agent (graphical user interface agent) is a specific LLM agent which processes user commands or requests in natural language, understands the current state of the GUI through screenshots or UI element trees, and performs actions that simulate human-computer interaction, thus spanning various software interfaces.

A GUI agent typically includes modules such as operating environment, prompt project, model inference, action, and memory.

GUI agent technology is still far from fully mature, but some OEMs, including Li Auto, Geely, and Xiaomi, have already started to deploy it.

In the ordering scenario aforementioned, Lixiang Tongxue leverages GUI agent technology when selecting a meal package, so that it can operate the screen components automatically without user intervention. The team of Lixiang Tongxue has pointed out that the operation accuracy of the GUI agent will also affect the final action of the CUA framework (because the payment process requires scanning screenshots, which involves the GUI agent). If the accuracy is too low, it may be difficult to guarantee a stable experience for complex tasks such as registering for parking and paying parking fees.

For example, Xiaomi has launched a GUI agent framework "BTL-UI", which uses a Group Relative Policy Optimization (GRPO) algorithm within a Markov decision process (MDP). The agent should receive the current screen state, user commands, and historical interaction records at each time step, and then output a structured BTL response, converting the input multimodal information into a comprehensive output that includes visual attention zones, reasoning processes, and command execution.

Its implementation methods and core technologies include:

Bionic interaction framework: Based on the BTL-UI model, it simulates human visual attention allocation (blinking), logical reasoning (thinking), and precise execution (action), supporting complex multi-step tasks (such as cross-application calls and multimodal interactions).

Automated data generation: It automatically analyzes screenshots, identifies the interface elements most relevant to user commands, and generates high-quality attention annotations for these zones.

BTL reward mechanism: It meticulously evaluates each cognitive stage in between, checking whether the AI correctly identifies the relevant interface elements, performs reasonable logical reasoning, and generates accurate operation instructions.

OEMs are currently transitioning from L2 reasoners to L3 agents, with L3 further divided into four stages.

According to OPEN AI's definition of AGI, Chinese OEMs are currently in the process of transitioning from L2 reasoners to L3 agents. At each different stage, different problems should be solved, with corresponding characteristics:

At present, most OEMs' cockpit AI assistants have delivered "professional services" to a certain extent. The next goal is to achieve "emotional resonance" and overcome the hurdle of "proactive prediction".

For "emotional resonance", NIO offers "Nomi" as a leading player.

In 2025, most AI assistants' emotional chats are implemented primarily through tone changes simulated by TTS technology, terminology from the knowledge base (such as colloquial interjections), and preset emotional scenario workflows. Compared to other cockpit agents, Nomi has two unique advantages:

1.Physical shell: Nomi can materialize more than 200 dynamic expressions through its shell "Nomi Mate" (upgraded to version 3.0 as of November 2025), giving emotional value in the real world. For example, when Nomi interacts with people via voice, it simulates the head movements that occur when people are talking to each other, and simulates the movement of a person's head turning towards the source of a sound when they hear a sound, thus achieving an arc-shaped head turning trajectory.

2.Emotional settings:

In terms of architecture, a dedicated "emotion engine" module is set up. Through three sub-modules, namely "contextual intelligence", "personalized intelligence" and "emotional expression", it uses voice, vision and multimodal perception technologies to achieve contextual arbitration, derive a series of understandings of the current situation, and realize natural human-like reactions in emotional scenarios.

In terms of settings, Nomi can have a personality. Based on the settings, it can perform search associations through a streaming prediction model similar to GPT, exhibiting unique situational responses and providing a personalized experience for each user (such as simulating multiple MBTI personalities, in contrast to Lixiang Tongxue set as ENFJ).

After achieving "proactive prediction," cockpit agents make a breakthrough from digital AI to physical AI.

Starting from L3.5+, generalization has become one of the limiting factors for agents' ability to flexibly cope with multi-scenario tasks. To improve generalization in different scenarios, agents should not only learn policies (what actions to take in a certain state), but also know about dynamic environmental models (how the world will change after performing an action) to make predictions in direct interaction with the environment.

To avoid limitations caused by the shortage of high-quality data, one solution is to learn in a real physical environment to achieve a breakthrough from digital AI to physical AI.

For example, the team of Lixiang Tongxue has found that the effect of data on improving the model's capabilities decreases after using massive Internet data for training the base model, namely the marginal benefit of scaling law in model pre-training declines.

Therefore, the team of Lixiang Tongxue has changed the training method for the next stage. it will focus on the interaction between the model and the physical world. Through reinforcement learning, the model will judge the correctness of the thinking process and accumulate experience and data in the interaction with the environment.

Fei-Fei Li's team from World Labs has proposed "augmented interactive agents," which feature multimodal capabilities with "cross-reality-agnostic" integration and incorporate an emergent mechanism.

In training intelligent agents, Fei-Fei Li's team has introduced an "in-context prompt" or "implicit reward function" to capture key features of expert behavior. The intelligent agents can be trained by physical world behavior data learned from expert demonstrations for task execution. The data is collected by gathering expert demonstrations in the physical world in the form of "state-action pairs".

In 2025, most OEMs chose a multi-agent approach to build their cockpit AI systems. Multi-agent collaboration is also one of the ways to improve the generalization of agents. Through "domain specialization + scenario linkage + group learning", the generalization limitations of existing agents can be broken through from multiple dimensions.

For example, GAC's "Beibi" agent can recognize intent in complex scenarios through multi-agent collaboration based on foundation model intent recognition, tackling the problems of vertical agents like "lack of unified interaction entry and inefficient collaboration". It eliminates the need for users to operate multiple agents separately (such as adjusting navigation and air conditioning individually), thus improving collaboration efficiency. Its principles include:

Build the core intelligent agent: Fine-tune the pre-trained language model using a pre-set dataset related to automotive scenarios (such as vehicle control, navigation, and other instruction records) to obtain an intent recognition model. Then, build an "intent understanding intelligent agent" based on this model, while adding a caching service to improve response speed.

Parse user intent: Receive user commands (such as voice or touch commands), and infer the intent recognition result (including 1-3 intents and their corresponding confidence scores, e.g., "Find a gas station" confidence score 0.85, "Adjust temperature" confidence score 0.9) from the intent understanding agent, and cache the commands and results.

Call collaborative agents: Make collaborative decisions based on the current scenario (such as driving status, weather), call on target agents related to the intent (such as navigation and vehicle control agents) to work together, and receive the action results of each agent.

Arbitrate, feed back and enforce: Arbitrate based on historical confidence scores (the past success rate of the agents) and the current action result; arbitrate based on the intent recognition model when there are no historical scores, and finally feed back the result to the actuation system (such as the IVI or voice broadcast) to complete the operation.

Table of Contents

Definition

1 Status Quo and Trends of Cockpit Agents

• 1.1 Overview of Cockpit Agents
• Definition and Value
• Functional Features and Workflow
• Reference Architecture (1): Classic Module Design Applied
• Reference Architecture (1): Derivative Module Design Applied
• Reference Architecture (1): Derivative Module Design Applied: Functional Module Design Requirements (1)-(2)
• Reference Architecture (2): Multi-Agent System Module Design
• Reference Architecture (2): Multi-Agent System Module Design: Components and Their Functions
• Reference Architecture (2): Multi-Agent System Module Design: Components and Their Features (1)-(8)
• Reference Architecture (2): Multi-Agent System Module Design: Architecture Diagram
• Reference Architecture (3): Agent Architecture Design: By Different Deployment Levels
• Collaboration Mechanism between Cockpit Agents, LLMs and OS
• 1.2 Overview of Cockpit Agent Scenarios
• Classification of Cockpit Agent Application Scenarios (1)
• Classification of Cockpit Agent Application Scenarios (2)
• Typical Agent Scenarios (1): Workflow Decomposition of MAS in Mobility Scenarios (1)-(5)
• Typical Agent Scenarios (2): Workflow Decomposition of MAS in Entertainment Scenarios (1)-(4)
• Typical Agent Scenarios (3): Workflow Decomposition of MAS in Children Scenarios (1)-(2)
• Typical Agent Scenarios (4): Workflow Decomposition of MAS in Emotional Scenarios (1)-(2)
• Typical Agent Scenarios (5): Workflow Decomposition of MAS in Q&A Scenarios (1)-(2)
• Typical Agent Scenarios (6): Workflow Decomposition of MAS in Education Scenarios (1)-(2)
• Typical Agent Scenarios (7): Workflow Decomposition of MAS in Parking Scenarios (1)-(3)
• Typical Agent Scenarios (8): Workflow Decomposition of MAS in Shopping Scenarios
• Typical Agent Scenarios (9): Workflow Decomposition of MAS in Medical Scenarios
• Typical Agent Scenarios (10): Workflow Decomposition of MAS in Office Scenarios (1)-(2)
• Agent Scenario Cases (1)
• Agent Scenario Cases (2)
• Agent Scenario Cases (3)
• 1.3 Status Quo of Cockpit Agents
• Development History of Agents
• OEM Agent Comparison
• Comparison of Three Development Models for Automotive AI Agents: Advantages/Disadvantages
• Comparison of Three Development Models for Automotive AI Agents: Cost
• 1.4 Development Trends of Cockpit Agents
• 5 Levels of AGI: Main Application Issues
• Four Stages of Cockpit Agent Iteration
• Agent Trends (1)
• Agent Trends (2)
• Agent Trends (3)
• Agent Trends (4)
• Agent Trends (5)
• Agent Trends (5): Cases
• Agent Trends (6): Key Goals of L3.5+ Agents: High-Frequency Emergence
• Agent Trends (6): Key Goals of L3.5+ Agents: Emergent Technology Foundation
• Agent Trends (6): Key Goals of L3.5+ Agents: Typical Emergent Scenarios
• Agents with Emergent Capabilities (1): Interactive Agents
• Agents with Emergent Capabilities (2): "Emergence" Mechanisms of Interactive Agents
• Agents with Emergent Capabilities (3): Training Methods of Interactive Agents
• Agents with Emergent Capabilities (4):
• Agents with Emergent Capabilities (5): Two Strategies Accelerate "High-Level Emergence"
• Agents with Emergent Capabilities (6):

2 OEM Agent Solutions

• Overview Diagram of Cockpit AI Agents/AI Assistants in 2025
• Overview Table of Cockpit AI Agent/AI Assistant in 2025
• 2.1 Lixiang Tongxue
• Upgrade to Agent
• Ordering Scenario Analysis
• Payment Scenario Analysis
• Agent Architecture: Two Paths
• R&D Insights (1): Focus of Agent Performance Improvement
• R&D Insights (2): Planning 2.0
• R&D Insights (3): Interactive Scenario Design and Evaluation
• Functional Module Diagram
• Underlying capabilities: Base Model Performance Improvement
• Underlying Capabilities: The Base Model Adds Agent Task Training (1)-(4)
• Underlying Capabilities: Different Paths to Enhance Base Model Capabilities (1)-(6)
• Underlying Capabilities: Base Model Engineering Capability Optimization Solution (1)-(6)
• Underlying Capabilities: Base Model Engineering Capability Optimization Solution - Training Platform
• Underlying Capabilities: Base Model Engineering Capability Optimization Solution - Inference Engine
• Underlying Capabilities: From Models to CUA
• Underlying Capabilities: Base Model Agent Capability Enhancement Solution
• Underlying Capabilities: Full-Modal Foundation Models
• Underlying Capabilities: Application Scenarios of Full-Modal Foundation Models (1): Speech Knowledge Q&A
• Underlying Capabilities: Application Scenarios of Full-Modal Foundation Models (2)
• Underlying Capabilities: Application Scenarios of Full-Modal Foundation Models (3)
• Underlying Capabilities: Model Capability Assessment of Full-Modal Foundation Models (1)-(2)
• Underlying Capabilities: Tool Capability Assessment of Full-Modal Foundation Models (1)-(2)
• 2.2 NIO
• NomiGPT and NomiAgent Deployment Architecture
• Functional Modules of NomiGPT
• Functions of NomiGPT (1): Multimodal Perception
• Functions of NomiGPT (2): Command Distribution
• Functions of NomiGPT (3):
• Highlights of NomiGPT (1): EAI (1)-(2)
• Highlights of NomiGPT(2): Emotional Interaction (1)-(4)
• Highlights of NomiGPT (3)
• 2.3 Xpeng
• Cockpit Focuses on Edge AI
• Cockpit AI Functions and Planning
• 2.4 Geely
• 5-Layer Architecture of Geely Agent System
• OS Functional Features of Geely Agent (1)-(3)
• Functions of Galaxy M9
• Architecture of ZEEKR Agent
• ZEEKR Cockpit Agent Scenarios (1): Life Services
• ZEEKR Cockpit Agent Scenarios (2)
• 2.5 Xiaomi
• Application Scenarios of "XiaoAi Tongxue"
• Architecture of "XiaoAi Tongxue"
• GUI Agent Technology (1)
• GUI Agent Technology (2)
• 2.6 Great Wall Motor
• Coffee Agent System (1): Application Scenarios
• Coffee Agent System (2): Built on AI OS
• Coffee Agent System: Cooperation Dynamics
• 2.7 BAIC
• Agent Platform Architecture: Baimo Huichuang
• Agent Architecture
• 2.8 SAIC
• IM Introduces Alibaba Agent System (1): Functions
• IM Introduces Alibaba Agent System (2): Features
• Roewe Intelligent Assistant Base: Doubao
• 2.9 Chery
• Agent Brain System
• Agent System Cooperation and Planning
• 2.10 Others
• Functions of GAC Agent
• Cooperation Dynamics of BYD Agent

3 Supplier Agent Solutions

• 3.1 Huawei
• Agent System
• HarmonySpace 5: MoLA
• Agent Underlying Capabilities: LLM Architecture
• Agent Underlying Capabilities: Multimodal Capabilities
• Agent Underlying Capabilities: Thinking Capabilities
• Xiaoyi Voice Technology
• 3.2 Alibaba Cloud
• Product System
• Model Studio Supports Agent Construction
• 3.3 Baidu Cloud
• Product System
• Multi-Agent Collaboration Mode
• 3.4 Tencent Cloud
• Product system
• Cockpit System Upgrade of TAI 6.0 (1)
• Cockpit System Upgrade of TAI 6.0 (2)
• Tencent (Inference Service Solution)
• Tencent (Generation Scenario Solution)
• Q&A Scenario Solution
• 3.5 ByteDance & Volcano Engine
• Doubao Model System
• Volcano Engine Cockpit Function Highlights
• 3.6 SenseTime
• Foundation Model System
• Model Layout
• Cockpit AI Product System
• Foundation Model Training Facility
• Customers
• 3.7 Zhipu AI
• Agent Evolves to LLM OS
• Agent Architecture
• Product System
• Agent Model
• Automotive Foundation Model Base
• Technical Highlights
• 3.8 iFLYTEK
• Product system
• Functional and Technical Highlights
• 3.9 Thundersoft
• Agent System Is Built Based on Aqua Drive OS
• Agent Dynamics
• 3.10 Kotei Agent
• 3.11 Lenovo
• Agent Architecture
• 3.12 TINNOVE
• Agent Architecture
• AI System Service Forms
• AI System Application Scenarios

4 Agent Practical Technology

• 4.1 Intent Recognition
• Cases (1)
• Cases (2)
• 4.2 Knowledge Graph and Search
• Cases (1)
• Cases (2)
• Cases (3)
• 4.3 Emotion Recognition
• Cases (1)
• Cases (2)
• Cases (3)
• 4.4 Inference Acceleration
• Cases (1)
• Cases (2)
• 4.5 Recommendation System
• Problems
• Patent Technology: Refueling Recommendation
• Cases (1)
• Cases (2)
• 4.6 Tool Calling
• Synergy and Differences between Function Calling, MCP and A2A
• MCP Application Cases (1)
• MCP Application Cases (2)
• A2A Application Cases
• 4.7 MAS
• Cases (1): Great Wall Motor
• Cases (2)
• 4.8 GUI Agent
• Principle
• Application
• Functions and Features (1)-(4)

5 Problems in Agent Application

• Problem 1: The Computing Power Balance Point of the "Edge-Cloud" Deployment
• Problem 2: Architecture Design of Multi-Agent Systems (1)
• Problem 2: Architecture Design of Multi-Agent Systems (2)
• Problem 2: Architecture Design of Multi-Agent Systems (3) - Base Model Selection
• Solution: "Intramodal Adaptive Fusion + Cross-modal Precise Interaction" are the Optimal Path for Multimodal Tasks
• Case: How Anthropic Designs MAS (1)
• Case: How Anthropic Designs MAS (2)
• Problem 3: Business Model Design
• OEM Agent Profit Model: Cost Types
• OEM Agent Profit Model: 7 Cost Recovery Mechanisms at the Current Stage
• Lixiang Tongxue Builds IP Ecosystem: Physical Goods
• Lixiang Tongxue Builds IP Ecosystem: Virtual Products
• NIO's Peripheral Product Sales
• Aion Beibi IP Peripheral Products
• OEM Agent Profit Model: Future Profit Methods
• Cockpit Agent Business Model from the Perspective of OEMs: Tiered Payment
• Problem 4: Effectiveness of Scenario Application
• Problem 5: Training Bias
• Problem 6: Data Privacy