DUBLIN: The "Cockpit AI Agent Research Report, 2024" report has been added to ResearchAndMarkets.com's offering.

Foundation models spawn Agent customized scenarios

Since AI foundation models were installed in cars, they have been widely used in multimodal interaction scenarios. OEMs have started from voice assistants to further improve intelligent processing of user commands:

Before foundation models got on cars, the wake-up rate and effective instructions of voice assistants in cockpits are at a low level.

After foundation models were available in cars, not only has voice recognition become more intelligent, but the application of expression monitoring and gesture control have also been improved, with effective commands and daily active rates increasing significantly.

For example, IM L6 has built Carlog and City Drive scenarios to enable the AI foundation models to proactively recommend food and attractions and allow users to post them on social media:

Carlog: Actively perceive the scenario during driving through AI vision foundation model, mobilize four cameras to take photos, automatically save and edit them, and support one-click share in Moments.

City Drive: Cooperate with Volcengine to model nearby food, scenic spots and landmarks in real time in the digital screen, and push them according to users' habits and preferences. The applicability of foundation models in various scenarios has stimulated users' demand for intelligent agents that can uniformly manage cockpit functions. In 2024, OEMs such as NIO, Li Auto, and Hozon successively launched Agent frameworks, using voice assistants as the starting point to manage functions and applications in cockpits.

Agent service frameworks can not only manage cockpit functions in a unified way, but also provide more abundant scenario modes according to customers' needs and preferences, especially supporting customized scenarios for users, which accelerates the advent of the cockpit personalization era.

For example, NIO's NOMI GPT allows users to set an AI scenario with just one sentence:

Core competence of cockpit Agents

AI Agents in the era of foundation models are based on LLMs, whose powerful reasoning expands the applicable scenarios of AI Agents that can improve the thinking capability of foundation models through feedback obtained during operation. In the cockpit, the Agent capability paradigm can be roughly divided into 'Understanding' + 'Planning' + 'Tool Use' + 'Reflection'.

When Agents first get on cars, cognitive and planning abilities are more important. The understanding of task goals and the choice of implementation paths directly determine the accuracy of performance results, which in turn affect the scenario utilization rate of Agents. For example, in Xiaomi's voice interaction process, semantic understanding is the difficulty of the entire automotive voice processing process. XiaoAi handles semantic parsing through a semantic parsing model.

After the mass production of Agents, the personalized cockpits that support users to customize scenario modes become the highlight, and Reflection becomes the most important core competence at this stage, so it is necessary to build an Agentic Workflow that is constantly learning and optimizing.

For example, Lixiang Tongxue offered by Li Auto supports the creation of one-sentence scenarios. It is backed by Mind GPT's built-in memory network and online reinforcement learning capabilities. Mind GPT can remember personalized preferences and habits based on historical conversations. When similar scenarios recur, it can automatically set scenario parameters through historical data to fit the user's original intentions.

At the AI OS architecture setting level, we take SAIC Z-One as an example

Z-One accesses the LLM kernel (LLM OS) at the kernel layer, which controls the interfaces of AI OS SDK and ASF with the original microkernel respectively, in which AI OS SDK receives the scheduling of the LLM to promote the Agent service framework of the application layer. The Z-One AI OS architecture highly integrates AI with CPU. Through SOA atomic services, AI is then connected to the vehicle's sensors, actuators and controllers. This architecture, based on a terminal-cloud foundation model, can enhance the computing power of the terminal-side foundation model and reduce operational latency.

Application Difficulty of Cockpit AI Agents

Agents connect to users and execute commands. In the application process, in addition to the technical difficulties of putting foundation models on cars, they also face scenario difficulties. In the process of command reception-semantic analysis-intention reasoning-task execution, the accuracy of the performance results and the delay in human-computer interaction directly affect the user's riding experience.

Humanization of interaction

For example, in the 'emotional consultant' scenario, Agents should resonate emotionally with car owners and perform anthropomorphism. Generally, there are three forms of anthropomorphism of AI Agents: physical anthropomorphism, personality anthropomorphism, and emotional anthropomorphism.

Foundation model performance

In the 'encyclopedia question and answer' scenario, Agents may be unable to answer the user's questions, especially open questions, accurately because of LLM illusion after semantic analysis, database search, answer generation and the like.

Current solutions include advanced prompting, RAG+knowledge graph, ReAct, CoT/ToT, etc., which cannot completely eliminate "LLM illusion". In the cockpit, external databases, RAG, self-consistency and other methods are more often used to reduce the frequency of "LLM illusion".

Some foundation model manufacturers have improved the above solutions. For example, Meta has proposed to reduce "LLM illusion" through Chain-of-Verification (CoVe). This method breaks down fact-checking into more detailed sub-questions to improve response accuracy and is consistent with the human-driven fact-checking process. It can effectively improve the FACTSCORE indicator in long-form generation tasks.

Key Topics Covered:

1 Introduction to Cockpit AI Agent

1.1 Status Quo of AI Agent

1.2 Cockpit AI Agent

1.3 Application Scenario of Cockpit Agent

1.4 Application Status of Cockpit Agent

2 AI Agent Technology Implementation Path

2.1 AI OS Architecture

2.2 Technology related to AI Vision Large Model

2.3 Application Technology of Cockpit Large Model

2.4 Application Trends of AI in Cockpit

2.5 Cockpit Agent Solution

3 Application Analysis of Cockpit AI Agent of Suppliers

3.1 ThunderSoft

3.2 Huawei

3.3 Alibaba Cloud

3.4 Tencent

3.5 Baidu

3.6 iFLYTEK

3.7 AISpeech

3.8 Lenovo

3.9 SAIC Z-ONE

3.10 Zhipu AI

3.11 Microsoft

3.12 TINNOVE

3.14 Rockchip Uses AI Sound Field Technology in Cockpit

3.15 NNG Applies AI to Navigation Technology

4 Application Analysis of Cockpit AI Agent of OEMs

4.1 Application Status of Cockpit Agent in OEMs

4.2 NIO

4.3 Xpeng

4.4 Li Auto

4.5 Xiaomi

4.6 BAIC

4.7 Hozon

4.8 Dongfeng

4.9 JAC

4.10 Chang'an

4.11 Volkswagen: Evolving to Agents through GPTs

4.12 Mercedes-Benz: Personalized Service with MBUX Virtual Assistant

4.13 Cockpit Application of GAC AI Large Model

4.14 Cockpit Application of Great Wall Large Model

4.15 Chery self-developed LION AI + iFLYTEK Spark Model

4.16 Geely

4.17 IM AI Large Model Builds Active Perception Scenario

For more information about this report visit https://www.researchandmarkets.com/r/a1s6dn

About ResearchAndMarkets.com

ResearchAndMarkets.com is the world's leading source for international market research reports and market data. We provide you with the latest data on international and regional markets, key industries, the top companies, new products and the latest trends.

Fonte: Business Wire