克服記憶體瓶頸：CXL擴展和KV快取壓縮創新

2026年上半年，KV快取需求激增，而記憶體供應卻十分緊張，導致記憶體嚴重瓶頸。為了解決KV快取瓶頸問題，業界各公司正從KV快取相容記憶體容量的供需兩端探索解決方案。在擴展可尋址記憶體容量方面，Penguin Solutions推出了「MemoryAI™ KV快取伺服器」，Marvell發布了「Structera S CXL交換器」，Meta開發了自己的「Vistara CXL交換器」以擴展記憶體層次結構。在需求端，NVIDIA推出了「KVTC」，Google推出了「TurboQuant」以壓縮KV快取。

本報告詳細分析了以下主題：（1）鍵值快取瓶頸；（2）透過 CXL 和鍵值快取卸載擴展可用鍵值快取容量；（3）透過注意力機制和鍵值快取量化降低鍵值快取容量需求；（4）提高解碼效率的技術（特別是 MTP 和 DiffusionGemma）；以及（5）對記憶體市場的廣泛影響。本報告目的是評估各種鍵值快取瓶頸解決方案的技術原理、效能指標和未來發展趨勢。

主要亮點

• 2026年初，由於 KV 快取需求增加和記憶體供應緊張，出現了嚴重的瓶頸。
• Penguin Solutions、Marvell 和 Meta 等供應商透過基於 CXL 的解決方案擴展支援的記憶體容量。
• NVIDIA 和 Google 透過壓縮技術降低對 KV 快取的需求。
• 本報告涵蓋 CXL 和 KV 快取卸載、注意力/量化技術、解碼效率（MTP、DiffusionGemma）及其對記憶體市場的影響。
• 分析重點在於瓶頸問題解決方法的技術原理和發展趨勢。

目錄

第1章 KV快取的瓶頸

第2章 透過 CXL 和 KV 快取卸載擴充可用 KV 快取容量

第3章 基於鍵值快取量化的注意力機制及鍵值快取容量需求壓縮

第4章 解讀效率提昇技術 - MTP 與擴散 Gemma

第5章 對記憶體市場的影響

第6章 TRI的觀點

During the first half of 2026, surging demand for KV Cache coupled with constrained memory supply resulted in severe memory bottlenecks. To resolve the KV Cache bottlenecks, industry players are seeking solutions from both the supply side of KV Cache-addressable memory capacity and the demand side. Regarding the expansion of the addressable memory capacity, Penguin Solutions launched the MemoryAI™ KV Cache Server, Marvell introduced the Structera S CXL switch, and Meta developed its proprietary Vistara CXL switch to expand the memory hierarchy. On the demand side, NVIDIA introduced KVTC, and Google launched TurboQuant to compress the KV Cache.

This report provides an in-depth analysis of: (1) the KV Cache bottleneck; (2) expanding available KV Cache capacity through CXL and KV Cache offloading; (3) reducing KV Cache capacity demand via attention mechanisms and KV Cache quantization; (4) methods for improving decode efficiency, specifically MTP and DiffusionGemma; and (5) the broader impact on the memory market. The objective is to evaluate the technical principles, performance metrics, and future development trajectories of various KV Cache debottlenecking technologies.

Key Highlights

• KV Cache demand growth alongside limited memory supply created significant bottlenecks in early 2026.
• Vendors (Penguin Solutions, Marvell, Meta) are expanding addressable memory capacity via CXL-based solutions.
• NVIDIA and Google are reducing KV Cache demand through compression technologies.
• Report covers CXL and KV Cache offloading, attention/quantization methods, decode efficiency (MTP, DiffusionGemma), and memory market impact.
• Analysis focuses on technical principles and development trends of debottlenecking approaches.

Table of Contents

1. The KV Cache Bottleneck

• Figure 1: Example of KV Cache in Use
• Figure 2: KV Cache Expansion Relative to Context Window Size (Using Llama 3 70B as an Example)

2. Expanding Available KV Cache Capacity via CXL and KV Cache Offloading

• Figure 3: Applications of CXL Switch
• Table 1: Evolution of the Specifications for CXL
• Figure 4: Applications for ACF-S
• Figure 5: Applications for EMFASYS
• Figure 6: MemoryAI KV Cache Server with 8 x 1TB CXL AICs from Penguin Solutions
• Figure 7: CXL AIC from Penguin Solutions
• Figure 8: Marvell’s Structera X
• Figure 9: Architecture of Meta’s Vistara
• Figure 10: Architecture of Meta’s MemServer

3. Compressing KV Cache Capacity Demand via Attention Mechanism and KV Cache Quantization

• Figure 11: Principles of Different Multi-Head Attention Mechanisms
• Figure 12: Workflow of KVTC Compression
• Figure 13: Process of Recursive Polar Coordinate Transformation
• Table 2: Comparison of KVTC and TurboQuant Technologies

4. Decode Efficiency Enhancement Methods－MTP and DiffusionGemma

• Figure 14: Example of Speculative Decoding
• Figure 15: Operating Principles of Meta’s MTP
• Figure 16: Operating Principles of DeepSeek’s MTP
• Table 3: Comparison of MTP Technologies Across Companies
• Figure 17: Sudoku as an Example of Google’s DiffusionGemma in Use
• Table 4: Comparison of MTP and DiffusionGemma Technologies

5. Impact on the Memory Market

6. TRI’s View