面向智能体时代大语言模型推理的新型存算解聚合架构

摘要：针对传统大模型推理在长上下文场景及新兴智能体时代推理中所面临的键值（KV）Cache膨胀、本地显存容量受限及跨节点通信开销高等问题，提出一种基于光输入/输出（OIO）拉远且以内存为中心的新型存算解聚合架构。该架构在物理层面实现计算与存储资源的解耦，支持二者的独立伸缩、池化共享与按需协同；在计算侧将Prefill与Decode阶段划分为异构计算簇，在存储侧构建基于异质材料的全局内存池，并借助近存计算单元对集合通信原语进行卸载。基于8个异构计算处理单元（8XPU）与8个内存处理单元（8MPU）全互联配置的系统级仿真分析表明，该架构在长上下文及高并发推理场景下，可有效降低首token时延与逐token时延，减少数据搬移开销，提升系统吞吐率，并在满足同等服务水平协议（SLA）的条件下显著降低对计算资源的需求，验证了其在智算基础设施中的潜在应用价值。

关键词：智能体时代；存算解聚合；内存为中心；光输入/输出；近存计算

Abstract: To address the challenges of Key-Value (KV) cache explosion, limited local video memory capacity, and high inter-node communication overhead in long-context inference for traditional large language models as well as emerging agent-based reasoning scenarios, a novel compute-storage disaggregated architecture centered on optical I/O (OIO)-attached remote memory is proposed. At the physical level, this architecture decouples compute and storage resources, enabling independent scaling, pooled sharing, and on-demand collaborative orchestration. On the compute side, heterogeneous clusters are designated separately for the Prefill and Decode phases, while on the storage side, a global memory pool composed of heterogeneous materials is constructed, with near-memory processing units offloading collective communication primitives. System-level simulation results based on a fully interconnected configuration of 8 X processing units (XPU) and 8 memory processing units (MPU) demonstrate that the proposed architecture effectively reduces both time-to-first-token and per-token latency, minimizes data movement overhead, and improves system throughput in long-context and high-concurrency inference scenarios. Moreover, under equivalent service-level agreement (SLA) constraints, the required compute resources are significantly reduced, confirming its potential applicability in intelligent computing infrastructure.

Keywards: agent era; compute-storage disaggregation; memory-centric; optical I/O; near-memory computing