语义通信性能评估体系及指标

摘要：鉴于语义通信系统缺乏统一的、具泛化价值的性能评估体系，对现有研究中评估方法的使用情况与不足进行分析，并提出语义通信效率指标Esc 和语义通信效用指标Usc。其中，指标Esc 用于衡量通信系统在给定通信资源下单位时间开销内的任务完成准确性，指标Usc 用于衡量通信系统在给定通信资源下与任务性能上限的接近程度。与现有评估指标相比，这两种指标具有更好的泛用性，可为不同任务场景、模态信息下的语义通信模型横向性能对比提供指导。此外，以语义图像重建与语音重建任务为例，分别搭建端到端语义通信仿真模型，并基于所提Esc 与Usc指标对仿真模型性能进行评估。

关键词：语义通信；人工智能；性能评估指标

Abstract: The absence of a unified and generalized performance evaluation system brings challenges to the realization of semantic communication systems. The usages and shortcomings of evaluation methods in existing research are analyzed. Two evaluation metrics, namely the semantic communication efficiency metric Esc and the semantic communication utility metric Usc, are proposed. Esc measures the accuracy of task completion within unit time cost of a communication system under certain resources. Usc measures how close a communication system is to the upper limit of task performance under certain resources. Compared with the existing evaluation methods, the two proposed metrics are of better generality, which can provide guidance for the performance comparison of semantic communication models under different scenarios and information modalities. In addition, the semantic image reconstruction and speech reconstruction are taken as examples to build end-to-end semantic communication simulation models respectively, and the performance of models based on the proposed Esc and Usc metrics is evaluated.

Keywords: semantic communication; artificial intelligence; performance evaluation metric