摘要
情绪识别研究从实验室环境推进到无约束的真实场景中时面临很多问题。真实场景中不受限制的个体活动和复杂环境使面部图像、语音等单一模态的数据无法可靠获取,并且在真实场景中人们自发的情绪更加微妙,表达强度不大,导致识别难度增加。因此,为了更加稳健地识别真实场景中的个体情绪,针对个体活动的特点,设计了特征提取网络充分挖掘面部、骨架、姿态及场景等多模态数据中的情绪信息进行相互补充;同时,关注不同数据间的联系,设计了特征融合模块融合多种特征。网络在具有挑战性的公共空间真实场景的PLPS-E数据集上取得了最佳识别性能,VAD维度情绪识别准确率达到了74.62%、79.15%、87.94%;网络在相对简单的真实场景FABE数据集上也达到了相当的性能,维度V的识别准确率达到了98.39%。实验表明了算法的有效性。
Emotion recognition research faces many problems when advancing from laboratory environments to unconstrained real-world scenarios.Unrestricted individual activities and complex environments in real-life scenarios make it impossible to reliably obtain single-modal data such as facial images and speech,and people’s spontaneous emotions in real-life scenarios are much more subtle and expressive in less intensity,leading to increased recognition difficulty.Therefore,in order to recognize individual emo-tions in real scenes more robustly,a feature extraction network is designed to fully mine the emotion information in multimodal data such as face,skeleton,posture and scene to complement each other for the characteristics of individual activities;at the same time,it pays attention to the connection between different data and designs a feature fusion module to merge a variety of features.The net-work achieves the best recognition performance on the challenging PLPS-E dataset of real scenes in public space,with VAD dimen-sion emotion recognition accuracies of 74.62%,79.15%,and 87.94%;the network also achieves comparable performance on the relatively simple FABE dataset of real scenes,with dimension V recognition accuracy of 98.387%.The experiments show the effec-tiveness of the proposed algorithm.
作者
熊昆洪
贾贞超
高峰
文虹茜
卿粼波
高励
Xiong Kunhong;Jia Zhenchao;Gao Feng;Wen Hongqian;Qing Linbo;Gao Li(College of Electronics and Information Engineering Information,Sichuan University,Chengdu 610065,China;Wangjiang Hospital,Sichuan University,Chengdu 610065,China;Department of Neurology,Chengdu Third People’sHospital,Chengdu 610031,China)
出处
《现代计算机》
2024年第1期18-25,共8页
Modern Computer
基金
四川省科技计划项目(2023YFS0195)。
关键词
情绪识别
真实场景
多模态
特征深度融合
emotion recognition
real scenes
multimodal
feature depth fusion
