基于深度情感唤醒网络的多模态情感分析与情绪识别

基于深度情感唤醒网络的多模态情感分析与情绪识别

DOI:

作者单位:

1. 河北大学 数学与信息科学学院,河北 保定 071002;2. 河北省机器学习与计算智能重点实验室,河北 保定 071002

中图分类号:

TP181

基金项目:

国家自然科学基金面上项目(61773150,61672205)；河北省自然科学基金面上项目(F2018201115)；河北省教育厅科学技术研究重点项目(ZD2019021)；河北大学高层次创新人才科研启动经费项目.

Deep emotional arousal network for multimodal sentiment analysis and emotion recognition

Author:

ZHANG Feng

1. College of Mathematics and Information Sciences,Hebei University,Baoding 071002,China;2. Hebei Key Laboratory of Machine Learning and Computational Intelligence,Baoding 071002,China
在期刊界中查找
在百度中查找
在本站中查找

LI Xi-cheng

1. College of Mathematics and Information Sciences,Hebei University,Baoding 071002,China;2. Hebei Key Laboratory of Machine Learning and Computational Intelligence,Baoding 071002,China
在期刊界中查找
在百度中查找
在本站中查找

DONG Chun-ru

1. College of Mathematics and Information Sciences,Hebei University,Baoding 071002,China;2. Hebei Key Laboratory of Machine Learning and Computational Intelligence,Baoding 071002,China
在期刊界中查找
在百度中查找
在本站中查找

HUA Qiang

1. College of Mathematics and Information Sciences,Hebei University,Baoding 071002,China;2. Hebei Key Laboratory of Machine Learning and Computational Intelligence,Baoding 071002,China
在期刊界中查找
在百度中查找
在本站中查找

Affiliation:

1. College of Mathematics and Information Sciences,Hebei University,Baoding 071002,China;2. Hebei Key Laboratory of Machine Learning and Computational Intelligence,Baoding 071002,China

摘要:

随着网络平台上各类图像、视频数据的快速增长,多模态情感分析与情绪识别已成为一个日益热门的研究领域.相比于单模态情感分析,多模态情感分析中的模态融合是一个亟待解决的关键问题.受到认知科学中情感唤起模型的启发,提出一种能够模拟人类处理多通道输入信息机制的深度情感唤醒网络(DEAN),该网络可实现多模态信息的有机融合,既能处理情绪的连贯性,又能避免融合机制的选择不当而带来的问题.DEAN网络主要由以下3部分组成:跨模态Transformer模块,用以模拟人类知觉分析系统的功能;多模态BiLSTM系统,用以模拟认知比较器;多模态门控模块,用以模拟情感唤起模型中的激活结构.在多模态情感分析与情绪识别的3个经典数据集上进行的比较实验结果表明,DEAN模型在各数据集上的性能均超越了目前最先进的情感分析模型.

Abstract:

Multimodal sentiment analysis and emotion recognition have become an increasingly popular research topic, with a large number of emerging of various images and videos on internet. Compared with the unimode sentiment analysis, one of the key issues in multimodal sentiment analysis is the fusion strategy. Inspired by the emotional arousal model in cognitive science, a deep emotional arousal network(DEAN) is proposed, which can simulate emotional coherence. The proposed DEAN consists of three parts: A crossmodal transformer module that simulates the functions of a human perception analysis system, a multimodal BiLSTM system that simulates the cognitive comparator, and a multimodal gating module that is used to simulate the activation mechanism in the physiological emotional arousal model. Extensive experimental comparisons on three benchmarks for multimodal sentiment analysis and emotion recognition are conducted, and the experimental results show that the DEAN outperforms the state-of-the-art methods.

张峰,李希城,董春茹,等.基于深度情感唤醒网络的多模态情感分析与情绪识别[J].控制与决策,2022,37(11):2984-2992

复制

相关知识

探索多模态情感识别：MER数据集的深度解析与应用
这是一篇关于「情绪分析」和「情感检测」的综述（非常详细）
面向掌纹掌静脉识别网络轻量化的非对称双模态融合方法
【综述研读】深度学习多模态图像语义分割前沿进展
基于深度学习的植物识别研究
【花卉种类识别】基于matlab GUI深度学习VCG19网络花卉种类识别检测系统（含识别率）【含Matlab源码 8773期】
香与情感记忆：香气如何触发人们的情感共鸣
实战测试：多模态AI在文档解析、图表分析中的准确率对比
心理学情绪情感理论
【花卉种类识别】 深度学习VCG19网络花卉种类识别检测系统（含识别率）【含GUI Matlab源码 8773期】

网址: 基于深度情感唤醒网络的多模态情感分析与情绪识别 https://www.huajiangbk.com/newsview2520547.html