Title:
MULTI-MODAL EMOTION RECOGNITION METHOD COMBINED WITH HIERARCHICAL POLICY
Document Type and Number:
WIPO Patent Application WO/2024/040793
Kind Code:
A1
Abstract:
Disclosed in the present invention is a multi-modal emotion recognition method combined with a hierarchical policy. The present invention provides the multi-modal emotion recognition method combined with the hierarchical policy. According to the emotion recognition method, audio features and text features are combined, and compared with single-audio and single-text emotion recognition methods, the emotion recognition accuracy is further improved; a multi-modal emotion recognition model is generally large, such that the reasoning prediction speed of the model is slow, and the response efficiency and concurrency of the model are affected. Therefore, the present invention provides the multi-modal emotion recognition method combined with the hierarchical policy, reasoning prediction of a sample which is easy to predict is performed in a shallow model, and reasoning prediction of a sample which is difficult to predict is performed in a deep model, thereby improving the overall response speed of multi-modal emotion recognition while ensuring the accuracy.
Inventors:
LIU BO (CN)
SUN PENG (CN)
XU XIAOLONG (CN)
SUN PENG (CN)
XU XIAOLONG (CN)
Application Number:
PCT/CN2022/136487
Publication Date:
February 29, 2024
Filing Date:
December 05, 2022
Export Citation:
Assignee:
CHINA TELECOM BESTPAY CO LTD (CN)
International Classes:
G10L25/63
Domestic Patent References:
| WO2021068843A1 | 2021-04-15 | |||
| WO2021174757A1 | 2021-09-10 |
Foreign References:
| CN110364185A | 2019-10-22 | |||
| CN114120978A | 2022-03-01 | |||
| CN114882522A | 2022-08-09 |
Other References:
WU LIANGQING, LIU QIYUAN, ZHANG DONG, WANG JIANCHENG, LI SHOUSHAN, ZHOU GUODONG: "Multimodal Emotion Recognition with Auxiliary Sentiment Information", BEIJING DAXUE XUEBAO ZERAN KEXUE BAN - ACTA SCIENTIARUMNATURALIUM UNIVERSITATIS PEKINENSIS, BEIJING DAXUE CHUBANSHE, BEIJING, CN, vol. 56, no. 1, 20 January 2020 (2020-01-20), CN , pages 75 - 81, XP093143266, ISSN: 0479-8023, DOI: 10.13209/j.0479-8023.2019.105
ZOU JIYUN, XU YUNFENG: "Emotion recognition neural network based on auxiliary modal supervised training", JOURNAL OF HEBEI UNIVERSITY OF SCIENCE AND TECHNOLOGY., vol. 41, no. 5, 1 October 2020 (2020-10-01), pages 424 - 432, XP093143267, DOI: 1008-1542(2020)05-0424-09
ZOU JIYUN, XU YUNFENG: "Emotion recognition neural network based on auxiliary modal supervised training", JOURNAL OF HEBEI UNIVERSITY OF SCIENCE AND TECHNOLOGY., vol. 41, no. 5, 1 October 2020 (2020-10-01), pages 424 - 432, XP093143267, DOI: 1008-1542(2020)05-0424-09
Download PDF: