Title:
METHOD AND APPARATUS FOR GENERATING ANIMATION
Document Type and Number:
WIPO Patent Application WO/2023/284435
Kind Code:
A1
Abstract:
Provided in the present application are a method and apparatus for generating an animation. The method for generating an animation comprises: processing acquired speech to be processed and an acquired video to be processed, so as to obtain data of key points of a face corresponding to said speech, wherein the key points of the face comprise key points of a first feature, the position of at least one of the key points of the first feature corresponding to at least two of a plurality of audio frames is different, and the first feature comprises at least one of an expression in the eyes, the posture of the head and the shape of the lips; then, obtaining a plurality of image frames according to the data of the key points of the face and said video; and then obtaining an animation according to the plurality of image frames. By means of the method and apparatus for generating an animation provided in the present application, facial expressions in a facial animation are enriched, so as to more vividly present emotional information of audio; and the degree of matching between the facial animation and speech is increased, such that a deaf person more accurately understands the meaning expressed by the audio, thereby improving the user experience of the deaf person.
Inventors:
LI MINGLEI (CN)
TANG JIE (CN)
WU YILING (CN)
HUAI BAOXING (CN)
YUAN JING (CN)
TANG JIE (CN)
WU YILING (CN)
HUAI BAOXING (CN)
YUAN JING (CN)
Application Number:
PCT/CN2022/096773
Publication Date:
January 19, 2023
Filing Date:
June 02, 2022
Export Citation:
Assignee:
HUAWEI CLOUD COMPUTING TECH CO LTD (CN)
International Classes:
G06T13/40
Foreign References:
| CN111862277A | 2020-10-30 | |||
| CN112329451A | 2021-02-05 | |||
| CN110570877A | 2019-12-13 | |||
| CN109446876A | 2019-03-08 | |||
| CN104732590A | 2015-06-24 | |||
| CN113077537A | 2021-07-06 | |||
| US8566075B1 | 2013-10-22 |
Other References:
XINYA JI; HANG ZHOU; KAISIYUAN WANG; WAYNE WU; CHEN CHANGE LOY; XUN CAO; FENG XU: "Audio-Driven Emotional Video Portraits", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 20 May 2021 (2021-05-20), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081955157
SONG NAN, PEI-WEN WU, HONG-WU YANG: "Gesture-to-emotional Speech Conversion Based on Gesture Recognigion and Facial Expression Recognition", SHENGXUE-JISHU : JIKAN = TECHNICAL ACOUSTICS, SHENG XUE JI SHU BIAN JI BU, CN, vol. 37, no. 4, 31 August 2018 (2018-08-31), CN , pages 372 - 379, XP093024719, ISSN: 1000-3630, DOI: 10.16300/j.cnki.1000-3630.2018.04.014
"Master Thesis", 3 June 2019, NORTHWEST NORMAL UNIVERSITY, CN, article NAN SONG: "Research on Sign Language-to-Mandarin/Tibetan Emotional Speech Conversion by Combining Facial Expression Recognition", pages: 1 - 45, XP093025249
SONG NAN, PEI-WEN WU, HONG-WU YANG: "Gesture-to-emotional Speech Conversion Based on Gesture Recognigion and Facial Expression Recognition", SHENGXUE-JISHU : JIKAN = TECHNICAL ACOUSTICS, SHENG XUE JI SHU BIAN JI BU, CN, vol. 37, no. 4, 31 August 2018 (2018-08-31), CN , pages 372 - 379, XP093024719, ISSN: 1000-3630, DOI: 10.16300/j.cnki.1000-3630.2018.04.014
"Master Thesis", 3 June 2019, NORTHWEST NORMAL UNIVERSITY, CN, article NAN SONG: "Research on Sign Language-to-Mandarin/Tibetan Emotional Speech Conversion by Combining Facial Expression Recognition", pages: 1 - 45, XP093025249
Attorney, Agent or Firm:
LONGSUN LEAD IP LTD. (CN)
Download PDF: