Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHOD AND DEVICE FOR GENERATING SPEECH VIDEO ON BASIS OF MACHINE LEARNING
Document Type and Number:
WIPO Patent Application WO/2020/256471
Kind Code:
A1
Abstract:
Disclosed are a method and a device for generating a speech video on the basis of machine learning. The disclosed device for generating a speech video according to an embodiment relates to a computing device comprising one or more processors and a memory for storing one or more programs executed by the one or more processors, and comprises: a first encoder for receiving a portrait background image that is a video part of a speech video of a predetermined person, and extracting an image feature vector from the portrait background image; a second encoder for receiving a speech audio signal that is an audio part of the speech video, and extracting a voice feature vector from the speech audio signal; a combination unit for generating a combination vector by combining the image feature vector output from the first encoder and the voice feature vector output from the second encoder; and a decoder for reconstructing the speech video of the person by configuring the combination vector as an input, wherein, in the portrait background image that is input to the first encoder, a part related to speech of the person is covered with a mask, and the face and upper body of the person are included.

Inventors:
CHAE GYEONGSU (KR)
HWANG GUEMBUEL (KR)
PARK SUNGWOO (KR)
JANG SEYOUNG (KR)
Application Number:
PCT/KR2020/007974
Publication Date:
December 24, 2020
Filing Date:
June 19, 2020
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
MONEYBRAIN INC (KR)
International Classes:
H04N5/265; G06N3/08; G10L13/027; G10L19/00; H04N21/2368; H04N21/439
Foreign References:
KR20060090687A2006-08-14
KR20140037410A2014-03-27
KR20190046371A2019-05-07
JP2016042362A2016-03-31
Other References:
KONSTANTINOS VOUGIOUKAS, PETRIDIS STAVROS, PANTIC MAJA: "Realistic Speech-Driven Facial Animation with GANs", INTERNATIONAL JOURNAL OF COMPUTER VISION., KLUWER ACADEMIC PUBLISHERS, NORWELL., US, vol. 128, no. 5, 1 May 2020 (2020-05-01), US, pages 1398 - 1413, XP055767229, ISSN: 0920-5691, DOI: 10.1007/s11263-019-01251-8
Attorney, Agent or Firm:
DOOHO IP LAW FIRM (KR)
Download PDF:
View/Download PDF      PDF Help



 
Previous Patent: IMAGE ENCODING/DECODING METHOD AND DEVICE, AND RECORDING MEDIUM FOR STORING BITSTREAM

Next Patent: METHOD AND DEVICE FOR GENERATING UTTERANCE VIDEO BY USING VOICE SIGNAL