Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHOD AND APPARATUS FOR GENERATING SPEECH VIDEO
Document Type and Number:
WIPO Patent Application WO/2022/045486
Kind Code:
A1
Abstract:
A method and an apparatus for generating a speech video are disclosed. The disclosed apparatus for generating a speech video according to an embodiment corresponds to a computing apparatus having one or more processors and a memory for storing one or more programs executed by the one or more processors, and comprises: a first encoder for receiving a first person background image of a predetermined person partially covered by a first mask and extracting a first image feature vector from the first person background image; a second encoder for receiving a second person background image of a person partially covered by a second mask and extracting a second image feature vector from the second person background image; a third encoder for receiving a speech audio signal of a person and extracting a voice feature vector from the speech audio signal; a combining unit for generating a combined vector by combining the first image feature vector output from the first encoder, the second image feature vector output from the second encoder, and the voice feature vector output from the third encoder; and a decoder for reconstructing a speech video of a person by using the combined vector as an input.

Inventors:
CHAE GYEONGSU (KR)
HWANG GUEMBUEL (KR)
Application Number:
PCT/KR2020/018374
Publication Date:
March 03, 2022
Filing Date:
December 15, 2020
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
DEEPBRAIN AI INC (KR)
International Classes:
G10L21/10; G06N20/00; G10L15/02; G10L15/04; G10L21/055; H04N21/854
Foreign References:
KR20060090687A2006-08-14
KR20140037410A2014-03-27
KR20200080681A2020-07-07
US20200135172A12020-04-30
KR20200145700A2020-12-30
KR20200145701A2020-12-30
Other References:
KONSTANTINOS VOUGIOUKAS; STAVROS PETRIDIS; MAJA PANTIC: "Realistic Speech-Driven Facial Animation with GANs", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 14 June 2019 (2019-06-14), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081381844
Attorney, Agent or Firm:
DOOHO IP LAW FIRM (KR)
Download PDF:
View/Download PDF      PDF Help



 
Previous Patent: APPARATUS AND METHOD FOR GENERATING SPEECH VIDEO THAT CREATES LANDMARKS TOGETHER

Next Patent: ANTI-ICING/DE-ICING HONEYCOMB CORE COMPOSITE MATERIAL AND METHOD FOR MANUFACTURING SAME