Title:
TRAINING METHOD AND APPARATUS FOR VIDEO GENERATION MODEL, AND STORAGE MEDIUM AND COMPUTER DEVICE
Document Type and Number:
WIPO Patent Application WO/2024/078243
Kind Code:
A1
Abstract:
Disclosed in the present application are a training method and apparatus for a video generation model, and a storage medium and a computer device. The method comprises: extracting a voice feature, an expression parameter and a head parameter from a training video of a target user, wherein the head parameter is used for representing head posture information and head position information of the target user; combining the voice feature, the expression parameter and the head parameter, so as to obtain a conditional input of the training video; and on the basis of the conditional input, three-dimensional coordinates and an angle-of-view direction, performing network training on a single neural radiance field, so as to obtain a video generation model, wherein the video generation model is obtained by means of performing training on the basis of a total loss, and the total loss comprises an image reconstruction loss. Head posture information and head position information are introduced during a training process, and thus a video generation model obtained by means of training can take a shoulder motion state into consideration, such that when video reconstruction is subsequently performed according to the video generation model, motion between a head and shoulders can be more harmonious and stable, thereby improving the display authenticity of a reconstructed video.
Inventors:
WU YANG (CN)
HU PENGFEI (CN)
QI XIAOJUAN (CN)
WU XIUZHE (CN)
SHAN YING (CN)
XU JING (CN)
HU PENGFEI (CN)
QI XIAOJUAN (CN)
WU XIUZHE (CN)
SHAN YING (CN)
XU JING (CN)
Application Number:
PCT/CN2023/118459
Publication Date:
April 18, 2024
Filing Date:
September 13, 2023
Export Citation:
Assignee:
TENCENT TECH SHENZHEN CO LTD (CN)
International Classes:
G06T17/00
Foreign References:
| CN114202604A | 2022-03-18 | |||
| CN113192162A | 2021-07-30 | |||
| CN113269872A | 2021-08-17 | |||
| CN113822969A | 2021-12-21 | |||
| CN114782596A | 2022-07-22 | |||
| US11295501B1 | 2022-04-05 | |||
| US20220044463A1 | 2022-02-10 |
Attorney, Agent or Firm:
SHENPAT INTELLECTUAL PROPERTY AGENCY (CN)
Download PDF:
Previous Patent: OPTICAL-POWER-ADJUSTABLE OPTICAL COMBINER/SPLITTER, RELATED DEVICE AND SYSTEM
Next Patent: BEAUTY INSTRUMENT
Next Patent: BEAUTY INSTRUMENT