Title:
NEURAL RENDERING METHOD BASED ON MULTI-RESOLUTION NETWORK STRUCTURE
Document Type and Number:
WIPO Patent Application WO/2023/225891
Kind Code:
A1
Abstract:
Disclosed in the present application is a neural rendering method based on a multi-resolution network structure. The method comprises: first photographing images of an object to be modeled under different viewing angles and illumination, and obtaining camera parameters, light source positions, proxy geometry, neural textures, foreground matting, radiance cues and UV image data, and building and training a neural rendering pipeline model; and finally photographing images under specified camera parameters and illumination conditions, processing same and obtaining the radiance clues and a UV image, and synthesizing a new image or animation by using the neural rendering pipeline model. Compared with a traditional neural network, different spatial frequency components are explicitly split in the present application, so that a sequence of a synthesized image has better stability in a time domain. According to the present application, the problem of high-frequency loss caused by hybrid coding of different frequency components in a traditional method is solved, so that more detailed textures are reserved for the synthesized image, and higher fidelity is achieved.
Inventors:
ZHOU KUN (CN)
WU HONGZHI (CN)
REN ZHONG (CN)
MA SHENGJIE (CN)
WU HONGZHI (CN)
REN ZHONG (CN)
MA SHENGJIE (CN)
Application Number:
PCT/CN2022/094877
Publication Date:
November 30, 2023
Filing Date:
May 25, 2022
Export Citation:
Assignee:
UNIV ZHEJIANG (CN)
FACEUNITY TECH CO LTD (CN)
FACEUNITY TECH CO LTD (CN)
International Classes:
G06N3/08; G06N3/04; G06T3/00; G06T13/20; G06T15/04
Foreign References:
| CN114514561A | 2022-05-17 | |||
| CN113538664A | 2021-10-22 | |||
| RU2764144C1 | 2022-01-13 |
Other References:
GAO DUAN; CHEN GUOJUN; DONG YUE; PEERS PIETER; XU KUN; TONG XIN: "Deferred neural lighting", ACM TRANSACTIONS ON GRAPHICS, ACM, NY, US, vol. 39, no. 6, 26 November 2020 (2020-11-26), US , pages 1 - 15, XP059134795, ISSN: 0730-0301, DOI: 10.1145/3414685.3417767
JUSTUS THIES ; MICHAEL ZOLLHöFER ; MATTHIAS NIEÃNER: "Deferred neural rendering", ACM TRANSACTIONS ON GRAPHICS, ACM, NY, US, vol. 38, no. 4, 12 July 2019 (2019-07-12), US , pages 1 - 12, XP058439437, ISSN: 0730-0301, DOI: 10.1145/3306346.3323035
JUSTUS THIES ; MICHAEL ZOLLHöFER ; MATTHIAS NIEÃNER: "Deferred neural rendering", ACM TRANSACTIONS ON GRAPHICS, ACM, NY, US, vol. 38, no. 4, 12 July 2019 (2019-07-12), US , pages 1 - 12, XP058439437, ISSN: 0730-0301, DOI: 10.1145/3306346.3323035
Attorney, Agent or Firm:
HANGZHOU QIUSHI PATENT OFFICE CO., LTD (CN)
Download PDF:
Previous Patent: QUALITY OF SERVICE (QOS) DIFFERENTIATION IN USER-PLANE PROCEDURES
Next Patent: AUTOMATIC FEEDING DEVICE FOR VACUUM PACKAGING MACHINE
Next Patent: AUTOMATIC FEEDING DEVICE FOR VACUUM PACKAGING MACHINE