Title:
METHOD AND APPARATUS FOR NEURAL NETWORK BASED ON ENERGY-BASED LATENT VARIABLE MODELS
Document Type and Number:
WIPO Patent Application WO/2022/077345
Kind Code:
A1
Abstract:
Methods and apparatuses for training neural networks based on energy-based latent variable models (EBLVMs) are provided. The method comprises bi-level optimizations based on a score matching objective. The lower-level optimizes a variational posterior distribution of the latent variables to approximate the true posterior distribution of the EBLVM, and the higher-level optimizes the neural network parameters based on a modified SM objective as a function of the variational posterior distribution. The method may be applied to train neural networks based on EBLVMs with nonstructural assumptions.
Inventors:
ZHU JUN (CN)
BAO FAN (CN)
LI CHONGXUAN (CN)
XU KUN (CN)
SU HANG (CN)
LU SILIANG (CN)
BAO FAN (CN)
LI CHONGXUAN (CN)
XU KUN (CN)
SU HANG (CN)
LU SILIANG (CN)
Application Number:
PCT/CN2020/121172
Publication Date:
April 21, 2022
Filing Date:
October 15, 2020
Export Citation:
Assignee:
BOSCH GMBH ROBERT (DE)
UNIV TSINGHUA (CN)
UNIV TSINGHUA (CN)
International Classes:
G06N3/08
Foreign References:
| US20060034495A1 | 2006-02-16 | |||
| US20160300134A1 | 2016-10-13 |
Other References:
XIE JIANWEN, LU YANG, GAO RUIQI, WU YING NIAN: "Cooperative Learning of Energy-Based Model and Latent Variable Model via MCMC Teaching", THE THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, 29 April 2018 (2018-04-29), XP055921839, Retrieved from the Internet
CHONGXUAN LI; CHAO DU; KUN XU; MAX WELLING; JUN ZHU; BO ZHANG: "To Relieve Your Headache of Training an MRF, Take AdVIL", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 24 January 2019 (2019-01-24), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081493492
DU YILUN, MORDATCH IGOR: "Implicit Generation and Modeling with Energy-Based Models", NEURIPS 2019, 30 June 2020 (2020-06-30), XP055921843, Retrieved from the Internet
CHONGXUAN LI; CHAO DU; KUN XU; MAX WELLING; JUN ZHU; BO ZHANG: "To Relieve Your Headache of Training an MRF, Take AdVIL", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 24 January 2019 (2019-01-24), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081493492
DU YILUN, MORDATCH IGOR: "Implicit Generation and Modeling with Energy-Based Models", NEURIPS 2019, 30 June 2020 (2020-06-30), XP055921843, Retrieved from the Internet
Attorney, Agent or Firm:
NTD PATENT AND TRADEMARK AGENCY LTD. (CN)
Download PDF:
Previous Patent: NOVEL SURFBOARD
Next Patent: CHANNEL TRANSMISSION METHOD, TERMINAL DEVICE AND NETWORK DEVICE
Next Patent: CHANNEL TRANSMISSION METHOD, TERMINAL DEVICE AND NETWORK DEVICE