Title:
TRAINING A NEURAL NETWORK USING PERIODIC SAMPLING OVER MODEL WEIGHTS
Document Type and Number:
WIPO Patent Application WO/2021/075735
Kind Code:
A1
Abstract:
A computer-implemented method includes: initializing model parameters for training a neural network; performing a forward pass and backpropagation for a first minibatch of training data; determining a new weight value for each of a plurality of nodes of the neural network using a gradient descent of the first minibatch; for each determined new weight value, determining whether to update a running mean corresponding to a weight of a particular node; based on a determination to update the running mean, calculating a new mean weight value for the particular node using the determined new weight value; updating the weight parameters for all nodes based on the calculated new mean weight values corresponding to each node; assigning the running mean as the weight for the particular node when training on the first minibatch is completed; and reinitializing running means for all nodes at a start of training a second minibatch.
Inventors:
TRIPATHI SAMARTH (KR)
LIU JIAYI (KR)
KURUP UNMESH (KR)
SHAH MOHAK (KR)
LIU JIAYI (KR)
KURUP UNMESH (KR)
SHAH MOHAK (KR)
Application Number:
PCT/KR2020/012379
Publication Date:
April 22, 2021
Filing Date:
September 14, 2020
Export Citation:
Assignee:
LG ELECTRONICS INC (KR)
International Classes:
G06N3/08; G06N3/04; G06N7/00
Domestic Patent References:
| WO2018140294A1 | 2018-08-02 |
Foreign References:
| US20160162781A1 | 2016-06-09 | |||
| KR20160102690A | 2016-08-31 | |||
| US20170278018A1 | 2017-09-28 |
Other References:
TRIPATHI SAMARTH, LIU JIAYI, KURUP UNMESH, SHAH MOHAK: "Robust Neural Network Training using Periodic Sampling over Model Weights", ARXIV.ORG/ABS/1905.05774V1, 14 May 2019 (2019-05-14), XP055803274, Retrieved from the Internet
See also references of EP 4046078A4
See also references of EP 4046078A4
Attorney, Agent or Firm:
HAW, Yong Noke (KR)
Download PDF:
Previous Patent: METHOD FOR SEPARATING DISODIUM 5'-INOSINATE
Next Patent: ELECTRONIC DEVICE AND METHOD FOR SHARING VOICE COMMAND THEREOF
Next Patent: ELECTRONIC DEVICE AND METHOD FOR SHARING VOICE COMMAND THEREOF