Title:
DEVICE AND METHOD FOR PERFORMING ADAM GRADIENT DESCENT TRAINING ALGORITHM
Document Type and Number:
WIPO Patent Application WO/2017/185257
Kind Code:
A1
Abstract:
A device and method for performing Adam gradient descent training algorithm, the device comprising a direct memory access unit (1), an instruction cache unit (2), a controller unit (3), a data cache unit (4) and a data processing module (5). The method comprises: firstly, reading a gradient vector and a to-be-updated value vector, and initializing a first-order vector, a second-order vector and a corresponding exponential decay rate; at each iteration, updating the first-order vector and second-order vector by using the gradient vector, and calculating a first-order biased estimation vector and a second-order biased estimation vector; updating to-be-updated parameters by using the first-order biased estimation vector and second-order biased estimation vector; and continuing with training till the vector of the to-be-updated parameters is converged. The present invention may achieve the application of the Adam gradient descent algorithm, and greatly improve the data processing efficiency.
Inventors:
GUO QI (CN)
LIU SHAOLI (CN)
CHEN TIANSHI (CN)
CHEN YUNJI (CN)
LIU SHAOLI (CN)
CHEN TIANSHI (CN)
CHEN YUNJI (CN)
Application Number:
PCT/CN2016/080357
Publication Date:
November 02, 2017
Filing Date:
April 27, 2016
Export Citation:
Assignee:
CAMBRICON TECH CO LTD (CN)
International Classes:
G06N3/063; G06N3/08
Foreign References:
| CN105184366A | 2015-12-23 | |||
| CN105184369A | 2015-12-23 | |||
| CN103956992A | 2014-07-30 | |||
| US20130325401A1 | 2013-12-05 |
Attorney, Agent or Firm:
CHINA SCIENCE PATENT & TRADEMARK AGENT LTD. (CN)
Download PDF:
Previous Patent: RMSPROP GRADIENT DESCENT ALGORITHM EXECUTION APPARATUS AND METHOD
Next Patent: FULLY AUTOMATIC VEGETABLE CUTTER
Next Patent: FULLY AUTOMATIC VEGETABLE CUTTER