Title:
RANK SELECTION IN TENSOR DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS
Document Type and Number:
WIPO Patent Application WO/2021/102679
Kind Code:
A1
Abstract:
Tensor decomposition can be advantageous for compressing deep neural networks (DNNs). In many applications of DNNs, reducing the number of parameters and computation workload is helpful to accelerate inference speed in deployment. Modern DNNs comprise multiple layers with multi-array weights where tensor decomposition is a natural way to perform compression-in which the weight tensors in convolutional layers or fully-connected layers are decomposed with specified tensor ranks (e.g., canonical ranks, tensor train ranks). Conventional tensor decomposition with DNNs involves selecting ranks manually, which requires tedious human efforts to finetune the performance. Accordingly, presented herein are rank selection embodiments, which are inspired by reinforcement learning, to automatically select ranks in tensor decomposition. Experimental results validate that the learning-based rank selection embodiments significantly outperform hand-crafted rank selection heuristics on a number of tested datasets, for the purpose of effectively compressing deep neural networks while maintaining comparable accuracy.
More Like This:
Inventors:
CHENG ZHIYU (US)
LI BAOPU (US)
FAN YANWEN (CN)
BAO YINGZE (US)
LI BAOPU (US)
FAN YANWEN (CN)
BAO YINGZE (US)
Application Number:
PCT/CN2019/120928
Publication Date:
June 03, 2021
Filing Date:
November 26, 2019
Export Citation:
Assignee:
BAIDU COM TIMES TECH BEIJING CO LTD (CN)
BAIDU USA LLC (US)
BAIDU USA LLC (US)
International Classes:
G06N3/04
Foreign References:
| CN105637540A | 2016-06-01 | |||
| CN107944556A | 2018-04-20 | |||
| US20190180144A1 | 2019-06-13 | |||
| US5293456A | 1994-03-08 |
Attorney, Agent or Firm:
INSIGHT INTELLECTUAL PROPERTY LIMITED (CN)
Download PDF: