SELF-DISTILLATION TRAINING METHOD AND DEVICE FOR CONVOLUTIONAL NEURAL NETWORK, AND SCALABLE DYNAMIC PREDICTION METHOD

WIPO Patent Application WO/2021/023202

A1

The present invention provides a self-distillation training method for a convolutional neural network, for use in significantly improving the performance of a convolutional neural network by reducing the size of the convolutional neural network instead of expanding the size of the network. When knowledge is distilled within a network itself, the network is first divided into several parts; then, knowledge in a deep part of the network is pressed into a shallow part. Without taking the response time as the cost, self-distillation greatly improves the performance of a convolutional neural network, achieving an average accuracy improvement of 2.65%; the 0.61% accuracy improvement for a data set ResNeXt is the minimum value and the 4.07% accuracy improvement for VGG19 is the maximum value. In combination with enhanced extraction of features of shallow classifiers by an attention layer, the accuracy of the shallow classifiers is significantly improved; thus, a convolutional neural network having multiple outputs can be regarded as multiple convolutional neural networks, and the output of each shallow classifier can be used according to different needs.

MA KAISHENG (CN)
ZHANG LINFENG (CN)

PCT/CN2020/106995

February 11, 2021

August 05, 2020

Click for automatic bibliography generation  

INSTITUTE FOR INTERDISCIPLINARY INFORMATION CORE TECH XIAN CO LTD (CN)

G06N3/04

CN110472730A	2019-11-19
CN107229942A	2017-10-03
CN103679185A	2014-03-26

BEIJING BAO HU INTELLECTUAL PROPERTY AGENCY CO., LTD. (CN)

View/Download PDF      PDF Help

Previous Patent: PIXEL ARRAY, ARRAY SUBSTRATE, AND DISPLAY DEVICE

Next Patent: OPERATION STRUCTURE OF CIRCUIT BREAKER