Title:
METHOD, SYSTEM AND DEVICE FOR PERFORMING KNOWLEDGE DISTILLATION ON NEURAL NETWORK MODEL, AND MEDIUM
Document Type and Number:
WIPO Patent Application WO/2022/037165
Kind Code:
A1
Abstract:
A method, system and device for performing knowledge distillation on a neural network model, and a medium. Said method comprises: acquiring a first neural network model on which knowledge distillation is to be performed; acquiring a second neural network model; generating a target sample set for training the second neural network model; and performing knowledge distillation on the second neural network model by taking the target sample set as an input of the second neural network model and taking an output of the first neural network model as an output of the second neural network model, so as to obtain a target neural network model. In the present application, in the process of performing knowledge distillation on a neural network model, a new target sample set is regenerated to perform knowledge distillation, rather than performing knowledge distillation by directly using a training sample set used in the process of training a first neural network model, and thus a training sample set does not need to be acquired, thereby increasing the efficiency of knowledge distillation of neural network models.
Inventors:
ZHU KEFENG (CN)
KAN HONGWEI (CN)
TONG PEILIN (CN)
KAN HONGWEI (CN)
TONG PEILIN (CN)
Application Number:
PCT/CN2021/096207
Publication Date:
February 24, 2022
Filing Date:
May 27, 2021
Export Citation:
Assignee:
INSPUR SUZHOU INTELLIGENT TECHNOLOGY CO LTD (CN)
International Classes:
G06N3/04
Foreign References:
| CN111985620A | 2020-11-24 | |||
| CN111160533A | 2020-05-15 | |||
| CN109800821A | 2019-05-24 |
Attorney, Agent or Firm:
UNITALEN ATTORNEYS AT LAW (CN)
Download PDF: