Title:
METHODS, DEVICES AND MEDIA FOR RE-WEIGHTING TO IMPROVE KNOWLEDGE DISTILLATION
Document Type and Number:
WIPO Patent Application WO/2022/217856
Kind Code:
A1
Abstract:
Methods, devices and processor-readable media for re-weighting to improve knowledge distillation are described. A reweighting module may be used to determine relative weights to assign to a ground truth label and dark knowledge distilled from the teacher (i.e. the teacher output logits used as soft labels). A meta-reweighting method is described to optimize the weights for a given labeled data sample.
Inventors:
LU PENG (CA)
RASHID AHMAD (CA)
REZAGHOLIZADEH MEHDI (CA)
GHADDAR ABBAS (CA)
RASHID AHMAD (CA)
REZAGHOLIZADEH MEHDI (CA)
GHADDAR ABBAS (CA)
Application Number:
PCT/CN2021/121751
Publication Date:
October 20, 2022
Filing Date:
September 29, 2021
Export Citation:
Assignee:
HUAWEI TECH CO LTD (CN)
International Classes:
G06N3/08
Domestic Patent References:
| WO2019240964A1 | 2019-12-19 |
Foreign References:
| US20200104717A1 | 2020-04-02 | |||
| CN111223483A | 2020-06-02 | |||
| US20200302295A1 | 2020-09-24 |
Download PDF: