Title:
METHOD AND SYSTEM FOR LIGHTENING DEEP LEARNING MODEL USING PRUNING
Document Type and Number:
WIPO Patent Application WO/2023/022255
Kind Code:
A1
Abstract:
Disclosed are a method and a system for lightening a deep learning model using pruning. A method for lightening a deep learning model according to one embodiment may comprise the steps of: determining a significance level for channels for each layer of a deep learning model to be compressed; assigning an initial value for each channel according to the significance level of the channel; and initiating compression of the deep learning model at a target compression rate on the basis of the assigned initial value.
Inventors:
KIM TAE-HO (KR)
KANG SINHAN (KR)
SHIN EUNSEOP (KR)
KANG SINHAN (KR)
SHIN EUNSEOP (KR)
Application Number:
PCT/KR2021/011008
Publication Date:
February 23, 2023
Filing Date:
August 19, 2021
Export Citation:
Assignee:
NOTA INC (KR)
International Classes:
G06N3/04; G06N3/08
Other References:
PAJUSCO NICOLAS, HUANG RICHARD, FARRUGIA NICOLAS: "LIGHTWEIGHT CONVOLUTIONAL NEURAL NETWORKS ON BINAURAL WAVEFORMS FOR LOW COMPLEXITY ACOUSTIC SCENE CLASSIFICATION", DETECTION AND CLASSIFICATION OF ACOUSTIC SCENES AND EVENTS 2020, 1 November 2020 (2020-11-01), pages 135 - 139, XP093037633
WANG, ZHE N ET AL.: "Boosting Lightweight CNNs Through Network Pruning and Knowledge Distillation for SAR Target Recognition", IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 11 August 2021 (2021-08-11), pages 8386 - 8397, XP011876473, DOI: 10.1109/JSTARS.2021.3104267
SEUL-KI YEOM; PHILIPP SEEGERER; SEBASTIAN LAPUSCHKIN; ALEXANDER BINDER; SIMON WIEDEMANN; KLAUS-ROBERT M\"ULLER; WOJCIECH SAME: "Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 12 March 2021 (2021-03-12), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081895140, DOI: 10.1016/j.patcog.2021.107899
MOON HYEON-CHEOL, LEE HO-YOUNG, KIM JAE-GON: " Acceleration of CNN Model Using Neural Network Compression and its Performance Evaluation on Embedded Boards", THE KOREAN SOCIETY OF BROADCAST ENGINEERS AUTUMN CONFERENCE, pages 44 - 45, XP093037699, Retrieved from the Internet [retrieved on 20230405]
LEE SEON-WOO, HO-JUN YANG, SEUNG-YEON OH, MUN-HYUNG LEE, JANG-WOO KWON: "Application and Performance Analysis of Double Pruning Method for Deep Neural Networks. Convergence Society for SMB", JOURNAL OF CONVERGENCE FOR INFORMATION TECHNOLOGY, vol. 10, no. 8, 28 August 2020 (2020-08-28), pages 23 - 34, XP093037630, DOI: 10.22156/CS4SMB.2020.10.08.023
WANG, ZHE N ET AL.: "Boosting Lightweight CNNs Through Network Pruning and Knowledge Distillation for SAR Target Recognition", IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 11 August 2021 (2021-08-11), pages 8386 - 8397, XP011876473, DOI: 10.1109/JSTARS.2021.3104267
SEUL-KI YEOM; PHILIPP SEEGERER; SEBASTIAN LAPUSCHKIN; ALEXANDER BINDER; SIMON WIEDEMANN; KLAUS-ROBERT M\"ULLER; WOJCIECH SAME: "Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 12 March 2021 (2021-03-12), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081895140, DOI: 10.1016/j.patcog.2021.107899
MOON HYEON-CHEOL, LEE HO-YOUNG, KIM JAE-GON: " Acceleration of CNN Model Using Neural Network Compression and its Performance Evaluation on Embedded Boards", THE KOREAN SOCIETY OF BROADCAST ENGINEERS AUTUMN CONFERENCE, pages 44 - 45, XP093037699, Retrieved from the Internet
LEE SEON-WOO, HO-JUN YANG, SEUNG-YEON OH, MUN-HYUNG LEE, JANG-WOO KWON: "Application and Performance Analysis of Double Pruning Method for Deep Neural Networks. Convergence Society for SMB", JOURNAL OF CONVERGENCE FOR INFORMATION TECHNOLOGY, vol. 10, no. 8, 28 August 2020 (2020-08-28), pages 23 - 34, XP093037630, DOI: 10.22156/CS4SMB.2020.10.08.023
Attorney, Agent or Firm:
YANG, Sungbo (KR)
Download PDF: