ARTIFICIAL NEURAL NETWORK OPTIMIZATION METHOD AND SYSTEM BASED ON ORTHOGONAL PROJECTION MATRIX, AND APPARATUSES

Title:

ARTIFICIAL NEURAL NETWORK OPTIMIZATION METHOD AND SYSTEM BASED ON ORTHOGONAL PROJECTION MATRIX, AND APPARATUSES

Document Type and Number:

WIPO Patent Application WO/2020/172974

Kind Code:

A1

Abstract:

The present invention belongs to the field of machine learning and artificial intelligence, particularly relates to an artificial neural network optimization method and system based on an orthogonal projection matrix, and apparatuses, and aims to solve the problem of catastrophic forgetting occurring during continuous learning by an artificial neural network. The method comprises: initializing an artificial neural network, and calculating an orthogonal projection matrix set of each layer of the network; using the orthogonal projection matrix set to update a weight matrix of the artificial neural network, and processing input data of the current task; using a recursive algorithm to calculate a new projection matrix set, and using same to update a weight matrix of an artificial neural network of the next task; and repeating the execution of a recursive operation of a projection matrix and the updating of the weight matrix until the execution of all tasks in a task queue has been completed. The method can be applied to different task spaces, and can also be applied to a specific weight of a local network, and even a specific network. The method is simple in terms of calculation and has significant effects, and prevents the problem of "catastrophic" forgetting of a traditional artificial neural network.

More Like This:

WO/2022/015874	ARTIFICIAL NEURAL NETWORK FOR SPARSE DATA PROCESSING
WO/2023/059743	MACHINE LEARNING SYSTEMS FOR TRAINING ENCODER AND DECODER NEURAL NETWORKS
WO/2023/072528	METHOD AND CONTROL DEVICE FOR CONTROLLING A TECHNICAL SYSTEM

Inventors:

CHEN YANG (CN)
ZENG GUANXIONG (CN)
YU SHAN (CN)

Application Number:

PCT/CN2019/083355

Publication Date:

September 03, 2020

Filing Date:

April 19, 2019

Export Citation:

Click for automatic bibliography generation Help

Assignee:

INST AUTOMATION CAS (CN)

International Classes:

G06N3/08

Foreign References:

CN103559696A	2014-02-05
CN107480777A	2017-12-15
US20110161267A1	2011-06-30

Other References:

ZENG, GUANGXIONG, ET AL.: "Continuous Learning of Context-dependent Processing in Neural Networks", ARXIV E-PRINTS, 5 October 2018 (2018-10-05), XP055729159, DOI: 20191106172921X

Attorney, Agent or Firm:

HENYOL INTELLECTUAL PROPERTY LAW CORPORATION (CN)

Download PDF:

View/Download PDF PDF Help

Previous Patent: POLYEPTIDE CROSSLINKED PROTEIN MOLECULAR IMPRINTING POLYMER, PREPARATION METHOD THEREFOR, AND APPLIC...

Next Patent: DISPLAY DEVICE