Title:
METHOD AND APPARATUS FOR DATA SET CACHING IN NETWORK TRAINING, DEVICE, AND STORAGE MEDIUM
Document Type and Number:
WIPO Patent Application WO/2022/110861
Kind Code:
A1
Abstract:
A method and apparatus for data set caching in network training, a device, and a storage medium. The method comprises the steps of: collecting statistics about disk performance overhead required by training nodes, which have not cached a data set to be trained, in a network training cluster during a process of caching a data set to be trained; monitoring current performance parameters of the training nodes that have not cached a data set to be trained; selecting, on the basis of the current performance parameters, from the network training cluster a target node meeting the disk performance overhead requirement; and using the target node to cache a data set to be trained transmitted from a source node, so as to perform network training on said data set on the basis of the target node. The present method can ensure the reliability of the training node caching the data set to be trained, thereby ensuring the overall reliability of the network training. The apparatus for data set caching, the device, and the storage medium also have the beneficial effects as described above.
Inventors:
ZHAO RENMING (CN)
CHEN PEI (CN)
CHEN PEI (CN)
Application Number:
PCT/CN2021/109237
Publication Date:
June 02, 2022
Filing Date:
July 29, 2021
Export Citation:
Assignee:
INSPUR SUZHOU INTELLIGENT TECHNOLOGY CO LTD (CN)
International Classes:
G06N3/04; G06N3/08
Foreign References:
| CN112446490A | 2021-03-05 | |||
| CN109710406A | 2019-05-03 | |||
| CN103905517A | 2014-07-02 | |||
| CN111625696A | 2020-09-04 | |||
| CN111459506A | 2020-07-28 |
Attorney, Agent or Firm:
UNITALEN ATTORNEYS AT LAW (CN)
Download PDF: