Title:
MODEL INFERENCE METHOD, CLOUD PLATFORM, DEVICE AND STORAGE MEDIUM
Document Type and Number:
WIPO Patent Application WO/2023/065656
Kind Code:
A1
Abstract:
The present application provides a model inference method, a cloud platform, a device and a storage medium, belonging to the field of artificial intelligence. The method comprises: acquiring a first sub-model and a second sub-model obtained by model segmentation; configuring a first instance set, the first instance set comprising multiple first instances each loading the first sub-model; configuring a second instance set, the second instance set comprising multiple second instances each loading the second sub-model; configuring a first load balancer for the first instance set, the first load balancer being used to distribute multiple inference samples to multiple first instances in the first instance set for model inference, so as to generate multiple first inference results; configuring a second load balancer for the second instance set, the second load balancer being used to distribute the multiple first inference results to multiple second instances in the second instance set for model inference. Using the present application, each sub-model is loaded by multiple instances, such that model inference has relatively high reliability.
Inventors:
LIAN YUNWEN (CN)
LI YI (CN)
LIU CHANG (CN)
LI YI (CN)
LIU CHANG (CN)
Application Number:
PCT/CN2022/093378
Publication Date:
April 27, 2023
Filing Date:
May 17, 2022
Export Citation:
Assignee:
HUAWEI CLOUD COMPUTING TECH CO LTD (CN)
International Classes:
G06F9/50
Foreign References:
| CN112508188A | 2021-03-16 | |||
| CN112183668A | 2021-01-05 | |||
| CN109685202A | 2019-04-26 | |||
| US20210089969A1 | 2021-03-25 |
Attorney, Agent or Firm:
BEIJING SAN GAO YONG XIN INTELLECTUAL PROPERTY AGENCY CO., LTD. (CN)
Download PDF: