Title:
INFERENCE SERVICE SYSTEM BASED ON KUBERNETES
Document Type and Number:
WIPO Patent Application WO/2021/238251
Kind Code:
A1
Abstract:
An inference service system based on Kubernetes, comprising a computing resource cluster and an inference service platform. The inference service platform comprises: a multi-framework model module used for supporting models exported from multiple frameworks; and a user-defined mirror image module used for obtaining a mirror image file sent by a user, performing deployment according to the mirror image file, and executing an inference service, wherein the mirror image file is a file obtained by packaging a trained model and a running environment by the user. Thus, in the present application, a trained model and a running environment are packaged in a mirror image form and then submitted to the inference service platform, the inference service platform deploys an online inference service in a parameter passing mode, inference tasks can be carried out without converting model types or considering model compatibility, and the inference service operation efficiency is improved.
Inventors:
WANG CHAO (CN)
WU SHAOHUA (CN)
CHEN QINGSHAN (CN)
ZHANG RONGGUO (CN)
LIN XIU (CN)
WU SHAOHUA (CN)
CHEN QINGSHAN (CN)
ZHANG RONGGUO (CN)
LIN XIU (CN)
Application Number:
PCT/CN2021/073345
Publication Date:
December 02, 2021
Filing Date:
January 22, 2021
Export Citation:
Assignee:
INSPUR SUZHOU INTELLIGENT TECHNOLOGY CO LTD (CN)
International Classes:
H04L29/08
Domestic Patent References:
| WO2019184750A1 | 2019-10-03 |
Foreign References:
| CN111629061A | 2020-09-04 | |||
| CN109272116A | 2019-01-25 | |||
| CN110058922A | 2019-07-26 |
Attorney, Agent or Firm:
UNITALEN ATTORNEYS AT LAW (CN)
Download PDF: