Title:
MODEL PERFORMANCE EVALUATION METHOD AND APPARATUS, DEVICE AND MEDIUM
Document Type and Number:
WIPO Patent Application WO/2023/216899
Kind Code:
A1
Abstract:
According to embodiments of the present disclosure, provided are a model performance evaluation method and apparatus, a device and a medium. The method comprises: at a client node, obtaining a plurality of prediction scores output by a machine learning model for a plurality of data samples, the plurality of prediction scores respectively indicating a prediction probability that the plurality of data samples belong to a first category or a second category; modifying a plurality of truth tags on the basis of a random response mechanism to obtain a plurality of protected tags, the plurality of truth tags respectively annotating that the plurality of data samples belong to the first category or the second category; determining, on the basis of the plurality of protected tags and the plurality of prediction scores, error metric information related to a predetermined performance index of the machine learning model; and transmitting the error metric information to a service node. Therefore, the purpose of privacy protection of local tag data of the client node is achieved while realizing model performance evaluation.
More Like This:
Inventors:
SUN JIANKAI (US)
YANG XIN (US)
WANG CHONG (US)
XIE JUNYUAN (CN)
WU DI (CN)
YANG XIN (US)
WANG CHONG (US)
XIE JUNYUAN (CN)
WU DI (CN)
Application Number:
PCT/CN2023/091142
Publication Date:
November 16, 2023
Filing Date:
April 27, 2023
Export Citation:
Assignee:
BEIJING BYTEDANCE NETWORK TECH CO LTD (CN)
LEMON INC (GB)
LEMON INC (GB)
International Classes:
G06F21/62
Foreign References:
| CN111488995A | 2020-08-04 | |||
| CN111861099A | 2020-10-30 | |||
| CN114169010A | 2022-03-11 | |||
| CN113222180A | 2021-08-06 | |||
| US20150379429A1 | 2015-12-31 |
Other References:
SUN JIANKAI, YANG XIN, YAO YUANSHUN, XIE JUNYUAN, WU DI, WANG CHONG: "Differentially Private AUC Computation in Vertical Federated Learning", ARXIV (CORNELL UNIVERSITY), CORNELL UNIVERSITY LIBRARY, ARXIV.ORG, ITHACA, 24 May 2022 (2022-05-24), Ithaca, XP093107217, Retrieved from the Internet [retrieved on 20231130], DOI: 10.48550/arxiv.2205.12412
CHEN, CHUAN ET AL.: "FedGL: Federated Graph Learning Framework with Global Self-Supervision", BAIDU, [ONLINE], [RETRIEVAL DATE 2023-7-6].[RETRIEVAL ON THE INTERNET]: URL: HTTPS://ARXIV.ORG/PDF/2105.03170.PDF, 7 May 2021 (2021-05-07)
CHEN, CHUAN ET AL.: "FedGL: Federated Graph Learning Framework with Global Self-Supervision", BAIDU, [ONLINE], [RETRIEVAL DATE 2023-7-6].[RETRIEVAL ON THE INTERNET]: URL: HTTPS://ARXIV.ORG/PDF/2105.03170.PDF, 7 May 2021 (2021-05-07)
Attorney, Agent or Firm:
SHIHUI PARTNERS (CN)
Download PDF: