Field of the Invention

The invention comprises a secure and privacy-protected analysis system that includes algorithms in which the performance of diagnostic tests developed as a single or multicenter can be evaluated, the performances of more than one diagnostic test can be compared, and the best cut-off values for the diagnostic test with high performance can be determined.

State of Art of the Invention (Prior Art)

Although many domestic and/or foreign companies develop diagnostic kits on the problem side in the technical field of the invention, these companies do not develop the algorithms/systems needed for diagnostic kits as a secure and privacy-preserved system. As for the solution side, commercial or open-source statistical software is used to evaluate the performance of diagnostic tests. However, there is no algorithm method, system, process, or software in the world where the performance of diagnostic tests can be analyzed in a secure and privacy-preserved way. Therefore, it is considered that the developed method will benefit many domestic and foreign companies, particularly the big ones that develop diagnostic kits.

It is very important to be able to intervene in diseases with early and accurate diagnosis in the field of medicine. In recent years, an increasing interest in medical decision-making methods is observed, and the applications of these methods take a wide place in the medical literature. Most of the studies on diagnostic tests are devoted to researching the reliability of these methods and comparing them.

A diagnostic test is a general term referring to evaluation methods based on laboratory techniques, clinical observations, or original material measurements used to identify a disease or condition. Distinguishing the cases (e.g., sick individuals) and controls (e.g., healthy individuals) is aimed to be achieved by making use of various diagnostic methods and the results of laboratory tests. To analyze the results of a diagnostic test comprehensively and reliably, the first thing to do is to control the actual level of effectiveness of the diagnostic test. Recently, there are many statistical decision-making methods used for this purpose. ROC (Receiver Operating Characteristic) curve is the most commonly used method for this purpose.

Today, ROC analysis is a method used to determine the performance of diagnostic tests applied in different clinical situations and to evaluate the accuracy of statistical models such as logistic regression models and linear classifiers (e.g., linear discriminant analysis). Additionally, by considering the performance of a diagnostic test, ROC analysis also enables the development of new diagnostic tests. It also allows for the evaluation of diagnostic tests and prediction models, the explanation of diagnostic accuracy with numerical results, and the comparison between the diagnostic accuracy of predictions. Additionally, it determines the threshold value in clinical studies and reveals the non-evaluation results that are in its structure and are in between sensitivity- specificity. Its use has been increasing rapidly recently in many clinical areas, mainly in diagnosis, screening and particularly laboratory tests, epidemiology, radiology, and bioinformatics. Many articles published on ROC analysis have taken place in the literature in recent years. The use of diagnostic tests and ROC analysis plays a major role in studies particularly in the field of radiology and cardiology.

ROC analysis is a statistical method with a specific methodology. This analysis can be performed with both commercial and open-source software such as SAS, SPSS, JMP, R, and MedCalc. Furthermore, these analyses can be performed in the cloud environment using cloud-based software such as Intellectus, TURCOSA, and easyROC. A comprehensive guide on how to evaluate the performance of a diagnostic test using ROC analyses has been developed by the Clinical Laboratory Standards Institute (CLSI). This guide no EP24-A2 provides researchers with the methodological details and sample applications of the entire process from planning to reporting of such studies. The large diagnostic, pharmaceutical, and biotechnology companies in the world act according to these guidelines when evaluating the performance of the diagnostic markers they have developed. Institutions such as the FDA and the European Union require that in product applications, the research processes are carried out according to these guidelines. Although ROC analysis is a commonly used approach in routine, it proves insufficient for the said companies because of technological developments and legal regulations. Technical issues related to these analyses are explained in the next section.

The patents in the state-of-art regarding secure and privacy-preserved health systems are listed below.

Patent application no US2015261917A1 provides storage, access, editing, and sharing of health records through a secure and privacy-preserved system based on collaboration.

Patent application no US2009156906A1 provides a patient-centered model for prognosis by storing clinical, imaging, genomics, and proteomic data in a federated database.

Patent application no US5964891A provides a data access system with a distributed network.

Patent no US 10607156B2 includes systems and software methods for sharing health data.

The large diagnostic, pharmaceutical, and biotechnology companies in the World carry out the diagnostic kit development process in a multicentered manner. A study protocol and a statistical analysis plan in addition to this protocol are constructed within this scope. In pre- clinical and clinical research projects where the performance of diagnostic tests is evaluated, statistical analysis plans include ROC analyses. In this context, the data of all partners of the project are analyzed under this statistical analysis plan.

At this stage, two important problems are encountered. First of all, most of the project partners are other large companies, hospitals, or universities in the World. It is very important to be the first developer in such diagnostic test development projects. Therefore, project partners often do not share their project data. Applications made to institutions such as the FDA involve only the data of centers that accept data sharing. The other problem is the GDPR legislation, which comprises strict rules regarding the storage and sharing of patient data. The equivalent of this legislation in our country is the Law on Protection of Personal Data (KVKK). It is apparent that the patent applications in the state-of-art on secure and privacy-preserved health systems are insufficient in terms of the said problems.

Brief Description and Aims of the Invention

The advantages of the invention are that the said analyses are carried out in a secure system, companies can evaluate the performance of diagnostic tests without sharing their data, and patient data can be analyzed following laws such as GDPR/KVKK. It is expected that changes will be made in standard practices and guidelines in the World with the increase in such inventions. With the proposed invention, it is planned to construct a system that the large diagnostic, pharmaceutical, and biotechnology companies in the World can use. Additionally, influencing the standard guideline CLSI EP24-A2 used in this context is aimed.

There are many stages in the process of evaluating the validity of an analytical method, and at each stage, a large variety of statistical analyses need to be performed. Evaluation of the performance of diagnostic tests is just one of these stages. It is predicted that the developed technique can overcome the problems in the other stage. For instance, it is envisaged that the developed technique can be used for different purposes, such as investigating whether there is a systematic error between the methods by comparing analytical methods, precision analysis, calibration analysis, etc. However, by performing technical analyses, it should be demonstrated that the invention can offer solutions to these problems as well.

The invention comprises an algorithm developed within the scope of secure data analysis, which is a very current topic. Analyses for diagnostic tests in statistical software such as SAS, SPSS, JMP, R, and MedCalc are performed in software installed on a personal computer. Software such as Intellectus, TURCOSA, and easyROC enable these analyses to be webbased performed in the cloud environment by making use of the advantage of cloud technology. In logic, cloud-based analyses include sending data to the cloud and performing analysis with algorithms in the cloud environment. However, in most cases, it may not be possible to upload data to the cloud environment within the scope of GDPR/KVKK. The developed invention brings a solution to this problem, and the analyses are carried out without transferring the data of the users/companies. Algorithms developed in the cloud environment send a request to the user's computer and perform the data analysis based on the responses received, thereby not required to upload the data to external environments. With this feature, the invention can multicentrally analyze the data of the world's leading companies in multicenter studies and enables these companies to securely evaluate their data together with the data of other companies.

Drawings Explaining the Invention

The figures used to better explain the steps of the working method of the secure and privacy- preserved system developed with this invention to evaluate the diagnostic accuracy of the laboratory test are given below.

Figure 1 Process Steps of the Secure and Privacy-Preserved Method to Evaluate Diagnosis Accuracy of Laboratory Tests

Definition of Elements and Parts Composing the Invention

The parts and elements in the figures to better explain the working method of the secure and privacy-preserved system to evaluate the diagnostic accuracy of the laboratory test developed with this invention are enumerated separately and listed below.

Ul: User 1

U2: User 2

U3: User 3

X : Analysis parameter of user 1

Y : Analysis parameter of user 2

Z : Analysis parameter of user 3

Stati: First candidate parameter

Stat ₂ : Second candidate parameter

Stats: Third candidate parameter

Stat _m : Last candidate parameter

Statu: Response of User 1 to the first candidate parameter

Stat 1 ₂ : Response of User 2 to the first candidate parameter

Stati ₃ : Response of User 3 to the first candidate parameter Stat2i: Response of User 1 to the second candidate parameter

Stat22: Response of User 2 to the second candidate parameter

Stat23: Response of User 3 to the second candidate parameter

Stat _m i: Response of User 1 to the last candidate parameter

Stat _m 2: Response of User 2 to the last candidate parameter

Stat _m 3: Response of User 3 to the last candidate parameter

Detailed Description of the Invention

Within the scope of the invention, a system that comprises secure and privacy-preserved statistical algorithms through which the performance of diagnostic tests can be evaluated and a method using this system is provided.

With the invention, a system that solves the problems in the patents in the state-of-art and comprises statistical algorithms through which the performance of diagnostic tests will be evaluated is provided. Therefore, the priority within the scope of the invention is to overcome the problems in the diagnostic kit development process by developing statistical methodologies that provide this opportunity.

The stages of the federated learning algorithm, which are in the working principle and can be used in the invention, are designed. Secure analysis approaches have been developed for each stage of the evaluation of the performance of diagnostic tests. The developed algorithms were coded on the R programming language and many trials were performed. The invention includes the following steps:

1. Developing a secure data analysis algorithm by which ROC analyses can be performed in evaluating the performance of a diagnostic test and coding it in the R programming language,

2. Developing secure data analysis algorithms to compare the performance of multiple diagnostic tests and coding them in the R programming language, 3. Developing secure data analysis algorithms and coding them in R programming language to determine the best cut-off value(s) of the diagnostic test,

4. Performing software trials, comparing the results with the results of current software, performing validations

Accordingly, the invention consists of a method comprising algorithms through which the performance of diagnostic tests is evaluated and software comprising these algorithms.

The system proposed within the scope of the invention is based on the idea that analyses can be performed without having the data of a user to evaluate the performance of the diagnostic test. Within the scope of the invention, it is possible to evaluate the performance of diagnostic tests without transferring the data. A system was created in which the performance of the diagnostic test can be evaluated without the need for project partners to share their data in projects related to multicenter diagnostic test development. Also, problems related to storage and sharing of data resulting from the GDPR/KVKK legislation can be overcome with this approach.

The user selects the marker that she/he wants to perform the ROC analysis and defines the distribution range to the system. She/he also chooses the number of values within the distribution range that is desired to scan, along with its random or systematic features. If no selection is made, 30 values are systematically determined in the data set. Each value is sent to the user's computer in turn. Information on how many observations is below and above this value is retrieved from the user's computer. When this process is completed for all values, sensitivity and selectivity statistics are calculated for each possible value. As a result of combining these statistics, the ROC analysis curve is obtained. Similar procedures can be easily performed when comparing multiple ROC curves, or when it is desired to determine the cut-off value for a diagnostic test. In the performance of these operations, the server sends various requests to the user's computer through the system installed on the computer. As a result of these requests, instead of data, various descriptive statistics are accessed. As using these descriptive statistics may be sufficient for the said analyses to be performed, the analyses are performed securely without needing to upload the data on the user's computer to the cloud environment. A secure and privacy-preserved system to evaluate the diagnostic accuracy of the laboratory test, wherein; it comprises

• A server in which the analysis algorithms are stored and the system is controlled,

• A device that enables the users to respond to the questions coming from the server, comprises the process steps of

• Server asking the analysis parameters to the user, (101)

• User determining analysis parameters and sending them to the server, (102)

• Server creating candidate parameters in accordance with the determined features, (103)

• Server sending the first candidate parameter to the users, (201)

• Users returning necessary statistics in response to request, (202)

• Server blending and storing the returned statistics, (203)

• Server sending the second candidate parameter to the users, (301)

• Users returning statistics in response to request, (302)

• Server blending and storing the returned statistics, (303)

• Server sending the last candidate parameter to the users, (401)

• Users returning necessary statistics in response to request, (402)

• Server blending and storing the returned statistics, (403)

• Server performing the analysis for the evaluation of the diagnostic test and its performance (404) using all the results it has.