CROSS REFERENCE TO OTHER APPLICATIONS

[0001] This application claims priority from U.S. Provisional Application 61/510,912, filed July 22, 2011. The disclosure of that application is incorporated herein by reference in its entirety.

BACKGROUND

[0002] Recent advances in the identification of numerous biomarkers that are linked to various diseases underscore the potential of their clinical utility in disease diagnosis, prognosis, identifying at-risk subjects, drug target engagement, monitoring of disease progression during therapy, and prediction of therapeutic drug response. Quality control methods and systems have been developed with the intent to accurately measure biomarker levels in cells and tissues.

[0003] What is needed, however, is an automated quality control system that can process and analyze assay data for controls and subject samples from multiple runs, across instruments, and over time to ensure the quality of the data.

SUMMARY OF THE INVENTION

[0004] Disclosed herein are automated systems, devices and methods for applying quality control to assay results from in situ analysis of cell, tissue or bodily fluid samples. The quality control reduces data errors caused by variations among subjects, instruments, samples, reagents, assays, and individual runs of an assay.

[0005] Systems and methods are provided herein for processing assay results from an assay run, wherein the assay determines the level of one or more biomarkers in a cell, tissue or bodily fluid sample. In some embodiments, a method includes a) performing quality control for each biomarker in said run, b) performing subject sample quality control for each subject in said run, and c) accepting assay results when (a) and (b) are within predefined quality limits.

[0006] In certain implementations, the method also includes utilizing detected levels of two or more biomarkers in a multivariate index assay (MIA). When an MIA is performed, a method of the invention also includes performing MIA quality control, and accepting assay results and/or MIA scores when MIA quality control data is within predefined quality limits.

[0007] In some embodiments, a system for processing assay results from at least one run of an assay that determines the level of at least one biomarker includes an analyzer that performs quality control for each biomarker in said run, performs subject sample quality control for each subject in said run, and accepts assay results when biomarker quality control and subject sample quality control are within predefined quality limits. The analyzer may also be configured to utilize detected levels of two or more biomarkers in a MIA. The analyzer may perform MIA quality control for a run, and may accept MIA results or scores when the MIA quality control is within predefined quality limits.

[0008] In some embodiments, a system for processing assay results from at least one run includes:

a) a biomarker control for quality control for each biomarker;

b) a subject control for quality control for each subject;

c) one or more autostainers configured to stain controls a) and b);

d) one or more scanners configured to scan the stained controls; and

e) an analyzer configured to accept assay results when the biomarker quality control and subject quality control are within predefined quality limits.

The analyzer may utilize detected levels of two or more biomarkers in a MIA, and one or more MIA controls may be included for MIA quality control. The analyzer may accept assay and/or MIA scores when MIA quality control data is within predefined limits.

[0009] In some embodiments, a kit for measuring the quality of a run from an assay that determines the level of one or more biomarkers includes:

a) a biomarker control for quality control for each biomarker;

b) a subject control for quality control for each subject in the run; and

c) reagents that detectably label the biomarkers.

[0010] Systems, methods, and devices discussed herein are useful for determining information with respect to diagnosis, risk of disease progression and recurrence, disease state, metastatic potential, metastatic stage, predicting or monitoring therapeutic response, and efficacy of therapy with precision and accuracy. Systems, methods, and devices discussed herein are also useful for identifying subjects at risk of a disease or identifying a therapy suitable for a particular subject.

[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Exemplary methods and materials are described below, although methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention. All publications and other references mentioned herein are incorporated by reference in their entirety. In case of conflict, the present specification, including definitions, will control. Although a number of documents are cited herein, this citation does not constitute an admission that any of these documents forms part of the common general knowledge in the art. Throughout this specification and claims, the word "comprise," or variations such as "comprises" or "comprising" will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers. The materials, methods, and examples are illustrative only and not intended to be limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The foregoing and other objects and advantages will be appreciated more fully from the following further description thereof, with reference to the accompanying drawings. These depicted embodiments are to be understood as illustrative and not as limiting in any way:

[0013] Figs. 1A and IB show flow diagrams of illustrative automated quality control systems;

[0014] Fig. 2 is an illustrative biomarker control with five levels of quantitated standards for a biomarker;

[0015] Fig. 3 is an illustrative MIA control with three levels of quantitated standards for a biomarker and MIA +/- control standards;

[0016] Fig. 4 is an illustrative slide with biomarker control standards and a subject sample;

[0017] Fig. 5 represents an illustrative daily quality control for a MIA with five biomarkers and six subjects;

[0018] Fig. 6 is an illustrative setup of an automated quality control system that performs biomarker- specific, subject- specific and MIA-specific checks;

[0019] Fig. 7 is an illustrative Levy- Jennings control chart indicating an out-of-range data point; and [0020] Fig. 8 is an illustrative Levy- Jennings multi-biomarker control chart useful for analyzing instrument trends.

DETAILED DESCRIPTION OF THE INVENTION

[0021] To provide an overall understanding of the systems, devices, and methods described herein, certain illustrative embodiments will now be described. For the purpose of clarity and illustration, the systems and methods will be described with respect to in situ assays performed on tissue samples. It will be understood by one of ordinary skill in the art that the systems and methods described herein may be adapted and modified as is appropriate, and that the systems and methods described herein may be employed in other suitable

applications, such as for other types of assays and samples, and that such other additions and modifications will not depart from the scope hereof. The present invention encompasses such additions, modifications, and uses.

[0022] The term "multivariate index assay," or "MIA" generally refers to an assay that includes measurements from two or more biomarkers and applies an interpretation function to yield a single result or score. An MIA may be an in vitro diagnostic multivariate index assay (IVDMIA).

[0023] A "biomarker," "marker," or "feature" refers to an analyte that can be objectively measured and evaluated as an indicator for a biological state. Examples include, but are not limited to, a protein, a lipid, a metabolite, a nucleic acid sequence, a glycolipid, a

glycoprotein, a polypeptide, an antigen, an antibody, an epitope, DNA, mRNA, cDNA, microRNA, or other suitable analyte.

[0024] The term "reagent" generally refers to, but is not limited to, nucleic acids,

oligonucleotides, antibodies, antigen -binding fragments of antibodies, aptamers or other naturally-occurring or synthetic molecules used for biomarker detection. The term also encompasses reagents that affect the quality of assay results including but not limited to buffers (e.g. antigen retrieval buffer), solvents (e.g. xylene, ethanol), fluorescent dyes, signal amplification systems (e.g. Tyrimade Signal Amplification, TSA kit from Perkin Elmer or Life Technologies), 4'6-diamidino-2-phenylindole (DAPI), mounting media with anti-fade reagents, or other suitable reagents. A reagent may optionally be detectably labeled.

[0025] The term "predefined quality limits," "predefined limits," or "tolerance" generally refers to an acceptable error for each controlled feature in a run, e.g., with respect to biomarker controls, subject controls or MIA controls. The measurement value is relative with respect to each control, e.g., for biomarker controls, subject controls or MIA controls. Data from an assay run may be plotted or compared, manually or automatically, on a chart, such as a Levy- Jennings control chart, to detect any runs that are outside of the acceptable range.

[0026] The term "out-of-range," "failed run" or "failed batch" generally refers to a run that is not within predefined quality limits or tolerances.

[0027] The term "subject sample," "tissue sample" or "subject-specific sample" refers to a tissue sample, a bodily fluid sample, circulating tumor cells, or other suitable bodily derived sample. Bodily fluid samples include blood, plasma, urine, saliva, lymph fluid, cerebrospinal fluid (CSF), synovial fluid, cystic fluid, ascites, pleural effusion, interstitial fluid and ocular fluid. Tissue samples include a solid tissue, a formalin-fixed paraffin embedded tissue sample, a snap-frozen tissue sample, an ethanol-fixed tissue sample, a tissue sample fixed with an organic solvent, a tissue sample fixed with plastic or epoxy, a cross-linked tissue sample, surgically removed tumor tissue and a biopsy sample. A tissue sample can be a cancerous tissue sample. A cancerous tissue sample can be from any solid or liquid tumor, including but not limited to melanoma, prostate cancer, breast cancer, colon cancer, lung cancer, kidney cancer, pancreatic cancer, brain cancer, leukemia, lymphoma or myeloma. Subject samples can be from a human or animal.

[0028] The term "run," "assay run" or "batch" refers to a set of samples, which may include subject samples and quality control samples, tested by contemporaneously to produce a set of assay data.

[0029] In various embodiments, an automated quality control system that can process and analyze assay data for controls and subject samples to ensure the quality of the data is provided. Results of a single assay run are released only when the appropriate controls are validated. The automated system applies clinical pathology quality control rules, which are typically time-dependent utilizing a single type of instrument, to analytical methods including but not limited to immunohistochemistry or to nucleic acid hybridization, protection, or amplification, traditional in situ hybridization (FISH, miRNA hybridization), as well as newer technologies that rely on nuclease protection assays (HTG) or branched amplification schemes (Panomics and Advanced Cell Diagnostics), which are batch-dependent and rely on multiple different instruments and different types of instruments to determine a result.

[0030] The above methodology is preferably operator-, instrument-, and site-independent, and thus inter-instrument reproducibility and inter-batch precision are achieved. Provided each result or score is within a predefined tolerance, the method reduces the need to correct for small system variations (e.g., one instrument with a slightly lower intensity light source or varied light path). Calibration between instruments is also obviated and thus reliance on correction factors is reduced. Larger variances, if detected, are corrected by optional calibration standards. The methodology is applicable to singleplex (detection of one biomarker per sample) and multiplex (detection of multiple biomarkers per sample) reactions. The method can also be applied to quantitative and semi-quantitative analysis methods. In some embodiments, the automated system detects out-of-range runs and attributes errors to a faulty autostainer, faulty hybridization chamber, faulty scanner or faulty reagent.

Additionally, the automated system utilizes quality control rules across different biomarkers to determine autostainer, hybridization chamber and scanner trends, thus allowing

preventative maintenance prior to a failed batch. Accordingly, a control system is provided that monitors data quality across different platforms (e.g., autostainers and hybridization chambers and scanners) thereby eliminating the need to predetermine or correct for the particular instrument combination used for a given run of an assay.

[0031] In some embodiments, an assay relates to the diagnosis and therapeutic management of cancer. Various cancer biomarkers have been elucidated for diagnosis and prognosis in cancer patients, and the methods discussed herein contemplate use of such biomarkers, and provide a more reliable diagnostic result obtained from analysis of the biomarkers. In some embodiments, methods are provided that can inform about the risk of disease progression and recurrence (e.g., the metastatic, recurrence, or lethal potential of a cancer) with precision and accuracy using the various biomarkers, and can produce more accurate and precise measurement of the expression or activity levels of the biomarkers in a tissue sample from a subject. The quality control methods also can be used to predict and/or monitor the efficacy of a cancer therapy (e.g., surgery, radiation therapy, chemotherapy, or targeted therapy) independent of, or in addition to, traditional, established risk assessment procedures. The quality control methods also can be used to identify subjects in need of aggressive cancer therapy (e.g., adjuvant therapy), or to guide further diagnostic tests (e.g., sentinel lymph node biopsy). The levels can also be used to inform subjects about which types of therapy they would be most likely to benefit from, and to stratify patients for inclusion in a clinical study. The quality control methods also can be used to identify subjects who will not benefit from and/or do not need cancer therapy (e.g., surgery, radiation therapy, chemotherapy, targeted therapy, or adjuvant therapy).

[0032] A biomarker can be measured in situ by various approaches. For example, one may measure the RNA transcript levels (e.g., mRNA or total RNA levels) or gene copy numbers of the biomarkers, or may measure the protein or activity levels of the biomarkers. In some embodiments, one may also measure metabolites (e.g., peptide fragments) of the biomarkers, or surrogates of the biomarkers (e.g., substrates or ligands of the biomarkers, or biological entities downstream in the signaling pathways of the biomarkers). The term "metabolite" includes any chemical or biochemical product of a metabolic process, such as any compound produced by the processing, cleavage or consumption of a biomarker. Metabolites can be detected in a variety of ways known to one of skill in the art including fluorescence analysis. In some embodiments, post-translational modifications of a biomarker may be relevant to some diseases or conditions. Such modifications include, without limitation, phosphorylation (e.g., tyrosine, threonine, or serine phosphorylation), methylation, acetylation, SUMOylation, ubiquitination and glycosylation (e.g., O-GlcNAc). Such modifications may be detected, for example, by antibodies specific for the modifications.

[0033] At the nucleic acid level, biomarkers may be measured by in situ hybridization (e.g., single or multiplex nucleic acid in situ hybridization technology such as Advanced Cell Diagnostic's RNAscope® technology), or quantitative nuclease protection assay (e.g.

Highthroughput Genomics qNPA™). RNAse protection assays, Panomics QuantiGene® Plex technology can also be used to assess the RNA levels of biomarkers.

[0034] Exemplary methods for proteins include, without limitation, immunoassays such as immunohistochemistry assays (IHC) and immunofluorescence assays (IF). In immunoassays, one may use, for example, antibodies that bind to a biomarker or a fragment thereof. The antibodies may be monoclonal, polyclonal, and may be non-human, human, or humanized. One may also use antigen-binding fragments of a whole antibody, such as single chain antibodies, Fv fragments, Fab fragments, Fab' fragments, F(ab') ₂ fragments, Fd fragments, single chain Fv molecules (scFv), bispecific single chain Fv dimers, diabodies, domain- deleted antibodies, single domain antibodies, and/or an oligoclonal mixture of two or more specific monoclonal antibodies. All of the foregoing antibodies and fragments may be detectably labeled, or detected with a detectably labeled secondary antibody or other signal- generating amplification scheme (e.g., TSA).

[0035] To determine whole cell and/or subcellular levels of a biomarker, one may use any of a number of computational methods that are well-known, such as AQUA® (see, e.g., U.S. Patents 7,219,016, and 7,709,222; Camp et al., Nature Medicine, 8(11): 1323-27 (2002)), and Definiens Developer or TissueStudio ^™ (see, e.g., U.S. Patents 7,873,223, 7,801,361,

7,467,159, and 7,146,380, and Baatz et al., Comb Chem High Throughput Screen, 12(9):908- 16 (2009)). [0036] One of skill in the art will appreciate that a sample utilized in the measurement of a biomarker can be any sample suitable for this purpose. In some embodiments, the sample is from a cancerous tissue. A cancerous tissue sample includes, for example, any sample derived from a cancerous tissue of a subject, or from a tissue that is suspected to be cancerous. A sample can be, by way of example, tissue biopsies, blood, serum, plasma, blood cells, endothelial cells, lymphatic fluid, ascites fluid, interstitial fluid, bone marrow, cerebrospinal fluid, saliva, mucous, sputum, sweat, urine, circulating tumor cells, and circulating endothelial cells.

[0037] A sample may be fresh, frozen (e.g., snap-frozen), fixed (e.g., by formalin, ethanol, or an organic solvent, or with plastic or epoxy), embedded (e.g., in paraffin or wax), and/or cross-linked. The sample may be taken as core biopsies, punch biopsies, fine needle aspirations, surgically removed tumor tissue, or tumor-derived cells grown in vitro or in live animals. In some embodiments, the sample may be formalin-fixed paraffin-embedded biopsies.

[0038] A tissue sample may be collected from a subject that is preferably a mammal. The mammal can be a human, non-human primate, mouse, rat, dog, cat, horse, or cow, but is not limited to these examples. Mammals other than humans can be advantageously used as subjects that represent animal models of the disease or condition for which the testing is performed. A subject can be male or female. In embodiments relating to cancer, a subject can be one who has been previously diagnosed or identified as having a primary tumor or a metastatic tumor, and optionally has already undergone, or is undergoing, a therapeutic intervention for the tumor such as surgery. Alternatively, a subject can be one who has not been previously diagnosed as having a primary or metastatic tumor, including one who exhibits one or more risk factors for a primary or metastatic tumor. In some embodiments, a subject has a primary tumor, a recurrent tumor, a metastatic tumor or a tumor of unknown primary ("TUP").

[0039] Fig. 1A is a flow diagram 100 illustrating a process for analyzing data from a run of an assay that determines the level of one or more detectably labeled biomarkers present in subject samples. Data obtained from the run may be analyzed for biomarker quality control, subject quality control, or both. If biomarker quality control data for a single biomarker in the run fails to meet set tolerances, all data in the run may be rejected or, alternatively, only data relating to the failed biomarker may be rejected. If subject quality control for a single subject fails to meet set tolerances, all data in the run may be rejected or, alternatively, only data relating to the failed subject may be rejected. [0040] Quality control for each biomarker is performed at step 10, in which biomarker controls are prepared and scanned for quality control evaluation. The biomarker controls contain a quantitated level of one or more biomarkers to be measured in the run. In some embodiments, the controls are stained with reagents to specifically detect the biomarkers on the slides and are analyzed to detect a quantitated level. The detection may be done by image analysis, for example by an analyzer that images the controls and analyzes the images to determine a detected biomarker level from the image by measuring pixel density or using any other suitable quantitative image analysis method.

[0041] The detected biomarker levels for the biomarker controls are compared to the known quantitated levels of the controls to obtain an error measurement, and the analyzer determines whether the determined error is within acceptable quality tolerances. If the error is not within set tolerances, data for subject samples in the run is rejected and flagged for review or re-run at step 15.

[0042] If biomarker control data passes quality control at step 10, then data for the subject samples is released at step 40 for analysis at step 50. The data may be analyzed, for example, using a laboratory information system (LIS). An LIS may be any suitable analysis module, and may be a computer or other system programmed to process the data or present the data to a user for manual analysis. The analysis performed at step 50 depends on the type and application of the assay being run. For example, the analysis may involve reading raw data scores obtained at step 10 to determine biomarker levels in subject samples and producing diagnostic information or a prediction for a particular disease. The analysis may also indicate disease risk, therapy efficacy, disease state, or any other suitable information. A report of the analysis, which may be a data output or a physical print out, is generated at step 60.

[0043] In addition to the biomarker quality control run at step 10, subject quality control is run at step 20. The assay run includes one or more subject controls that are stained and processed at step 20. The subject controls may be negative control slides that are either unstained samples or samples stained identically as the test samples except the biomarkers are not stained or are stained with an antibody specific to the biomarker from a different species. The controls are processed by the same methods used to analyze subject samples to detect any signal present in the controls. The detected signal, if any, may be attributed to background signal from the subject samples or an interaction between the subject samples and a reagent that may interfere with the assay readings for subject samples.

[0044] The detected levels for the controls are used to obtain a background error

measurement, and an analyzer determines whether the error is within acceptable quality tolerances. If the error is not within the tolerances, data from the run is rejected and flagged for review or re-run at step 25. The rejected data may be all data from the assay run, or may be only data relating to the particular subjects associated with the controls that failed the quality control check. If the subject control data passes quality control at step 20, data for subject samples is released at step 40 for analysis at step 50 and report generation at step 60.

[0045] In certain implementations, a quality control method includes utilizing detected levels of two or more biomarkers in a MIA. In the MIA, detected levels for multiple biomarkers are obtained and processed to produce a single data result or score, for example a high or low risk indication for a particular disease. In addition to biomarker and subject quality control, a MIA quality control may be applied to ensure the quality of the MIA data. The MIA data results are accepted when MIA quality control data is within predefined quality limits, and the limits can be set as desired for a particular application. MIA quality control for a certain assay run may include analysis of controls stained for a single biomarker, but preferably includes analysis of controls stained for two or more biomarkers or controls each stained for one of a plurality of biomarkers utilized in the MIA.

[0046] Fig. IB is a flow diagram 200 illustrating a process for performing quality control for a MIA. In addition to biomarker quality control performed at step 110 and subject quality control performed at step 120, process 200 includes MIA quality control performed at step 130. The MIA quality control is run on MIA controls that have at least one standard, and preferably more than one standard, that is known to produce a particular MIA result or score when run correctly. For example, for an MIA run that determines high or low risk for a particular disease, each MIA control may include one standard known to be a high risk standard and one standard known to be a low risk standard.

[0047] The MIA controls are stained and analyzed along with subject samples. During initial validation of the system, multiple sets of MIA controls may be run for the quality control, or a set of MIA controls may be run multiple times to produce an adequate number of MIA quality control data points. For each data point obtained, the MIA quality control output is compared to the expected output for the MIA controls to determine an error measurement. In certain implementations, the error measurement may be taken from the correlation between detected MIA scores and expected MIA scores. For example, the error measurement for a high-risk/low-risk MIA may be the percentage of known high-risk standards and low-risk standards that were read correctly. The error measurement is compared to quality control tolerances, which may be a minimum percentage correlation between detected MIA scores and expected MIA scores, and the data for the assay is released at step 140 if the error is within the tolerances. The error measurement may also be a single MIA score obtained from the controls in the run, and the data for the assay is released at step 140 if the score is read correctly. If the error measurement is not within the tolerances, the data from the run is rejected and flagged for review or re-run at step 135.

[0048] The released data is analyzed at step 150, for example using a LIS, to determine an MIA score for the subjects in the run. If all subject controls passed the subject quality control at step 120, then an MIA score is determined for each subject. If subject controls for one or more subjects did not pass subject quality control at step 120, then data for the failed subjects may be excluded, and no MIA score may be given for the failed subjects at step 150. The results of the analysis are used to generate a report at step 160.

[0049] Advantages of the systems, methods, and devices discussed herein include but are not limited to the ability to fully leverage automation, reduce systematic variations (e.g., by eliminating subjective readings from one or more pathologists, and obviating the need to account for light intensity or path variations) and more importantly, automatically assess quality prior to calculating a subject result thereby ensuring results will not be reported if any part of the quality control for the results is outside of its predefined tolerances.

Initial Validation Studies

[0050] Quality control for assay results incorporates single-batch or multi-batch biomarker quality control, subject quality control, and in some embodiments MIA quality control. In some embodiments, a biomarker control comprises a sample that is isolated or purified biomarker, or may be a sample of a cell line, a genetically-engineered cell line (e.g.

engineered to express increased or decreased levels of the biomarker(s) of interest), and/or a xenograft that expresses a quantitated level of one or more biomarkers. In other

embodiments, a biomarker control is composed of normal or diseased standards that express a quantitated level of the biomarkers. Biomarker controls may also contain one or more standards with quantitated levels of the biomarkers being checked for quality control, and preferably contain between three and five standards with different quantitated levels. In certain implementations, the true value of a standard is determined by measuring the expression level of each biomarker by a quantitative reference method (e.g.; real-time PCR, western blot, chromatography, or ELISA). The biomarker level may be measured with a reference method separately in standards with varying levels of a biomarker, or it may be measured in a single standard which is then diluted, and the lower levels then inferred. [0051] Fig. 2 depicts an illustrative quantitated control for a biomarker to determine analytical accuracy. In some embodiments, the control has calibrated standards of the biomarker at one or more levels, for example at one level, at two or more levels, at three or more levels or at five or more levels. Accuracy is determined for each biomarker on any autostainer or hybridization chamber in an assay using such quantitated standards. Images are captured on a primary slide scanner instrument and scored by a computerized image analysis system. Analytical accuracy is the correctness of the result. In this case, correctness is determined by comparison of the obtained score to the known amount of the biomarker on the control. If the biomarker measurement using the in situ technology is accurate, then the measured levels will be close to the true values as measured by the reference method, and the correlation between the two will be linear within the measureable range. This may be performed only once initially during setup or repeated periodically to confirm calibration.

[0052] The calibrated control shown in Fig. 2 also includes a barcode that indicates identifying information for the control. The calibration information may include the type of biomarker present in the standards on the slide, the quantitated levels of the standards, an identification number of the particular control, or any other suitable identifying information. While a barcode is shown in Fig. 2, a control may include a barcode, a QR code, text, color coding, or any other suitable indication for the identifying information. The code can be used to track the control, and may be automatically read by an analysis system to obtain calibration information for the biomarker and the quantitated standards of the biomarker that are on the control.

[0053] Fig. 3 depicts an illustrative control used to measure precision, another aspect of initial validation. Precision, also referred to as reproducibility, is the degree to which repeated measurements give the same result. Precision is determined by measuring the same sample several times and calculating the coefficient of variation, which is the standard deviation of the repeated measurements divided by the mean. The controls shown in Fig. 3 have three levels of biomarker standards, but a control may include one level, two levels, three levels, or more than three levels of a biomarker. Precision for a given instrument combination may vary depending on biomarker level because precision is a function of a mean of measurements, and thus differing levels of the biomarker standards may be desired to measure precision at multiple biomarker levels. A standard used for measuring precision must be intrinsically consistent so that any variation reflects the precision of the measurement rather than standard variation. Preferably, the biomarker controls are composed of tissue, cell lines, engineered cell lines, purified biomarkers, and/or xenografts that express predetermined differing levels of the biomarkers. Precision studies may be performed once initially to determine the reproducibility of the instruments and to establish control charts for each biomarker and/or instrument.

[0054] Controls can also have positive and negative standards for the characteristic being tested in the assay. For example, a control may include low and high risk prognostic or negative and positive diagnostic standards that are scored across all biomarkers. The positive diagnostic standards are samples known to produce a positive MIA score when the assay process is working correctly. Likewise, the negative diagnostic standards are samples known to produce a negative MIA score when the assay process is correct. Each control in a set of controls can have the same MIA standards but each control may be stained for a different biomarker or combination of biomarkers.

[0055] Controls used for biomarker, subject, or MIA quality control in an assay may also contain subject samples. Fig. 4 shows a slide having biomarker control standards and a subject sample. In some embodiments, the slide may have MIA control standards in place of or in addition to the biomarker control standards shown. The slide shown in Fig. 4 may be provided to a technician with the biomarker control standards included on the slide, and the technician may apply the subject sample to the slide for use in a particular analysis. When an analyzer reads the slide, the barcode on the slide is read to obtain calibration information for the type and quantitated levels of the biomarker on the slide. The analyzer then images the slide and processes the image to evaluate the biomarker standards for quality control analysis and to evaluate the subject sample to obtain a biomarker reading for the subject.

[0056] In further aspects, a quality control method enables determining precision in a system where the quantitative results depend on two or more different types of instruments, in some embodiments a set of autostainers and/or hybridization chambers and a set of scanners is used in any combination. For intra- and inter-instrument precision, a series of about 10-30 controls for each biomarker can be run on each autostainer or hybridization chamber instrument. For inter-batch precision, a control for each biomarker can be run on each autostainer or hybridization chamber instrument on a sufficient number of consecutive work days. Images are captured on a primary scanner instrument and scored with a computerized image analysis system. The mean and standard deviation are determined on each instrument for each level of biomarker control (e.g., for three levels). These values are plotted on a control chart to compare daily measurements of controls. For inter- instrument precision, additional autostainers or hybridization chambers are compared to a designated primary autostainer or hybridization chamber. [0057] In certain embodiments, precision is determined for each biomarker on a designated primary scanner by replicate testing of controls. Images are acquired on additional or secondary scanning instruments for a subset of controls from the autostainer or hybridization chamber precision study. The mean and standard deviation of each score are determined for each secondary scanner for each level of biomarker control. For inter- instrument precision each secondary scanner is compared to the primary scanning instrument.

[0058] In certain embodiments, a contingency table can be constructed containing a MIA score for high/low risk determination or +/- presence of disease for all precision runs. The MIA is based on computational analysis across raw biomarker scores. Preferably, approximately >95% concordance is expected across all runs without having to apply scanner-, autostainer-, or hybridization chamber- specific correction factors.

[0059] Accordingly, the systems, methods and devices discussed herein provide a more precise and accurate assay as a result of accepting assay results only when data from a run falls within a predefined tolerance.

Daily Quality Control

[0060] In various embodiments, a quality control method links the biomarker control slides, runs, and subject sample slides across all instruments used for analysis so that a computerized system can process the assay data. Fig. 5 shows an exemplary daily quality control setup for "MIA #1" with five biomarkers, indicated by feature numbers, and six patients, indicated by patient ID numbers. In this example, automated quantitative quality control procedures are applied to quantitative immunohistochemistry. Such an automated quality control system can be used to check batch-specific, subject- specific, and MIA-specific values prior to releasing assay results and/or MIA scores.

[0061] Fig. 6 shows three sets of slides in an assay indicated for MIA, biomarker and subject quality control to qualify and calculate a subject result. Each MIA control slide has three biomarker- specific spots with different levels of a biomarker and two MIA spots that each produce a known MIA score when analyzed across the five control slides shown. MIA- dependent quality control is determined from the MIA spots for this set of slides from the same set of runs. Calculated MIA scores must correspond to the expected high-risk (or positive) and low-risk (or negative) MIA scores within a set percentage error tolerance.

[0062] For biomarker-dependent quality control, calculated levels for the spots on the MIA control slides are obtained by image acquisition and analysis. Each calculated biomarker level must fall within a pre-determined range relating to the known quantitated levels for each spot as established by initial precision studies for the batch to pass biomarker quality control for each biomarker.

[0063] For subject quality control, a control slide from each subject is included in the run. A set of sample slides for each subject is linked to the appropriate subject- specific control slide and must pass biomarker quality control, subject quality control, and MIA quality control. Accordingly, the quality control process provides methods for using data to assess runs, and then using data from multiple runs to calculate MIA scores. The method allows for automatic assessment prior to calculating assay results and MIA scores and, furthermore, prevents results from being reported if any part of the quality control is not within a predefined tolerance. If biomarker-dependent or MIA-dependent quality control does not meet acceptability criteria, then all of the assay results relating to any of the unacceptable quality control results are prevented from being reported. If only a subject- specific quality control result does not meet acceptability criteria, then only data relating to that specific subject may be prevented from being reported.

[0064] A quality control chart (e.g., Levy- Jennings chart) can be constructed for each biomarker from data obtained in precision studies. The same control slides can be included in every biomarker run independent of which instruments (for example, autostainer, hybridization chamber, or scanner) are used. The mean and standard deviation are established with the initial precision studies. An image captured from a slide-mounted standard can be converted into numerical biomarker values by an image analysis system and plotted in a chart. It is then determined whether the run passes the predefined level of the biomarker value set for LIS analysis and release of the results. Failure to conform to the predefined value level indicates an error in the system, therefore, the run does not pass the quality control and the results are rejected. In Fig. 7, the biomarker low-level control chart depicts a failed run that is above a predefined standard deviation range that contains the mean and e.g., +1 SD, +2 SD, +3 SD ranges. Each point on the chart is a reading of the control biomarker level standard on a specific batch, and quality control for each biomarker must pass for the batch prior to release of data. Additional quality checks can be run by monitoring a rule -based algorithm that tracks trends over multiple consecutive data points on the control chart.

[0065] Optionally, if tighter calibration is required between instruments, then calibration standards (measured value vs. expected value) can be included in every batch. The expected and measured results can be determined automatically for each batch and a batch- specific correction factor also can be calculated. The correction factor can be compared to preset criteria to ensure that it is acceptable. If the correction factor is acceptable, then it can be applied to all biomarker results for that run, including the control slide values, prior to any quality control checks on the data.

[0066] Biomarker data from an instrument (for example an autostainer, hybridization chamber, or scanner) can be used to detect instrument trends. For example, a failing light in a scanner would be reflected in a pattern of consecutive measurements across all of the biomarkers captured by the scanner. In Fig. 8, each individual point is within the set control parameters, but four points in a row are more than 1 SD below the mean signaling a possible trend in an instrument used to obtain the data.

[0067] Accordingly, in preferred embodiments, the method further comprises analyzing data from a plurality of runs to identify trends for individual instruments or reagents that may indicate an improperly functioning analysis system. The method detects out-of-range runs that may be caused by a fault in an autostainer, a hybridization chamber, a scanner or a reagent as shown in Figs. 7 and 8.

[0068] In various embodiments, any number of biomarkers may be analyzed to obtain assay data. In certain implementations, one, two, three, four, or five or more biomarkers are analyzed. An MIA comprises analysis of at least two biomarkers and, while there is no predefined optimum number of biomarkers, generally three to ten biomarkers may be preferable. Each run also contains a negative subject control slide. Therefore, each assay depends on mating multi-run/biomarker- specific quality controls and a subject- specific negative control. In some embodiments, the assay further incorporates MIA quality control.

Quality Control for Cancer Assay

[0069] In certain embodiments, a method provides for analyzing a tissue sample from a cancer patient or potential cancer patient. A tissue sample is obtained from a subject and the levels of two or more biomarkers in the sample are measured as described in Example 1. The methods employed can be used for diagnosis or prognosis, to define metastatic potential or stage, predict therapeutic response, or to select a treatment for a patient in need thereof.

[0070] According to the method, one obtains a tissue sample from the subject patient and measures the levels of biomarkers in the sample. Example 1 describes an assay using a formalin-fixed paraffin embedded primary melanoma tumor sample. Generally, biomarker control slides contain a quantitated standard of a biomarker. In some embodiments, control slides further include high risk or positive ("+") and a low risk or negative ("-") control standards. For each subject, the slides are prepared for the biomarkers and the negative control. A separate run with a slide from each subject and a biomarker- specific control slide is performed for each biomarker. The runs for one set of biomarkers and the negative subject control are performed on one autostainer and the runs for the other set of biomarkers are performed on a second autostainer. In preferred embodiments, a quantitative assay employs fluorescence from Alexa dyes, Cy dyes, or Atto dyes or radioactivity, e.g., a radiolabeled ligand for signal detection. Semi-quantitative assays can be used and utilizes chromogenic stains selected from 3,3' diaminobenzidine [DAB], 3-Amino-9-ethyl carbazol (AEC), 5- bromo-4-chloro-3-indoyl phosphate and Nitroblue tetrazolioum (BCIP/NBT), Vector Blue and Fast Red. Subject and biomarker control slides are then scanned using fluorescence or bright field image scanners, and the images are captured in a computer readable media. Slides stained for a first set of biomarkers and the negative subject control are scanned on one scanner, and slides stained for a second set of biomarkers are scanned on a second scanner. The levels of biomarkers can be quantitated using a customized computer algorithm (e.g., Definiens, AQUA, inForm), and a score is calculated for each biomarker on each sample and control slide. The standard and subject data can be transferred to a laboratory information system (LIS) for quality analysis.

[0071] In a run for a particular biomarker, a result is determined for the quantitated standard or standards. Each biomarker control result is compared to predetermined tolerances to perform quality control. The results may be used for a quality control chart in some cases by plotting the results, manually or automatically, in a control chart such as a Levy- Jennings graph. If the control result falls within a certain predefined quality limit of the previously established mean for this control, the assay run is accepted. The accepted results for the subject samples are then used or stored for use in the assay analysis. The quality controls are specific for each biomarker level and type, however, they are independent of which autostainer or hybridization chamber and scanner combination is used for any run. A significant advantage is the ability to minimize or eliminate correction factors and calibration between instruments. The method enables determining precision in a system where the quantitative results depend on multiple different types of instruments, in this case a series of autostainers and a series of scanners in any combination. The methods provide for a quality control system that is effective across different platforms (autostainers and hybridization chambers and scanners) without having to predetermine which instrument combination will be used for a given test. [0072] In embodiments that involve a MIA, the final calculation combines results from all of the biomarkers for a given set of subject samples. This result or score links the high and low risk human tissue controls (MIA +/-) on control slides and therefore is quantitated in each separate biomarker run. The next quality control step therefore involves performing the final calculation on the MIA +/- by an algorithm. If the correlation of the scores for the high risk control is within the predefined quality limit and the correlation of the scores for the low risk control is within the predefined quality limit, the MIA run is acceptable.

[0073] Quality control also is performed separately on each subject- specific negative control to minimize or eliminate falsely-elevated biomarker levels due to non-specific antibody, nucleic acid, or other subject- specific feature interactions. Accordingly, the results for each subject include validated results for all the biomarkers, validated MIA high/low or +/- scores and validated subject- specific negative control results. The LIS performs the final calculation for each subject using the biomarker results and produces an MIA score for each subject that is ready for release. In preferred embodiments, a MIA score indicates at least one of a diagnosis, prognosis, disease state, metastatic potential, metastatic stage, therapeutic response, and efficacy of therapy.

Kits

[0074] In some implementations, a kit is provided for measuring the quality of an assay from one or more assay runs, wherein the assay comprises determining the level of one or more biomarkers in said run, said kit comprising:

a) a biomarker control for performing quality control for each biomarker in said run, wherein each biomarker control includes at least one quantitated standard of a biomarker;

b) a subject control for performing subject quality control for each subject in said run; and c) reagents that specifically detect or label said biomarkers.

[0075] The levels of biomarkers may be measured using a kit with detection reagents that specifically detects and quantify the biomarkers. The detection reagents may have been detectably labeled, or the kit may provide labeling reagents for conjugation to the detection reagents. The kit may comprise detection reagents, e.g., antibodies and/or oligonucleotides, that can bind to biomarker proteins (or fragments thereof) or nucleic acids, respectively. In some embodiments, the biomarkers are proteins and the kit contains antibodies that bind to the biomarkers. In other embodiments, the biomarkers are nucleic acids and the kit contains oligonucleotides or aptamers that bind to the biomarkers. In some embodiments, the oligonucleotides may be fragments of the biomarker genes. For example the oligonucleotides can be 200, 150, 100, 50, 25, 15, or fewer nucleotides in length.

[0076] A kit also may contain in separate containers a nucleic acid or antibody control formulation (positive and/or negative), and/or a detectable label such as fluorescence from Alexa dyes, Cy dyes, Atto dyes, or the detectable label may be a radiolabel. For a semiquantitative assay, the kit may include chromogenic stains selected from 3,3'

diaminobenzidine [DAB], 3-Amino-9-ethyl carbazol (AEC), 5-bromo-4-chloro-3-indoyl phosphate and Nitroblue tetrazolioum (BCIP/NBT), Vector Blue and Fast Red. Instructions for carrying out the assay may be included in the kit.

EXAMPLE

[0077] Further details of the invention will be described in the following non-limiting example. It should be understood that this example, while indicating preferred embodiments, is provided by way of illustration only, and should not be construed as limiting. From the present disclosure and this example, one skilled in the art can ascertain certain characteristics of this invention, and without departing from the spirit and scope thereof, can make various changes and modifications to adapt it to various usages and conditions.

Quality Control Melanoma Multivariate Index Assay

[0078] An assay for melanoma detects ten biomarkers (ANLN, MMPl, SPARC, CDH2, FSCNl, CD117, KIF2C, DEPDC1, CD44, PCNA) with quantitative fluorescent

immunohistochemistry. Subject samples are formalin-fixed paraffin embedded from primary melanoma tumors. Control slides contain biomarker- specific cell lines with three different levels of expression of each of the above-mentioned biomarkers, plus human tissue controls containing a high and a low risk score calculated across all of the biomarkers. For each subject, 11 slides are prepared for the ten biomarkers and the negative control (no primary antibody applied) using five-micron thick sections, and the slides are deparaffinized with xylene/ethanol rinses. Antigen retrieval is performed using the Lab Vision PT module (Lab Vision, Fremont, CA). A separate run with a slide from each subject and a biomarker- specific control slide is performed for each biomarker. In this example six subjects are included, so each run contains six patient slides and one control slide.

Immunohistochemistry is performed using Lab Vision Autostainer 360s (Lab Vision Corp. Fremont, CA). The runs for biomarkers ANLN, MMPl, SPARC, CDH2, and FSCNl and the negative subject control are performed on Autostainer#l and the runs for biomarkers CD117, KIF2C, DEPDC1, CD44, and PCNA are performed on Autostainer#2. All ten primary antibodies were mouse anti-human (custom manufactured) against each biomarker.

Biomarker detection is performed with secondary HRP-labeled goat anti-mouse antibodies combined with Alexa Fluor 647-tyramide signal amplification (Life Technologies, Carlsbad, CA). Slides are counter- stained with DAPI and cover slipped using Pro Long® Gold mounting media (Life Technologies, Carlsbad, CA).

[0079] After drying, the subject and control slides are scanned using Aperio ScanScopeFL fluorescence image scanners (Aperio Technologies, Vista, CA) and the images are stored on the local server. The slides for biomarkers ANLN, MMPl, SPARC, CDH2, and FSCNl and the negative subject control are scanned on ScanScopeFL#l and the slides for biomarkers CD117, KIF2C, DEPDC1, CD44, and PCNA are scanned on ScanScopeFL #2. The biomarkers on each image are quantitated using the AQUA computer algorithm (HistoRX, New Haven, CT) and a normalized AQUA score is calculated for each biomarker on each subject and control slide. The quality control and subject AQUA score results are transferred to the Orchard Harvest laboratory information system (LIS) (Orchard Software, Carmel, IN) for quality analysis and final calculations.

[0080] Quality control is performed in an automated manner within the Orchard Harvest LIS. For each run, the biomarker- specific quality control AQUA score results are plotted on a Levy- Jennings graph and the results are compared to the mean, e.g., +/- 3SD for the specific biomarker. For example, in the run for ANLN, an AQUA score is determined for the high, medium, and low expression level controls. The high-level ANLN control AQUA score is plotted on the high-level ANLN Levy- Jennings graph. If the AQUA score falls within 3SD of the previously established mean for this control, the high-level controls is acceptable. The same process is repeated for the medium and low expression controls and if they also fall within e.g., 3SD of the mean for their respective Levy- Jennings graphs, the ANLN run is accepted as a valid run. Once the ANLN run is accepted, the LIS transmits the subject- specific ANLN AQUA scores to the results file for each of the six subject samples in the run. Of note the Levy- Jennings graphs are specific for each level and type of biomarker control, but they are independent of which Autostainer and ScanScopeFL combination are used for any run. The same quality control procedure is repeated for the MMPl, SPARC, CDH2,

FSCNl, CD117, KIF2C, DEPDC1, CD44, and PCNA runs and each run was independently validated. At completion of this quality control portion, each subject record contains a validated AQUA score result for each of the ten biomarkers, but the results are not released for final calculations until additional quality control is performed. [0081] The melanoma assay is a multivariate index assay and the final calculation combines AQUA score results from all ten of the biomarkers. All of the biomarker runs for a given set of subject samples together form a master run. The master run is linked by the high and low risk human tissue controls that are embedded onto each control slide and therefore quantitated by each separate biomarker run. The next quality control step therefore involves performing the final calculation on the high and low risk human tissue controls in the master run. If the FINAL SCORE for the high risk control is above the established cut-off and the FINAL SCORE for the low risk control is below the established cut-off so the master run is acceptable.

[0082] Quality control is also performed separately on each subject- specific negative control AQUA score result to assure that no falsely-elevated biomarker levels due to non-specific antibody or reagent interactions occur. For example for Subject#l, if the negative control AQUA score is below the previously established cut-off, the LIS transmits the negative control results to the results file for Subject#l. Since the results file for Subject#l contains validated AQUA scores for ANLN, MMP1, SPARC, CDH2, FSCN1, CD117, KIF2C,

DEPDC1, CD44, and PCNA, validated high and low master results, and validated negative control results, the LIS performs the final calculation for Subject#l using the ten biomarker AQUA scores and records the FINAL SCORE in the Subject#l report for release. The same process is repeated for the remaining five subjects.

[0083] The LIS system also uses the AQUA scores from the control slides to monitor result trends and alert operators, based on established rule sets, to possible process errors or instability (reagents, instruments, or technical concerns). For example, if five consecutive AQUA scores for the ANLN control are below the mean on the ANLN Levy-Jennings graph, or two consecutive scores are more than 2SD below the mean, then the operator is alerted to check for anti-ANLN degradation. If five consecutive AQUA scores from any biomarker controls, where the slides are scanned with ScanScopeFL#l, are below the mean, then the operator is alerted to check for light source degradation in ScanScope#l. If five consecutive AQUA scores from any biomarker controls, where the slides are stained in Autostainer#l, are below the mean, then the operator is alerted to check for reagent integrity in Autostainer#l.

[0084] It is to be understood that the foregoing description is merely illustrative and is not to be limited to the details given herein. While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems, devices, and methods, and their components, may be embodied in many other specific forms without departing from the scope of the disclosure. [0085] Variations and modifications will occur to those of skill in the art after reviewing this disclosure. The disclosed features may be implemented, in any combination and

subcombinations (including multiple dependent combinations and subcombinations), with one or more other features described herein. The various features described or illustrated above, including any components thereof, may be combined or integrated in other systems. Moreover, certain features may be omitted or not implemented. Examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the scope of the information disclosed herein.