BACKGROUND

Several studies have demonstrated the potential value of measures of microvasculature in the management of diseases of the retina. Microvascular perfusion of the macula is particularly relevant for diabetic retinopathy (DR) progression and vision loss. The two vision- threatening complications of DR, proliferative retinopathy and center-involving macular edema, have been shown to be directly related to poor microvascular perfusion and increased leakage, respectively. Alterations of the microvascular perfusion have been associated with retinal microthrombosis, vascular remodeling, and increased microaneurysm turnover. Such alterations may occur predominantly in a specific phenotype of DR associated with more rapid progression to sight-threatening. Identification of eyes with DR that show lower vascular density in the macula may contribute to improved and individualized management of diabetic retinal disease with closer follow-up periods and earlier and more timely treatment of the sight-threatening complications of diabetes.

In particular, the density of vessels in the macula and the size of the foveal avascular zone (FAZ) are known to be affected by the presence of DR, and capillary closure may provide relevant information regarding progression of retinopathy in individual diabetic patients. However, the FAZ is known to have a wide range of values in normal eyes, which prevents it from being useful in detection of early disease. This is only one example of an avascular area/region that is present in normal eyes, which will tend to decrease the correlation between the total avascular area detected in an eye and the disease status. Besides the fovea avascular zone (FAZ), which is an avascular region present in most normal eyes, normal eyes also have other avascular regions, such as around retinal arteries. Even in the most densely vascular (e.g., microvascular) perfused areas of any eye, there are the regions between capillaries which are technically avascular.

SUMMARY

Eliminating nonessential areas of non-perfusion (e.g., avascular regions similar to those present in normal eyes) may improve the ability to use measures (e.g., metrics) of reduced perfusion to distinguish eyes with pathology from normal eyes. For example, a sum of all avascular areas that are not typical of a normal retina may be used to analyze a disease status. In some embodiments, a method is provided for detecting/identifying avascular regions in angiography image data, such as optical coherence tomography angiography (OCTA) image data, and optionally in more general vascular image data (such as that provided by an en face optical coherence tomography (OCT) image). The identified avascular regions

constitute/define a collection of identified avascular regions. The method may review the (e.g., collection of) identified avascular regions in an angiography image and further identify (or classify or flag) as nonessential any avascular region that is not used in (or pertinent to) a particular application/algorithm being implemented (and optionally removed (e.g., from consideration) the nonessential avascular regions, such as removing them from the collection of identified avascular regions). The remaining avascular regions may be termed (e.g., classified as) test avascular regions since they may be used in testing for an eye pathology. Nonessential avascular regions may include avascular regions typically found in normal healthy eyes (e.g., those not indicative of pathology) and/or avascular regions that correspond to areas/regions of an angiography image that are of low confidence (e.g., not visible/clear enough to provide useful information). For example, descriptive (or characteristic) features may be extracted from imaged avascular regions, and avascular regions whose descriptive feature(s) match(es) (or corresponds to) one or more predefined descriptive feature previously identified as being characteristic of a population of healthy eyes may be identified as nonessential, and optionally removed from further consideration. Image areas of low confidence may correspond to blurry image areas (e.g., areas having an opacity above a predefined opacity threshold) or to low contrast image areas (e.g., areas having a signal-to- noise ratio (SNR) below a predefined SNR level). It is to be understood that the predefined descriptive feature, predefined opacity threshold, predefined SNR level, and any other predefined value used in the present embodiments may be determined statistically (as described below), or manually defined/entered, or adjusted/updated in a feedback sequence, such as in a calibration stage.

Metrics based on such detection may be used for eye disease detection. Some embodiments of the present invention may involve generating OCTA image data, detecting a collection of avascular regions in the OCTA images, identifying as nonessential the detected avascular regions that are similar to those found in normal eyes (e.g., by use of extracted image descriptive features), removing the identified nonessential regions from the collection of avascular regions (or from the OCT image data), identifying the remaining avascular regions as "test avascular regions," and displaying or storing the resulting image and/or test avascular regions, or a further analysis thereof. Descriptive features identified as characteristic of a population of healthy eyes may include features that identify (or characterize) avascular regions typically found in normal/healthy eyes, such as the fovea avascular zone (FAZ), or avascular regions near (or substantially adjacent or around) large blood vessels (e.g. arteries) or the FAZ, or avascular regions smaller than a predefined size threshold. In some embodiments, the further analysis may include calculating a (e.g., cumulative) avascular region size (e.g., area) remaining after the removal of nonessential avascular regions and correlating the size to an eye disease status.

In some embodiments, characteristic features of avascular regions (and/or vascular regions) may be compared between two or more angiography images (or vascular images) of the same eye to identify changes in vasculature between the images (e.g., changes over time) that may be indicative of an eye pathology. In some embodiments, a database of avascular regions (and/or one or more of their corresponding characteristic features such as shape, size, relative location, etc.) may be maintained and used to aid in identifying pathologies in an angiography image. In some embodiments, the database may be used as part of a training set to train an artificial neural network to help analyze an angiography image.

The features and advantages described herein are not all-inclusive and many additional features and advantages may be apparent to one of ordinary skill in the art in view of the figures and description. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes and not to limit the scope of the inventive subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a generalized optical coherence tomography (OCT) system suitable for use with the present invention.

FIG. 2 is a flowchart of an example method for detecting and removing certain avascular regions from OCT angiography (OCTA) images for evaluating an eye pathology (or eye disease) according to one aspect of the present invention.

FIG. 3(A) shows an example original OCTA vasculature image. FIG. 3(B) shows the corresponding binarized vasculature map of the OCTA vascular image of Fig. 3(A), and FIG. 3(C) shows an inverse image of FIG. 3(B).

FIG. 4(A) shows an example resultant image after removal of the foveal avascular zone (FAZ) region. FIG. 4(B) shows the resulting image after removal of avascular regions around large vessels from the image of FIG. 4(A). FIG. 4(C) shows the resulting image after removing avascular regions that are smaller than a predefined (size) threshold from the image of FIG. 4(B).

FIG. 5 illustrates a plot of avascular region size distributions in normal/healthy eyes.

FIG. 6 (A) demonstrates the average and standard deviation of normal and diabetic retinopathy (DR) subjects. FIG. 6(B) shows a receiver operating characteristic (ROC) curve obtained from an avascular index of normal and DR subjects.

FIG. 7 demonstrates an avascular region detection comparison between normal and DR subjects.

FIG. 8 illustrates an example of a multilayer perception (MLP) neural network.

FIG. 9 shows a simplified neural network consisting of an input layer, a hidden layer, and an output layer.

FIG. 10 is an example of a registered image (e.g., a registered OCTA image) highlighting avascular regions, as may result from registering together two or more scans (e.g., a previous scan and a later scan) of the same retinal area.

FIG. 11 illustrates an example method for analyzing avascular regions of a retina in a current optical coherence tomography angiography (OCTA) image of an eye.

FIG. 12 is a block diagram of a general computer system that may perform functions discussed in this disclosure according to at least one aspect of the present invention.

DETAILED DESCRIPTION

All patent and non-patent references cited within this specification are herein incorporated by reference in their entirety to the same extent as if the disclosure of each individual patent and non-patent reference was specifically and individually indicated to be incorporated by reference in its entirety.

Example OCT System

A generalized FD-OCT system used to collect 3-D image data of the eye suitable for use with the present invention is illustrated in FIG. 1. An FD-OCT system 100 includes a light source, 101, typical sources including but not limited to broadband light sources with short temporal coherence lengths or swept laser sources. Abeam of light from source 101 is routed, typically by optical fiber 105, to illuminate the sample 110, a typical sample being tissues in the human eye. The source 101 can be either a broadband light source with short temporal coherence length in the case of SD-OCT or a wavelength tunable laser source in the case of SS-OCT. The light is scanned, typically with a scanner 107 between the output of the fiber and the sample, so that the beam of light (dashed line 108) is scanned laterally (in x and y) over the region of the sample to be imaged. Light scattered from the sample is collected, typically into the same fiber 105 used to route the light for illumination. Reference light derived from the same source 101 travels a separate path, in this case involving fiber 103 and retro-reflector 104 with an adjustable optical delay. Those skilled in the art recognize that a transmissive reference path can also be used and that the adjustable delay could be placed in the sample or reference arm of the interferometer. Collected sample light is combined with reference light, typically in a fiber coupler 102, to form light interference in a detector 120. Although a single fiber port is shown going to the detector, those skilled in the art recognize that various designs of interferometers can be used for balanced or unbalanced detection of the interference signal. The output from the detector 120 is supplied to a processor 121 that converts the observed interference into depth information of the sample. The results can be stored in the processor 121 or other storage medium or displayed on display 122. The processing and storing functions may be localized within the OCT instrument or functions may be performed on an external processing unit (e.g., the computer system 800 shown in FIG. 12) to which the collected data is transferred. This unit could be dedicated to data processing or perform other tasks which are quite general and not dedicated to the OCT device. The processor 121 may contain for example a field-programmable gate array (FPGA), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), a system on chip (SoC) or a combination thereof, that performs some, or the entire data processing steps, prior to passing on to the host processor or in a parallelized fashion. The sample and reference arms in the interferometer could consist of bulk-optics, fiber-optics or hybrid bulk-optic systems and could have different architectures such as Michelson, Mach- Zehnder or common-path based designs as would be known by those skilled in the art. Light beam as used herein should be interpreted as any carefully directed light path. Instead of mechanically scanning the beam, a field of light can illuminate a one or two-dimensional area of the retina to generate the OCT data (see for example, U.S. Patent 9332902; D. Hillmann et al, "Holoscopy- holographic optical coherence tomography" Optics Letters 36(13): 2390 2011; Y. Nakamura, et al, "High-Speed three dimensional human retinal imaging by line field spectral domain optical coherence tomography" Optics Express 15(12):7103 2007; Blazkiewicz et al, "Signal-to-noise ratio study of full-field Fourier-domain optical coherence tomography" Applied Optics 44(36):7722 (2005)). In time-domain systems, the reference arm needs to have a tunable optical delay to generate interference. Balanced detection systems are typically used in TD-OCT and SS-OCT systems, while spectrometers are used at the detection port for SD- OCT systems. The invention described herein could be applied to any type of OCT system.

In Fourier Domain optical coherence tomography (FD-OCT), each measurement is the real- valued spectral interferogram (Sj(k)). The real-valued spectral data typically goes through several postprocessing steps including background subtraction, dispersion correction, etc. The Fourier transform of the processed interferogram, results in a complex valued OCT signal output Aj(z)=|Aj|e ^ltp . The absolute value of this complex OCT signal, |Aj|, reveals the profile of scattering intensities at different path lengths, and therefore scattering as a function of depth (z-direction) in the sample. Similarly, the phase, cpj can also be extracted from the complex valued OCT signal. The profile of scattering as a function of depth is called an axial scan (A-scan). A set of A-scans measured at neighboring locations in the sample produces a cross-sectional image (tomogram or B-scan) of the sample. A collection of B-scans collected at different transverse locations on the sample makes up a data volume or cube. For a particular volume of data, the term fast axis refers to the scan direction along a single B-scan whereas slow axis refers to the axis along which multiple B-scans are collected. We use the term "cluster scan" herein to refer to a single unit or block of data generated by repeated acquisitions at the same location for the purposes of analyzing motion contrast. A cluster scan can consist of multiple A-scans or B-scans collected with relatively short time

separations at approximately the same location(s) on the sample. A variety of ways to create B-scans are known to those skilled in the art including but not limited to along the horizontal or x-direction, along the vertical or y-direction, along the diagonal of x and y, or in a circular or spiral pattern. The majority of the examples discussed herein refer to B-scans in the x-z dimensions but the invention would apply equally to any cross sectional image.

The OCT system may use any one of a number of OCT Angiography processing algorithms on one or more cluster scans of OCT data collected at the same or approximately the same transverse locations on a sample at different times. As previously mentioned, motion contrast algorithms can be applied to the intensity information derived from the image data (intensity- based algorithm), the phase information from the image data (phase-based algorithm), or the complex image data (complex-based algorithm). An en face vasculature image is an image displaying motion contrast signal in which the data dimension corresponding to depth is displayed as a single representative value, typically by summing or integrating all or an isolated portion of the data.

The OCT system discussed herein may provide 2D (i.e. cross-sectional) images, en- face images, 3-D images, metrics related to a health condition, and the like. This system may be used with any other system. For example, the OCT system may be used with a surgical system or surgical microscope system for diagnostic or treatment purposes. The OCT system may be used to analyze any sample. For example, the OCT system may be used in analysis, e.g. formation of images, of, for example, any type of life forms and inanimate objects.

Examples of life forms may be animals, plants, cells or the like.

Detection of Avascular Regions from Angiography (OCTA) Images

FIG. 2 is a flowchart of an example method 200 for detecting and removing certain avascular regions from angiography images (e.g., OCTA, Fluorescein Angiography, Fundus

Autofluorescence (FAF), etc.) for evaluating a diseased eye according to one aspect of the present invention. For ease of illustration, the present example of Fig. 2 assumes the use of OCTA images, but the present example may be extended to general vascular images, such as those provided by an en face OCT image. It should also be understood that the method 200 is not limited to the steps and/or operations referenced herein and that other steps and/or operations are also possible and are within the scope of the present disclosure. It should also be understood that not every step described herein must be performed.

The method 200 may begin by collecting three-dimensional OCT image data of an eye from an OCT system (step 202), such as the OCT system 100 shown in FIG. 1, or from a collection/library of OCT image data. The three-dimensional OCT image data may include multiple B-scans taken at each location of a particular region of the eye. The method 200 may calculate motion contrast information in the three-dimensional OCT image data and determine vasculature from the motion contrast information to obtain/define/generate an OCTA vascular image (step 204) using an OCT angiography (OCTA) processing technique (see for example, the OCTA vascular image of FIG. 3(A)). The OCTA processing technique may include, without limitation, an intensity-based processing technique, a phase-based processing technique, a combination of the intensity-based and phase-based techniques, and a complex- based processing technique.

Next, in step 206, a binarized vascular map of the OCTA vascular image may be generated, which could be used to distinguish the vessel region(s) (e.g., arteries or veins) from the background. FIG. 3(B) represents one such binarized vascular map of the OCTA vascular image of FIG. 3(A). Binarization may be performed by removing values below a noise value, rescaling the image, and then remapping the image intensities to use the full dynamic range of the grayscale image. Following this, one or more filters may be applied to enhance the vessels in the image and to reduce the image noise (examples of appropriate filters may include Hessian, Gabor and Frangi filters). After this, any values below a threshold may be set to zero and any values above the threshold may be set to 1. Morphological operations may be applied to clean the resulting binary image including removing small isolated dots and holes that contain only a few pixels.

In step 208, an inverse image may be calculated from the binarized vascular map. The inverse image may help label areas without vessels (avascular regions) in a way that allows them to be summed to give an overall area. FIG. 3(C) shows the inverse image. The images of FIGS. 3(B) and 3(C) both highlight avascular regions. In the present example, the image of FIG. 3(C) may be used to delineate the overall avascular regions (e.g., may be an example of a collection of identified avascular regions), including those avascular region that are typical of normal eyes (e.g., nonessential avascular regions in some embodiments), such as the foveal avascular zone 21, avascular regions around large vessels 23 (i.e., vessels wider than a predefined threshold), and avascular regions smaller than a predefined "normal" size 25, explained more fully below. At this point, it is also possible to multiply the inverted binary image by weightings so that the sum of non-zero pixels accounts for factors besides just presence or absence of vasculature in a particular pixel. The weighting can be zero if the avascular region is within the foveal avascular zone 21 (FAZ) or near an artery (such as avascular regions 23) or smaller than a predefined size threshold (such as avascular regions 25), and 1 otherwise, or it can be calculated to account for such things as distance from the fovea, shape of regions of interest, and other factors. Other image processing steps such as skeletonization, combinations of processed binarized images, or additional filtering might also be performed on the binarized image at any point in the processing pipeline to enhance the accuracy of the detection of areas that are truly avascular as a result of pathology.

After obtaining the overall binarized image of avascular areas, those regions typical of normal eyes (e.g., nonessential avascular regions) may then be eliminated from motion contrast step-by-step. For instance, in step 210, the FAZ region may be detected and removed from the overall avascular region image (demonstrated in FIG. 4(A)). Next, in step 212, the avascular regions around large vessels may be detected and eliminated

(demonstrated in FIG. 4(B)). This may include detecting and removing avascular regions hugging (e.g. contacting, or substantially adjacent to, or within a predefined distance from) large vessels (e.g. vessels larger (e.g., thicker/wider) than a predefined value (e.g., larger than a percentage (e.g., 65%) of the remaining vessels in the image or of vessels in a typical angiography image). In step 214, the avascular regions that are smaller than a certain size threshold may be detected as nonessential and removed (e.g., removed from the collection of identified avascular regions of Fig. 3(C)), as illustrated in FIG. 4(C). The avascular regions that are not removed from consideration may be termed "test avascular regions" and used for further analysis/testing.

In order to obtain a size threshold that characterizes normal, small avascular regions and that can work for most of the cases, the avascular images after removing the FAZ and removing avascular regions around larger vessels were collected from 53 normal subjects. The distribution of remaining, normal avascular regions was analyzed, as shown in FIG. 5. Based on the distributions of these normal avascular sizes, 99% of them are smaller than 290 pixels for 3x3 vasculature images. The size threshold for normal avascular regions could be set as 290 pixels for 3x3 vascular images. It is to be understood that that this pixel size (or density or area) may be scaled in accordance with a change in image size, and/or converted to a standard unit of measure (e.g., microns or square microns), and/or converted to a ratio (e.g., ratio of avascular area to vascular area, or ratio of avascular area to total image area). The avascular regions in the angiography image that are smaller (or not greater than this size threshold) may be identified as nonessential and optionally removed from the final result. FIG. 4(C) demonstrates the avascular region map after removal of all the avascular regions with a size smaller than the size threshold, as well as removal of the FAZ and avascular regions around large vessels.

Note that not all of the avascular regions in steps 210, 212, and 214 need to be removed and one or more of these steps may be performed or omitted, as required. Also note that additional nonessential regions may be identified and optionally removed from consideration. In some embodiments, areas with low confidence in a scan (or image) may be identified as nonessential. These nonessential avascular regions may be identified in an additional avascular region detection and optional removal step prior to step 216 in the flowchart of FIG. 2. Such areas may be found, for example, by using an OCT structural en face image or using fundus imaging. Either method, or a combination of them, may be used to identify regions of low confidence. These regions may be identified as regions of low signal reflectivity (e.g. below a predefined reflectivity threshold), such as due to opacity in the eye caused by eye floaters or cataracts. Low confidence areas may also be identified as areas that appear blurry or have low contrast (e.g., have a low signal-to-noise (SNR) below a predefined threshold). For example, this cut-off SNR threshold may be determined by analyzing multiple en face sample scans having different SNR values, and noting at what SNR value errors in avascular region detection are reduced to a predefined error rate.

Once the nonessential avascular regions (e.g., avascular regions typical of normal eyes or of low confidence) are eliminated, the resulting inverse image shows test avascular regions and can be used for further analysis, e.g. to determine an avascular index or area for disease eye evaluations (step 216 in FIG. 2), as discussed in further detail below. The avascular index may be a simple (e.g., area, size, and/or count) sum of all test avascular regions (e.g., avascular areas remaining after removing those typical of normal eyes or of low confidence, as discussed above), a sum adjusted by known demographic factors such as age or ethnicity, or a weighted sum where the weight depends on the location in the eye, or characteristics of the area being included in the sum.

In order to evaluate the performance of an avascular index on predicting the status of eye diseases, 83 healthy eyes and 54 diabetic retinopathy eyes were imaged using an OCT system (such as one depicted in FIG. 1). The obtained vasculature images were generated by following the steps discussed in reference to FIG. 2. The avascular index obtained from the healthy group and the disease group were analyzed. The results are illustrated in FIGS. 6(A) and 6(B). In particular, FIG. 6 (A) demonstrates the average and standard deviation observed in normal and diabetic retinopathy (DR) subjects. As depicted in FIG. 6(A), normal subjects have a much smaller average and standard deviation of the avascular index than DR subjects. FIG. 6(B) shows the receiver operating characteristic (ROC) curve obtained from avascular index of normal and DR subjects. The area under the receiver operating characteristic curve (AROC) is 0.9512, which shows that the avascular index is able to achieve clear distinction between normal and DR subjects.

FIG. 7 demonstrates an avascular region detection comparison between normal and DR subjects. The left two columns (LC) are image results captured from normal subjects. The right two columns (RC) are image results captured from DR patients. The top row (TR) shows the vasculature image overlaid with avascular regions (dark gray color). The middle row (MR) demonstrates the avascular regions after removal of the FAZ. The bottom row (BR) shows the avascular regions after removal of the smaller regions. As depicted in FIG. 7, the DR patients have a much larger cumulative avascular area (e.g., a larger sum of relevant, e.g., not normal, avascular areas) compared to normal subjects.

The smaller avascular areas typical of normal eyes may be affected by several factors related to the subject and eye of interest. First, because OCT creates its image using a beam that travels through a specified angle rather than a specified distance, larger eyes (such as the longer eyes typical of myopes) may be observed to have larger separations between capillaries. Refraction through the lens and cornea may also affect the total size of avascular areas measured as well, so that the overall refractive error might affect the size of regions observed independently of the length of the eye. These effects might be accounted for by regressing the size of typical avascular regions with the length of the eye or the refractive error, and taking such factors into account when choosing the size threshold. Second, it is observed that spaces (e.g. avascular regions) closer to the FAZ may tend to be larger than spaces away from the FAZ. This can be addressed by having the size threshold depend on eccentricity or distance from the FAZ. In this case, the method would eliminate 'normal' areas of non-perfusion (e.g., avascular areas) from the assessment based on excluding areas using more than one parameter. The illustration in FIG. 5 shows exclusion based only on sizes. If one plots size versus distance from fovea, it may be possible to eliminate from consideration larger avascular areas (e.g., larger than the avascular index) near the fovea that may still be typical of normal eyes (see for example, FIG. 4(C)). Another possibility is to use additional parameters such as shape of regions and distance from vessels or specific vessels such as arteries.

To account for the different factors that may affect the expected range of normal avascular areas, it is also reasonable to consider that the avascular area could be constructed from a weighted sum of areas where the weight may depend on location relative to the fovea, disc or retinal arteries, length of the eye, age of the patient, and avascular region shape

characteristics.

The method described in the figures may use an OCTA en face image that contains all (or a majority) of the vasculature in the retina. The method could also be applied to individual capillary plexi of the retina from standard structural OCT data, for instance, the deeper retinal layer. The characteristics of such a layer are different. For instance, it may not have an FAZ, and the spaces found in a normal retina may result in a different threshold. The method could also be applied to the choriocapillaris and choroid. In these cases, sometimes an en face image built from the structural OCT information rather than the motion contrast information might yield a better result for the contrasting step. For the choroid, the areas in a normal eye that are typically avascular will not include a foveal zone, but may include other watershed zones typical of the distribution of this vascular network.

The above method(s) for identifying avascular regions may further be used to construct a database of (normal and/or not-normal) avascular regions (and/or their associated

angiography images). Each avascular region in the database may be labeled as normal or not- normal and include corresponding characteristic features, such as shape, size and location, including positional relationships to prominent avascular regions, e.g. the FAZ. The database may be used as a further aid to diagnose a vasculature-related eye pathology. For example, a newly detected avascular region in a current angiography image (e.g., a current OCTA scan) being examined may be compared with the existing database. Any variance(s) in avascular regions in the current OCTA scan in relation to the normal avascular regions in the database may be an indicator of an abnormal avascular region. Alternatively, similarities in avascular regions in the current scan in relation to the not-normal avascular regions in the database may also be an indicator of an abnormal avascular region. These indicators may be assigned weights for evaluation. For example, these weights may be associated with specific pathologies (e.g., ischemia) and either provided separately for evaluation, or combined with the weighted diagnoses described above, such as that used to evaluate avascular regions for diabetic retinopathy. The database may further aid diagnosis by displaying for review by a clinician (e.g., as a reference), the scan from which the previously cataloged/stored and matched avascular region originated from and/or providing clinical information associated with the matched avascular region.

The database of avascular region shapes (and/or associated angiography images) may further be used with machine learning to aid diagnosis. That is, a machine learning module may be trained to preferably identify ophthalmic maladies based on a training set of angiography images and/or vascular images and/or avascular regions (optionally including both normal and not-normal avascular regions) and/or their respectively extracted characteristic features. The training set may include both labeled (e.g., labeled with diagnosed maladies and/or identified retinal/eye structures) or unlabeled training examples, as needed. Examples of machine learning (ML) architectures (or models or algorithms) include: nearest neighbor, naive Bayes, decision trees, linear regression, support vector machines (SVM), neural networks, etc. For example, the database may be used to train a neural network to discern between normal and not-normal vasculature in an angiography image and/or vascular image.

For the sake of completeness, it may be beneficial to provide some background information regarding neural networks in general. A neural network, or neural net, is a (nodal) network of interconnected neurons, where each neuron represents a node in the network. Groups of neurons may be arranged in layers, with the outputs of one layer feeding forward to a next layer in a multilayer perception (MLP) arrangement. MLP may be understood to be a feedforward neural network model that maps a set of input data onto a set of output data.

FIG. 8 illustrates an example of a multilayer perception (MLP) neural network. Its structure may include multiple hidden (e.g. internal) layers HL1 to HLn that map an input layer InL (that receives a set of inputs (or vector input) in l to in_3) to an output layer OutL that produces a set of outputs (or vector output), e.g., out l and out_2. Each layer may have any given number of nodes, which are herein illustratively shown as circles within each layer. In the present example, the first hidden layer HLl has two nodes, while hidden layers HL2, HL3, and HLn each have three nodes. Generally, the deeper the MLP (e.g. the greater the number of hidden layers in the MLP), the greater its capacity to learn. The input layer InL receives a vector input (illustratively shown as a three-dimensional vector consisting of in l, in_2 and in_3), and may apply the received vector input to the first hidden layer HLl in the sequence of hidden layers. The output layer OutL receives the output from the last hidden layer, e.g. HLn, in the multilayer model, processes its inputs, and produces a vector output result (illustratively shown as a two-dimensional vector consisting of out l and out_2).

Typically, each neuron (or node) produces a single output that is fed forward to neurons in the layer immediately following it. But each neuron in a hidden layer may receive multiple inputs, e.g., from the input layer or from the outputs of neurons in an immediately preceding hidden layer. In general, each node may apply a function to its inputs to produce an output for that node. Nodes in hidden layers (e.g., learning layers) may apply the same function to their respective input(s) to produce their respective output(s). Some nodes, however, such as the nodes in the input layer InL receive only one input and may be passive, meaning that they simply relay the values of their single input to their output(s), e.g., they provide a copy of their input to their output(s), as illustratively shown by dotted arrows within the nodes of input layer InL.

For illustration purposes, FIG. 9 shows a simplified neural network consisting of an input layer InL', a hidden layer HLl ', and an output layer OutL'. Input layer InL' is shown having two input nodes il and i2 that respectively receive inputs Input l and Input_2 (e.g. the input nodes of layer InL' receive an input vector of two dimensions). The input layer InL' feeds forward to one hidden layer HLl ' having two nodes hi and h2, which in turn feeds forward to an output layer OutL' of two nodes ol and o2. Interconnections, or links, between neurons (illustrative shown as solid arrows) have weights wl to w8. Typically except for the input layer, a node (neuron) may receive as input the outputs of nodes in its immediately preceding layer. Each node may calculate its output by multiplying each of its inputs by each input's corresponding interconnection weight, summing the products of it inputs, adding (or multiplying by) a constant defined by another weight or bias that may be associated with that particular node (e.g., node weights w9, wlO, wl 1, wl2 respectively corresponding to nodes hi, h2, ol, and o2), and then applying a non- linear function or logarithmic function to the result. The non-linear function may be termed an activation function or transfer function. Multiple activation functions are known the art, and selection of a specific activation function is not critical to the present discussion. It is noted, however, that operation of a machine learning (ML) model, or behavior of a neural network, is dependent upon weight values, which may be learned so that the neural network (or neural net) provides a desired output for a given input.

The neural net learns (e.g., is trained to determine) appropriate weight values to achieve a desired output for a given input during a training, or learning, stage. Before the neural net is trained, the weights may be individually assigned an initial (e.g., random and optionally nonzero) value. Various methods of assigning initial weights are known in the art. The weights are then trained (optimized) so that for a given training vector input, the neural network produces an output close to a desired (predetermined) training vector output. For example, the weights may be incrementally adjusted in thousands of iterative cycles by a technique termed back-propagation. In each cycle of back-propagation, a training input (e.g., vector input) is fed forward through the neural network to determine its actual output (e.g., vector output). An error for each output neuron, or output node, is then calculated based on the actual neuron output and a target training output for that neuron. One then propagates back through the neural network (in a direction from the output layer back to the input layer) updating the weights based on how much effect each weight has on the overall error so that the output of the neural network moves closer to the desired training output. This cycle is then repeated until the actual output of the neural network is within an acceptable error range of the desired training output.

Thus, construction of a neural network model may include a learning (or training) stage and a classification (or operational) stage. In the learning stage, the neural network may be trained for a specific purpose and may be provided with a set of training examples, including training (sample) inputs and training (sample) outputs, and optionally including a set of validation examples to test the progress of the training. During this learning process, various weights associated with nodes and node-interconnections in the neural network are incrementally adjusted in order to reduce an error between an actual output of the neural network and the desired training output. In this manner, a multi-layer feed- forward neural network (such as discussed above) may be made capable of approximating a measurable function to a desired degree of accuracy. The result of the learning stage is a (neural network, machine learning) model that has been learned (e.g., trained). In the operational stage, a set test inputs (or live inputs such as an OCTA image or avascular regions (and/or their respective characteristic features) identified in an OCTA image) may be submitted to the learned (trained) ML model, which may apply what it has learned to produce an output prediction based on the test inputs.

The neural network may be trained to identify particular shapes, sizes, or positions of avascular regions that may be indicative of a pathology. The use of the neural network may include the identification of avascular region, and their descriptive features, as described above. The database may include the FAZ from different angiography images of healthy and diseased eyes. These angiography images may include multiple angiography images of the same eye and taken at different times so that changes in displayed vasculature may be descriptive of pathology progression in a specific eye. In this manner, training of the neural network may include examples of FAZ regions from normal and diseased eyes, and the neural network may be trained to identify progression (e.g. change over time) in size, shape, etc., of an FAZ (or other avascular region) that is indicative of a pathology.

Thus, any of the above-described methods may be used to analyze the progression of an avascular area over time and/or over multiple scans. For example, monitoring changes (e.g. growth) of avascular regions may be used to track disease progression, and to detect the onset of some diseases/conditions, such as ischemia. To track changes in avascular regions, two or more OCT en face images or angiography (e.g., OCTA) images (e.g., images/scans taken at different doctor visits) that contain vasculature information may be registered together by identifying corresponding features in the two (or more) images. That is, corresponding areas (e.g. corresponding avascular areas) in the images may be identified by registering (e.g., matching and correlating) similar features found in the images, such as by matching vessel crossings and/or vessel branchings. A difference map (or deviation map) highlighting differences in avascular regions between two or more scans may be constructed. In this manner, corresponding avascular areas in the images may be identified, and changes in their characteristics may be quantified. For example, changes in characteristics between two images (e.g., between two doctor visits) may be quantified by measuring the difference (e.g., change in area and/or shape) between corresponding avascular area(s). In a manner similar to that described above, a determination of what constitutes a normal (e.g., small) change in an avascular area (e.g., over time) versus an abnormal change may be made by comparing changes in avascular regions in a population of normal, healthy eyes with changes in avascular regions in diseased eyes (e.g., diabetic retinopathy). Thus, small changes in avascular areas between scans, that may be typical of normal eyes, may be ignored in a manner similar to that described above.

FIG. 10 is an example of a registered image 11 (e.g., a registered OCT en face image/scan or registered OCTA image/scan) highlighting avascular regions, as may result from registering together two, or more, en face (OCTA) scans (e.g., a previous scan l and a later scan_2, not shown) of the same (or overlapping) retinal area. That is, the white regions 15 and parallel- hatch regions 13 highlight (e.g., represent) a combination of avascular regions, of the same retinal area, identified in either of scan l or scan_2. In the present case, the white regions 15 represent avascular regions found in both scan l and scan_2, whereas the parallel-hatch regions 13 regions may represent differences in avascular regions between scan_l and scan_2 (e.g. avascular regions found in one scan, but not the other). For example, parallel-hatch regions 13 may represent newly detected avascular regions found only in scan_2, and the white regions 15 regions may represent avascular regions found in both scan l and scan_2. Alternatively, parallel- hatch regions 13 (or a differently highlighted region, not shown) may identify shrinkage in avascular regions between scan_l and scan_2. This might be the case, for example, if some region in scan l was erroneously identified as avascular (such as due to poor image quality) in that area of scan l, but scan_2 is of better quality and correctly shows that region as a vasculature region or a normal avascular region to be ignored.

Changes in avascular regions (e.g. size, shape, location etc.) in multiple scans may affect the weighting assigned to those avascular regions. For instance, returning to the example where the parallel- hatch regions 13 regions represent newly detected avascular regions found in scan_2 but not found in scan l, one may search registered image 11 for white regions 15 that abut (or are within a predefined distance from, or overlap, or intersect, or substantially adjacent) parallel- hatch regions 13, which may indicate a growth of an avascular region from its original white size (e.g., in scan l) to a larger size encompassing the combined white and parallel-hatch regions (e.g., in scan_2). In the example of FIG. 10, parallel-hatch region 13' indicates an avascular region that intersects (or is adjacent to, or abuts, or overlaps, or adjoins) a previously detected white (avascular) region 15'. That is, neighboring avascular white regions and parallel-hatch regions may be connected. For example, avascular regions 13' and 15' may be combined into a hybrid avascular region that includes a higher (or additional) weight than typically assigned to indicate that this is a change over time in a previously identified avascular region. Thus, increasing avascular regions may be assigned weights dependent upon their amount of increase, or their rate of increase over time (e.g., over multiple scans). The resultant higher weight may influence the avascular index. Thereby, changes in the characteristics of individual avascular regions (as detected by registering multiple scans) may be quantified. In some embodiments, changes in avascular regions can also be compared with that of normal subjects to detect potentially suspicious changes.

The methods described herein may further be extended to detect meaningful avascular regions around the foveal avascular zone (FAZ), and to monitor changes in avascular regions about the FAZ. For example, the FAZ in multiple scans (of the same eye) may be compared to quantify changes in the shape and/or size of the FAZ in a manner similar that described above. FAZ measurement variances of a population of normal eyes may be compared with FAZ measurement variances of a population of diseased eyes, as described above, to identify changes that may be ignored and changes that should be flagged (and weighted) as indicators of possible ophthalmic pathology.

This may be extended to regions surrounding the FAZ to identify and monitor changes in avascular regions in the parafovea and perifovea. Avascular regions in the parafovea and perifovea may be distinguished by a different highlighting (e.g., different color, outlining, etc.) and/or be assigned different weights than typically assigned to other avascular regions. In this case, the assigned weight may be dependent upon the location of the avascular regions (e.g., different weights for avascular regions in the parafovea and in the perifovea).

FIG. 11 illustrates an example method 60 for analyzing avascular regions of a retina in a current optical coherence tomography angiography (OCTA) image of an eye. The method may begin at step S61, where avascular regions are identified in the current OCTA image to define a collection of identified avascular regions, for example as illustrated in FIG 3(C). It is to be understood that the current OCTA image may have been generated by use of an optical coherence tomography (OCT) system. The current OCTA image may further have been selected (e.g., accessed) from a collection/library of OCTA images. At step S63, some avascular region may be classified, flagged, designated, or otherwise identified as a nonessential (or normal or healthy) avascular region, e.g., as illustrated in FIGs. 4(A) and 4(B). Nonessential avascular regions may be identified as avascular regions that have one or more descriptive feature characteristic of a population of healthy eyes. The descriptive features extracted from each identified avascular region may include such features as size, shape, location relative to prominent retinal structures/landmarks (such as the FAZ or large vessels), etc. In step S65 the classified nonessential avascular regions may be removed from the collection of identified avascular regions, such as illustrated in FIG. 4(C). Any remaining avascular regions may be classified, or designated, as test avascular regions, and displayed or submitted to further analyses (step S67). The further analysis may include any or all of the above-described methods for identifying pathologies, such as the use of an avascular index, the use of a database to match a current avascular region under test to a previously identified normal or diseased avascular regions or to monitor a progression (e.g., change in size and/or location) of a test avascular region over time, and/or the use of a machine learning module.

Some embodiments may repeat one or more steps of the methods of FIGs. 2 and 11, where appropriate. Although this disclosure describes and illustrates particular steps of the methods of FIGs. 2 and 11 as occurring in a particular order, this disclosure contemplates any suitable steps of the methods of FIGs. 2 and 11 occurring in any suitable order.

Example Computer System

The processing unit 121 that has been discussed herein in reference to FIG. 1 can be implemented with a computer system configured to perform the functions that have been described herein for this unit. For instance, the processing unit 121 can be implemented with the computer system 800, as shown in FIG. 12. The computer system 800 may include one or more processors 802, one or more memories 804, a communication unit 808, an optional display 810, one or more input devices 812, and a data store 814. The display 810 is shown with dotted lines to indicate it is an optional component, which, in some instances, may not be a part of the computer system 800. In some embodiments, the display 810 discussed herein is the display 122 that has been discussed herein in reference to FIG. 1.

The components 802, 804, 80ff8, 810, 812, and 814 are communicatively coupled via a communication or system bus 816. The bus 816 can include a conventional communication bus for transferring data between components of a computing device or between computing devices. It should be understood that the computing system 800 described herein is not limited to these components and may include various operating systems, sensors, video processing components, input/output ports, user interface devices (e.g., keyboards, pointing devices, displays, microphones, sound reproduction systems, and/or touch screens), additional processors, and other physical configurations. The processor(s) 802 may execute various hardware and/or software logic, such as software instructions, by performing various input/output, logical, and/or mathematical operations. The processor(s) 802 may have various computing architectures to process data signals including, for example, a complex instruction set computer (CISC) architecture, a reduced instruction set computer (RISC) architecture, and/or architecture implementing a combination of instruction sets. The processor(s) 802 may be physical and/or virtual, and may include a single core or plurality of processing units and/or cores. In some embodiments, the processor(s) 802 may be capable of generating and providing electronic display signals to a display device, such as the display 810, supporting the display of images, capturing and transmitting images, performing complex tasks including various types of feature extraction and sampling, etc. In some embodiments, the processor(s) 802 may be coupled to the memory(ies) 804 via a data/communication bus to access data and instructions therefrom and store data therein. The bus 816 may couple the processor(s) 802 to the other components of the computer system 800, for example, the memory(ies) 804, the communication unit 808, or the data store 814.

The memory(ies) 804 may store instructions and/or data that may be executed by the processor(s) 802. In the depicted embodiment, the memory(ies) 804 stores at least an avascular region detection algorithm 806, which may include software, code, logic, or routines for performing any and/or all of the techniques described herein. For instance, the avascular region detection algorithm 806 may perform all or some of the operations depicted in FIG. 2. In some embodiments, the memory(ies) 804 may also be capable of storing other instructions and data including, for example, an operating system, hardware drivers, other software applications, databases, etc. The memory(ies) 804 are coupled to the bus 816 for communication with the processor(s) 802 and other components of the computer system 800. The memory(ies) 804 may include a non-transitory computer-usable (e.g., readable, writeable, etc.) medium, which can be any apparatus or device that can contain, store, communicate, propagate or transport instructions, data, computer programs, software, code, routines, etc. for processing by or in connection with the processor(s) 802. Anon-transitory computer-usable storage medium may include any and/or all computer-usable storage media. In some embodiments, the memory(ies) 804 may include volatile memory, non-volatile memory, or both. For example, the memory(ies) 804 may include a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, flash memory, a hard disk drive, a floppy disk drive, a CD ROM device, a DVD ROM device, a DVD RAM device, a DVD RW device, a flash memory device, or any other mass storage device known for storing instructions on a more permanent basis.

The computer system for the processing unit 121 may include one or more computers or processing units at the same or different locations. When at different locations, the computers may be configured to communicate with one another through a wired and/or wireless network communication system, such as the communication unit 808. The communication unit 808 may include network interface devices (I/F) for wired and wireless connectivity. For example, the communication unit 808 may include a CAT-type interface, USB interface, or SD interface, transceivers for sending and receiving signals using Wi-Fi™; Bluetooth® , or cellular communications for wireless communication, etc. The communication unit 808 can link the processor(s) 802 to a computer network that may in turn be coupled to other processing systems.

The display 810 represents any device equipped to display electronic images and data as described herein. The display 810 may be any of a conventional display device, monitor or screen, such as an organic light-emitting diode (OLED) display, a liquid crystal display (LCD). In some embodiments, the display 810 is a touch-screen display capable of receiving input from one or more fingers of a user. For example, the device 810 may be a capacitive touch-screen display capable of detecting and interpreting multiple points of contact with the display surface.

The input device(s) 812 are any devices for inputting data on the computer system 800. In some embodiments, an input device is a touch-screen display capable of receiving input from one or more fingers of the user. The functionality of the input device(s) 812 and the display 810 may be integrated, and a user of the computer system 800 may interact with the system by contacting a surface of the display 810 using one or more fingers. In other embodiments, an input device is a separate peripheral device or combination of devices. For example, the input device(s) 812 may include a keyboard (e.g., a QWERTY keyboard) and a pointing device (e.g., a mouse or touchpad). The input device(s) 812 may also include a microphone, a web camera, or other similar audio or video capture devices.

The data store 814 can be an information source capable of storing and providing access to data. In the depicted embodiment, the data store 814 is coupled for communication with the components 802, 804, 808, 810, and 812 of the computer system 800 via the bus 816, and coupled, via the processor(s) 802, for communication with the avascular region detection algorithm 806. In some embodiments, the avascular region detection algorithm 806 is configured to manipulate, i.e., store, query, update, and/or delete, data stored in the data store 814 using programmatic operations.

In the above description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the specification. It should be apparent, however, that the subject matter of the present application can be practiced without these specific details. It should be understood that the reference in the specification to "one embodiment", "some embodiments", or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the description. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment(s).

Furthermore, the description can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

The foregoing description of the embodiments of the present subject matter has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the present embodiment of subject matter to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the present embodiment of subject matter be limited not by this detailed description, but rather by the claims of this application. As will be understood by those familiar with the art, the present subject matter may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.