SYSTEM AND METHOD FOR SPINAL CORD AND VERTEBRAE

SEGMENTATION

Cross Reference to Related United States Applications

This application claims priority from "Parametric model based vertebrae segmentation", U.S. Provisional Application No. 60/838,048 of Liu, et al., filed August 16, 2006, "RANSAC-based Algorithm for Spinal Cord Detection", U.S. Provisional Application No. 60/839,291 of Jerebko et al, filed August 22, 2006, and "Band-pass filtering for segmentation and characterization of structures", U.S. Provisional Application No. 60/839,280 of Anna Jerebko, filed August 22, 2006, the contents of all of which are herein incorporated by reference.

Technical Field

This disclosure is directed to the detection and segmentation of vertebrae structures in the spine.

Discussion of the Related Art

Many anatomical structures in human body exhibit natural repetitiveness. For example, vertebrae of the spinal column have heights of certain range, and inter- vertebrae spaces and disks have a certain periodicity. Another example is the colon surface, which has periodic folds. The folds of the human brain also have a known range of frequencies. Besides the normal structures in the body, some lesions, such as breast cancer masses, have features such as specularity, where radial spikes that are strong indicators of malignancy can be captured at several ranges of frequency.

Whole-body MRI has high sensitivity and specificity for detection of bone- marrow metastases. Spine metastases detection is an important and time-consuming task. In recent years, computer aided detection (CAD) has proven useful for radiologists if used as a second reader in various applications of computed tomography (CT) and X-rays, such as colonic polyp, lung nodule, breast mass calcification detection, etc. Although MRI, including Tl, T2 HASTE (half-fourier

acquisition single-shot turbo spinecho) and T2-weighted Short Inversion Time Inversion Recovery (STIR) sequences, is becoming a popular screening modality, development of MRI CAD applications is more complicated than that of CT because of intensity in-homogeneities and higher noise level. In order to automatically detect metastasis in the vertebrae column and label vertebrae, a reliable segmentation of vertebrae body is needed. Robust segmentation of the spinal cord as a reliable reference object is useful for automatic vertebrae segmentation, metastasis detection, multi-modal registration, and curved MPR visualization. The few methods for vertebrae column segmentation in MR images available in recent literature, such as normalized cuts, are not suitable in the case of severe metastases where dissimilarity between healthy and affected vertebrae could be much greater than between vertebrae and other tissue.

Summary of the Invention

Exemplary embodiments of the invention as described herein generally include methods and systems for automatically detecting the spinal cord in T2- weighted STIR images, segmenting vertebrae column and vertebrae body based on a given parametric representation of the spinal cord, and modeling naturally periodic structures using the domain knowledge of their periodicity. A method according to an embodiment of the invention involves constructing a model based on geometric primitives from purely anatomical knowledge of organ shape and rough variability limits. The basic intensity range of primary 'simple ¹ objects is derived from expert knowledge of image formation for certain tissue types. A method according to an embodiment of the invention can be used for segmentation, characterization and feature extraction, abnormality detection, alignment of anatomical structures or lesions, spinal cord examination, metastasis screening, multi-modal registration, and curved MPR visualization. A 3D-polynomial model via RANSAC (random sample consensus algorithm) provides a fully automatic rapid and reliable estimate of the spinal centerline position and segmentation of the spinal cord.

Methods according to embodiments of the invention are suited to work with alterations in scanning protocols and pulse sequences, such as changes in the level of fat suppression, image resolution, acquisition plane (sagittal or coronal) and tolerant

to severe pathological changes in organ appearance. A method according to an embodiment of the invention was successfully applied to the segmentation and alignment of human vertebrae column in MRI images of various pulse sequences as well as detection of collapsed vertebrae and vertebrae metastases. A method according to an embodiment of the invention can be applied to breast tomosynthesis and mammography images to allow extraction of a highly discriminative set of features which in conjunction with a classifier, such as a support vector machine, can reduce the number of false positives.

According to an aspect of the invention, there is provided a method for segmenting vertebrae in digitized images, including providing a plurality of digitized whole-body images, each the image comprising a plurality of intensities corresponding to a 3-dimensional grid of points, detecting and segmenting a spinal cord using 3D polynomial spinal model in each of the plurality of images, finding a height of each vertebrae in each image from intensity projections along the spinal cord, and building a parametric model of a vertebrae from the plurality of images.

According to a further aspect of the invention, detecting and segmenting a spinal cord in an image includes splitting the image into a plurality of sampling bins, randomly selecting a subset of these bins, randomly selecting one sample point from each selected bin, computing coefficient vectors for the 3D polynomial spinal model evaluated at each selected sample point, computing extrema points for the 3D polynomial spinal model, and evaluating the 3D polynomial spinal model in a local neighborhood volume of bright points within a curved cylinder built around the polynomial, if the extrema points satisfy pre-defined constraints.

According to a further aspect of the invention, if the volume is greater than a predefined minimum spinal cord volume, using least-squares to refine the coefficient vectors, taking into account only those points within a neighborhood of the 3D polynomial spinal model, evaluating the 3D polynomial spinal model in the neighborhood, and comparing the evaluation results to the total volume of the bright points within the curved polynomial spinal model.

According to a further aspect of the invention, the method comprises repeating the steps of randomly selecting a subset of these bins, randomly selecting one sample point from each bin, computing coefficient vectors, and evaluating the 3D polynomial

spinal model until either the ratio of the volume to the total volume bright points is greater than a pre-determined threshold or the number of repeats has exceed a predetermined maximum.

According to a further aspect of the invention, the 3D polynomial spinal model is defined by

wherein n is the polynomial order, x(t) represents a variation in the spinal curve in a coronal orientation, and z(t) represents a variation in the spinal curve in a saggittal orientation, the extrema points are defined by

(3)

where (x _cJ , y _cj ) are extrema points in coronal orientation, {z _sj , y _sj ) in the saggittal orientation, and j=J, ..., n-1, and wherein the subset of bins has (n+J) bins.

According to a further aspect of the invention, the coefficient vectors are constrained to satisfy

→ and the constraints satisfied by the extrema are defined by '

DjJ+l min < ysj ~ y*j+l I "^ Djj+i max wherein [D _j J _{+1 min} ,D _{jJ+l ma} J are natural limits of a longitudinal distance and LK _{j j + l} is a poterior-anterior (lordotic and kyphotic) distance in a spinal curve, ^' and \ ^< $c '

wherein S _c is a scoliosis pathology limit observed from the plurality of images, and wherein the [D _{Jj+l min} ,jD, _iJ+ i _max ] and LK _{jt J+ l} limits were set by an expert radiologist.

According to a further aspect of the invention, finding a height of a vertebrae from intensity projections along the spinal cord includes extracting a one-dimensional (ID) signal representing the spinal cord by projecting median intensity values along the spinal cord inside small sample circles adjacent to the front edge of the spinal cord within planes orthogonal to the spinal cord, filtering the ID signal with a minimum rank filter having a width between a largest inter-vertebra space and a shortest vertebrae to be detected, and finding upper and lower boundaries of the vertebrae from intensity minima and maxima of the filtered ID signal.

According to a further aspect of the invention, the method comprises removing spurious local minima caused by noise by setting an adaptive threshold U for each vertebra satisfying /(u,- ) < t _n f(d _t ) < t _; , wherein/is the filtered ID signal, (U ₁ , di) are two local minima about the middle of each vertebra, and t _j = mean(f(n _i _ _l ),...,f(n _M )) -std(f («,._, ),...,/ (n _l+ ,)) , where std is a standard deviation, and n,- is a local minima of/, and imposing constraints «,. — d _; < T _v and d _l+i — U _/ > T _s on the upper and lower vertebra boundary to maintain a minimum height of the vertebrae T _v and inter-vertebrae space T _s .

According to a further aspect of the invention, building a parametric model of a vertebrae includes aligning all vertebrae in the plurality of images based on an extrema of a thoracic curve of the 3D polynomial spinal model, acquiring measurements of minor and major axis for each vertebra throughout plurality of images, wherein a typical sample vertebrae is represented by

whereinj is a sample number, a _/ ,- are the major axes, b _j ,- are the minor axes, for m i = 1, ... , n , the number of vertebrae, computing a mean x = -^] x ■ and covariance S

of the major and minor axes for each vertebrae across the plurality of images, using principle component analysis to find a set of modes representing the sample axes by solving Sp _k = λ _k p _k , wherein an arbitrary model sample r is defined in terms of the principle modes as x = x+ Pd , where P=(pj,...,p,) is a matrix of t eigenvectors, and d = Cd ₁ d, ) ^τ is the model parameter.

According to a further aspect of the invention, the method comprises retaining on ily t _u hose t mod jes t _u hat sati -s _J fy _V >- ¹ λ _O , _^ ≥ confidence _ level 4 > ² _^ /t, for a pre-determi .ned ^y % ³ 100% y ^{} F} value of confidence_level, and wherein the model parameter d satisfies

According to a further aspect of the invention, the method comprises providing a new digitized whole-body image including a spinal cord, fitting an ellipse to each vertebrae of the spinal cord to find the major and minor axes, and applying constraints to the major and minor axes in the new image based on the parametric model to segment the vertebrae.

According to a further aspect of the invention, fitting an ellipse to each vertebrae includes detecting edges of the vertebrae, plotting a straight line into the vertebrae in a plane orthogonal to the spinal cord starting from the edge of the spinal cord, selecting a point on the line, projecting a plurality of rays from the selected point at substantially equally spaced angles over a substantially semi-circular span, locating edge points lying on or close to the projected rays and selecting a point closest to the selected line point, locating an opposite point to the selected point at a same distance on a ray projected in an opposite direction, for each ray, collecting the 2D coordinates of all selected and opposite points on all rays, and fitting an ellipse to the collected points using a least squares technique.

According to a further aspect of the invention, the method comprises repeating the steps of selecting a point on the line, projecting a plurality m of rays, locating edge points, locating an opposite point, collecting the 2D coordinates, and fitting an ellipse for all points on the line, and selecting a fitted ellipse with a maximum area as a vertebrae model.

According to a further aspect of the invention, applying constraints to the major and minor axes includes projecting a vector x of major and minor axes for the

whole vertebrae into the model space according to d = P ^~] (x'~ x) wherein x is a

— m mean x = ^■ £■ ^ X _j , P—(pι,...,p _t ) is a matrix of f model eigenvectors, and

d = (λ?, ,...,(I ₁ ) ^τ is a model parameter, wherein if all components of d do not satisfy the model constraints —^JX ^~ ≤ d _t . < yfλ ^~ , for / = \,...,t , wherein X ₁ are eigenvalues of Sp _k = λ _k p _k wherein 5 is a covariance matrix of x , then restricting values of d to be in the range, and computing a new axes vector x" = x +Pd using the restricted values of d.

According to a further aspect of the invention, if a vertebrae is missing in the new image, computing axis lengths of the missing vertebrae as an average from corresponding vertebrae in the plurality of images, and after applying constraints to detected ellipses, removing those components from the axes vector that correspond to the missing vertebrae.

According to another aspect of the invention, there is provided a method for segmenting vertebrae in digitized images including providing a digitized whole-body image including a spinal column, the image comprising a plurality of intensities corresponding to a 3-dimensional grid of points, fitting an ellipse to each vertebrae in the spinal column to find the major and minor axes of each vertebrae, and projecting a vector x ^r of the major and minor axes for each vertebrae into a model space

according to d = P ^'1 (x'—x) wherein x is a mean x = jz∑X _j , P=(Pi, --,Pt) is a

matrix of t model eigenvectors, and d = (d _l ,...,d, ) ^τ is a model parameter, wherein if all components of d do not satisfy model constraints - ^[λ ^~ < d _t ≤ -y[X ^~ , for i = 1, ... , t , wherein A ₁ are eigenvalues of Sp _k — λ _k p _k wherein S is a covariance matrix of 3c , then restricting values of d to be in the range, and computing a new axes vector jc ^* = 3c +Pd using the restricted values of d.

According to a further aspect of the invention, fitting an ellipse to each vertebrae in the spinal column includes detecting edges of the vertebrae, plotting a straight line into the vertebrae in a plane orthogonal to the spinal cord starting from the edge of the spinal cord, selecting a point on the line, projecting a plurality m of rays from the selected point at substantially equally spaced angles over a substantially

semi-circular span, locating edge points lying on or close to the projected rays and selecting a point closest to the selected line point, locating an opposite point to the selected point at a same distance on a ray projected in an opposite direction, for each ray, collecting the 2D coordinates of all selected and opposite points on all rays, and fitting an ellipse to the collected points using a least squares technique.

According to a further aspect of the invention, the method includes providing a plurality of digitized whole-body images, each the image comprising a plurality of intensities corresponding to a 3-dimensional grid of points, detecting and segmenting a spinal cord using a 3D polynomial spinal model in each of the plurality of images, finding a height of each vertebrae in each image from intensity projections along the spinal cord, aligning all vertebrae in the plurality of images based on an extrema of a thoracic curve of the 3D polynomial spinal model, acquiring measurements of minor and major axis for each vertebra throughout plurality of images, wherein a typical sample vertebrae is represented by

wherein j is a sample number, CLy ₁ are the major axes, b# are the minor axes, for m i = 1,..., n , the number of vertebrae, computing a mean x — -^∑^X _j and covariance S

of the major and minor axes for each vertebrae across the plurality of images, and using principle component analysis to find a set of modes representing the sample axes by solving Sp _k = λ _k p _k , wherein an arbitrary model sample x is defined in terms of the principle modes as x is a matrix of t eigenvectors, and d — (rf, ,...,d _t ) ^τ is the model parameter.

According to a further aspect of the invention, the method includes retaining

, . , , . ^ v- ¹ _O -. confidence level ^, . only those t modes that satis » wherein

conβdence_level is predefined, and wherein the model parameter d satisfies

According to a further aspect of the invention, the 3D polynomial spinal model is defined by

wherein n is the polynomial order, x(t) represents a variation in the spinal curve in a coronal orientation, and z(t) represents a variation in the spinal curve in a saggittal orientation, and wherein the coefficient vectors are constrained to satisfy

\n is even

According to another aspect of the invention, there is priovided a program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for segmenting vertebrae in digitized images.

Brief Description of the Drawings

FIG. 1 depicts a side view of an exemplary vertebral column, according to an embodiment of the invention.

FIG. 2 depicts spinal cord detection result in a maximum intensity projection image, according to an embodiment of the invention.

FIGS. 3(a)-(b) illustrates plots of a projected median intensity along a spinal cord and a raw image slice of the spinal column, according to an embodiment of the invention.

FIG. 4 illustrates a manual measurement of the major and minor axes, according to an embodiment of the invention.

FIG. 5 depicts schematic vertebrae and spinal cord edges with projected points on the rays, according to an embodiment of the invention.

FIGS. 6(a)-(c) depict a thoracic vertebrae and an axial slice through the middle of a vertebrae and the spinal cord with fitted ellipses, according to an embodiment of the invention.

FIGS. 7(a)-(c) depict exemplary vertebrae column segmentation results, according to an embodiment of the invention.

FIGS. 8(a)-(b) depict a breast mass feature extraction using projected median intensity, according to an embodiment of the invention.

FIG. 9 is a flow chart of a vertebrae segmentation method according to an embodiment of the invention includes five stages.

FIG. 10 is a flow chart of a spinal cord model fitting method according to an embodiment of the invention.

FIG. 11 is a flow chart of a method for determining the vertebra boundary, according to an embodiment of the invention.

FIG. 12 is a flow chart of a method for creating a parametric vertebrae model, according to an embodiment of the invention.

FIG. 13 is a flow chart of a method for fitting ellipses to find the major and minor axes for all vertebrae, according to an embodiment of the invention.

FIG. 14 is a flow chart of a method for applying model constraints to detected ellipses to improve vertebrae segmentation in a new image, according to an embodiment of the invention.

FIG. 15 is a block diagram of an exemplary computer system for implementing a method for detection and segmentation of vertebrae structures in the spine, according to an embodiment of the invention.

Detailed Description of Exemplary Embodiments

Exemplary embodiments of the invention as described herein generally include systems and methods for detection and segmentation of vertebrae structures in the spine. Accordingly, while the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the invention to the particular forms disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

As used herein, the term "image" refers to multi-dimensional data composed of discrete image elements (e.g., pixels for 2-D images and voxels for 3-D images). The image may be, for example, a medical image of a subject collected by computer tomography, magnetic resonance imaging, ultrasound, or any other medical imaging system known to one of skill in the art. The image may also be provided from nonmedical contexts, such as, for example, remote sensing systems, electron microscopy, etc. Although an image can be thought of as a function from R ³ to R, the methods of the inventions are not limited to such images, and can be applied to images of any dimension, e.g., a 2-D picture or a 3-D volume. For a 2- or 3-dimensional image, the domain of the image is typically a 2- or 3-dimensional rectangular array, wherein each pixel or voxel can be addressed with reference to a set of 2 or 3 mutually orthogonal axes. The terms "digital" and "digitized" as used herein will refer to images or volumes, as appropriate, in a digital or digitized format acquired via a digital acquisition system or via conversion from an analog image.

A method for vertebrae segmentation according to an embodiment of the invention assumes that (1) the spinal cord, tumors and blood vessels are among the brightest structures in the image for number of MRI pulse sequences (e.g. T2, T2 STIR, HASTE), (2) bones are surrounded by cortical bone that does not generate a signal in MR images, and therefore appears black, and looks like black contour around each bone, (3) vertebrae are perpendicular to the spinal cord, and that (4) each vertebra can be represented by a cylinder. A side view of an exemplary vertebral column, from Gray's Anatomy of the Human Body, published in 1918, is depicted in

FIG. 1. This view indicates the cervical 11, thoracic 12, lumbar 13 and pelvic 14 regions, and the individual vertebrae in each region are numbered. The pelvic region includes the sacrum and coccygeal vertebrae.

A vertebrae segmentation method according to an embodiment of the invention includes five stages, depicted in the flow chart of PIG. 9. Referring now to the figure, given a training set of MRI images, a first step 91 is to detect the spinal cord from the MRI sequences. Second, at step 92, the height of a cylinder for representing a vertebrae is determined from intensity projections along spinal cord. Third, at step 93, a parametric model of the vertebrae is built from the training set. Then, given a new image, the major and minor axes of each vertebrae are determined from ellipse fitting at step 94. Finally, at step 95, the model constraints are applied to the detected ellipses in the new image to improve the segmented vertebrae. There is also an optional preprocessing stage that includes image intensity inhomogeneity correction, scaling and thresholding.

The spinal cord is one of the most reliable reference objects in MR images and appears consistently bright in common MRI scanning protocols, such as HASTE, T2, and T2 STIR. Other tissue intensities however vary significantly with minor changes in the scanning protocols and levels of fat suppression. The number and shape of hyper-intense objects in the image, other than spinal cord, is not known a priori, as it varies from patient to patient depending on the primary tumor location, severity of the metastatic process, amount of body fat, condition of inter-vertebrae disks, pathological changes in the organ appearance, etc. A method according to an embodiment of the invention allows robust fitting and segmentation of spinal cord without extraction of a full collection of other objects present in the image.

An exemplary spinal cord is shown in FIG. 1. A typical human spinal cord includes three curves: the cervical curve 1 1 , which is convex forward, the thoracic curve 12, which is concave forward, and the lumbar curve 13, which is convex forward. It is convenient to model it with a curve having 3 extrema points. According to an embodiment of the invention, the spinal cord is modeled as a global 4 ^th -order 3D-polynomial:

where n=4, x(t) represents the variation in the spinal curve from patient side to side (coronal orientation), and z(t) represents the variation in the spinal curve from patient's back to front (saggittal orientation). It is to be understood, however, that embodiments of the invention are not limited to using a 4 ^th -order polynomial, and any polynomial model having the desired extrema can be used according to other embodiments of the invention. Furthermore, in this parametric representation, the polynomial can also be constrained to have the sacrum, the end of the vertebrae column, always pointing back:

Polynomial extrema points in the saggittal orientation correspond to cervical, thoracic and lumbar curves. If a patient's spine also has a lateral curvature, a pathology known as scoliosis, it will be reflected in x(t) variations. Polynomial extrema points are computed from:

(3)

where (x _cj , y _cj ) are extrema points in coronal orientation, (_z _sj , y _sj ) in the saggittal orientation, and j=J,...,n-J.

Next, the distances between polynomial extrema points in saggittal orientaion are constrianed to be within natural limits of longitudinal [D _{JJ+l m} j _n »£^ _/+ i _ma χ] ^an d the

poterior-anterior (lordotic and kyphotic) LK _{Jt J + l} distances between cervical, thoracic and lumbar curves:

z s _j ~ ^z s _j+ ι ^< LK _JtJ+]

Dj,j+ι nun < y _sj +l I I ** Dj.j+1 max ^{• (4)}

The [D _{j J+1 mm} ,D _{J J+1 ma} J and LK _{Jt J + l} limits were set by an expert radiologist.

The lateral distances between coronal extrema points are also constrained to be within the scoliosis pathology limit S _c , observed from our training set:

A scanned patient section does not necessarily contain all three curves described above, but this modeling approach allows one to extrapolate and guess their approximate location. The 4 ^th short curve, the pelvic curve, extends from sacrovertebral articulation to coccyx and, most often, is not visible in thoracic section of the whole body scan. It could be segmented together with pelvic bones using the spinal cord points as reference points.

The model parameters are estimated using random sample consensus (RANSAC) algorithm with subsequent least squares based fitting refinement. The RANSAC method is used for parametric shape fitting with a priori knowledge of the approximate object scale in the presence of highly correlated outliers that often constitute more than 50% of the image.

At least n+J sample points are needed to define an n ^th order polynomial. To accelerate fitting of a polynomial of an approximately known scale, defined by the limits on the distances between the extrema points, defined in EQ. (4), above, the image is split into M x(n + l) sampling bins. Bins are longitudinally evenly spaced throughout the image; each bin b contains all axial (horizontal) slices in the region [y*. y _b ⁺ max(y)/(M(n + l))]. The factor M constrains the scaling/warping degree of freedom for the polynomial.

A flow chart of a spinal cord model fitting according to an embodiment of the invention is presented in FIG. 10. Given an image that has been split into sampling bins, the model fitting begins at step 101 by randomly selecting (n+1) of these bins. At step 102, one sample point is randomly selected from each bin. The coefficient vectors ά ^~ _x and 5 ^" . are computed at step 103 by solving EQS. (1) for the sample points. If the conditions in EQS. (2) are satisfied at step 104, the polynomial extrema points are computed from EQS. (3) at step 105, otherwise the model fitting goes back to step 102 to select another set of sample points. If, at step 106, all extrema constraints (EQS. (4) and (5)) are satisfied, then the fitting function (EQS. (I)) is evaluated step 107 within a local neighborhood of the parametric model as a volume Vof bright voxels within a curved cylinder built around the polynomial, otherwise the model fitting returns to step 102. The size of the neighborhood is the average human spinal cord radius R + delta, a small increment. At step 108, if V is greater than a minimum spinal cord volume, the least-squares method is used to refine the coefficient vectors, taking into account only those voxels within the R + delta neighborhood of the spinal cord model. This shifts the model more towards the actual spinal cord center-line. The fitting function is evaluated again and compared to the current best fitness model, the total volume of the bright voxels in the curved polynomial spinal model. At step 109, the model fitting returns to step 101 for another iteration unless either the ratio of V to the total volume of bright voxels is greater than a pre-determined threshold or the number of iterations has exceed a predetermined maximum k.

The convergence speed depends on the percentage of outliers in the image: that is, hyperintense points in the image not belonging to the spinal cord, which, in turn, depends on the exact pulse sequence, quality of fat suppression and degree of metastatic process. The maximum number of iterations k is estimated as:

where m=n+l, Po is the probability of seeing only bad samples (possibly it was up to 80% for some data sets), and w is probability of good (inlier) samples.

The spinal cord is segmented as a curved 3D cylinder around the detected centerline. A spinal cord detection result 21 in a maximum intensity projection image is shown in FIG. 2.

The detection results were visually evaluated in 77 T2 STIR and 5 HASTE images from different hospitals. Results appeared satisfactory in all images. Segmentation accuracy was numerically evaluated using manually segmented ground truth (GT) in 35 T2 STIR images with average voxel size of 1.25xl.25x6mm. The average segmentation accuracy, estimated as ratio of overlapping of automatically detected (AD) and GT spinal cord volumes to the GT volume, was 91% with standard deviation (STD) of 14%. The accuracy of centerline position evaluated as average distance from all GT spinal cord voxels to the AD centerline was 4.4mm with STD of 1.9mm. The presence of collapsed vertebrae and edema in 1 patient and multiple vertebrae metastasis in 12 patients only affected the segmentation accuracy in one case, where all vertebrae had severe metastatic changes of image intensity similar to the spinal cord and with no visible boundary. As a result, the spinal cord centerline in this one case was shifted toward the center of the vertebral body. This had a slight negative effect on the test set statistics.

The second stage, step 92 of FIG. 9, determines the vertebra boundary in a vertical direction using intensity projections along the spinal cord and bandpass filter frequency analysis. A vertebrae model according to an embodiment of the invention only fits the vertebrae body, excluding the processes and pedicles. It can be represented by a section of an elliptical cylinder adjacent to the spinal cord. It can be assumed that the imaginary planes that separate vertebrae from each other and from inter-vertebrae disks are orthogonal to the spinal cord. A flow chart of a method according to an embodiment of the invention for determining the vertebra boundary is shown in FIG. 11. Referring now to the figure, vertebrae separating planes are detected by analyzing a one-dimensional signal representing the spinal column. This ID signal is extracted at step 111 by projecting median intensity values along the spinal cord inside small sample circles adjacent to the front edge of the spinal cord within the planes orthogonal to it. This extracted ID signal is referred to herein as the projected median intensity.

Normal vertebra is composed of spongy bone, containing bone marrow, which is surrounded by compact (cortical) bone. The most interesting property of cortical bone from an MRI point of view is that it does not generate a signal in MRI and therefore appears consistently hypointense in any pulse sequence, while vertebra and disks may change their appearance depending on the presences of metastases or other diseases. Sometimes inter-vertebrae disks are not visible in the whole vertebral column.

To extract reliable information from the projected signal and skip inconsistent high intensity peaks like disks and lesions, the minimum rank filter with a width between the largest inter-vertebra space and shortest vertebrae to be detected is applied to the projected median intensity at step 112. The upper and lower frequency thresholds for capturing the periodicity of the structures of interest are established using domain knowledge. These thresholds define frequency bands for filtering the original signal.

While different bands of frequencies or height thresholds could be used in cervical, thoracic and lumbar areas for higher precision, according to an embodiment of the invention the same band is used for the whole vertebrae column. A discrete Fourier transform (DFT) of the projected median intensity is computed using a fast Fourier transform (FFT) algorithm. Next, all elements of the resulting Fourier-space vector that correspond to the frequencies outside of desired range are set to 0. Finally, the inverse Fourier transform is obtained from the Fourier-space vector. The filtered signal is smooth and differentiable, thus finding a minimum and maximums in this signal is easy. For vertebrae segmentation, the minima will correspond to inter- vertebrae spaces and the maxima will correspond to the vertebrae body. FIG. 3(a) illustrates a plot of an unfiltered 31 and a filtered 32 projected median intensity signal along a spinal cord. FIG. 3(b) depicts a raw image slice of the spinal column, with the spinal cord 33 indicated by the sinusoidal curve. The arrows from FIG. 3(b) to the curves graphed in FIG. 3(a) show the correspondence of dark regions to intensity signal minima, and of bright regions to intensity signal maxima.

Referring again to FIG. 11, the upper and lower boundaries of the vertebrae are found at step 113 from the intensity minima and maxima. In general, given a

scalar function /of several variables, the conditions for finding local maxima /n,- of/, are

SL = o < o /OO >o (7a)

where y is the subsampled location of the spinal cord in the y direction. If two consecutive local maximums are too close, they can be merged together to form a new maximum.

Similarly, the conditions for finding local minima n,- of/ are

dy dy ²

(7b)

Referring again to FIG. 3(a), in the original, unfiltered signal 12, one searches for two local minima («;, dj) about the middle of each vertebra, ra,-, which will represent the upper and lower boundary of each vertebra. From m,- to n,-.;, one searches for the local minimum U ₁ as the upper boundary for I ^th vertebra. Similarly, from m,- to m.i, one searches for the local minimum di as lower boundary for i ^th vertebra. Spurious local minima caused by noise are removed by setting an adaptive threshold t _t for each vertebra. That is, the following conditions should be satisfied at M,- and dr. 1(.U ₁ ) K t _n /(</.) < *,- , where t, = mββn(/(n _M ),..., /(n _/+l )) - ji5rf(/(n _w ),..., /(«, _+.1 )) , where std is a standard deviation. Finally, additional constraints are imposed on the searched upper and lower vertebra boundary to maintain minimum height of the vertebrae T _v and inter-vertebrae space T _s : U ₁ — d _i < T _v and d _M - u. > T _s .

The next stage, step 93 of FIG. 9, is creating a parametric model for estimating the horizontal extent of the vertebrae through fitting ellipses to the middle sections of each vertebra. A flow chart of a method according to an embodiment of the invention for creating a parametric vertebrae model is depicted in FIG. 12. Referring to the figure, first, at step 121, all training samples x are aligned based on the second extrema (the thoracic curve) of the polynomial models of each spinal cord, that

approximately corresponds to 8th thorasic vertebrae. Measurements of minor b and major axis a for each vertebra though out the training set are manually acquired at step 122. FIG. 4 shows 3 orthogonal views of a vertebrae depicting a manual measurement of the vertebrae axes. Each view shows a pair of perpendicular dashed lines that a clinician uses for extracting a measurement from the image, and the x- marks 40 in the views indicate the positions in the image from which measurements are made.

For an incomplete data set with missing vertebrae, the missing vertebrae can be modeled based on data from the longest vertebrae column training sample having all vertebrae present. Alternatively, the missing vertebrae can be modeled based on a mean of those vertebrae present in other training samples.

A typical sample vertebrae is represented by

where j is the sample number, the aμ are the major axes, the bμ are the minor axes, for i = 1, ..., n , the number of vertebrae. Usually n=24, the number of cervical, thoracic, and lumbar vertebrae.

Referring again to FIG. 12, a sample mean x and co variance S are computed at step 123 as

where m is the number of training samples. Principle component analysis (PCA) is applied at step 124 for all aligned and completed samples X _j to solve the following eigenvalue system:

SP 'k - K Pk

where p _k is a k ^th principle component mode, X ₁₁ . is the k ^th eigenvalue, and k ~ l,...,2n . Only the t modes that satisfy

^ ₃ confidence _ level £p ₅

are retained. An exemplary, non-limiting value for confidence J.ev el is about 95%,

An arbitrary model sample can be defined in terms of the principle modes as

x = x + Pd , (10)

where is a matrix of t eigenvectors, and d = (d _i ,...,d ₁ ) ^τ is model parameter satisfying

(H)

for y=7,...,f and some weighting factor w. According to an embodiment of the invention, the weight factor w can. be set tol because vertebrae typically show little variation.

Therefore, a new x can be created by x — x + Pd . The new x generated by these equations will be similar to the training samples during training since model parameters di have limited variation.

A fourth stage, step 94 of FIG. 9, of a vertebrae segmentation according to an embodiment of the invention involves fitting ellipses to find the major and minor axes for all vertebrae in a new image. FIG. 6(a) depicts an exemplary thoracic vertebrae, taken from Gray's Anatomy of the Human Body (1918), while FIG. 6(b) is a schematic representation of the vertebrae and the spinal cord. The spinal cord 62 is

represented as a cylindrical tube, and the vertebrae 61 is represented as an elliptical cylinder with the major and minor axes as indicated.

A flow chart of a method according to an embodiment of the invention for fitting ellipses to find the major and minor axes for all vertebrae is depicted in FIG. 13. Referring now to the figure, given a new image, using an edge detection algorithm, the edges of each vertebrae are found at step 131 in the region of interest around the spinal cord in an axial slice perpendicular to the spinal cord. One exemplary, non-limiting edge detection technique according to an embodiment of the invention uses the Canny edge detection algorithm. At step 132, a straight line is plotted into the vertebrae starting from the edge of the spinal cord in a plane orthogonal to the spinal cord. According to an embodiment of the invention, an exemplary, non-limiting line is 4 voxels long. At step 133, a voxel on the line is selected and a plurality m of rays are cast from the voxel in the line at substantially equally spaced angles. According to one non-limiting embodiment of the invention, the rays are cast every 12 degrees in a 180 degree span, for a total 15 rays.

The edge voxels lying on or close to the projected rays are located at step 134. For each ray, the point closest to the ray's beginning point is selected, and at step 135 another point is located at the same distance on a ray projected in the opposite direction. All x and y coordinates of all 2m points, including the original edge points and projected coordinates, are collected on all m rays at step 136. FIG. 5 depicts schematic of a vertebrae boundary 51 and spinal cord 52 with the projected points 53 on the rays 54 cast from point P, with an ellipse 55 embedded within the vertebrae.

Referring again to FIG. 13, at step 137, an ellipse can be fitted to the collected points using a least squares technique. A least squares technique for fitting an ellipse involves minimizing an error function

where x and y are the coordinates of the detected and projected edge points translated to the origin p, and m is the number of rays projected from the current center. The

error function is minimized by setting the partial derivatives with respect to a and b to 0:

da db

Then a and b are computed as:

and

At step 138, this minimization is repeated for all points on the straight line from the edge of the spinal cord, and the fitted ellipse with the maximum area is selected at step 139 as the vertebrae model. FIG. 6(c) depicts an axial slice through the middle of a vertebrae 61 and spinal cord 62, both with fitted ellipses, according to an embodiment of the invention.

A final stage, step 95 of FIG. 9, of a vertebrae segmentation according to an embodiment of the invention involves applying the model constraints to detected ellipses to improve vertebrae segmentation in a new image. A flow chart of a method according to an embodiment of the invention for applying model constraints to detected ellipses to improve vertebrae segmentation in a new image is depicted in FIG. 14. Referring now to the figure, given a new image, an ellipse for each vertebra can be estimated at step 141 by using local intensity information. An ellipse fitting algorithm according to an embodiment of the invention such as that described above in connection with FIG. 13 is an example of the use of local intensity information for estimating an ellipse. If the vertebrae column image is incomplete, the axis lengths of

missing vertebrae can be computed as an average from the training sample. That will yield a vector x of ellipse axes

The vector x' can be projected at step 142 into the model space according to d = P ^~l [pc — x) . If, at step 143, x' is reasonable, d' will satisfy the model constraints: that is, all components of d' will satisfy the condition — WλJλi , for i = l,...,t. If the condition is not satisfied, the values of d' are restricted at step 144 to be in the range — . Then, x _aew = x + Pd' is computed at step 145, and

b = P- ^l {x _new -x), -Jλ _k ≤b _k ≤ 4λ _k ^~ .

(12)

The vector x now contains all axes of all vertebrae. Then, for those samples missing vertebrae, the axes lengths for the missing vertebrae that were inserted in the previous step as the average from the training samples are removed.

A vertebrae segmentation algorithm according to an embodiment of the invention was visually evaluated on 77 T2 STIR images with different levels of fat suppression and on 5 HASTE images (the HASTE pulse sequence is mainly used for abdominal organs like the liver). In all cases segmentation results were satisfactory. Although intensity distributions for a majority of the organs are very different for T2 STIR and HASTE images, it had no effect on the segmentation results. No changes were made to the algorithm developed originally for T2 STIR images to adapt it for the HASTE pulse sequence. FIGS. 7(a)-(c) depict vertebrae segmentation results for a coronal T2 STIR image in FIG. 7(a), an original (left) and segmented (right) T2

STIR image with metastases in FIG. 7(b), and vertebrae segmentation results for a HASTE image in FIG. 7(c).

The bandpass filtering can be applied to other anatomical structures. In the _. case of breast masses, the ID signal is extracted by a polar transformation of the intensity values inside a circle super-scribing the mass and centered on its center mass of intensity. For a colon fold periodicity analysis, an unwrapped colon surface, a local profile or a distance map of colon surface to either the colon centerline or the local profile can represent the signal for evaluation. For the upper and lower thresholds, for breast masses different bands of frequencies were used to capture the mass specularity at different scales. In the beast mass example, the maximums will be located in the spikes of higher intensity. Colon thresholds can be based on minimum and maximum distances between colon folds. Different sets of thresholds can also be used in different colon segments, as the rectum, sigmoid, descending, transverse, ascending colon and cecum, all have slightly varying fold spacing.

FIGS. 8(a)-(b) depict breast mass feature extraction using projected median intensities. FIG. 8(a) depicts a raw image slice from a breast image, while FIG. 8(b) illustrates a plot of an unfiltered 81 and a filtered 82 projected median intensity signal using a lower band pass filter, and a filtered 83 projected median intensity signal using a higherr band pass filter. The arrows from the curves of FIG. 8(b) to the raw image in FIG. 8(a) show the correspondence of dark regions to intensity signal minima, and of bright regions to intensity signal maxima.

It is to be understood that embodiments of the present invention can be implemented in various forms of hardware, software, firmware, special purpose processes, or a combination thereof. In one embodiment, the present invention can be implemented in software as an application program tangible embodied on a computer readable program storage device. The application program can be uploaded to, and executed by, a machine comprising any suitable architecture.

FIG. 15 is a block diagram of an exemplary computer system for implementing a method for detection and segmentation of vertebrae structures in the spine according to an embodiment of the invention. Referring now to FIG. 15, a

computer system 151 for implementing the present invention can comprise, inter alia, a central processing unit (CPU) 152, a memory 153 and an input/output (JIO) interface 154. The computer system 151 is generally coupled through the I/O interface 154 to a display 155 and various input devices 156 such as a mouse and a keyboard. The support circuits can include circuits such as cache, power supplies, clock circuits, and a communication bus. The memory 153 can include random access memory (RAM), read only memory (ROM), disk drive, tape drive, etc., or a combinations thereof. The present invention can be implemented as a routine 157 that is stored in memory 153 and executed by the CPU 152 to process the signal from the signal source 158. As such, the computer system 151 is a general purpose computer system that becomes a specific purpose computer system when executing the routine 157 of the present invention.

The computer system 151 also includes an operating system and micro instruction code. The various processes and functions described herein can either be part of the micro instruction code or part of the application program (or combination thereof) which is executed via the operating system. In addition, various other peripheral devices can be connected to the computer platform such as an additional data storage device and a printing device.

It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures can be implemented in software, the actual connections between the systems components (or the process steps) may differ depending upon the manner in which the present invention is programmed. Given the teachings of the present invention provided herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the present invention.

While the present invention has been described in detail with reference to a preferred embodiment, those skilled in the art will appreciate that various modifications and substitutions can be made thereto without departing from the spirit and scope of the invention as set forth in the appended claims.