METHOD, DEVICE , AND COMPUTER PROGRAM PRODUCT FOR SEGMENTING AN IMAGE OF AN AREA

FIELD OF THE INVENTION

The invention relates to a method for obtaining an image of an area comprising the following steps: illuminating the area with light from a light source; - detecting light coming from the area in response to illuminating the area with light from the light source.

The invention also relates to a device for obtaining an image of an area comprising: a light source for illuminating the area; - a detection unit for detecting light coming from the area in response to illuminating the area with light from the light source.

BACKGROUND OF THE INVENTION

An embodiment of the method and device is known from US patent application US 2006/0108502 Al. The document describes a control system that maintains the overall intensity of a microscope's view viewed at a constant level. The system is further enhanced to match the colour character of a microscope illuminator to a user-established colour reference for replication and comparison purposes. Normal use of a microscope requires constant changing of lenses, filters and diaphragms to optimise the particular objectives. Varying degrees of attenuation are correspondingly introduced into the illumination optical network. To compensate for these alterations in viewing intensity, the user is required to continually readjust the level of the system's illuminator. Associated with these intensity changes in incandescent illuminators are unwanted colour temperatures shifts in the illumination spectra. The document describes how a microscope viewed image is maintained at both a constant level of intensity and of spectral quality by monitoring the illuminator(s) output and specimen images at the end of their optical travel where the final image is formed for viewing. Utilising an array of spectrally matched pairs of detector/LED combinations attains colour compensation. A discreet primary colour is represented by each of the three detector/LED combinations. Each detector senses the colour level of a reference

image (or stored value) and develops a difference error signal to drive its respective LED until the difference is eliminated. The LED outputs are thereby either added or subtracted to the basic illuminator output to eliminate any relative spectral deviations. Intensity compensation is obtained by using a small sample of the final image to provide error signals for a closed loop servo system that alters the attenuation level of an electronically controlled variable neutral density filter.

A microscope as described above may be used, for instance, in the diagnosis of malaria and tuberculosis. In such applications, the microscope can be used to observe blood and sputum samples. During the preparation of smear samples, a staining agent is added to a biological specimen in order to highlight specific objects, such as parasites or bacteria, on the slide. The stain gets fixated on to the specific objects and show up in colours that can be distinguished from the background in the microscope image. Automatic analysis of such a microscopic image enables rapid detection of diseases. This requires image processing algorithms to robustly detect objects of interest specific to the disease, such as malarial parasites in blood smears and tuberculosis bacilli in sputum smears.

It is a drawback of the known method and device that the segmentation of an image obtained through the method and device is often inaccurate.

SUMMARY OF THE INVENTION It is an object of the invention to provide a method enabling better segmentation of an image obtained through use of the method. According to the invention this object is realised in that the method further comprises the following additional steps: changing the hue of the light from the light source over a range of hues; - obtaining a hue histogram for each hue; determining peak separation in each hue histogram; determining the segmentation hue corresponding to maximum peak separation; determining the valley between peaks in the hue histogram obtained for the segmentation hue; segmenting the image as foreground and background based on the hue histogram obtained for the segmentation hue.

The invention is based on the recognition that varying the hue of the light generated by the light source results in a different hue histogram for each hue value of the

light from the light source. In a hue histogram the occurrence of a specific to value is plotted as a function of the hue value. Ideally, a hue histogram comprises two peaks, one of which corresponds to the foreground of the image and one of which corresponds to the background of the image. However, in reality a hue histogram is often more complex for instance because the light source produces white light which comprises a wide range of hues. By obtaining hue histograms for different hue values, the segmentation hue for which the foreground peak and the background peak in the hue histogram are maximally separated can be determined. With this information, obtaining an image with the light from the light source having the segmentation hue then allows improved segmentation as compared to the prior art. Hue is quantified with a value (0 to 360 degrees) whereas colour is a more general term. The current embodiment described uses hue value to represent colour. It may however be possible to use alternative colour space representations such red-green-blue, chrominance, LAB, etc.

It is an additional advantage of the invention that overcomes the problem that the staining of a microscope sample is sensitive to the pH of the solution used in the sample. It is known that during slide preparation the staining of a microscope sample is sensitive to the pH of the solution used. This means that minor changes in pH can cause colour variations. The invention overcomes this problem by adaptive feed changing the hue of the light source in order to provide the best contrast. It is a further additional advantage of the invention that it improves the segmentation of a microscope image irrespective of the stain that is used in preparing the microscope sample. Different stains may be used for observing the same type of (biological) sample. For instance, malarial parasite can be highlighted using either Leishman or Giemsa stain, which have different colour characteristics and thus may need different thresholds for segmentation. The invention overcomes the drawback of retuning different threshold parameters for different staining agents.

It is a further additional advantage of the invention that is allows automatic segmentation of an image.

An embodiment of the method according to the invention wherein the method further comprises the following additional step: illuminating the area for a second time with light from the light source, the light having the segmentation hue.

This embodiment has the advantage that a device used to execute the method does not require a memory to store a view and the associated histogram. Instead, after the

segmentation hue value has been found by varying the hue value of the light generated by the light source (which includes illuminating the area to be viewed with light having the segmentation hue value), the area to be viewed is illuminated for a second time with light having the segmentation hue value. A further embodiment of the method according to the invention wherein the method further comprises the following additional step: post-processing of the segmented image to calculate counts of an object of interest.

This embodiment has the advantage that it enables to determine the occurrence of an object of interest in the area imaged. Determining the occurrence of an object of interest, such as malaria parasites or tuberculosis bacilli, is often helpful in diagnosing a disease.

The object of the invention is also a realised with a device for obtaining an image of an area comprising: - a light source for illuminating the area; a detection unit for detecting light coming from the area in response to illuminating the area with light from the light source, characterised in that the light source is a variable colour light source; - the device further comprises a control unit for:

(a) changing the hue of the light from the light source over a range of hues;

(b) obtaining a hue histogram for each hue;

(c) determining peak separation in each hue histogram;

(d) determining the segmentation hue corresponding to maximum peak separation;

(e) determining the valley between peaks in the hue histogram obtained for the segmentation hue;

(f) segmenting the image as foreground and background based on the hue histogram obtained for the segmentation hue. The device enables execution of the method according to the invention.

An embodiment of the device according to the invention, wherein the control unit further controls:

(g) illuminating the area for a second time with light from the light source, the light having the segmentation hue.

This embodiment has the advantage that a device according to this embodiment does not require a memory to store a view and the associated histogram. A further embodiment of the device according to the invention wherein the control unit further controls:

(g) post-processing of the segmented image to calculate counts of an object of interest.

This embodiment has the advantage that it enables to determine the occurrence of an object of interest in the area imaged.

The object of the invention is also realised with a computer program product for causing a processor to enable a device to execute the method according to any one of claims 1-3. A device to any one of the previous embodiments comprising a processor comprising, for instance, an integrated circuit would benefit from a computer program product enabling the device to execute the method according to the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 schematically shows an ideal bimodal histogram;

Fig. 2 schematically shows a hue histogram of a microscope blood smear image obtained with white light;

Fig. 3 schematically shows an embodiment of a method according to the invention;

Fig. 4 schematically shows an embodiment of a device according to the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 schematically shows an ideal bimodal histogram. Plotted along the x- axis is the hue value. Plotted along the y-axis is the occurrence of a specific hue value. The two peaks 5 and 10 correspond to the foreground and the background of a fictitious microscope image (not shown) of which fig. 1 schematically shows the hue histogram. The fictitious microscope image may be obtained by, for instance, inserting a stained blood or sputum specimen into a microscope fitted with a C-mount (for capturing images using a camera). The valley point 15 between peaks 5 and 10 is taken as a threshold for image segmentation. Valley detection methods in histograms are described in Bernaud F. Buxton et

al, 'Development of extension of Otsu algorithm for multidimensional image segmentation of thin- film blood slides', Proc. Int. Conf. Computing: Theory and Applications (ICCTA) 2007 and N. Otsu, 'A threshold selection method from gray-level histograms', IEEE Trans. Systems Man and Cybernetics, vol. 9, no. 1, pp. 62-66, 1979. In most cases, due to white light illumination, a hue histogram is not bimodal (see fig. 2).

Fig. 2 schematically shows a hue histogram of a microscope blood smear image obtained with white light. Plotted along the x-axis is the hue value normalised in range 0 to 1. Plotted along the y-axis is the occurrence of a specific hue value. Due to white light illumination the hue histogram does not resemble the ideal bimodal histogram shown in fig. 1. The problem of segmenting an image of an area to which the histogram shown in fig. 2 corresponds into a foreground and background is overcome by the invention by controlling the hue of the light source in the microscope used to obtain the image such that a bimodal hue histogram is obtained for the area imaged. According to the invention this bimodal hue histogram is obtained by changing the hue of the light generated by the microscope light source, obtaining a hue histogram for each hue value of the light generated by the light source, and selecting the bimodal histogram from the set of histograms obtained. The bimodal histogram with maximum foreground-background peak separation is selected from the group of histograms.

As a result of obtaining a bimodal hue histogram, foreground/background thresholds, which would otherwise have to be set manually, can be calculated automatically. Calculating foreground/background thresholds is described in Bernaud F. Buxton et al, 'Development of extension of Otsu algorithm for multidimensional image segmentation of thin- film blood slides', Proc. Int. Conf. Computing: Theory and Applications (ICCTA) 2007. Fig. 3 schematically shows an embodiment of a method according to the invention. In step 20 and area to be imaged, for instance by a microscope, is illuminated with light from a light source, the light having a certain hue value. Next, in step 25, an image of the area is obtained the area by detecting light coming from the area in response to illuminating the area with light from the light source. In step 30 a hue histogram of the image is constructed. Then, in step 35, the peak separation in the histogram is calculated. In step 40 the hue of the light from the light source is changed to another value within the range of values enabled by the light source. Next, steps 20-35 are repeated until a segmentation hue value is obtained for which the peak separation is maximal. This is indicated by the arrow 45. In step 50, the area to be imaged is illuminated with light from the light source, the light having the segmentation hue value. As during the search for the maximum peak separation

the hue of the light generated by the light source is varied over a range, the search includes illuminating the area to be viewed with light having the segmentation hue value. Hence, in step 50 the area to be viewed is illuminated with light having the segmentation hue value for the second time. However, this is not always necessary depending on the memory available in the device used to execute the method, for instance a microscope. If illuminating the area to be viewed with light having the segmentation hue value for a second time is not necessary, step 50 can be deleted. However, if all the images acquired during the hue-sweep have to be stored along with the histogram, then execution of the method would require a large memory in the device used to execute the method. If such memory capacity is not available, the area to be viewed has to be illuminated a second time with the segmentation hue value and step 50 should be performed. Next, in step 55 the valley between peaks in the hue histogram obtained for the segmentation hue is determined. In step 60, the image of the area to be imaged is segmented as foreground and background based on the hue histogram obtained for the segmentation hue. Finally, in step 65, the segmented image is post-processed to calculate counts of an object of interest in the image, such as malarial parasites or tuberculosis bacilli.

Fig. 4 schematically shows an embodiment of a device according to the invention. The device 70, for instance a microscope, comprises a light source 75 for illuminating an area 90 to be imaged. The light source 75 may, for instance, be an LED light source. The light source 75 is arranged such that the light generated by the light source 75 can vary over a range of hue values. The device further comprises a detection unit 80 for detecting light coming from the area 90 to be imaged in response to illuminating the area 90 with light from the light source 75. The device is still further comprises a control unit 85 for:

(a) changing the hue of the light from the light source 75 over a range of hues;

(b) obtaining a hue histogram for each hue; (c) determining peak separation in each hue histogram;

(d) determining the segmentation hue corresponding to maximum peak separation;

(e) illuminating the area 90 with light from the light source, the light having the segmentation hue; (f) determining the valley between peaks in the hue histogram obtained for the segmentation hue;

(g) segmenting the image as foreground and background based on the hue histogram obtained for the segmentation hue.

The control unit 85 may be further arranged for post-processing the segmented image to calculate counts of an object of interest, such as malarial parasites or tuberculosis bacilli.

As previously stated, the invention can be used in, for instance, a microscope. However, the invention is generally applicable to any kind of optical system where a sample area has to be observed under illumination, which can be varied to highlight the object of interest. Other possible usage scenarios than microscopy include defect detection in a manufacturing process where the defect is the object of interest.

In a microscope, light from a light source is transmitted through the sample under observation. However, the invention can also be used in cases in which light is reflected from a sample under observation.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the system claims enumerating several means, several of these means can be embodied by one and the same item of computer readable software or hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.