Technical field

The invention relates to spectral imaging, in particular - to imaging of distribution of the surface reflection attenuation coefficient at a number of fixed wavelength bands by means of a single RGB image data set.

Back round Art

Spectral attenuation of reflection or the spectral reflectance

(http://rsgislearn.blogspot.com/2007/04/spectral-reflecta nce.html) k( ) is defined as the ratio Ι(λ)/Ι ₀ (λ), wherein Ι(λ) is reflected intensity from the object surface area at a fixed wavelength λ, and Ι ₀ (λ) is the reflected intensity at this wavelength from a fully reflective surface (so-called white reference) under the same lighting conditions. Classical optics offers two options for experimental identification of k (λ) - either by recording the Ι(λ) and Ι ₀ (λ) values at broadband lighting conditions via monochromator or a narrowband spectral filter, selecting only reflection at the wavelength λ, or by recording the two values at monochromatic lighting conditions, where reflection is possible only at one λ-value (http://en.wikipedia.org/wiki/Reflectivity). To determine k( ) values at several fixed wavelengths in such a way, the corresponding number of measurements has to be taken - e.g. 3 measurements if 3 different wavelengths are of interest.

Spectral imaging allows identifying areas of object surface with different values of spectral reflectance; it can be helpful for art work expertise, satellite surveillance of Earth surface, clinical diagnostics of skin, etc. Digital sensors - two-dimensional photo matrices equipped with external spectral filters, e.g. a rotating disk with a set of different colour filters (E.C. Ruvolo et al, Proc. SPIE, Vol. 7548, 75480A, 2010) or spectrally tuneable acousto-optical or liquid crystal filters

(http://www.usgs. gov/science/science.php?term=765) - are often used for obtaining spectral images.

Digital RGB image sensors comprise integrated three colour (blue, green and red) spectral filters that can be used for spectral imaging in various combinations with external spectral filters (US 7612822 B2, US 2009290124 Al, JP 2008136251 A). Another way of obtaining spectral images is sequential illumination of the object by a number of light sources, each emitting at different spectral band (for example, LEDs of different colours - WO 2008093988 Al) and taking an image at each spectral band of the illumination.

These mentioned methods are useful, but their drawback is the necessity to take a number of consecutive images of the same object at different spectral bands. First, the process is time-consuming. Second, the object properties may change during this process, for instance, it can move - in that case incorrect results are obtained, or additional procedures for image stabilization should be included in the processing algorithm, so slowing-down the data processing. Third, huge amount of unnecessary information on image details is being stored. Fourth, the design of such equipment is complicated and expensive, since additionally to the RGB sensor with objective it includes external filtering devices and/or sets of spectrally different illumination sources.

Such drawbacks are avoided if the methodology of single snapshot RGB image spectral analysis is applied. The image taken by a digital RGB sensor (e.g. CCD or CMOS, http://broadcastengineering.com/hdtv/ccd-cmos/) is transformed into the colour format by determining first the numerical values of Rj, Gi and Bj (where„i" is the pixel number) signals detected in this particular pixel at the red (R), green (G) and blue (B) spectral bands, and then adjusting for each pixel the colour corresponding to the specific RiGiBj combination. Spectral sensitivities of individual R-, G- and B- channels are determined by absorption properties of the filtering coatings of three types and of the material of photomatrix detector (e.g. silicon). The spectral sensitivity curves of the three bands usually are provided in the specifications of serially produced RGB sensors; they can also be experimentally measured. Thus it is possible to extract additional three spectrally selective sub-images for the R-, G- and B-bands (corresponding to the emitted or reflected from the object light in red, green and blue part of the spectrum) from a single RGB colour image data set and to perform multi- spectral analysis by division, subtraction and/or other manipulation with these sub-images (D. Kapsokalyvas et al., Proc. SPIE, Vol. 7548, 754808, 2010). Logical analysis of the RGB data set allows separating six narrower spectral bands, if a certain discrimination level of the sensor output signals is fixed, above which at any wavelength of the sensor spectral sensitivity range only one or two of the R, G or B values are recorded (J. Spigulis et al, Proc. SPIE, Vol. 7557, 75570M, 2010). If the discrimination level is changed, the number of spectral intervals that can be extracted from a single digital RGB data set essentially increases (WO 2012/002787). These technical solutions can be implemented at poly-chromatic object illumination, i.e. by illuminating at only few narrow spectral bands with precisely known wavelengths.

In various technical, environmental and medical fields it is important to determine spectral attenuation for each image pixel simultaneously at several radiation wavelengths and to form parametric images based on the corresponding distributions. Such images are of practical importance, for example, in dermatology to map distribution, of skin chromophores (melanin, haemoglobin, bilirubin, etc.) for clinical assessment of pathologies. The possibility of single snapshot RGB mapping of skin oxy-haemoglobin at bi-chromatic laser illumination has been demonstrated, assuming that the G-channel signal is fully representing the intensity of reflected green laser light and the R-channel signal - the intensity of reflected red laser light (J. Spigulis et al., Proc. SPIE, Vol. 7557, 75570M, 2010). Based on the ratio of these two signals, parametric image of the oxy-haemoglobin index formed approximate skin oxy-haemoglobin map. However, it is impossible to determine the values of particular spectral reflection coefficients at each image pixel by this methodology.

Usually the three spectral sensitivity curves of practically available RGB image sensors partially overlap, so monochromatic radiation can be simultaneously registered in two or three colour channels (R, G and/or B) of the sensor; in this case none of colour channel signals adequately describes the perceived intensity of monochromatic radiation. The impact of inter-channel crosstalk is even greater in bi-chromatic lighting situation; it can be corrected, if RGB channel spectral sensitivity curves are known and linear photo- response is ensured (J.Spigulis et al., Proc. SPIE, v.8216, 82160L, 2012). The impact of RGB inter-channel crosstalk should be observed also in other poly-chromatic lighting situations and it is necessary to know if a single RGB image data set contains information, sufficient to determine spectral reflectance in all image pixels at all (or at least some) wavelengths applied for poly-chromatic lighting of the object.

Goal of this invention - to provide imaging of the spectral reflectance numerical values at several fixed wavelengths exploiting a single snapshot RGB image data set.

Disclosure of the invention To reach the goal, a new digital RGB sensor's output data processing method for imaging of spectral reflectance and a device implementing this method are proposed. The method is characterized in that k( ) numerical value in each object image pixel at several selected wavelengths within the RGB sensor spectral sensitivity range is determined, taking into account RGB inter-channel crosstalk, and parametric imaging of the k( ) values at each selected wavelength is provided. The following conditions for implementing this method are to be observed:

at the moment of taking a single snapshot RGB image, an object is evenly poly- chromatically illuminated at a number of selected wavelengths;

lighting conditions provide linear RGB image sensor's photo-response;

image sensor's RGB spectral sensitivity curves are known,

- the displayed object area contains relatively small, fully reflective within the selected spectral range element - the white reference.

Description of the figures

Fig. 1 explains the imaging of k( ) values at two fixed lighting wavelengths λ _\ and λ ₂ ; Fig. 2 illustrates situation with tri-chromatic lighting, when object is simultaneously illuminated at the wavelengths λ _\ , λ ₂ and λ ₃ and the respective signals are recorded at all three RGB channels;

Fig. 3 shows the block diagram of the device for spectral reflectance imaging at a number of spectral bands.

First, the method for imaging of the k k) values at two fixed illumination wavelengths λ _\ and λ ₂ (Fig. 1) is examined; the respective values are denoted as ki and k ₂ . The selection of λι and λ ₂ is determined by the proposed application - for example, for skin haemoglobin mapping λι =532 nm (high haemoglobin absorption) and λ ₂ =635 nm (low haemoglobin absorption) can be chosen. For the relative distribution mapping of skin haemoglobin concentration, the ratio k = k ₂ /ki values at each image pixel or group of connected pixels can be used. Assuming that the B-channel spectral sensitivity Sx at both selected wavelengths is negligibly small, only spectral sensitivities of the G and R bands have to be considered (Fig. 1). In this situation the signal detected in G-channel mainly displays the reflection intensity with wavelength λι (signal Gl) and the signal detected in R-channel - reflection intensity with wavelength λ ₂ (signal R2). However, because of partial overlap of the spectral sensitivity bands, a certain signal with wavelength λ _\ is detected also in R-channel (signal Rl), and some signal with wavelength λ ₂ is detected also in G-channel (signal G2) - this is the channel _^ crosstalk" effect, due to which the detected signal correction is necessary.

If the conditions above are met, signals in G- and R-channels received by all pixels are directly proportional to the channel spectral sensitivities S at the selected wavelengths λ _\ and λ ₂ ; they are denoted by S(G1), S(G2), S(R1) and S(R2). Thus, the„monochrome" signal' s ratio at X _\ and λ ₂ are strictly fixed values: S _r = ^—l∑ _anc j e __^R ^j ^ δ ^y S(G ₂ ) ^R S(R ₂ )

Consequently, the signals received in the G- and R-channels can be described by the following equation system:

G = G _{l +} G ₂ (1),

R = R, + R ₂ (2),

G _l = S _G · G ₂ (3),

R!= R ₂ (4)·

The system has an explicit solution:

GS _r G RS _R „ R

It allows to correct errors caused by channel interaction and to find exact intensity values of reflected signals for each wavelength. For example, "pure" Gi value could be found from the known (measured) G and S _r values: G, = G

(l + 5 _r ) 1 + S _r

If the above can be attributed to the signals, reflected by the white reference, the real object (e.g. skin) reflection intensity of each surface point - image pixel or pixel group has to be associated to the respective spectral reflectance or reflection spectral attenuation coefficients ki and k ₂ , which alters the values of signals received in the G- and R-channels:

G'= k _l ^■ G _l + k ₂ - G ₂ = + (5),

(1 + 5 _G ) (1 + 5 _G )

k,RS

(l + S _R ) (l + S _R )

The solution of the system (5)-(6) gives exact spectral attenuation coefficient values for each RGB image pixel or group of pixels: _k _G'(l + S _G )R-R'(l + S _R )G

GR(S _G -S _R )

k _GS _G R'(l + S _R )-RS _R G'(l + S _G )

GR(S _G -S _R )

k_k ₂ _GS _G R'(l + S _R )-RS _R G'(l + S _G )

k, G \ + S _G )R-R \ + S _R )G

Consequently, the expressions (7) and (8) could be used for spectral imaging at two fixed wavelengths λ _\ and λ ₂ by the measured integrated" R, G, R' and G' values, and the expression (9) - for imaging of the distribution of two spectral attenuation coefficient's ratio, all by using only one RGB image data set.

The above mentioned regarding the pair of R-G bands is also valid for the other two RGB band combinations (R-B and G-B), if the two selected wavelengths are within the spectral range of these bands.

The method is also applicable to the tri-chromatic illumination situation where the object simultaneously is illuminated at the wavelengths λ _1; λ ₂ and λ ₃ and the respective signals are recorded at all three RGB channels (Fig.2). In general, the signals recorded at each fixed wavelength will be summarized in each of the three colour channels; then the signals reflected by the white reference can be described by the system of nine equations:

R = R _l +R ₂ +R ₃ (10) G = G,+G ₂ +G ₃ (11) B = B _l +B ₂ +B ₃ (12)

S _R2 , =^- (15) S ²³ S(R ₃ )

S _G12 =^ (16) S(G ₂ )

S _G13 =^ (17)

5(G ₃ ) s ⁼ ^ ^{(18) 5} - ⁼ S ⁽ⁱ⁹⁾ S _B13 = (20)

S(B ₃ )

^{5B23 =} I¾) ⁽²¹⁾ ' where R1,G1 and Bl are the digital values of signals, recorded respectively in R-, G- and B-channels at the first wavelength R2, G2 and B2 - the signals recorded at the second wavelength λ ₂ , and R3, G3 and B3 - the signals recorded at the third wavelength λ ₃ . Ratios of the R, G and/or B channel spectral sensitivities (i.e. ratios of the intensities of signals received) at the selected wavelengths λ _1; λ ₂ and λ ₃ in all possible combinations are described by equations (13) - (21).

There is exact solution of the system (10)-(21) representing nine component values of the integrated RGB signals:

1 +— 1 +— 1 S _{c 2} + l +— 1— ^13 ++ ^23 ++ 1

^ G12 ^ G13 ^ G23

B

These values apply to signals from a white reference. If at the same time the object RGB image is received, it is possible precisely to establish also the spectral reflectance values k _l5 k ₂ and k in each object image area (a pixel or pixels group) at all three fixed wavelengths, by solving the equation system:

LR k ₂ R k ₃ R

R'^ k^ + k ₂ R ₂ + k ₃ R ₃

(1 + - + 1 2)

' «12 ₊ -L ₎ ( ⁵ «13 + ^S R23 +^ (2 (23)

B = k ₁ B _l + k ₂ B ₂ + k ₃ B ₃ = K ^B — + ^k 2 ^B — +—— ₍₂₄ )

(1 +— +—) +—) ™ ^{+ !} >** ⁺ » As a result, exact spectral reflectance values are obtained from the data of RGB measurements for each image pixel or selected group of pixels at the wavelengths λ _\ , λ ₂ and λ ₃ :

B"G ₃ R ₂ — B ₃ G" R ₂ — B" G ₂ R ₃ + B ₂ G "R ₃ + B ₃ G ₂ R "—B ₂ G _2i R " _l

(25),

B G ₂ R _X - - B ₂ G R _X - B^G _i R ₂ + B _i G ₃ R ₂ + B ₂ G _i R ₃ ^{~ B} iG ₂ R ₃

B 'G ₂ R ₁ - B ₂ G"R ₁ - B"G _i R ₂ + Bfi" R ₂ + B ₂ G _i R "- -BiG ₂ R"

(27).

B G ₂ R _X - B ₂ G R _X - B^G _i R ₂ + B _i G ₃ R ₂ + B ₂ G _i R ₃ ^{~ B} iG ₂ R ₃

According to expressions (25)-(27), it is possible to calculate the spectral reflectance values for each of the RGB image pixel or group of pixels at any combination of three fixed wavelengths, and to construct three respective parametric images or maps of spectral reflectance, for example, using a certain colour scale for display of the numerical values.

In general case of poly-chromatic illumination comprising n spectral bands with wavelengths λ _{1 ?} λ ₂ , ..., λ _η , the appropriate spectral reflectance values can be found by solvin a linear equation system represented as a product of matrices A*X = D, where

A the detector matrix formed by the values of spectral sensitivities

of the R G and B channels at the selected wavelengths λ _{1 ?} λ ₂ , ..., λ _η ;

X the matrix formed by the spectral reflectance coefficients k; (i = 1, 2, n);

(R'\

D the matrix formed by the signal values recorded at the R, G and B channels.

B'j

The expression:

follows.

To calculate the values of spectral reflectance (k ₁ ,k ₂ ,...,k _n ) from the expression

(28) for each object image area (pixel or group of pixels), for example, the Gaussian method (http://www.math-linux.com/spip.php7article53) can be applied.

The device for imaging of spectral reflectance at several wavelength bands that implements the method above is illustrated in Fig. 3. The device contains:

a poly-chromatic light source (1) comprising, for example, a set of lasers and/or LEDs, each emitting at different spectral band, by which the examined object (2), for example, skin surface is illuminated;

a white reflector/reference (3), adapted to occupy only a small portion of the displayed object surface;

an objective-equipped digital RGB sensor (4) which is adapted to transform the object image into digital format, attributing to each pixel a certain set of the R-, G- and B-values, with their output to the RGB data storage device (5);

a converter (6), adapted to transform the stored RGB data set into the set of spectral reflectance values for each pixel or group of pixels;

a spectral reflectance imager (7) which uses the obtained spectral reflectance values for each pixel or group of pixels and a colour scale or other identifier to create several spectral reflectance images of the object, accordingly to the number of spectral bands used for the poly-chromatic illumination;

an output device (8), such as computer monitor, to which the obtained spectral reflectance image set can be transmitted.

Further processing of the spectral reflectance images can be performed by already known methods.