GENERATING NARROW-BAND SPECTRAL IMAGES FROM BROADBAND SPECTRAL IMAGES

CROSS REFERENCE

This application claims priority from US provisional patent 62/644538 filing date March 18 2018 which is incorporated herein by its entirety.

BACKGROUND

Small tunable filters include microelectromechanical system (MEMS) filters such as the MEMS Fabri-Perot filter (hereinafter - FP etalon).

There is a growing need to provide narrow band images from a spectral imager that includes a FP etalon.

SUMMARY

There may be provided methods, spectral imagers and computer program products for generating narrow-band spectral images from broad-band spectral images. The generating of narrow-band spectral images from the broad-band spectral images may include constructing the narrow-band spectral images by a set of linear transformations applied on the broad-band spectral images through the use of a reconstruction matrix.

There may be provided a method for generating narrow-band spectral images, the method may include acquiring, by a spectral imager that may include a tunable filter, a first plurality (N) of broad-band spectral images associated with respective states of the tunable filter; and processing, by a processing circuitry the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images.

The reconstruction matrix when multiplied by H, which is a matrix that represents spectral responses associated with respective states of the tunable filter, can result in a sparse matrix which ideally resembles the unity matrix associated with an ideal narrow- band (e.g. hyperspectral) filter.

The each broad-band image of the first plurality of broad-band spectral images may be associated with a respective broad-band transmission curve having a respective broad-band full width half maximum value; and wherein each narrow-band spectral image of the second plurality of narrow-band spectral images may be associated with a respective narrow-band transmission curve having a narrow-band full width half maximum smaller than the broad-band full width half maximum value of the broad- band transmission curve.

The spectral imager may include an image sensor having a filter array with a third plurality (C) of filter types, wherein filters of different types differ from each other by transfer function; and wherein C exceeds two.

C may equal three and the filter array may be a color filter array.

C may equal three and the filter array may be a red, green and blue color filter array.

C may equal four and the filter array may be a red, green, blue and infrared filter array.

C may equal four and the filter array may be a red, green, blue and white filter array.

C may equal three and the filter array may be a red, blue and white filter array

The method may include applying an expansion process for generating, from the first plurality of broad-band spectral images, a fourth plurality (Q) of broad-band spectral images and processing the fourth plurality of broad-band spectral images into the second plurality of the narrow-band spectral images; and wherein Q exceeds each one of N and M.

The ratio between Q and N may be an integer.

The expansion process may be a demosaicing process.

The expansion process differs from a demosaicing process.

The processing of the fourth plurality of broad-band spectral images into the second plurality of narrow-band spectral images may include constructing the second plurality of narrow-band spectral images by linearly transforming the fourth plurality of broad-band spectral images using a reconstruction matrix.

The method may include constructing the second plurality of narrow-band spectral images by using a reconstruction matrix; wherein the reconstruction matrix represents a cost function and spectral responses associated with respective states of the tunable filter.

The reconstruction matrix may be calculated based on a Tikhonov matrix. The Tikhonov matrix may be a multiple of the identity matrix.

The Tikhonov matrix may be a lowpass operator.

The Tikhonov matrix may be selected out of a group of Tikhonov matrices.

The group of Tikhonov matrices may be associated with a group of reconstruction matrices; wherein different reconstructions matrices of the group may be associated with different values of one or more spectral imager performance attributes.

The one or more spectral imager performance attributes may include a signal to noise ratio.

The one or more spectral imager performance attributes may include a resolution. The reconstruction matrix may equal ( H ^T H + G ^T G ) ^~1 H ^T , wherein H may be a matrix that represents spectral responses associated with respective states of the tunable filter, and G a Tikhonov matrix.

Alternatively, the reconstruction matrix may include 0 ^T H[ H ^T H + G ^T G) ^_1 ] ^T wherein 0 may be a general matrix.

The method may include selecting the Tikhonov matrix out of a group of

Tikhonov matrices.

The selecting may be based on at least one property of at least one of the first plurality of broad-band spectral images.

The at least one property may be a signal to noise ratio.

The at least one property may be a resolution.

The method may include calculating the reconstruction matrix.

The cost function may be a Tikhonov matrix.

The Tikhonov matrix may be a multiple of the identity matrix.

The Tikhonov matrix may be a lowpass operator.

The method may include selecting the Tikhonov matrix out of a group of

Tikhonov matrices.

The group of Tikhonov matrices may be associated with a group of reconstruction matrices; wherein different reconstructions matrices of the group may be associated with different values of one or more spectral imager performance attributes.

The one or more spectral imager performance attributes may include a signal to noise ratio.

The one or more spectral imager performance attributes may include a resolution. The reconstruction matrix may equal (b ^t H + Y ^T Y ¹ H ^T , wherein H may be a matrix that represents spectral responses associated with respective states of the tunable filter, and G a Tikhonov matrix.

Alternatively, the reconstruction matrix may include 0 ^T H[(H ^T H + G ^T G) ^_1 ] ^T wherein 0 may be a general matrix.

The image sensor may be a monochromatic image sensor.

The tunable filter may be a Fabri Perot etalon.

The tunable filter may be a single Fabri Perot etalon that operates, when acquiring the first plurality of broad-band spectral images in a wavelength range of 400 - 1000 nanometers.

The method further may include displaying one or more of the M narrow-band spectral images.

There may be provided a spectral imager that may include a tunable filter and a sensor; wherein the sensor may be configured to acquire a first plurality (N) of broad- band spectral images associated with respective states of the tunable filter; and

a processing circuitry that may be configured to process the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images.

Each broad-band image of the first plurality of broad-band spectral images may be associated with a respective broad-band transmission curve having a respective broad-band full width half maximum value; and wherein each narrow-band spectral image of the second plurality of narrow-band spectral images may be associated with a respective narrow-band transmission curve having a narrow-band full width half maximum smaller than the broad-band full width half maximum value of the broad band transmission curve.

The spectral imager may include an image sensor having a filter array with a third plurality (C) of filter types, wherein filters of different types differ from each other by transfer function; and wherein C exceeds two.

C may equal three and the filter array may be a color filter array.

C may equal three and the filter array may be a red, green and blue color filter array.

C may equal four and the filter array may be a red, green, blue and infrared filter array.

C may equal four and the filter array may be a red, green, blue and white filter array. C may equal three and the filter array may be a red, blue and white filter array

The processing circuitry may be configured to apply an expansion process for generating, from the first plurality of broad-band spectral images, a fourth plurality (Q) of broad-band spectral images and process the fourth plurality of broad-band spectral images into the second plurality of the narrow-band spectral images; and wherein Q exceeds each one of N and M.

The ratio between Q and N may be an integer.

The expansion process may be a demosaicing process.

The expansion process differs from a demosaicing process.

The processing circuitry may be configured to construct the second plurality of narrow-band spectral images by linearly transforming the fourth plurality of broad-band spectral images using a reconstruction matrix.

The processing circuitry may be configured to construct the second plurality of narrow-band spectral images by using a reconstruction matrix; wherein the reconstruction matrix represents a cost function and spectral responses associated with respective states of the tunable filter.

The reconstruction matrix may be calculated based on a Tikhonov matrix.

The Tikhonov matrix may be a multiple of the identity matrix.

The Tikhonov matrix may be a lowpass operator.

The Tikhonov matrix may be selected out of a group of Tikhonov matrices.

The group of Tikhonov matrices may be associated with a group of reconstruction matrices; wherein different reconstructions matrices of the group may be associated with different values of one or more spectral imager performance attributes.

The one or more spectral imager performance attributes may include a signal to noise ratio.

The one or more spectral imager performance attributes may include a resolution.

The reconstruction matrix may equal (b ^t H + Y ^T Y ¹ H ^T , wherein H may be a matrix that represents spectral responses associated with respective states of the tunable filter, and G a Tikhonov matrix.

Alternatively, the reconstruction matrix may include 0 ^T H[ H ^T H + G ^T G) ^_1 ] ^T wherein 0 may be a general matrix.

The Processing circuitry may be configured to perform a selection of the Tikhonov matrix out of a group of Tikhonov matrices. The selection may be responsive to at least one property of at least one of the first plurality of broad-band spectral images.

The at least one property may be a signal to noise ratio.

The at least one property may be a resolution.

The processing circuitry may be configured to calculate the reconstruction matrix.

The cost function may be a Tikhonov matrix.

The Tikhonov matrix may be a multiple of the identity matrix.

The Tikhonov matrix may be a lowpass operator.

The processing circuitry may be configured to select the Tikhonov matrix out of a group of Tikhonov matrices.

The group of Tikhonov matrices may be associated with a group of reconstruction matrices; wherein different reconstructions matrices of the group may be associated with different values of one or more spectral imager performance attributes.

The one or more spectral imager performance attributes may include a signal to noise ratio.

The one or more spectral imager performance attributes may include a resolution.

The reconstruction matrix may equal (b ^t H + Y ^T Y ¹ H ^T , wherein H may be a matrix that represents spectral responses associated with respective states of the tunable filter, and G a Tikhonov matrix.

Alternatively, the reconstruction matrix may include 0 ^T H[(H ^T H + G ^T G) ^_1 ] ^T wherein 0 may be a general matrix.

The image sensor may be a monochromatic image sensor.

The tunable filter may be a Fabri Perot etalon.

The tunable filter may be a single Fabri Perot etalon that operates, when acquiring the first plurality of broad-band spectral images in a wavelength range of 400 - 1000 nanometers.

There may be provided a method for generating narrow-band spectral images, the method may include : acquiring, by a spectral imager that may include a tunable light source, a first plurality (N) of broad-band spectral images associated with respective states of the tunable light source; and processing, by a processing circuitry the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images. There may be provided a method for generating narrow-band spectral images, the method may include : receiving by a processing circuitry, a first plurality (N) of broad band spectral images; wherein the N broad-band spectral images were acquired by a spectral imager that may include a tunable filter; wherein the N broad-band spectral images may be associated with respective states of the tunable filter; and processing, by the processing circuitry, the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images.

There may be provided a method for generating narrow-band spectral images, the method may include : receiving by a processing circuitry, a first plurality (N) of broad- band spectral images; wherein the N broad-band spectral images were acquired by a spectral imager that may include a tunable filter; wherein the N broad-band spectral images may be associated with respective states of the tunable filter; and processing, by the processing circuitry, the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images.

There may be provided a non-transitory computer readable medium that stores instructions that once executed by a processing circuitry causes the processing circuitry to receive a first plurality (N) of broad-band spectral images; wherein the N broad-band spectral images were acquired by a spectral imager that may include a tunable filter; wherein the N broad-band spectral images may be associated with respective states of the tunable filter; and process the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images.

There may be provided a non-transitory computer readable medium that stores instructions that once executed by a device that may include a spectral imager and a processing circuitry causes the device to acquire, by a spectral imager that may include a tunable filter, a first plurality (N) of broad-band spectral images associated with respective states of the tunable filter; and process, by the processing circuitry the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images. BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting examples of embodiments disclosed herein are described below with reference to figures attached hereto that are listed following this paragraph. The drawings and descriptions are meant to illuminate and clarify embodiments disclosed herein and should not be considered limiting in any way. Like elements in different drawings may be indicated by like numerals.

FIG. 1 illustrates schematically a known spectral imaging system based on operation of a FP etalon;

FIG. 2A shows one half of a broad-band system response obtained at different states of operation of a FP etalon based spectral imaging system disclosed herein;

FIG. 2B shows another half of a broad-band system response obtained at different states of operation of a FP etalon based spectral imaging system disclosed herein;

FIG. 3 shows schematically in a flow chart two general steps of a method disclosed herein;

FIG. 4A shows schematically in a flow chart more details of a method disclosed herein, for a FP etalon based spectral imager with a color image sensor;

FIG. 4B shows schematically in a flow chart more details of a method disclosed herein, for a FP etalon based spectral imager with a monochromatic image sensor;

FIG. 5A shows one third of a total system response obtained after applying a method disclosed herein to the broad-band system response of FIGS. 2A and 2B;

FIG. 5B shows another third of a total system response obtained after applying a method disclosed herein to the broad-band system response of FIGS. 2A and 2B;

FIG. 5C shows yet another third of a total system response obtained after applying a method disclosed herein to the broad-band system response of FIGS. 2A and 2B;

FIG. 6 illustrates an example of a method;

FIG. 7 illustrates an example of a method;

FIG. 8A illustrates an example of a system in which a tunable filter is placed between the object and the image sensor;

FIG. 8B illustrates an example of a system in which a tunable filter is placed in front of a light source which illuminates the object; and

FIG. 8C illustrates an example of a system in which a first tunable filter is placed in front of a light source which illuminates the object and a second tunable filter is placed between the object and the image sensor. DETAILED DESCRIPTION

Any reference to a spectral imager should be applied, mutatis mutandis to a method that is executed by a spectral imager and/or to a computer program product that stores instructions that once executed by the spectral imager will cause the spectral imager to execute the method.

Any reference to method should be applied, mutatis mutandis to a spectral imager that is configured to execute the method and/or to a computer program product that stores instructions that once executed by the spectral imager will cause the spectral imager to execute the method.

Any reference to a computer program product should be applied, mutatis mutandis to a method that is executed by a spectral imager and/or a spectral imager that is configured to execute the instructions stored in the non-transitory computer readable medium.

The term“and/or” is additionally or alternatively.

A narrow-band FP etalon can be designed (a) either using metallic single layer coatings, or (b) using multilayer dielectric coatings.

On one hand, since metallic coatings have a very high absorption, the transmission of a FP etalon having such filter coatings is practically too low (~5%). On the other hand, in order to get sufficiently narrow transmission bands with a multi-layer dielectric coatings design, the number of coating layers needs to be high (~22 layers), leading to a thick coating stack that causes very high residual stress on MEMS substrates and deforms them mechanically.

Thus, due to the constrains on the total permissible stress in the coatings, the number of such coating layers on a FP etalon has to be significantly reduced, inevitably causing the transmission bands to become much wider in comparison to those achievable when using the significantly thicker substrates commonly used in non- MEMS Fabri-Perot filters.

To qualify as "hyperspectral" , a tunable filter needs to provide very narrow- band transmission curves (e.g. with a full width half maximum (FWHM) of l-lOnm). In practice, very narrow-band FP etalon filters (and in particular MEMS based filters) are limited to work either (approximately) in the 400-600 nm wavelength range or (approximately) in the 600-900nm wavelength range, since their transmission curves cannot be made narrow over a wide wavelength range such as the 400-1000 nm range without introducing higher order“blobs” in the transmission curves.

There is therefore a need for, and it would be desirable to have a MEMS FP etalon-based spectral imagers that provide narrow-band filters/ transmission curves over a wide (e.g. 400-1000 nm) wavelength range.

In the following detailed description, the term "broad-band spectra" refers to spectra resembling that (for example - equal) of a spectrum of a single band pass filter with a FWHM equal to or greater than a predefined value - such as but not limited to 50 nm, or that of a multi-band pass filter) and the term "narrow-band spectra" refers to spectra resembling (for example- equals) that of a single band pass filter with a FWHM smaller than the predefined value - such as but not limited to 50 nm). When used, the term "narrower-band spectra" refers to spectra that is narrower than broad band spectra by a FWHM in the range of 0-lnm, l-lOnm, l0-50nm, 50-l00nm and even more than lOOnm.

Narrow-band transmission curves and even hyperspectral transmission curves may be obtained in an extended wavelength range (e.g. 400-1000 nm) using a spectral imager such as the spectral imager of FIG. 1 and by converting broad-band spectral images to narrow-band spectral images.

Broad-band spectral images can be obtained using a FP etalon that provides only broad-band transmission spectra (for example, spectra with FWHM greater than about 75 -100 nm). These broad-band spectral images can be converted using methods and processes disclosed herein into narrow-band spectral images and even hyperspectral spectral images. The suggested methods, spectral imagers and computer program products provide a cheap, highly effective (low absorbance) and high resolution spectral imager.

A FP etalon can be designed such that each of its states produces a transmission curve with a form that resembles that of a multi-band transmission filter which partially overlaps the transmission curves of the filter’s neighboring states. Each state of the FP etalon provides a broad-band spectrum with a given shape, from which one can obtain a "broad-band spectral image" that includes a number of mixed wavelengths. It is disclosed herein that the broad-band spectral image can then be transformed into one or more narrower-band spectral (and even hyperspectral) images. The transformation may include reconstruction of an imaged object's radiosity (also referred to as "flux density" and "irradiance" and expressed in Watts/m2) as if obtained with a series of narrow-band filters, by manipulating the image data obtained through the multi-band transmission curves.

In an example there is provided a method, that may include providing a spectral imager that includes a FP etalon, acquiring a first plurality (N) of broad-band spectral images associated with respective FP etalon states, a processing of the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images. N and M are positive integers.

In an example, each broad-band image of the first plurality is associated with a respective broad-band transmission curve having a respective broad-band full width half maximum value FWHM _BB and each narrow-band spectral image of the second plurality is associated with a respective narrow-band transmission curve having a narrow-band full width half maximum FWHMNB smaller than FWHMBB.

In an example, the spectral imager includes a color image sensor having a color filter array (CFA) with a third plurality (C) of filter types.

In an example, C = 3, the first plurality includes N broad-band spectral images w, and the method may include applying an expansion process (such as but not limited to demosaicing) to the ^ _N images to obtain A fourth plurality (Q) of broad- band spectral images (for example 3N broad-band spectral images) I _3/v and processing the _3/v broad-band spectral images into the second plurality (M) M < 3 N narrow-band spectral images R _{x M} .

In an example, C = 4, the first plurality includes N broad-band spectral images w, and the method further comprises applying demosaicing to the I _{1 N} images to obtain 4N broad-band spectral images I _{1 4/v} and processing the I _{1 4/v} broad-band spectral images into M < 4 N narrow-band spectral images R _{4 M} .

Yet in another example, the spectral imager includes a monochromatic image sensor without a CFA, the first plurality of broad-band spectral images includes N broad-band spectral images I _{1 w ,} and the method further comprises processing the ... N broad-band spectral images into a M < N narrow-band spectral images R _{4 M} .

Figure 1 illustrates an example of a spectral imager that is based on the operation of a tunable filter such as the FP etalon. Figure 1 illustrates a spectral imager 100 that includes an optical unit 102 and image sensor 104 and a control system 105.

Control system 105 includes a control unit 106 configured for data communication with a readout circuit of image sensor 104 for receiving image data therefrom and processing the received data.

Optical unit 102 includes a tunable filter such as a tunable dispersive unit/element that functions as a wide spectral filter 108.

In an example, the tunable dispersive unit/element is a FP etalon with a wide transmission peak. Control unit 106 includes data input and output utilities (not shown), a memory module 106A and an analyzer module 106B adapted for analyzing the image data from the pixel matrix unit 104.

Control system 105 further includes a controller 107, which is configured to control the tuning of FP etalon 108 and to provide data about the variation of a tunable parameter. The controller may be part of control unit 106, a separate module, or part of etalon 108.

The tuning of the FP etalon is aimed at controllably varying its spectral transmission profile (transmission function), i.e. at changing the dispersive pattern of light passing therethrough. In case of a FP etalon, the tunable parameter of FP etalon 108 is a gap between its reflective surfaces. Different gaps correspond to different states.

Controller 107 operates the tuning procedure and provides data about the different values of the tunable parameter, e.g. gapi, .. .gapN, or provides data about the corresponding transmission functions of the etalon, Ti, .. . TN. FP etalon 108 is exemplarily located along a common optical axis in front of an imaging lens module, while image sensor 104 is located at an imaging plane that coincides with a back focal plane.

A spectral imaging system such as system 100 can provide N different broad band images which are taken at different FP etalon states (operation modes).

System 100 can include or be otherwise operatively connected to a processing device (including for example a hardware processor such as a CPU, an FPGA, an ASIC, an image processor, and the like) configured to execute the operations related to generating narrow band images from broad band images as disclosed herein. According to one example, the processing device is part of control unit 106. According to another example, the processing device is part of etalon 108. According to yet another example, the processing unit is part of some other computerized device externally connected to system 100

FIGS. 2A and 2B show two halves of the broad-band system response obtained at different states of operation of the FP etalon and using a color image sensor (e.g. with a Bayer RGB CFA).

The Bayer RGB CFA includes red pixels, blue pixels and green pixels. A red pixel includes (or is preceded by) by a red filter, a green pixel includes (or is preceded by) a green filter, and a blue pixel includes (or is preceded by) a blue filter.

Assuming that the FB etalon has N states then a red pixel is associated with N different transfer functions that are associated with (a) the transfer function of the red filter and (b) the N different states of the FB etalon.

Assuming that the FB etalon has N states then a green pixel is associated with N different transfer functions that are associated with (a) the transfer function of the green filter and (b) the N different states of the FB etalon.

Assuming that the FB etalon has N states then a blue pixel is associated with N different transfer functions that are associated with (a) the transfer function of the blue filter and (b) the N different states of the FB etalon.

The spectral response is also represented by H, see Eq. (4) below. The example illustrated in these figures shows 60 spectral bands obtained with N=20 etalon states. Each etalon state may have a different spectral response, marked as Rl, R2, ... R20, Gl, G2, ... G20 and Bl, B2, ... and B20.

The broad-band and asymmetrical system response (spectral transmission) contrasts with the response of an ideal hyperspectral filter that provides a narrow spectrum (e.g. with FWF1M < 50nm) and substantially symmetrical band spectral response.

As mentioned, the present inventors have determined that the broad-band asymmetric transmission curves of FIGS. 2 A and 2B may be converted into relatively narrow-band transmission curves (referred to hereinafter as "hyperspectral transmission curves" or "hyperspectral bands") in an extended wavelength range (e.g. 400-1000 nm.

When the broad-band transmission curves are obtained with a color image sensor, then N broad-band spectral images obtained in respective N Fabri Perot etalon states of the spectral imager may be processed to provide more than N intermediate broad-band images.

A non-limiting example of an expansion process that is used to generate more than N intermediate broad-band images is the demosaicing process. This is merely a non-limiting example and other expansion processes that differ from the demosaicing process may be used to generate more than than N intermediate broad-band images. The expansion process may use any extrapolation, interpolation, estimation or evaluation operations.

For simplicity of explanation the following examples will refer an expansion process that is a demosaicing process.

The demosaicing algorithm may be used to reconstruct all color channels in full resolution. The methods disclosed herein are applicable to all known CFAs. For example, for an RGB Bayer CFA, a demosaicing algorithm may be applied as follows: for each pixel of an image 7 _fe with indices [m, n] , reconstruct the red pixel, the green pixel and the blue pixels of corresponding red image, green image and blue image as a function of the neighboring pixels values (up to 2 pixels away in each direction in the acquired image).

In the following example of a demosaicing process applies different functions (fR for the red image, fG for the green image, and fB for the blue image) of sets of five by five adjacent pixels to generate the different images.

Pixels of the red image are denoted / _3fc-2 [ ^m , ⁿ ]·

Pixels of the green image are denoted / _3fe _i [m, n].

Pixels of the blue image are denoted / _3fe [m, n]

In this case, the total number of images triples and is equal to N = 3 N. In general, for CFAs having more than three distinct colors, e.g. RGB-IR or RGBW CFAs, the total number of images after demosaicing is equal to N = cN

This is true for CFA having any number of c distinct colors, including two or three. It should be noted that the CFA may be sensitive to two or more frequency ranges - and that the frequency ranges may be visible color frequency ranges and/or may include at least one non-visible frequency range (for example- infrared, near IR and the like.)· For a monochromatic (without CFA) sensor, I _k = I _k and the total number of images N is preserved.

A "spectral cube" is obtained, including set of images defined by two spatial coordinates and a third coordinate determined by the state of the FP etalon and the color channel. The goal is to transform the spectral cube into a hyperspectral cube such that the third coordinate correlates with a wavelength of a narrow-band filter.

In the general case (e.g. monochromatic sensors or sensors with any CFA), for each state k of the FP etalon, the corresponding system signal, I _k , is given by: i _k [m, n] = H _k R + _k [m, n] (2) where R (size M X 1) is the discretization of the radiosity (i.e. the influx of photons on the image sensor) of the object, ri _k is the“noise” at each gap and H _k (size 1 X M) is the discretization of the transmission spectrum at each gap of the etalon given by: tfix _M (k) = (Tigap _k . · ·· T(gap _k , X _M )) (3) where T gap _k , X ₁ ) is the transmission of the equivalent filter at gap k, in vicinity of wavelength l ₁ . Matrix H, which is the total response of the system, is given by:

In H _NxM , N ³ M, i. e. the number of etalon states is equal to or larger than the number of discrete wavelengths. It is assumed that the noise ή is only a weak function of R, i.e. ή = ή(l,). It should be noted that other assumptions about the noise may be used.

In the case of a color sensor, the previously described demosaicing procedure (given by eq. (1) and shall be defined henceforth by D{I }) (or any other process that may be used to generate many broad-band images) is applied to the system signal:

where H _3NxM = D{H _NxM } is the result of applying the demosaicing function to the total response function of the system, H, and is now a function of both gap and color channel (e.g. of Rl, R2.., Gl, G2,.., Bl, B2.. in Fig. 2A and 2B). For CFAs having c > 1 distinct colors:

I _C NX i ⁼ D{I _Nx1 ] = D{H _NXM R _Mx1 + ^ή _Nc1 ] = H _C NXM ^R MX _l + ⁿ cNxi (5b)

For the monochromatic sensor case, the symbol H used henceforth is simply H _NxM = HNxM-

The object radiosity R is estimated exemplarily by a reconstruction matrix H _inv such that:

^R = H _inv I (6)

When reconstructing the object radiosity R, it should be remembered that for most cases the noise has a Gaussian statistical distribution, thus the maximum likelihood estimator R for R coincides with a least square estimator which minimizes ||/ - HR II over R.

According to some non-limiting examples a cost function can be used as follows:

||/— HR\\ + Regularization_term(a, R) (7)

In which the second term is some general regularization function with s being some tuning parameter that is potentially by itself a function of wavelength.

The regularization term could be constructed to impose some characteristics on the reconstructed signal, e.g. smoothness of the signal. It may be exemplarily derived based on the required spectral resolution and signal-to-noise ratio (SNR) per each wavelength l.

For example, using Tikhonov’s regularization method, the following cost function in which G is some suitably chosen Tikhonov matrix may be used for minimization: The solution to the problem of minimizing this cost function is then given by:

R = (H ^T H + G ⁷ G) ¹ Iί ⁷ 7 (9) and thus:

n ₌ (H ^T H ₊ r ^T rr ¹ H ^T (10)

Tikhonov’s matrix G could be chosen, for example, as some multiple a of the identity matrix (G = al), giving preference to solutions with smaller norms, which is known as the L2 type regularization. Alternatively, since the object radiosity R is assumed to be continuous, G could be constructed as some lowpass operator (e.g. a difference operator or a weighted Fourier operator) which enforces smoothness on the reconstructed R.

For any choice of G, the system performance may then be assessed in terms of the resulting resolution and the SNR. By assigning different weights to these parameters, an optimal G may be constructed.

For example, by sweeping through values of a in a certain predetermined range, it is possible to create maps of the system’s resolution and SNR vs. wavelength. The system’s performance at each reconstructed wavelength as a function of a is assessed by a weighted sum of these maps. The optimization is then performed by choosing, for each wavelength, a different value of a; which minimizes the weighted sum at that wavelength. In this case a is in fact a diagonal matrix a = diag(a.i ) multiplying the Tikhonov operator matrix G such that: G = at.

According to another non-limiting example, when choosing a reconstruction matrix H _inv , goal can be set, for example, to minimize the magnitude of the following cost function:

The first term (||/ί _ίti17 HR— R\\ _{2 )} on the right-hand side of this expression (data term) accounts for the deviations of the reconstruction. Minimizing this term is equivalent to minimizing its square, which is often more convenient to use for calculations. It is beneficial for the total system response H _inv H to resemble an ideal narrow-band (e.g. hyperspectral) filter, independently of the object radiosity. By using compatibility of induced norms, the following could be said about the data term in eq. (11):

\\R - ft _inv HR\\ ² ₂ £ \\ldentity _MxM - H _inv H\\ ₂ \\R\\ ₂ (12) Where Identity _MxM is the identity matrix of size M x M.

With a possible goal of limiting noise amplification, e.g. < s, the following cost function could be used in this exemplary method:

\\ldentity _MxM H _inv H \\ ² + Regularization_term(a, H _inv ) (13)

As previously, here s is some tuning parameter, which may be constructed as a function of wavelength. This general regularization term could be constructed to impose some characteristics on the reconstructed signal, e.g., noise amplification, or smoothness of the signal. The regularization term may be exemplarily derived based on the required spectral resolution and signal-to-noise ratio (SNR) per each wavelength l.

Since using an identity matrix poses a very strict requirement on the minimization, a general matrix 0 _MxM might be used instead and the general cost function is written as:

In this minimization problem, the FWHM of the rows of matrix O in fact represents the reconstruction spectral resolution.

As exemplified before, using Tikhonov’s regularization method, the following cost function in which G is some suitably chosen Tikhonov matrix may be used for this minimization:

Here, the fact that an L2 norm of a matrix is equivalent to the norm of its transpose was used. The solution to the problem of minimizing this cost function is then given by:

H _ίhn = O ^t H H ^t H + T ^t T) ^~1 ] ^t (16)

In addition to the examples mentioned above, various regression methods may be implemented to estimate the spectral bands of the image, both linear and nonlinear. Furthermore, various methods other than regression such as machine learning, neural network or artificial intelligence methods may be implemented to estimate the object radiosity R.

The mentioned algorithms could be tuned for any specific application requirements. It is noted that the required spectral resolution of reconstruction can be tunable - and accordingly the number of broadband images, i.e. for different applications data would be pre-processed and processed differently. For example, different optimal reconstruction matrices (Hinv) could be used for different SNR requirements.

FIG. 8A shows an example of a system which includes a sensor and a tunable filter between an object and the image sensor. The system’s signal is I _k [m, n\ = H _k R + n _k [m, n] in which /?(L;) is the radiosity of the object’s surface which can be estimated, for example, by the methods described thus far.

The system can be enhanced, for example with a tunable light source, e.g led array or a white source with tunable filter. Such tunable light source may improve the results of the spectrum estimation, by refining the collected data and enhance the estimation.

FIG. 8B shows an example of a system that includes a sensor and a tunable light source - without filter in front of the sensor. The tunable light source can either comprise an array of LEDs (or other types of light sources), each illuminating at a different wavelength band, or a broad-band light source with a tunable filter in front of it (as exemplified in the figure). In either case, the methods for reconstructing the object’s radiosity are practically identical to the methods discussed thus far. Furthermore, this still enables to use the same methods to get narrower bands relative to the original light source bands.

Assuming that the emissivity of the surface of an object is e(l ), then the radiosity of the object’s surface is H _k Es in which H _k relates to the tunable filter’s response, E(l ) is the pre-filtered light source intensity, and H _k E is the irradiance on the object’s surface. In case of an array of LEDs (or other types of light sources) it is still possible to write irradiance on the object’s surface as H _k E. Following the exemplary methods described thus far, the object’s spectral response can be estimated from the system’s signal I _k [m, n] = H _k R + n _k [m, n ] in which R is defined as R = Ee.

FIG. 8C shows another example of a system which is a combination of the examples in FIGS. 8A-B and includes a tunable light source (as described above) with an additional tunable filter in front of the image sensor. In this example, R = Ee and the total response of the filters is H _k = H ₂ ,k2Hi,ki i ⁿ which H _{l kl} is the response of the filter in front of the light source set at gap k ₁ and _2,f c2 > ^s the response of the filter in front of the image sensor set at gap k ₂

In this case, it is possible to obtain narrow band images solely by alternating the filter and the light source spectra. In this method, the light that reaches the sensor will include only the common wavelengths transmittable by both filters at their set gaps.

In a more concise manner, FIG. 3 shows schematically in a flow chart the two general steps of a method disclosed herein: in step 300, a series of broad-band spectral images of an object is acquired by operating the tunable FP etalon in a series of sequential states. In step 302, the broad-band spectral images are processed into narrower band images.

Step 302 may be followed by a step of -using the narrow band images e.g. displaying one or more of the images (for example - one image at a time) on a computer display device, storing one or more of the images, transmitting one or more of the images, and the like - or some other usage.

FIG. 4A shows schematically in a flow chart more details of a method.

In step 400, N broad-band spectral images ί _{1 N} are acquired. In step 402, a demosaicing algorithm is applied on the N broad-band spectral images to reconstruct all color channels in full resolution, thereby obtaining cN broad-band spectral images (e.g. 3N when C = 3 for a Bayer CFA) or 4N broad-band spectral images (where C = 4 when using RGB-IR or RGBW CFAs).

In step 404, a series of narrow-band spectral images, R _{1 M} , is constructed by

M - f'A _ linearly transforming ^ # using the matrix H _inv : \ · I = H _ίhn I : I , N = cN, M £

cN.

FIG. 4B shows schematically in a flow chart more details of a method according to an example disclosed herein, for a FP etalon based spectral imager with a monochromatic image sensor. In step 410, N broad-band spectral images _w are acquired. In step 412, a series of narrow-band spectral images, R _{1 M} , is constructed by

FIGS. 5A-5C show a "modified" total system response H _inv H in three separate graphs. The "modified" term refers to the fact that the total system response is changed from the response shown in FIGS. 2 A and 2B as a result of the processing applied in the methods described above. Each graph provides a number of narrow band HHi curves (spectra).

In summary, the present inventors have managed to perform narrow-band spectral imaging and even hyperspectral imaging using a FP etalon-based spectral imager that was not known to be able to provide such spectral or hyperspectral imaging.

Figure 6 illustrates an example of a method 600, according to the presently disclosed subject matter. Some operations described below with reference to Fig. 6 (as well as Fig. 7 and Fig. 3, 4a and 4b described above) can be performed by a processing circuitry configured for this purpose. As mentioned above such processing circuitry can be part of system 100 (e.g. as part of control unit 106 and/or etalon 108) or according to other examples be part of a computerized device externally connectable to system 100.

Method 600 may involve generating narrow-band spectral images.

Method 600 may include steps 610 and 620.

Step 610 may include acquiring, by a spectral imager that comprises a tunable filter, a first plurality (N) of broad-band spectral images associated with respective states of the tunable filter. N is an integer that exceeds one. The tunable filter may be a Fabri-Perot tunable filter such as a MEMS Fabri-Perot etalon (as shown above by way of example with reference to Fig. 1). The respective states of the tunable filter may include N different states. The respective states differ from each other by their transfer function.

Step 610 may be followed by step 620 of processing, by a processing circuitry (such as a hardware processor)), the first plurality of broad-band spectral images into a second plurality (M) of narrow-band spectral images. M is an integer that exceeds one. The processing may include executing instructions and/or code by the processing circuitry .

The processing circuitry may be a central processing unit, a graphic processor, a hardware accelerator, an FPGA, an ASIC, and the like.

The spectral imager may include an image sensor that may be a monochromatic sensor or a color filter that has a third plurality (C) of filter types filters of different types differ from each other by transfer function. C is an integer than exceeds two. The image sensor may include any combination of any number of filter types - for example - (a) red, green and blue, (b) red, green, blue and white, (c) red, green, blue and infrared, (d) red, blue and white, and the like.

Step 620 may include step 622 of applying an expansion process for generating, from the first plurality of broad-band spectral images, a fourth plurality (Q) of broad band spectral images. Q is a positive integer. Q exceeds N. Q exceeds M. The ratio between Q and N may be an integer or not.

The expansion process increases the amount of information. The expansion process may be a demosaicing process but may differ from a demosaicing process.

Step 622 may be followed by step 624 of processing the fourth plurality of broad-band spectral images into the second plurality of the narrow-band spectral images.

Step 624 may include constructing the second plurality of narrow-band spectral images by linearly transforming the fourth plurality of broad-band spectral images using a reconstruction matrix.

Examples of calculations that are executed during step 624 are illustrated in the pages above- especially equations (1)- (10) and the associated text.

Step 624 may include constructing the second plurality of narrow-band spectral images by using a reconstruction matrix; wherein the reconstruction matrix represents a cost function and spectral responses associated with respective states of the tunable filter. The reconstruction matrix may be calculated based on a Tikhonov matrix. The Tikhonov matrix may be a multiple (alpha) of the identity matrix.

The Tikhonov matrix may be selected out of a group of Tikhonov matrices. See, for example, the selection of alpha.

The group of Tikhonov matrices may be associated with a group of reconstruction matrices. Different reconstruction matrices of the group may be associated with different values of one or more spectral imager performance attributes. The one or more spectral imager performance attributes may be signal to noise ratio and/or resolution.

Step 624 may include calculating the reconstruction matrix.

Step 624 may include selecting the Tikhonov matrix out of a group of Tikhonov matrices. According to one example, the calculation or selection of the reconstruction matrices (e.g. Tikhonov matrices) may be executed before generating the fourth plurality of broad-band spectral images. According to another example, the calculation or selection may be executed after generating the fourth plurality of broad-band images but before processing the fourth plurality of broad-band spectral images into the second plurality of the narrow-band spectral images. According to yet another example, the calculation or selection may be executed during the processing of the fourth plurality of broad-band spectral images into the second plurality of the narrow-band spectral images.

The selection may be based on at least one property of at least one of the first plurality of broad-band spectral images.

Accordingly - after one or more broad-band spectral images are acquired during step 610 - the method may evaluate one or more property and then decide which Tikhonov matrix to select. For example- assuming that a first broad-band spectral image is acquired and has a low signal to noise ratio then a Tikhonov matrix that is associated with low signal to noise ratio should be selected. In this example there may be a need to increase the signal to noise ratio of one or more narrow-band spectral images generated during step 620 - and more emphasis may be assigned to the signal to noise ratio.

Figure 7 illustrates an example of method 700.

Method 700 may involve generating narrow-band spectral images.

Method 700 may include steps 610 and 621.

Step 610 may include acquiring, by a spectral imager that comprises a tunable filter, a first plurality (N) of broad-band spectral images associated with respective states of the tunable filter. N is an integer that exceeds one. The tunable filter may be a Fabri-Perot tunable filter such as a MEMS Fabri-Perot etalon. The respective states of the tunable filter may include N different states. The respective states differ from each other by their transfer function.

Step 610 may be followed by step 621 of processing, by a processing circuitry , the first plurality of broad-band spectral images into a second plurality (M) of narrow- band spectral images. M is an integer that exceeds one.

Step 621 differs from step 620 by including step 626 instead of steps 622 and 624. Notably, step 621 does not include performing the expansion process. Step 626 may include constructing the second plurality of narrow-band spectral images by linearly transforming the first plurality of broad-band spectral images using a reconstruction matrix.

Step 626 may include constructing the second plurality of narrow-band spectral images by using a reconstruction matrix; wherein the reconstruction matrix represents a cost function and spectral responses associated with respective states of the tunable filter. The reconstruction matrix may be calculated based on a Tikhonov matrix. The Tikhonov matrix may be a multiple (alpha) of the identity matrix.

The Tikhonov matrix may be selected out of a group of Tikhonov matrices. See, for example, the selection of alpha described above.

The group of Tikhonov matrices may be associated with a group of reconstruction matrices. Different reconstructions matrices of the group may be associated with different values of one or more spectral imager performance attributes. The one or more spectral imager performance attributes may be signal to noise ratio and/or resolution.

Step 626 may include calculating the reconstruction matrix.

Step 626 may include selecting the Tikhonov matrix out of a group of Tikhonov matrices.

The selecting may be executed during the processing of the first plurality of broad-band spectral images into the second plurality of the narrow-band spectral images.

The selection may be based on at least one property of at least one of the first plurality of broad-band spectral images.

The various features and steps discussed above, as well as other known equivalents for each such feature or step, can be mixed and matched by one of ordinary skill in this art to perform methods in accordance with principles described herein. Although the disclosure has been provided in the context of certain embodiments and examples, it will be understood by those skilled in the art that the disclosure extends beyond the specifically described embodiments to other alternative embodiments and/or uses and obvious modifications and equivalents thereof. Accordingly, the disclosure is not intended to be limited by the specific disclosures of embodiments herein. For example, any digital computer system can be configured or otherwise programmed to implement a method disclosed herein, and to the extent that a particular digital computer system is configured to implement such a method, it is within the scope and spirit of the disclosure. Once a digital computer system is programmed to perform particular functions pursuant to computer-executable instructions from program software that implements a method disclosed herein, it in effect becomes a special purpose computer particular to an embodiment of the method disclosed herein. The techniques necessary to achieve this are well known to those skilled in the art and thus are not further described herein. The methods and/or processes disclosed herein may be implemented as a computer program product such as, for example, a computer program tangibly embodied in an information carrier, for example, in a non-transitory computer- readable or non-transitory machine -readable storage device and/or in a propagated signal, for execution by or to control the operation of, a data processing apparatus including, for example, one or more programmable processors and/or one or more computers. The term "non-transitory" is used to exclude transitory, propagating signals, but to otherwise include any volatile or non-volatile computer memory technology suitable to the application including, for example, distribution media, intermediate storage media, execution memory of a computer, and any other medium or device capable of storing for later reading by a computer program implementing embodiments of a method disclosed herein. A computer program product can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

Unless otherwise stated, the use of the expression“and/or” between the last two members of a list of options for selection indicates that a selection of one or more of the listed options is appropriate and may be made.

It should be understood that where the claims or specification refer to "a" or "an" element, such reference is not to be construed as there being only one of that element.

It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments or example, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination or as suitable in any other described embodiment of the invention. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments, unless the embodiment is inoperative without those elements.

All patents and patent applications mentioned in this application are hereby incorporated by reference in their entirety for all purposes set forth herein. It is emphasized that citation or identification of any reference in this application shall not be construed as an admission that such a reference is available or admitted as prior art.

The terms "including", "comprising", "having", "consisting" and "consisting essentially of" are used in an interchangeable manner. For example- any method may include at least the steps included in the figures and/or in the specification, only the steps included in the figures and/or the specification. The same applies to the spectral imager and the mobile computer. It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.

In the foregoing specification, the invention has been described with reference to specific examples of embodiments of the invention. It will, however, be evident that various modifications and changes may be made therein without departing from the broader spirit and scope of the invention as set forth in the appended claims.

Moreover, the terms "front, " "back, " "top, " "bottom, " "over, " "under " and the like in the description and in the claims, if any, are used for descriptive purposes and not necessarily for describing permanent relative positions. It is understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in other orientations than those illustrated or otherwise described herein.

Those skilled in the art will recognize that the boundaries between logic blocks are merely illustrative and that alternative embodiments may merge logic blocks or circuit elements or impose an alternate decomposition of functionality upon various logic blocks or circuit elements. Thus, it is to be understood that the architectures depicted herein are merely exemplary, and that in fact many other architectures can be implemented which achieve the same functionality.

Any arrangement of components to achieve the same functionality is effectively "associated” such that the desired functionality is achieved. Flence, any two components herein combined to achieve a particular functionality can be seen as "associated with” each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being "operably connected,” or "operably coupled,” to each other to achieve the desired functionality.

Furthermore, those skilled in the art will recognize that boundaries between the above described operations merely illustrative. The multiple operations may be combined into a single operation, a single operation may be distributed in additional operations and operations may be executed at least partially overlapping in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be altered in various other embodiments.

Also for example, in one embodiment, the illustrated examples may be implemented as circuitry located on a single integrated circuit or within a same device. Alternatively, the examples may be implemented as any number of separate integrated circuits or separate devices interconnected with each other in a suitable manner.

Also for example, the examples, or portions thereof, may implemented as soft or code representations of physical circuitry or of logical representations convertible into physical circuitry, such as in a hardware description language of any appropriate type.

Also, the invention is not limited to physical devices or units implemented in non-programmable hardware but can also be applied in programmable devices or units able to perform the desired device functions by operating in accordance with suitable program code, such as mainframes, minicomputers, servers, workstations, personal computers, notepads, personal digital assistants, electronic games, automotive and other embedded systems, cell phones and various other wireless devices, commonly denoted in this application as‘computer systems’.

However, other modifications, variations and alternatives are also possible. The specifications and drawings are, accordingly, to be regarded in an illustrative rather than in a restrictive sense.

In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word‘comprising’ does not exclude the presence of other elements or steps then those listed in a claim. Furthermore, the terms "a” or "an,” as used herein, are defined as one as or more than one. Also, the use of introductory phrases such as "at least one " and "one or more " in the claims should not be construed to imply that the introduction of another claim element by the indefinite articles "a " or "an " limits any particular claim containing such introduced claim element to inventions containing only one such element, even when the same claim includes the introductory phrases "one or more " or "at least one " and indefinite articles such as "a " or "an. " The same holds true for the use of definite articles. Unless stated otherwise, terms such as "first” and "second” are used to arbitrarily distinguish between the elements such terms describe. Thus, these terms are not necessarily intended to indicate temporal or other prioritization of such elements the mere fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage.

Any system, apparatus or device referred to this patent application includes at least one hardware component.

While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents will now occur to those of ordinary skill in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.

Any combination of any component of any component and/or unit of spectral imager that is illustrated in any of the figures and/or specification and/or the claims may be provided.

Any combination of any spectral imager illustrated in any of the figures and/or specification and/or the claims may be provided.

Any combination of any set of spectral imagers illustrated in any of the figures and/or specification and/or the claims may be provided.

Any combination of steps, operations and/or methods illustrated in any of the figures and/or specification and/or the claims may be provided.

Any combination of operations illustrated in any of the figures and/or specification and/or the claims may be provided.

Any combination of methods illustrated in any of the figures and/or specification and/or the claims may be provided.

While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. The disclosure is to be understood as not limited by the specific embodiments described herein, but only by the scope of the appended claims.