CROSS REFERENCE TO OTHER APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No.

62/397,876 (Attorney Docket No. CBIOP020+) entitled RGB CALIBRATION FOR A FABRY PEROT INTERFEROMETER filed September 21, 2016 which is incorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

[0002] Traditional methods for acquiring three-dimensional hyper-spectral measurements typically require intensive computational efforts and expensive optical components. However, recent advances in fabrication techniques have allowed the creation of very precise tunable Fabry-Perot etalons that can be used for making low-cost hyper-spectral measurements. These etalons have a transmission spectrum that exhibit peaks of transmission as a function of a settable gap between two reflective glass optical flats. By collecting images using a sensor that collects light that has passed through the Fabry-Perot etalon for a defined set of gaps, it is possible to reconstruct the full three-dimensional (χ,γ,λ) hyper-spectral data cube of what is being imaged by the camera. However, because the etalon typically transmits multiple narrow peaks in the spectral range of interest and the camera pixel sensitivity spectra are broad and fixed, there is not a simple one-to-one mapping of pixel measurements to source spectra for a given gap, making it difficult or impossible to directly interpret the spectral content of raw image measurements.

BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.

[0004] Figure 1 is a block diagram illustrating an embodiment of a calibrated spectral measurement system.

[0005] Figure 2 is a flow diagram illustrating an embodiment of a process for reading a calibrated spectral measurement.

[0006] Figure 3 is a block diagram illustrating an embodiment of a system to calibrate a spectral measurement system of Figure 1. [0007] Figure 4 is a flow diagram illustrating an embodiment of a process for calibrating a spectral measurement system of Figure 1.

DETAILED DESCRIPTION

[0008] The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term 'processor' refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.

[0009] A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention

encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

[0010] A system for determining a calibrated spectral measurement is disclosed. The system includes a tunable Fabry-Perot etalon, a detector, and a processor. The tunable Fabry-Perot etalon has a settable gap. The detector measures the light intensity transmitted through the tunable Fabry-Perot etalon. The processor is configured to determine the calibrated spectral measurement. The calibrated spectral measurement is based at least in part on a set of detected light intensities for a plurality of settable gaps and a reconstruction matrix. The reconstruction matrix is based at least in part on calibration measurement using multiple source wavelengths and multiple settable gaps. [0011] A system for determining a calibrated spectral measurement comprises an illumination source, a Fabry-Perot etalon with a settable gap, an optical system filter, and an optical sensor array (e.g., a camera). The optical system filter sets the range over which the target spectrum is acquired and helps to constrain and thereby improve the spectral reconstruction associated with the target. To acquire a three-dimensional hyper-spectral cube measurement of the target, a controller sets the illumination source and acquires a set of images from the camera for a set of settings of the etalon gap. These images are indexed by the gap value and stored in a data scan memory. Once a complete set of images has been acquired (e.g., one for each of the set of settings of the etalon gap), the controller sends a READ signal to a spectrum calculator. The spectrum calculator uses a previously calculated calibrated reconstruction matrix and a set of gap images to estimate the target's optical spectrum. A matrix product, comprising the indexed gap images and the reconstruction matrix, is used to calculate an optical spectrum estimate.

[0012] A system for determining a calibrated reconstruction matrix comprises a settable structured illumination source, a calibrated spectrometer for measuring the illumination source spectrum, a Fabry-Perot etalon with a settable gap, an optical system filter, and an optical sensor array such as a camera. The optical system filter sets the range over which the target spectrum is acquired. A calibration controller sets the settable structured illumination source to one of a set of illumination structure types (e.g., a narrow band illumination defined by its peak frequency) and to one of a set of settable gaps of the Fabry-Perot etalon. For each gap and illumination type, the controller stores the measured illumination spectrum and an image from the camera in a calibration scan memory. Once all sets of illumination types and etalon gaps have been acquired in the calibration scan memory, the controller sends a calibration signal to the calibration processor to process the calibration scan data. The calibration processor calculates the reconstruction matrix using a linear model with regularization (e.g. Li _, L _2, or Lo regularization) and stores the resulting reconstruction matrix (e.g., in a reconstruction matrix memory).

[0013] In contrast to traditional Fabry-Perot etalon-based spectrometers that attempt to estimate the source spectrum from independent gap measurements, the disclosed system for hyper- spectral cube measurements is able to resolve ambiguities and eliminate spectral estimation artifacts by using information from the full set of gap measurements. In addition, the framework disclosed is able to work with a wide variety of illumination source types including narrow band, multiband, and broadband, as well as a wide variety of cameras such as a red-green-blue (RGB) camera, a red-green-blue and infrared (RGB+IR) camera, a monochrome camera, or a multispectral camera. [0014] Figure 1 is a block diagram illustrating an embodiment of a calibrated spectral measurement system. In the example shown, target 102 is illuminated with illumination source 100. The parameters specifying illumination source 100 are provided by a signal L. Light from illumination source 100 is scattered by target 102 with spectrum S*(X) and passes through Fabry- Perot etalon 104 with a settable gap specified by a signal δ. The system includes optics (e.g., a lens to collimate the light from the illumination source) to collimate light that enters the Fabry-Perot etalon. Light coming out of Fabry-Perot etalon 104 passes through an optical filter 106 (e.g., a broad band-pass filter) and is detected by optical sensor 108. Filter 106 may incorporate a fixed spectral bandpass filter (or a low pass and high pass combination which creates an effective bandpass filter). Reader controller 1 10 receives image data I[x,y,c] from optical sensor 108 for each setting of the Fabry-Perot gap δ and light source L and stores the image data I[x,y,c] for each gap setting δ in data scan memory 1 12. For example, the image data I contains c pixel types (e.g. red, green, blue) at locations x,y. After a complete set of data has been acquired for each gap setting δ, reader controller 1 10 sends a READ signal to a spectrum calculator instructing it to calculate the target spectrum estimate S[X\. Spectrum calculator 1 14 reads in the data from data scan memory 1 12 as matrix ϋ[δ,Ι] and reads in a reconstruction matrix R[0, ] from reconstruction matrix memory 1 16. Spectrum calculator 114 then multiplies matrix ϋ[δ,Ι] by matrix R[¾ ] to get the target spectrum estimate S[X\. In some embodiments, where there are multiple pixel values (x,y) processed by spectrum calculator 1 14, the spectral estimate S k] will be a three-dimensional S[x,y, ] hyper-spectral measurement.

[0015] In some embodiments, illumination source 100 is a broadband source such as a halogen lamp. In some embodiments, illumination source 100 is a composite source such an array of LEDs. In some embodiments, illumination source 100 is optimized for the detection of particular target by matching the illumination source spectrum to the target's reflection spectrum. In some embodiments, Fabry-Perot etalon 104 is a pair of cascaded Fabry-Perot etalons with settable gaps specified by a pair of signals [δι, δ ₂ ]. In contrast to a single Fabry-Perot etalon, which typically has multiple peaks of transmission in the range of interest, a pair of cascaded Fabry-Perot etalons with different gap settings can be set to pass a single narrow band of light in the range of interest. For example, if the gaps [δι, δ ₂ ] specify different transmission orders for the two etalons, a single narrow band of transmission is passed. Detection of such a single band of light from a pair of cascaded Fabry-Perot etalons by optical sensor 108 resolves the ambiguity associated with assigning spectral estimates to multiple bands when using a single Fabry-Perot etalon. [0016] In various embodiments, optical sensor 108 comprises a red-green-blue (RGB) camera, a red-green-blue and infrared (RGB+IR) camera, a monochrome camera, a multispectral camera, or any other appropriate sensor.

[0017] In some embodiments, the spectral estimate S[k] will be a spatially weighted average. For example, the spectral estimate S k] is an average over the central region of optical sensor 108. In some embodiments, the three-dimensional spectral estimate S[x,y, ] will be a spatially weighted average from a plurality of sensor regions.

[0018] In some embodiments, spectrum calculator 1 14 uses a priori knowledge of the target spectrum to estimate spectral features of the target. For example, if the target spectra S*( ) is known to have a finite number of peaks, spectrum calculator 1 14 may use a reconstruction matrix designed to map the peaks in the optical sensor measurement to peaks in the spectral estimate S[X\.

[0019] In some embodiments, reconstruction matrix memory 1 16 contains multiple reconstruction matrices optimized for various conditions of the device. For example, spectrum calculator 1 14 for various temperature conditions of the device may use a second reconstruction matrix.

[0020] In some embodiments, reconstruction matrix memory 1 16 contains reduced descriptions of the reconstruction matrix for faster spectral estimation and better noise immunity. For example, the reconstruction matrix is estimated using a small number of principle components. This estimated reconstruction matrix comprises a reduced reconstruction matrix (determined using the reduced description of the reconstruction matrix) that is used in place of the reconstruction matrix.

[0021] Figure 2 is a flow diagram illustrating an embodiment of a process for reading a calibrated spectral measurement. In some embodiments, the process of Figure 2 is executed by the system shown in Figure 1. In the example shown, in 200, a scan is initialized. For example, the spectral measurement system is initialized; the index i for the set of Fabry-Perot gap settings δ; is set to zero and the number of gap settings Ng is defined. In 202, the illumination source is changed to type L. For example, the illumination source is turned on and specified by parameters L. L may specify a narrow or broadband source. In 204, the gap for the Fabry-Perot etalon is set to δ;. The setting of the gap for the Fabry-Perot etalon results in spectrally filtered light, which is then imaged at an optical sensor. For example, δ; might set the Fabry-Perot etalon to a 500nm gap for i=0. In 206, an optical sensor image I(x,y,c) is captured. For example, an image I[x,y,c] is captured of the light scattered from a target that is illuminated using the light coming out of the Fabry-Perot etalon. In some embodiments, the optical sensor is a high definition RGB camera and the data image I has dimensions I[1920, 1080,3]. In 208, the data image I is stored in the data scan memory indexed by 5i. For example, each image I in the data scan memory is indexed by a specific gap setting δ;. In 210, the gap setting index i is incremented; for example, i is increased by 1. In 212, it is determined whether i is less than Ng. For example, the index i is compared to the maximum value Ng. In the event that i is less than Ng, control passes to 204. For example, in the event that the index i is less than the maximum Ng, then the scanning dataset is not yet complete and the next Fabry-Perot etalon gap is set in 204 and the data capturing process continues in 206 and 208. In the event that i is not less than Ng, then control passes to 214. For example, in the event that the index i is greater than or equal to the maximum Ng, the data capturing process loop ends. In 214, a READ signal is sent to the spectrum calculator to signal a full data scan is ready for processing. For example, the reader controller sends a signal READ to the spectrum calculator to signal that the data in the data scan memory is ready for processing. In 216, a target spectrum estimate S ] is calculated using data scan memory G[5,I] and reconstruction matrix memory R[5, ]. For example, the spectrum calculator calculates a hyperspectral data cube estimate S(x,y, ) of the target using the data G[5,I] stored in the data scan memory and the reconstruction matrix R[5, ] stored in the reconstruction matrix memory. In some embodiments, the hyperspectral data cube estimate S(x,y, ) can be calculated by multiplying the data scan matrix G[5,I] by the reconstruction matrix R[5, ] .

[0022] Figure 3 is a block diagram illustrating an embodiment of a system to calibrate a spectral measurement system of Figure 1. In the example shown, light from input source 300 is set to one of a plurality of spectral structure types specified by λ;. For example, λ; may specify a narrow band light source (e.g., a light source with half-width of 2 nm) with center wavelength equal to λ;. In some embodiments, the input source includes multi-peaked types or broadband spectra types where the types are specified by the parameter λ;. The light from input source 300 with spectra type λ; is passed to spectrometer 308 and Fabry-Perot etalon 302. Spectrometer 308 records the measured spectrum [ki] generated by input source 300 with spectral structure specified by λ _ί . Light from input source 300 with spectrum ki) passes through Fabry-Perot etalon 302 with a settable gap specified by a signal 5 _j . Light coming out of Fabry-Perot etalon 302 passes through optical filter 304 (e.g., a broad band-pass filter) and is detected by optical sensor 306. Calibration controller 310 receives image data I[x,y,c] from optical sensor 306 for each setting of the Fabry- Perot gap 5 _j and light source type λ _ί and stores the image data I[x,y,c] and measured spectrum S[V| for each gap setting 5 _j and input source type λ; in calibration scan memory 312. After a complete set of data F[S'( i),r(5J, i)] has been acquired for each gap setting 5 _j and each input source type λ;, calibration controller 310 sends a CAL signal to calibration processor 314 instructing it to calculate the reconstruction matrix R[5, λ] using calibration scan data memory 312. Calibration processor 314 then reads in the calibration data F[S'( ),F(5, )] from calibration memory 312 and computes a reconstruction matrix R using a linear model S = RI to estimate the input source spectrum S. For example, the optimal reconstruction matrix can be computed using a least squares cost function between the linear model S=RI and the measured spectrum together with Li regularization. In various embodiments, the optimal reconstruction matrix is computed using a least squares cost function between the linear model S=RI and the measured spectrum together with Li or L ₀ regularization. In various embodiments, least absolute shrinkage and selection operator (LASSO), stochastic gradient decent, or expectation-maximization methods are used for Li regularization, or any other appropriate method may be used to solve the linear model with a convex cost function. In various embodiments, orthogonal matching pursuit or locally competitive algorithms are used for L ₀ regularization or any other appropriate method may be used to solve the linear model.

Calibration processor 314 then stores the optimal reconstruction matrix R[ ] in reconstruction matrix memory 316. In some embodiments, the linear model has a constant term. For example, the linear model is S = RI + R0 where R0 is a constant term. In some embodiments, R is determined using a pseudo-inverse For example, a radiometer is used to measure the transmitted power at each wavelength λ _ί and gap setting 5 _j . For each gap setting 5 _j , a narrowband source λ _ί is scanned over the spectral range and the average red, green, and blue values from the sensor 306 at each wavelength setting λ _ί are scaled by the radiometer output and saved. The transmission spectrum for each gap 5 _j is placed into a row of a matrix that when inverted using the pseudo-inverse, results in the reconstruction matrix R.

[0023] In some embodiments, the spectrum of illumination source 300 comprises multiple narrow bands. For example, illumination source 300 may be generated from broadband source such as a halogen lamp that is passed through a second Fabry-Perot etalon. In some embodiments, illumination source 300 is a composite source such as an array of LEDs.

[0024] In various embodiments, optical sensor 306 comprises a red-green-blue (RGB) camera, a red-green-blue and infrared (RGB+IR) camera, a monochrome camera, a multispectral camera, or any other appropriate sensor.

[0025] In some embodiments, the image data I will be a spatially weighted average. For example, the image data is an average over the central region of optical sensor 306. In some embodiments, the image data I will be a spatially weighted average from a plurality of sensor regions. [0026] In some embodiments, calibration processor 312 uses a priori knowledge of the target spectrum to estimate spectral features of the target. For example, if the target spectra S*(X) is known to have a finite number of peaks, calibration processor 312 may calculate a reconstruction matrix designed to map the peaks in the optical sensor measurement to peaks in the spectral estimate S ].

[0027] In some embodiments, reconstruction matrix memory 316 contains multiple reconstruction matrices optimized for various conditions of the device. For example, calibration processor 312 may calculate a reconstruction matrix for various temperature conditions of the device.

[0028] In some embodiments, reconstruction matrix memory 316 contains reduced descriptions of the reconstruction matrix for faster spectral estimation and better noise immunity. For example, the reconstruction matrix is estimated using a small number of principle components. This estimated reconstruction matrix comprises a reduced reconstruction matrix (determined using the reduced description of the reconstruction matrix) that is used in place of the reconstruction matrix.

[0029] Figure 4 is a flow diagram illustrating an embodiment of a process for calibrating a spectral measurement system of Figure 1. In some embodiments, the process of Figure 4 is executed by the system shown in Figure 3. In the example shown, in 400, a calibration system is initialized. For example, the calibration process is initialized; the index j for the set of Fabry-Perot gap settings 5 _j and the index i for the set of input source spectral structure types λί is set to zero and the number of gap settings Ng and the number of source types Νλ is defined. In 402, the

illumination source type is set to λί. In 404, the spectrum S[ ] of input source 300 with source type λ _ί is measured using spectrometer 308. In 406, the spectrum measured in process 404 is stored in calibration scan memory 312 and indexed by the spectrum type λ _ί . In 408, the gap for the Fabry- Perot etalon is set to 5 _j . The setting of the gap for the Fabry-Perot etalon results in spectrally filtered light, which is then imaged at an optical sensor. For example, 5 _j might set the Fabry-Perot etalon to a 500nm gap for j=0. In 410, an optical sensor image I(x,y,c) is captured and stored in a calibration scan memory. For example, an image I[x,y,c] is captured of the light from input source type λ _ί that has passed through of the Fabry-Perot etalon. In some embodiments, the optical sensor is a high definition RGB camera and the data image I has dimensions I[1920,1080,3]. The data image I is stored in the calibration scan memory indexed by input source type λί and etalon gap type 5 _j . For example, each image I in the data scan memory is indexed by a specific gap setting 5 _j and specific source type λί. In 412, gap setting index j is incremented; for example, j is increased by 1. In 414, it is determined whether j is less than Ng. For example, the index j is compared to the maximum value Ng. In the event that j is less than Ng, control passes to 408. For example, in the event that the index j is less than the maximum Ng, then the scanning dataset is not yet complete and the next Fabry-Perot etalon gap is set in 408 and the data capturing process continues in 410. In the event that j is not less than Ng, then control passes to 416. For example, in the event that the index j is greater than or equal to the maximum Ng, the data capturing process loop ends for scanning over all gaps. In 416, the source type index i is incremented; for example, i is increased by 1. In 418, the gap setting index j is reset to zero in preparation for a scan over all Ng gaps for a new input source type λ;. In 420, it is determined whether i is less than Ν _λ . For example, the index i is compared to the maximum value Ν _λ . In the event that i is less than Ν _λ , control passes to 402. For example, in the event that the index i is less than the maximum Ν _λ , then the scanning dataset is not yet complete and the next input source type is selected and the data capturing process continues in 404, 406, 408, and 410. In the event that i is not less than N _¾ ,, then control passes to 422. For example, in the event that the index i is greater than or equal to the maximum Ν _λ , the data capturing process loop ends. In 422, a CAL signal is sent to the calibration processor to signal that the calibration data F[S'( i),I'(5 _j , i)] is ready for processing. In 424, optimal reconstruction matrix R is calculated for linear model S=RI using least squares and LI regularization. For example, calibration processor then reads in the calibration data F[S'( ),F(5, )] from the calibration memory and computes a reconstruction matrix R using a linear model S = RI to estimate the input source spectrum S. In 426, reconstruction matrix R is stored in reconstruction matrix memory. For example, the calibration processor stores the optimal reconstruction matrix R k] in the

reconstruction matrix memory.

[0030] Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.