INFORMATION

Cross-Reference to Related Applications

This application claims the benefit of U.S. Provisional Patent Application No. 62/343,314, filed on May 31, 2016, which is herein incorporated by reference in its entirety.

Background

[OOOlJImaging arrays for use in color photography typically use four photodiodes per pixel in the final image. The photodiodes are typically arranged in a 2x2 array with a red filter over one photodiode, a blue filter over a second photodiode and a green filter over the remaining two photodiodes. The color filters select a relatively broad range of wavelengths, and hence, the photodiodes measure an average of the light intensity over a broad range of wavelengths in the red, blue, and green wavelength bands. Hence, these photodiodes are relatively insensitive to wavelength variations within the acceptance band of the filters. In addition, the use of four photodiodes for each image pixel increases the cost of the imaging array over a monochrome array by a factor of approximately four. Finally, the filters reduce the light reaching the photodiodes, and hence, either lower the sensitivity of the array in low light conditions or require larger photodiodes to compensate for the loss in intensity.

Summary

[0002] The present invention includes an apparatus for forming a color image of a scene and a method for utilizing that apparatus. The apparatus includes a plurality of pixel sensors. Each of the pixel sensors includes a first photodetector that includes a first main photodiode and a first floating diffusion node. The first main photodiode is characterized by a first light conversion efficiency as a function of wavelength of a light signal incident thereon. The first floating diffusion node includes a parasitic photodiode characterized by a second light conversion efficiency as a function of the wavelength. The second light conversion efficiency is different from the first light conversion efficiency as a function of wavelength. The floating diffusion node also is illuminated by the light signal. The apparatus also includes a controller that generates an intensity of light in each of a plurality of wavelength bands for the pixel sensor utilizing a measurement of the light signal by each of the first main photodiode and the first parasitic photodiode in that photodetector.

[0003] In one aspect of the invention, the second light conversion efficiency is greater than three percent of the first light conversion efficiency. In another aspect, the second light conversion efficiency is greater than ten percent of the first light conversion efficiency.

[0004] In a still further aspect, the apparatus includes a plurality of second

photodetectors, each pixel sensor being associated with one of the plurality of second photodetectors, the second photodetectors having a second main photodiode with a third light conversion efficiency as a function of the wavelength, the third light conversion efficiency being different from the first and second light conversion efficiencies. The second photodiodes provide a third measurement of the light signal, the third measurement being used by the controller to generate the intensities of light in the plurality of wavelength bands.

[0005] In one aspect, there is one second photodetector for each pixel sensor.

[0006]In another aspect, there are fewer second photodetectors then pixel sensors, at least one of the second photodetectors being shared by a plurality of pixel sensors. In a still further aspect, the controller interpolates intensity measurements from a plurality of the second photodetectors to arrive at the third measurement.

[0007] In another aspect, the second photodiode is part of a second photodetector in each pixel sensor and the second photodetector further includes a second floating diffusion node that is separate from the first floating diffusion node, the second floating diffusion node includes a second parasitic photodiode, the processor using a signal from the second parasitic photodiode to generate the intensities in each of the plurality of wavelength bands.

[0008]The present invention also includes a method for generating a color image of a scene that includes a plurality of color pixels. The method includes projecting the image on an array of pixel sensors, each pixel sensor generating one color pixel of the color image. Each pixel sensor includes a first photodetector having a main photodiode and a floating diffusion node. The first main photodiode is characterized by a first light conversion efficiency as a function of wavelength of a light signal incident on the pixel sensor. The first floating diffusion node includes a parasitic photodiode characterized by a second light conversion efficiency as a function of the wavelength, the second light conversion efficiency being different from the first light conversion efficiency as a function of wavelength.

[0009] First and second light intensity values are generated for each pixel sensor from the first main photodiode and the first floating diffusion node in each of the pixel sensors. A third light intensity value corresponding to each pixel sensor is generated utilizing a corresponding one of a plurality of second photodetectors having a third light conversion efficiency as a function of wavelength that is different from first and second light conversion efficiencies. Each color pixel is generated utilizing the first, second, and third light intensity values corresponding to that pixel.

[0010] In one aspect of the invention, the second photodetector is part of each pixel sensor.

[0011]In another aspect, the plurality of second photodetectors includes one second photodetector that is shared by a plurality of the pixel sensors.

[0012]In another aspect, intensity values from two or more of said plurality of second photodetectors are interpolated to arrive at said third light intensity value for one of said pixel sensors.

[0013]In a further aspect, each second photodetector includes a floating diffusion node that includes a second parasitic photodiode, and a light intensity value from the second parasitic photodiode is used also used to generate the color pixel.

Brief Description of the Drawings

[0014] Figure 1 is a schematic drawing of a typical prior art photodetector in one column of photodetectors in an imaging array.

[0015]Figure 2 illustrates a photodetector in which the parasitic photodiode is utilized in an image measurement.

[0016] Figure 3 A illustrates a conventional imaging array.

[0017] Figure 3B illustrates which illustrates an imaging array using four

photodetector pixel sensors according to one embodiment of the present invention.

[0018] Figure 3C illustrates a pixel sensor having two photodetectors of different sizes.

[0019] Figure 4 illustrates the ratio of the two conversion efficiencies in one embodiment.

[0020] Figure 5 illustrates a pixel sensor having two main photodiodes that share a common floating diffusion node that has a parasitic photodiode response. Detailed Description

[0021] For the purposes of the present discussion, a digital camera converts an image of a scene into an array of pixels representing the scene. In the case of a color image, each pixel can be represented by a color vector having components in three or more color channels. Each color component typically represents the intensity of the light in a corresponding band of wavelengths. Typically, the color components represent the intensities in the red, blue, and green wavelength bands in the optical spectrum.

[0022]The imaging array that captures the light typically includes a number of photodetectors that are used to measure the light intensities that are used to generate the color vector. In the following discussion, the group of photodetectors that provides the information for a single pixel in the final image will be referred to as the pixel sensor for that pixel. In conventional color imaging detectors, there are four photodetectors in each pixel sensor. Each photodetector is covered by a filter that selects light in a particular band of wavelengths. In systems that use three color components, one photodetector is covered by a filter that passes red light and blocks other bands, one photodetector is covered by a filter that passes blue light, and two photodetectors are covered by filters that pass green light.

[0023]In conventional imaging arrays, each photodetector has a single photodiode that converts the incident light on that photodetector during an exposure period to a charge and circuitry for generating a voltage signal from that charge. The voltage signal is typically digitized to produce an intensity reading for the light incident on that photodetector. In one aspect of the present invention, each photodetector includes two photodiodes, while requiring only the same amount of silicon surface area of a single conventional photodetector having one photodiode. In addition, the two photodiodes of the photodetector have different light sensitivities as a function of wavelength of the incident light, and hence, can be used to provide color information without requiring wavelength filters that reduce the light intensity.

[0024] Figure 1 is a schematic drawing of a typical prior art photodetector in one column of photodetectors in an imaging array. Photodetector 21 includes a photodiode 22 that measures the light intensity at a corresponding pixel in the image. As noted above, in prior art pixel sensors, there are typically four such photodetectors corresponding to each pixel sensor. Initially, photodiode 22 is reset by placing gate 25 in a conducting state and connecting floating diffusion node 23 to a reset voltage, Vr. Gate 25 is then closed and photodiode 22 is allowed to accumulate photoelectrons during an exposure period. A potential on an optional anti-blooming circuit 27 sets the maximum amount of charge that can be accumulated on photodiode 22. If more charge is accumulated than allowed by anti- blooming circuit 27, the excess charge is removed from photodiode 22.

[0025] After photodiode 22 has been exposed, the charge accumulated in photodiode 22 is typically measured by noting the change in voltage on floating diffusion node 23 when the accumulated charge from photodiode 22 is transferred to floating diffusion node 23. For the purposes of the present discussion, a floating diffusion node is defined to be an electrical node that is not tied to a power rail, or driven by another circuit. In the photodetector shown in Figure 1, one source/drain side of the transfer gate transistor and the drain side of the photodetector reset transistor are tied together. When neither gate of these transistors is active, the connected source/drain diffusions are considered to be one floating diffusion node. Floating diffusion node 23 is characterized by a capacitance represented by capacitor 23'. In practice, capacitor 23' is charged to a voltage, Vr, and isolated by pulsing the reset line of gate 24 prior to floating diffusion node 23 being connected to photodiode 22. The charge accumulated on photodiode 22 is transferred to floating diffusion node 23 when gate 25 is opened. The voltage on floating diffusion node 23 is sufficient to remove all of this charge, leaving the voltage on floating diffusion node 23 reduced by an amount that depends on the amount of charge transferred and the capacitance of capacitor 23'. Hence, by measuring the change in voltage on floating diffusion node 23 after gate 25 is opened, the accumulated charge can be determined.

[0026] This scheme assumes that the reset potential on the floating diffusion node prior to transferring the charge from photodiode 22 is known. In practice, the actual reset voltage will vary from the voltage, Vr, due to noise. If this noise is significant, a correlated double sampling algorithm can be utilized to reduce the errors caused by the noise. In the correlated double sampling processing, the actual voltage on the floating diffuse node after the reset operation is performed is measured prior to connecting photodiode 22 to floating diffusion node 23. The difference between this measured reset voltage and the voltage after the charge has been transferred is used to calculate the charge generated during the exposure. The procedure starts by connecting floating diffusion node 23 to Vr using reset gate 24. The potential on floating diffusion node 23 is then measured by connecting source follower 26 to readout line 31 by applying a select signal to word line 28. This reset potential is stored in column amplifier 32. Next, gate 25 is placed in a conducting state and the charge

accumulated in photodiode 22 is transferred to floating diffusion node 23. It should be noted that floating diffusion node 23 is effectively a capacitor that has been charged to Vr. Hence, the charge leaving photodiode 22 lowers the voltage on floating diffusion node 23 by an amount that depends on the capacitance of floating diffusion node 23 and the amount of charge that is transferred. The voltage on floating diffusion node 23 is again measured after the transfer. The difference in voltage is then used to compute the amount of charge that accumulated during the exposure.

[0027] A "parasitic photodiode" is an inherent property of the floating diffusion node in many photodetector designs. A depletion region is formed by the floating diffusion node and the gates of other transistors in the photodetector. The parasitic photodiode collects photoelectrons from photons that are converted in the silicon under the floating diffusion node. In the prior art, care is taken to minimize the conversion efficiency of the parasitic photodiode. For example, in some designs, a mask is provided over the floating diffusion node to block light from striking the node to reduce the light conversion efficiency of the parasitic photodiode. In contrast, the present invention is based on the observation that a photodetector of the type discussed above can be modified to enhance the light conversion efficiency of the parasitic photodiode to provide a second photodiode that does not significantly increase the size of the photodetector, and hence, the present invention provides the advantages of a two- photodiode photodetector in which the two photodiodes have different color responses without significantly increasing the photodetector.

[0028]Refer now to Figure 2, which illustrates a photodetector in which the parasitic photodiode is utilized in an image measurement. To distinguish the parasitic photodiode 42, photodiode 22 will be referred to as the "main photodiode". To simplify the following discussion, those elements of photodetector 21 that serve functions analogous to those discussed above with respect to Figure 1 have been given the same numeric designations and will not be discussed further unless such discussion is necessary to illustrate a new manner in which those elements are utilized. In general, parasitic photodiode 42 that is part of floating diffusion node 43 has a detection efficiency that is significantly less than that of main photodiode 22. For the purposes of the present discussion, the detection efficiency of a photodiode is a function of the wavelength of the light that irradiates that photodiode and is defined to be the average number of photoelectrons generated per unit of energy of photons of a particular wavelength. In co-pending US Patent Application 14/591,873, filed on January 7, 2015, this difference in detection efficiency between the main and parasitic photodiodes is utilized to increase the dynamic range of the photodetector by adjusting the conversion efficiencies of the two photodiodes such that the parasitic photodiode provides an intensity measurement at incident light intensities that are too great to be measured in the main photodiode. For example, the parasitic photodiode could have a conversion efficiency that is three percent of the conversion efficiency of the main photodiode. In another example the parasitic photodiode has a conversion efficiency that is greater than ten percent of the conversion efficiency of the main photodiode. Other examples of the detection efficiency and methods for adjusting the relative conversion efficiencies of the main and parasitic photodiodes are discussed in this US Patent Application, which is incorporated by reference herein, and hence, will not be discussed in detail here.

[0029] The present system is based on the observation that the parasitic photodiode has a different color response than the main photodiode in the photodetector. That is, the ratio of the conversion efficiency of the parasitic photodiode to the conversion efficiency of the main photodiode is not a constant as a function of the wavelength of the incident light. In fact, as will be discussed in more detail below, this ratio can be used to provide information on the average wavelength of the light incident on the two photodiodes. This information can be used to refine the color measurement without the need to provide a color filter over the photodetectors. This information can also be used to reduce the number of photodetectors per pixel sensor in the imaging array. In one aspect of the present system, controller 52 includes a processor that performs the color computations that generate the intensities in the color channels utilizing the parasitic photodiode as one photodetector in the pixel sensor.

[0030] The number of photoelectrons generated by a photon interacting with the photodiode as a function of the photon wavelength depends on a number of factors. The main photodiode may have a significantly different depth profile than the parasitic photodiode, hence, the wavelength response of the two different photodiodes will be different. In addition, the main photodiode is typically a pinned photodiode and the parasitic photodiode is not. Other factors affecting the relative parasitic response include a difference in the spatial position of the parasitic photodiode relative to the micro-lens that is placed over the pixel detector, and the difference in electrical and optical crosstalk as a result of a number of geometric, layout, implant, and structural differences. In general, it is advantageous to adjust the overall conversion efficiency of the parasitic photodiode to be close to that of the main photodiode; however, in practice the conversion efficiency of the parasitic photodiode is less than that of the main photodiode. The ratio of the two conversion efficiencies in one embodiment of the present invention is shown in Figure 4.

[0031]Denote the signal from the main photodiode by S _m and the signal from the parasitic photodiode by S _P . Then,

Sm— IrCmr"tTgCmg"t"IbCmb (1)

Here, Cmr, Cmg, and Cmb are calibration constants for the main photodiode that are related to the conversion efficiency of the main photodiode in the red, green, and blue wave bands, respectively, and C _pr , C _pg , and C _P b are calibration constants for the parasitic photodiode that are related to the conversion efficiency of the parasitic photodiode in the red, green, and blue wave bands, respectively. Eqs. (1) provide two equations in three unknowns, i.e., the intensity of the incident light in the three wavelength bands. Hence, these equations cannot be solved to provide the desired intensities without some additional information in the general case.

[0032] Refer now to Figure 3 A which illustrates a conventional imaging array. Each pixel sensor 81 has four photodetectors. It is assumed that the light intensity does not vary significantly over the pixel, and hence, each photodetector receives the same incident light intensity. Typically, each photodetector has a filter that limits the light reaching the photodetector to light in a particular wavelength band. The filters are denoted by R, B, and G in the figure. Typically, two photodetectors are used to measure the intensity in the green wavelengths in the Bayer RGB color filter pattern scheme shown in the figure. As noted above, the filters remove a significant fraction of the light, and hence, the shot noise associated with each photodetector is increased relative to the shot noise that would be experienced without the filters. In essence, this prior art pixel design provides four intensity measurements from which to obtain the intensity of light in the three wavelength bands.

[0033]Refer now to Figure 3B, which illustrates an imaging array using four photodetector pixel sensors according to one embodiment of the present invention. Each pixel sensor 82 now includes one photodetector 83 that has a color filter, F, and three photodetectors 84 that are unfiltered. Since each photodetector now provides two intensity measurements having different color weightings, there are effectively eight useful intensity measurements. Suppose a filter is provided over one of the photodetectors to provide a third response. The remaining photodetectors do not include filters. Then,

Sfm— IrCfmr"tTgCfmg"tTbCfmb where i runs from 1 to 3. Here, Cfmr, Cftn _g , and Cftnb are conversion efficiencies for the main photodiode in the filtered pixel, and Cfpr, Cfp _g , and Cfpb are conversion efficiencies for the parasitic photodiode in the filtered pixel. A least squares fit to the eight measurements can now be done to provide an improved estimate of the intensities of the incident light in the three color bands. The filter could select a particular color channel such as a green, red, or blue filter. Alternatively, one or more filters could merely partially attenuate light in one of the filter channels to provide the required independent measurement while sacrificing less light.

[0034] In principle, a pixel sensor could be utilized that has only two photodetectors, one with a filter and one without. This would provide four intensity measurements to determine the color intensities. Refer now to Figure 3C, which illustrates a pixel sensor having two photodetectors of different sizes. Photodetector 85 lacks a filter, and

photodetector 86 has a filter. The two photodetectors provide four intensity measurements that can be used to determine the three color intensities. The total area of the pixel for a given signal-to-noise level will be less than that of the conventional pixel sensor for the same signal-to-noise level, as the amount of light received per unit area is greater due to the lack of a filter in photodetector 85.

[0035] The above-described embodiments utilize a three-channel color representation system to determine the intensities in channels corresponding to the red, blue, and green wavelength bands. Even a two-photodiode pixel sensor embodiment according to the present invention provides more intensity measurements than needed to determine the three channel intensities. The additional measurements can be used to reduce noise or to provide intensities in other color schemes having more color channels.

[0036] In the above embodiments, a second photodetector that provides the information needed to uniquely solve for the three or more color channel intensities is included in every pixel. However, the response from a third photodetector that has a different light conversion efficiency as a function of wavelength than that provided by the main photodiode and a parasitic photodiode in a single photodetector can be provided in other configurations. Refer now to Figure 5, which illustrates a pixel sensor having two main photodiodes that share a common floating diffusion node that has a parasitic photodiode response. To simplify the following discussion, those elements of pixel sensor 73 that provide the same functions as corresponding elements in photodetector 41, discussed above with reference to Figure 2, have been given the same numeric designations. Pixel sensor 73 includes a second main photodiode 71 that can be connected to floating diffusion node 43 by gate 72. The second main photodiode 71 has a different color response than photodiode 22 and parasitic photodiode 42. For example, second main photodiode 71 could be covered by a color filter. For example, second main photodiode 71 could be filtered such that second main photodiode 71 detects light preferentially in the blue or green portions of the spectrum. The cost of adding the second photodiode to pixel sensor 73 is much smaller than providing a complete second photodetector to implement a pixel sensor, since second main photodiode 71 shares all of the readout and reset circuitry. If the intensity of light is provided by main photodiode 22, then second main photodiode 71 need not be as large as photodiode 22, since the purpose of second main photodiode 71 is to provide the remaining color information, and the human eye is less sensitive to errors in color than errors in intensity. Controller 74 combines the color information form the three photodiodes to provide the intensities in the three color channels.

[0037] For the purposes of the present discussion, this second main photodetector that provides the third color equation, will be referred to as the secondary photodetector in the following discussion. If the incident light on a pixel were of a single pure color, i.e., a spectrum consisting of light at one wavelength or narrow band of wavelengths, the secondary photodetector would not be needed to determine the wavelength. It can be shown that the ratio of the signals from the parasitic photodiode to the main photodiode is a function of the average wavelength of the incident light on the photodetector. The function in question depends on the details of the main and parasitic photodiodes and can be calibrated for any particular design, provided the ratio of light response of the main photodiode to the secondary photodiode is a monotonic function of wavelength in the region of the spectrum of interest. In one embodiment, the ratio of light response is a linear function of the average wavelength.

[0038] The ratio of the parasitic photodiode response to the main photodiode response is shown in Figure 4 for one exemplary photodetector. In this embodiment, this ratio is a linear function of the average wavelength of the light incident on the photodetector over a significant fraction of the optical spectrum. The functional relationship between the ratio of the responses and the wavelength of the incident light depends on the details of the construction of the photodiode. If there is only one wavelength in the incident light, this ratio can be used to determine that wavelength and one of the equations in Eqs. (1) can be used to determine the intensity in the one RGB triplet color that is non-zero. It should be noted that if the average wavelength corresponds to red or blue, and camera filters remove wavelengths above and below the visual range, then there is only one wavelength in the incident light, and hence, the two equations can provide a unique solution for the wavelength. If, however, the average wavelength is within the intermediate wavelength band, the incident light could be the combination of two spectral lines, one in the red and one in the blue. In this case, information from a second photodiode or photodetector is needed.

[0039] If the information in one of the color channels changes slowly over the image, the density of secondary photodetectors can be reduced in the array. In this case, one secondary photodetector can be provided for every N pixel sensors, where N>1. The results from these sparse secondary photodetectors can be interpolated to provide a value at each pixel detector.

[0040] The above-described embodiments of the present invention have been provided to illustrate various aspects of the invention. However, it is to be understood that different aspects of the present invention that are shown in different specific embodiments can be combined to provide other embodiments of the present invention. In addition, various modifications to the present invention will become apparent from the foregoing description and accompanying drawings. Accordingly, the present invention is to be limited solely by the scope of the following claims.