UNDERWATER IMAGE APPARATUS WITH RED BIAS FILTER ARRAY

BACKGROUND

Cameras are commonly used to capture an image of a scene. Additionally, some cameras are waterproof and are used to capture an image of a scene that is underwater.

It is well known that water absorbs longer wavelength light more rapidly then shorter wavelength light. As a result, underwater, at shallow depths, red structures in the scene no longer appear red. This effect continues for increasing depths, and longer wavelength (visible) colors. As a result thereof, typical underwater photographs are dominated by short wavelength colors, e.g. blue and the longer wavelength colors, e.g. red are absorbed proportionally to the depth underwater.

SUMMARY

The present invention is directed to an image capturing apparatus for capturing an image of a scene. The image capturing apparatus can include an image sensor assembly and a filter array. The image sensor assembly includes a plurality of photosites. The filter array filters light that is directed at the image sensor assembly. The filter array includes a plurality of blue filters, a plurality of green filters, and a plurality of red filters that are arranged in an array. In one embodiment, the number of red filters is greater than the number of blue filters, and the number of red filters is greater than the number of green filters. As a result of this design, in certain embodiments, the image capturing apparatus compensates for the overwhelming amount of green and blue colors caused by the absorption of shorter wave length light (red) underwater, and the image capturing apparatus is better able to capture the actual or true colors of .the an underwater scene.

As utilized herein, the actual or true colors of the underwater scene shall mean colors that are present with no light attenuation at the scene and even white light illumination of the scene.

In alternative, non-exclusive embodiments, the number of red filters is approximately 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 150, 200, 300, 400, 500,

600 or 1000 percent greater than the number of blue filters, and/or the number of red filters is approximately 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 150, 200, 300, 400, 500, 600 or 1000 percent greater than the number of green filters. In another embodiment, the number of red filters is approximately greater than the number of blue filters plus the number of green filters.

In another embodiment, the image capturing apparatus can include a substantially waterproof housing and a capturing system for capturing the image. The capturing system can include a plurality of blue pixels, a plurality of green pixels, and a plurality of red pixels that are arranged in an array. In one embodiment, the number of red pixels is greater than the number of blue pixels, and the number of red pixels is greater than the number of green pixels.

The present invention is also directed to a method for capturing an image of a scene.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of this invention, as well as the invention itself, both as to its structure and its operation, will be best understood from the accompanying drawings, taken in conjunction with the accompanying description, in which similar reference characters refer to similar parts, and in which:

Figure 1 is a simplified side plan illustration of a scene and an image capturing apparatus having features of the present invention;

Figure 2A is a simplified front perspective view of one embodiment of the image capturing apparatus;

Figure 2B is a simplified rear perspective view of the image capturing apparatus of Figure 2A;

Figure 3 is a simplified cut-away view of another embodiment of an image apparatus;

Figure 4 is a simplified front perspective plan view of a filter array and an image sensor assembly having features of the present invention;

Figure 5 includes a graph that illustrates the attenuation of light as a function of wavelength and a graph that illustrates the percentage of light reaching certain depths; and

Figure 6 is a simplified flowchart that illustrates one example of the operation of the image capturing apparatus.

DESCRIPHON

Figure 1 is a simplified side plan illustration of an image capturing apparatus 10 having features of the present invention and a scene 12. The image capturing apparatus 10 is useful for providing an image 214 (illustrated in Figure 2B) of the scene 12. The type of scene 12 captured by the image capturing apparatus 10 can vary. In certain embodiments, the image capturing apparatus 10 is waterproof and is adapted to capture images of one or more scenes 12 that are partly or fully under a liquid 16 such as water. For example, each scene 12 can include one or more underwater animals, plants, mammals, fish, coral, objects, and/or environments. In Figure 1, the scene 12 includes a starfish 18 that is a subject 20, e.g. the focal point of the scene 12.

In certain embodiments, the image capturing apparatus 10 can be any device capable of providing the image 214, including (i) a digital camera that electronically stores the image 214, (ii) a digital camera in video mode, and/or (iii) a digital video recording device that electronically records still or moving images 214. As provided herein, in certain embodiments, the image capturing apparatus 10 includes one or more features that allow the image capturing apparatus 10 to more accurately capture the true colors of the underwater scene 12.

In Figure 1 , the focal point 20 of the scene 12, e.g. the center of the starfish 18 is at a subject depth SDep below a fluid surface 21, and an optical assembly 22 (illustrated in phantom) of the image capturing apparatus 10 is at an apparatus depth AD below the fluid surface 21. For example, the subject depth SDep can be greater than, less than or approximately equal to the apparatus depth AD. The apparatus depth AD at which the image capturing apparatus 10 is still waterproof can vary according to the design of the image capturing apparatus 10. For example, in nonexclusive alternative embodiments, the image capturing apparatus 10 can be waterproof up to an apparatus depth AD of at least approximately 3, 5, 10, 30, 40, 50, or 100 meters.

Moreover, the subject 20 of the scene 12 is separated a separation distance SDist away from optical assembly 22. The acceptable amount of separation distance SDist can be varied according to the type of optical assembly 22 utilized in the image capturing apparatus 10, the visibility of the water 16, and other factors including, but not limited to, the amount of available light. In one non-exclusive embodiment, the separation distance SDist can be between approximately 1 centimeter and 30 meters. However, separation distances can be used.

Figure 2A illustrates a simplified, front perspective view of one, non-exclusive embodiment of the image capturing apparatus 210. In this embodiment, the image capturing apparatus 210 is a camera that includes a housing 228, an optical assembly 222, a capturing system 230 (illustrated as a box in phantom), a power source 232 (illustrated as a box in phantom), an illumination system 234, and a control system 236 (illustrated as a box in phantom). The design of these components can be varied to suit the design requirements and type of image capturing apparatus 210. Further, the image capturing apparatus 210 could be designed without one or more of these components. For example, the image capturing apparatus 210 could be designed without the illumination system 234.

The housing 228 can be rigid and support at least some of the other components of the image capturing apparatus 210. In one embodiment, the housing 228 defines a cavity that receives and retains at least a portion of the capturing system 230, the power source 232, and the illumination system 234. Further, the optical assembly 222 is fixedly secured to the housing 12.

In one embodiment, the housing 228 is watertight and forms a watertight compartment that protects the electronic components of the image capturing apparatus 210. Alternatively, as illustrated in Figure 3 and described below, the image capturing apparatus 310 can include an inner housing 328A, and an. outer housing 328B that forms an outer shell that surrounds and encloses the inner housing 328A. In this embodiment, the outer housing 328B provides a watertight barrier around the electronic components of the image capturing apparatus 310.

Referring back to Figure 2A, the image apparatus 210 can include an aperture (not shown) and a shutter mechanism (not shown) that work together to control the amount of light that reaches the capturing system 230. The shutter mechanism can include a pair of blinds that work in conjunction with each other to allow the light to be focused on the capturing system 230 for a certain amount of time. Alternatively, for example, the sWutter mechanism can be all electronic and contain no moving parts. For example, an electronic capturing system can have a capture time controlled electronically to emulate the functionality of the blinds. The time in which the shutter mechanism allows light to be focused on the capturing system 230 is commonly referred to as the capture time or the exposure time. The shutter mechanism is activated by a shutter button 238.

The optical assembly 222 is secured to the housing 228 near the aperture. The optical assembly 222 can include a single lens or a combination of lenses that work in

conjunction with each other to focus light onto the capturing system 230.

The imaging apparatus 10 can include an autofocus assembly (not shown) including one or more lens movers that move one or more lenses of the optical assembly 222 in or out to focus the light on the capturing system 230. For example, the autofocus assembly can be an active or passive type system.

The capturing system 230 captures the image 214 (illustrated in Figure 2B), is positioned within the housing 228, and is coupled to the housing 228. The design of the capturing system 230 can vary according to the type of image capturing apparatus 210. For a digital type camera, the capturing system 230 includes an image sensor assembly 240 (illustrated in phantom), a filter array 242 (illustrated in phantom), and a storage system 244 (illustrated in phantom).

The image sensor assembly 240 receives the light that passes through the aperture and the filter array 242, and converts the light into electricity. The type of image sensor assembly 240 can vary. One non-exclusive example of an image sensor assembly 240 for digital cameras is known as a charge coupled device ("CCD"). A CCD consists of an integrated circuit containing an array of tiny, light-sensitive photosites or pixels 446 (illustrated in Figure 4), which are capable of accumulating varying amounts of charge in proportion to the amount of light they receive. A CCD can contain thousands or even millions of these photosites 446, each of which is individually light-sensitive.

An alternative image sensor 240 that may be employed in digital cameras uses complementary metal oxide semiconductor ("CMOS") technology. CMOS devices use several transistors at each photosite 446.

The imaπe sensor assembly 240, by itself, produces a grayscale image as it only keeps track of the total intensity of the light that strikes the surface of the image sensor assembly 240. In order to produce a full color image, the filter array 242 positioned in front of and filters the light that is directed at the image sensor assembly 240. With the filter array 242 filtering the light that is directed at the image sensor assembly 240, certain pixels 446 measure the red light, certain pixels 446 measure the blue light, and certain pixels 446 measure the green light from the scene 12 (illustrated in Figure 1). In certain embodiments, filter array 242 disclosed herein compensates for the overwhelming amount of green and blue colors caused by the absorption of shorter wave length light (red) underwater, and the image capturing apparatus 210 is better able to capture the actual or true colors of the an underwater scene 12. The filter array 242 is discussed in more detail below.

The storage system 244 stores the various captured images 214. The storage system 244 can be fixedly or removable coupled to the housing 228. Non-exclusive examples of suitable storage systems 244 include flash memory, a floppy disk, a hard disk, or a writeabfe CD or DVD.

The power source 232 provides electrical power to the electrical components of the image capturing apparatus 210. For example, the power source 232 can include one or more batteries.

The illumination system 234 can selectively provide a flash of light that illuminates at least a portion of the scene 12.

The control system 236 is electrically connected to and controls the operation of the electrical components of the image apparatus 210. Additionally, the control system 236 receives information from the capturing system 230 to generate the image 214. For example, the control system 236 receives the color information from the pixels 446, and interpolates the red, green, and blue values to generate the image 214. The control system 236 can include one or more processors and circuits.

Figure 2B illustrates a simplified, rear perspective view of the image capturing apparatus 210. In this embodiment, the image apparatus 210 includes an image display 250 that displays the image 214. Additionally, the image display 250 can display other information such as the time of day, and the date. Moreover, the image apparatus 210 can include one or more control switches 252 electrically connected to the control system 236 (illustrated in Figure 2A) that allow the user to control the functions of the image apparatus 210.

Figure 3 is a simplified side plan illustration of another embodiment of an image capturing apparatus 310 that includes an inner housing 328A and a selectively removable outer housing 328B. In this embodiment, the inner housing 328A is somewhat similar to the corresponding housing 228 described above. However, in this embodiment, the inner housing 328A is not waterproof. Instead, in this embodiment, the outer housing 328B forms an outer shell that surrounds and encloses the inner housing 328A and provides a watertight barrier around the electronic components of the image capturing apparatus 310.

In one embodiment, the outer housing 328B is at least partly made of a clear material. Moreover, the outer housing 328B can include one or more pass through switches 354 that can be used to control the operation of the control switches 252 of the image capturing apparatus 310. For example, each pass through switch 354 can be a button that is aligned with and engages one of the control switches 252. Further,

each button extends through the outer housing 328B and is movably sealed to the outer housing 328B. With this design, the user can control the control switches 252 when the outer housing 328B encircles the rest of the image capturing apparatus 310.

Figure 4 is a simplified perspective view of one non-exclusive embodiment of the image sensor assembly 240 and the filter array 242. More specifically, Figure 4 illustrates the two dimensional array of pixels 446 of the image sensor assembly 240. The number and design of pixels 446 in the image sensor assembly 240 can varied to achieve the desired resolution. In alternative, non-exclusive embodiments, the image sensor assembly 240 includes at least approximately 2, 3, 4, 5, 6, 7, 8, 9, 10, or 12 million pixels 446. It should be noted that only a relatively small number of pixels 446 are illustrated in Figure 4 for ease of illustration.

The filter array 242 filters light that is directed at the image sensor assembly 240.. Further, the two dimensional filter array 242 includes a plurality of individual filters 456. The number of filters 456 in the filter array 242 can be varied to match the design of the image sensor assembly 240. In one embodiment, the number of filters 456 in the filter array 242 is equal to the number of pixels 446 in the image sensor assembly 240. In alternative, non-exclusive embodiments, the filter array 242 includes at least approximately 2, 3, 4, 5, 6, 7, 8, 9, 10, or 12 million filters 456. It should be noted that only a relatively small number of filters 456 are illustrated in Figure 4 for ease of illustration.

The filter array 242 is composed of a plurality of blue filters ("B"), a plurality of green filters ("G"), and a plurality of red filters ("R") that are arranged in the filter array 242. In one embodiment, the number of red filters R is greater than the number of blue filters B, and the number of red filters R is greater than the number of green filters G. As a result of this design, in certain embodiments, the capturing system 230 compensates for the overwhelming amount of green and blue colors caused by the absorption of shorter wave length light (red) underwater, and the capturing system 230 is better able to capture the actual or true colors of the underwater scene 12 (illustrated in Figure 1). With this design, the filter arrays 242 disclosed herein can be useful for a dedicated underwater image capturing apparatus.

The ratio of color filters 456 in the filter array 242 can be varied pursuant to the teachings provided herein. In alternative, non-exclusive embodiments, the number of red filters (R) is approximately 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 150, 200, 300, 400, 500, 600 or 1000 percent greater than the number of blue filters (B), and/or the number of red filters (R) is approximately 10, 20, 30, 40, 50, 60, 70, 80, 90, 100,

110, 150, 200, 300, 400, 500, 600 or 1000 percent greater than the number of green filters (G). In another embodiment, the number of red filters (R) is approximately greater than the number of blue filters (B) plus the number of green filters (G). In still another embodiment, the number of green filters (G) is greater than the number of blue filters (B).

In one embodiment, the ratio of the color filters 456 and the pattern of the color filters 456 can be optimized to increase the sensitivity of the image sensor assembly 240 in proportion to one or more of (i) a common underwater depth (apparatus depth AD illustrated in Figure 1) at which the image apparatus 10 is typically used, (ii) a common subject depth SDep (illustrated in Figure 1), (iii) a common separation distance SDist (illustrated in Figure 1), and/or (iv) a common type of fluid 16 (illustrated in Figure 1) in which the image apparatus 10 is used. For example, 5-10 feet is a common depth for snorkeling, while 30 feet, 60 feet and 100 feet are common depths for scuba diving. In these examples, the pattern and ratio of filters 456 can be optimized for a 5, 10, 30, 60 or 100 foot apparatus depth AD. Further, the pattern and ratio of filters 456 can be optimized for fresh or salt water.

For example, the pattern and ratio of filters 456 can be optimized for a 20 meter depth. In one simplified example, if red is absorbed approximately 60% per meter, green is absorbed approximately 40% per meter, and blue is absorbed approximately 10% per meter. At 20 meters, red has been absorbed approximately 1200%, green has been absorbed approximately 800%, and blue has been reduced approximately 200%. So to normalize this to blue; red is 600%, green is 400%, and blue is 100%. Red has a 6 times gain, green has a 4 times gain, and blue has a 1 times gain versus the distribution at sea level.

Because, the number of pixels has a fixed pattern, with a fixed number of pixels, a repeating, or alternating pattern of filters 456 needs to be established which represents the new number of filters 456.

Additionally, the most common absorption curves encountered at the most common depths can be utilized to determine the ratio and the pattern of filters 456.

Further, the control system 236 has the ability to provide extra gain.

Additionally, the image capturing apparatus 210 with the filter array 242 disclosed herein can also be used at or above sea level by using the control system 236 to attenuate the signals inversely. However, in this example, light is lost, which means that the overall signal to noise is decreased, and (ISO) is equivalents decreased at sea-level. In this embodiment, for example, one or more of the control

switches 252 can be used to instruct the control system 236 that the image is being captured at or above sea level.

Figure 5 includes a first graph that illustrates- the attenuation of light in a fluid (the ocean) in percent per meter as a function of wavelength and a second graph that illustrates the percentage of 465 nm light reaching certain depths. In these graphs, line I represents extremely pure ocean water; line Il represents turbid tropical- subtropical water; line III represents mid-latitude water; and lines 1-9 represent coastal waters of increasing turbidity. The incidence angle is 90 degrees for lines l-lll and the incidence angle is 45 degrees for lines 1-9. The graphs in Figure 5 are reproduced from Jerlov N. G. 1976, Marine Optics. Amsterdam: Elsevier Scientific . Publishing Company ISBN .0444414908.

The pattern and ratio of filters 456 can be optimized using information from the charts illustrated in Figure 5.

Figure 4 illustrates one non-exclusive pattern and ratio of filters 456. In this embodiment, the pattern alternates a row of red (R) and green filters (G) with a row of red (R) and blue filters (B). Further, in this embodiment, the number of red filters (R) is approximately equal to the number of blue filters (B) plus the number of green filters (G).

The design of each of the filters 456 can vary. In one embodiment, (i) each of the blue filters (B) transmits blue light (wavelengths of approximately 450 to 490 nm), and blocks light that is not blue (wavelengths that are greater than 490 nm and less than 450 nm), (ii) each of the green filters (G) transmits green light (wavelengths of approximately 490 to 570 nm), and blocks light that is not green (wavelengths that are greater than 570 nm and less than 490 nm) and (iii) each of the red filters (R) transmits red light (wavelengths of approximately 630 to 750 nm), and blocks light that is not red (wavelengths that are greater than 750 nm and less than 630 nm). Each of the filters 456 can include a transparent substrate (e.g. glass) that is coated with the appropriate coatings.

As a result of the filter array 242, (i) some of the pixels 446 can be classified as blue pixels 446B that measure the amount of blue light, (ii) some of the pixels 446 can be classified as green pixels 446G that measure the amount of green light, and (iii) some of the pixels 446 can be classified as red pixels 446R that measure the amount of red light. With the filter array 242 designs provided herein, the number of red pixels 446 R is greater than the number of blue pixels 446B, and the number of red pixels 446R is greater than the number of green pixels 446G.

In alternative, non-exclusive embodiments, the number of red pixels 446R is approximately 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 150, 200, 300, 400, 500, 600 or 1000 percent greater than the number of blue pixels 446B, and/or the number of red pixels 446R is approximately 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 150, 200, 300, 400, 500, 600 or 1000 percent greater than the number of green pixels . 446G. In another embodiment, the number of red pixels 446R is approximately greater than the number of blue pixels 446B plus the number of green pixels 446G. As a result of these designs, the capturing system 230 captures more information relating to the amount of red that is present in the scene in order to create an image 214 that is closer to the true color.

Figure 6 is a simplified flowchart that illustrates one non-exclusive example of the operation of the image capturing apparatus. It should be noted that one or more of the steps can be omitted or the order of steps can be switched. First, the image capturing apparatus is aimed toward the scene 610. Second, the user adjusts the zoom so as to adjust the size of the image as desired 620. Next, the user presses lightly on the shutter button to enable the image capturing apparatus to automatically focus on the object(s) 630. Subsequently, the image capturing apparatus sets the aperture and shutter speed 640. Next, the user presses the shutter button all the way, which resets the image sensor assembly, and exposes the image sensor assembly to light 650. Next, the charge at each photosite of the image sensor assembly is measured and a digital signal that represents the values of the charge at each photosite is created 660. Data is gained, or attenuated for color balance purposes. Subsequently, the control system interpolates the data from the different photosites, to create the color image 670. Data is gained, or attenuated for color balance purposes. Next, the image is displayed on the image display 680.

While the current invention is disclosed in detail herein, it is to be understood that it is merely illustrative of the presently preferred embodiments of the invention and that no limitations are intended to the details of construction or design herein shown other than as described in the appended claims.