VIDEO CAMERAS

RELATED APPLICATIONS

[ 0001 ] This application claims the benefit under 35 USC 119(e) of U.S. Provisional Application No.: 61/766,333, filed on February 19, 2013, which is incorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

[ 0002 ] Video surveillance or security cameras are often used for monitoring locations inside or outside of buildings. These buildings include government, residential, and commercial buildings, such as hospitals, casinos, retail establishments (e.g., shopping malls and department stores), offices, or warehouses, to list a few examples. The video cameras typically capture video data as a series of images (or frames) and send the video data to a control system, workstation, and/or network video recorder (NVR). The video data may be monitored in real time by security personnel or stored in the network video recorder and viewed at a later time.

[ 0003 ] Wide dynamic range (WDR) cameras are often used in security systems because the video cameras many times operate in locations with varying lighting conditions and must image objects of widely different intensities. Dynamic range refers to the ratio between the maximum and minimum measurable light intensities. For example, video cameras installed to monitor a parking lot at night will see bright headlights and area/street lights and dark shadows at the edges.

[ 0004 ] Currently, wide dynamic range (WDR) cameras utilize a number of different technologies. For example, some WDR video cameras implement multi-frame imaging. In this method, the video cameras capture multiple images of the same scene with different exposure settings (e.g., iris setting and shutter speed). The different images are then combined into one image. Other cameras use logarithmic sensors, which have different sensitivity at different intensity levels. Because the logarithmic sensors are less sensitive to changes in bright objects and more sensitive to changes in darker objects, the sensors provide better contrast in the darker parts of a scene. [ 0005 ] The video cameras use different automatic exposure modes (metering modes) to calculate the exposure settings for a scene. For example, one metering mode is average metering. In this mode, light from the entire scene is averaged to determine the exposure settings. Conversely, in spot metering, only light from a small focus area (typically 1-5%) or "spot" is used to determine the exposure settings. The light from other areas of the scene is not factored into the exposure settings. Yet another metering mode is center-weighted metering. In this mode, light from a center focus area is weighted significantly more than the light from areas outside of the center area.

[ 0006 ] Partial metering and multi-zone metering are two additional metering modes. Partial metering is similar to spot metering, but includes a larger focus area (typically 10- 15%)). Similar to spot metering, light outside of the focus area is not used to determine the exposure settings. Additionally, multi-zone (or matrix) metering uses light from multiple areas in the scene to determine exposure settings.

SUMMARY OF THE INVENTION

[ 0007 ] One problem with video cameras is that light from bright areas in a scene will typically affect the exposure settings of the video cameras and the bright areas skew the exposure settings. This leads to a loss of detail in the darker areas. However, in many situations, the important objects or details are in those darker areas, especially in security applications. By way of example, while monitoring a parking lot at night, the most important objects from a security standpoint might be located in the shadows. However, these objects are often dimly lit compared to the light sources (e.g., area/street lights or headlights of cars) in the scene. If the exposure settings are calculated based on the small bright areas, the important objects and details (e.g., the people) may be lost in the darker areas of the image.

[ 0008 ] In general, the present system and method are directed to enhancing the darker areas of scenes and preventing bright areas from distorting or skewing the exposure settings of the video cameras. That is, the present system and method prevent skewing of the exposure settings calculations by very small, but very bright "point" sources that are generally not of particular interest in security applications. These point sources typically include bright light sources such as the sun, light reflections, and/or small windows to the outside in an otherwise dark room, to list a few examples. Additional point sources include headlights or street lights, for example. Enhancing the darker areas and preventing the bright areas from skewing the exposure settings ensures that better detail is provided for darker areas in the captured image.

[ 0009 ] In more detail, imaging sensors of the video cameras capture video data as a series of images (or frames). Digital signal processors (DSP) of the video cameras segment captured images into window segments and identify bright areas (if any) in the window segments. If the bright areas of the frames cover less than some percentage, e.g., 15 percent of that captured image, then those bright areas are excluded from the exposure settings calculations of the captured image. On the other hand, if the bright areas cover more than 15 percent of that captured image, then the bright areas are factored into the exposure settings calculations.

[ 0010 ] In general, according to one aspect, the invention features a method for setting exposure of video cameras. The method includes capturing images with the video cameras and identifying bright areas in the captured images. The method further includes calculating exposure settings from the captured images excluding the bright areas in the captured images. Finally, camera control parameters are adjusted based on the calculated exposure settings.

[ 0011 ] In embodiments, the captured images are segmented into window segments by digital signal processors of the video cameras. The bright areas in the captured images are identified by generating a histogram for each window segment to determine a number of pixels at different light intensity levels for each window segment.

[ 0012 ] Preferably, the window segments are assigned weighting factors that correspond to automatic exposure modes of the video cameras. The weighting factors include center- weighted, spot, target, region of interest (ROI), average, partial-zone, or multi-zone metering, to list a few examples.

[ 0013 ] In the preferred embodiment, each captured image is analyzed to determine if the bright areas in the captured image cover less than 15 percent of the captured image. The bright areas are excluded from the exposure settings calculations if the bright areas of that captured image are less than 15 percent of the captured image. Additionally, the bright areas are excluded from the exposure settings calculations when a brightness threshold is exceeded. Typically, the brightness threshold is exceeded when the brightness a threshold that is greater than 50% of the full sampling range. In one example, the threshold is 700 out of a maximum of 1023 of a 10-bit analog to digital converter (ADC). [ 0014 ] In the preferred embodiment, the signal processor modifies the calculated exposure settings based on user adjustable compensation parameters and adjusts the camera control parameters based on the modified exposure settings. Typically, the camera control parameters include automatic gain control, shutter speed, and/or iris settings.

[ 0015 ] In general, according to another aspect, the invention features a video camera. The video camera includes an imaging sensor that captures images and a signal processor that identifies bright areas in the captured images and calculates exposure settings from the captured images excluding the bright areas in the captured images. A processing unit of the video camera adjusts camera control parameters based on the calculated exposure settings.

[ 0016 ] The above and other features of the invention including various novel details of construction and combinations of parts, and other advantages, will now be more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be understood that the particular method and device embodying the invention are shown by way of illustration and not as a limitation of the invention. The principles and features of this invention may be employed in various and numerous embodiments without departing from the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[ 0017 ] In the accompanying drawings, reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale; emphasis has instead been placed upon illustrating the principles of the invention. Of the drawings:

[ 0018 ] Fig. 1 A is a hybrid block diagram and perspective view illustrating a security system that includes security devices.

[ 0019 ] Fig. IB is a block diagram illustrating components of a video camera.

[ 0020 ] Fig. 2A is a flow diagram illustrating the steps performed by the camera including its digital signal processor to adjust the exposure settings.

[ 0021 ] Fig. 2B is a flow diagram of an alternative method performed by the camera to adjust the exposure settings.

[ 0022 ] Fig. 3 A illustrates an example of an image captured by the video camera.

[ 0023 ] Fig. 3B illustrates how the captured image is segmented into window segments by the digital signal processor of the video camera. [ 0024 ] Fig. 3C illustrates an example of an alternative image captured by a video camera.

[ 0025 ] Fig. 3D illustrates how the alternative captured image is segmented into window segments by the digital signal processor.

[ 0026 ] Fig. 3E illustrates how histograms are generated for each window segment of the captured image.

[ 0027 ] Fig. 4A is a flow diagram that illustrates how users are able to manually adjust exposure settings of the video camera.

[ 0028 ] Fig. 4B is an example of a graphical user interface that enables users to manually adjust exposure settings of the video camera.

[ 0029 ] Fig 5 is a flow diagram illustrating an alternative method to adjust the exposure settings of the video camera by correlating bright pixels of an entire window histogram with bright pixels of the window segments.

[ 0030 ] Fig. 6A illustrates how histograms are generated for each window segment of the captured image.

[ 0031 ] Fig. 6B illustrates an example of an entire window histogram for a captured image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[ 0032 ] The invention now will be described more fully hereinafter with reference to the accompanying drawings, in which illustrative embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[ 0033 ] As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. Further, the singular forms are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms: includes, comprises, including and/or comprising, when used in this

specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Further, it will be understood that when an element, including component or subsystem, is referred to and/or shown as being connected or coupled to another element, it can be directly connected or coupled to the other element or intervening elements may be present.

[ 0034 ] Fig. 1 A is a hybrid block diagram and perspective view illustrating a security system 100 that includes security devices such as video cameras 110, 111-1 to 111-n, card readers 54, and touchpad/keypads 113, to list a few examples.

[ 0035 ] In the illustrated example, a building 50 includes an entry/exit 56 and an access controlled entry/exit 52, which leads to a restricted area. A user validation system such as card reader 54 reads identification badges or keycards of users (e.g., person 112). If the badge is validated, then the access controlled entry/exit 52 is unlocked to enable the person 112 to access the restricted area.

[ 0036 ] In a typical implementation, the security system 100 includes additional security devices such as keypad/card readers 113, additional video cameras 111-1 to 111-n, or motion sensors, for example. The security devices are connected to a network 114, which is typically a public and/or private data network.

[ 0037 ] The building 50 might include windows 106, 108, which provide light to the interior area of the building 50 depending on the time of day and can thus contribute to the dynamic range of the captured scene.

[ 0038 ] The video camera 110 monitors a specific area within or outside the building 50. The video camera 110 captures video data as a series of images, which are transferred via the network 114 to a control system 116 in a control room 115. The video camera 110 implements different types of lenses such as fish-eye, wide angle, or panoramic, to list a few examples. Different lenses enable video cameras to capture different fields of view of a scene.

[ 0039 ] In a typical implementation, the security system 100 further includes a control room 115, which houses the control system 116, a network video recorder (NVR) 118, and a workstation 122.

[ 0040 ] The control system 116 receives the video data from the video cameras 110, 111 of the security system 100. The video data are transferred to the NVR 116, which stores the video. Typically, time and date information are added to video data to enable the data to be indexed and reviewed at a later date. The workstation 122 is operated by a user 120. The workstation 122 enables the user 120 to view the video data from the video camera 110.

[ 004 1 ] Fig. IB is a block diagram illustrating the components of the video camera 110. The video camera 110 includes a camera housing (or enclosure). In terms of electronics, the video camera 100 includes a microprocessor or central processing unit (CPU) 150, which controls the camera control parameters (e.g., shutter speed, automatic gain control, and iris) of the video camera 110. The CPU 150 includes the DSP controller 151, which controls the video processing of the video camera 110 by communicating with the digital signal processor (DSP) 156. The CPU 150 is connected to memory 152, which stores programs used to calculate exposure settings, for example.

[ 0042 ] The video camera 110 further includes a lens system 160, which typically has one or more lenses, and iris and a focusing mechanism. The lens system 160 captures light and images it onto an imaging sensor 158.

[ 0043 ] The imaging sensor 158 converts captured light into a series of electronic images (i.e., video data) bases on a shutter setting. Generally, the imaging sensor 158 is a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) device. The imaging sensor 158 is generally installed directly behind the lens system 160 to receive the captured light from the lens system 160.

[ 0044 ] The video data are sent to the digital signal processor (DSP) 156. The DSP 156 also combines the video data with possibly corresponding audio captured by a microphone 164 and an audio amplifier circuit 162. Additionally, the DSP 156 may compress the audio and video data to make the video data easier to store, analyze, and/or stream over the network 114.

[ 0045 ] A network interface controller 174 enables the video camera 110 to

communicate with the control system 116 over the network 114. The network 114 is typically a public and/or private data network. The network may also include an enterprise network, university network, government network, and/or mobile broadband data network, to list a few examples. [ 004 6 ] Fig. 2A is a flow diagram illustrating the steps performed by the digital signal processor (DSP) 156 to adjust the exposure settings of the video camera 110, such as the iris and/or shutter settings, for example.

[ 0047 ] In the first step 302, a video camera 110 captures an image using current exposure settings. The digital signal processor (DSP) 156 of the video camera 110 segments the captured image into window segments in step 304.

[ 004 8 ] In the next step 306, the DSP 156 generates a histogram for each window segment to determine the number of pixels at different intensity levels. Then, the DSP analyzes the histograms of the window segments to identify bright areas in the captured image based on a brightness threshold in step 308. In a current embodiment, the brightness threshold greater than 50% of the full range of the sensor 158. Currently, it is about 700 out of 1023 for a 10-bit analog to digital converter (ADC).

[ 004 9 ] The DSP 156 determines if the bright areas of the captured images are less than a predetermined percentage of the captured image in step 310. That is, the number of bright pixels is less than a predetermined percentage of the total number of pixels in the two dimensional array of pixels of the imaging sensor.

[ 0050 ] If the bright areas of the captured images are less than the predetermined percentage of the captured image, then the window segments exhibiting bright areas are excluded from the exposure setting calculations in step 312. In a current implementation, the predetermined percentage is 5 percent. In general, the predetermined percentage is less than 15 percent, and usually less than 10 %.

[ 0051 ] In the next step 314, the DSP assigns weighting factors to the remaining window segments corresponding to an automatic exposure mode (e.g., center-weighted, spot, target, region of interest (ROI), average, partial, or multi-zone). Then, the DSP 156 applies the weighting factors to the window segments in step 316. The new exposure settings are calculated in step 318. If, however, the bright areas of the captured images are not less than the predetermined percentage of the captured image, then the DSP assigns weighting factors to all window segments, i.e., without exclusion, corresponding to the automatic exposure mode in step 320.

[ 0052 ] The DSP applies weighting factors to the window segments in step 322 and calculates exposure settings in step 324. In the next step 326, the calculated exposure setting is modified based on any user dictated exposure value compensation parameters. This step enables users to manually adjust the exposure and dynamic range settings of the video camera 110. For example, users that care more about the dark area can increase the exposure settings. Alternatively, users may decrease the exposure settings if they are more concerned with details in the bright areas.

[ 0053 ] In one current implementation, there are nine different level settings (e.g., levels 0 through 8), which are adjustable through a graphical user interface (GUI). Alternative embodiments, could implement greater or fewer levels. Each level enhances/decreases approximately 4-5 dB in wide dynamic range. Typically, level '0' disables the wide dynamic range of the video camera. The lower WDR range provides increased contrast for a smaller range of image intensities. Alternatively, higher levels improve the image contrast of dark areas for reduced exposure of bright areas. The result is more detail in these dark areas, but with increased noise.

[ 0054 ] Lastly, in step 328, the current exposure settings are used to adjust camera control parameters (e.g., such as shutter speed, automatic gain control (AGC) and iris settings). Returning to step 302, these camera control parameters and exposure setting are used to capture a subsequent image.

[ 0055 ] Fig. 2B is a flow diagram of an alternative method performed by the digital signal processor 156 to adjust the exposure settings of the video camera 110.

[ 0056 ] In the previous embodiment described with respect to Figure 2A, entire window segments were excluded from the exposure settings calculations. In this embodiment, however, only the pixels of bright areas are excluded from the exposure settings calculations.

[ 0057 ] As shown in Fig. 2B, steps 302-310, and 320-328 are identical to the similarly numbered steps described with respect to Fig. 2A.

[ 0058 ] If the bright areas of the captured images are less than the predetermined percentage of the captured image (step 310), then the pixels of bright areas are excluded from the exposure settings calculations in step 350. The DSP 156 then assigns weighting factors to the window segments corresponding to the automatic exposure mode in step 352. Then, in step 354, the DSP 156 applies the weighting factors to the window segments (including segments with excluded pixels). In the next step 356, the DSP calculates the exposure settings.

[ 0059 ] Fig. 3 A illustrates an example of an image 200 captured by the video camera 110 for step 302 of Figs. 2A and 2B. In the illustrated example, the captured image 200 is of an object (e.g., person 112) in front of windows 106, 108 within the building 50.

[ 0060 ] Fig. 3B illustrates how the captured image 200 is segmented into window segments 204-1 through 204-576 by the digital signal processor 156 of the video camera 110 in step 304 of Figs. 2 A and 2B.

[ 0061 ] In a preferred embodiment, the image is segmented into 32 columns by 18 rows (576 windows segments). In alternative embodiments, however, the captured images may be segmented into greater or fewer row and/or columns.

[ 0062 ] In the illustrated example, light enters the building 50 via the windows 106, 108. Because the windows cover such a large portion of the scene (e.g., greater than 5 or 15%) as determined in step 310 of Figs. 2 A and 2B, the window segments for these windows 106, 108 will not be excluded in the exposure setting calculations. They will be factored into the exposure settings calculations (e.g., steps 320-324 in Figs. 2A and 2B).

[ 0063 ] Fig. 3C illustrates an example of an alternative image 220 captured by an alternative video camera of the security system 100.

[ 0064 ] In this example, a car 230 is driving on a road 232, possibly at night. A person 228 and trees 222, 224, 226 are different objects of the scene.

[ 0065 ] Fig. 3D illustrates how the alternative image 220 is segmented into window segments 205-1 to 205-576 by a digital signal processor of the video camera according to step 304 of Figs. 2 A and 2B.

[ 0066 ] Similar to the previous example, the alternative captured image 220 is segmented into 32 columns by 18 rows (576 total windows segments) by a digital signal processor of the video camera.

[ 0067 ] In this example, headlights 234 provide the light source of the scene. Because the headlights only cover a small portion of the scene, the window segments for the headlights 234 will be excluded in the exposure setting calculations (e.g., steps 312-318 in Fig. 2A). In the alternative embodiment, only the bright pixels (the pixels that detect the light of the headlights 234) of the image 220 are excluded in step 350 of Fig. 2B.

[ 0068 ] These bright areas are excluded from the exposure settings calculations to prevent the small bright areas from skewing the exposure setting of the image. This helps to ensure that the objects and details in the darker areas are not lost in the captured image.

[ 0069 ] Fig. 3E illustrates example histograms 250-256 for window segments 204-a, 204-b, 204-c, and 204-d, respectively that are generated in step 306 of Figs. 2A and 2B.

[ 007 0 ] The window segments 204-a and 204-b are of areas of the image that do not include bright areas. Windows segments 204-c and 204-d, in contrast, exhibit spikes (or peaks) 255 and 257, respectively. These spikes 255, 257 are caused by the bright areas in the scene. Thus, these windows segments are excluded in the exposure settings calculations of the exposure settings in the Fig. 2A embodiment. In an alternative Fig. 2B embodiment, only the bright pixels (255, 257) are excluded in the exposure settings calculations.

[ 007 1 ] Fig. 4A is a flow diagram that illustrates how users are able to manually adjust or compensate exposure settings of the video cameras.

[ 0072 ] In the first step 402, a user selects a video camera to monitor via the

workstation 122. Then, the captured image is displayed from the selected video camera on a display of the workstation 122 in step 404. Next, the user accesses a settings menu of the graphical user interface (GUI) for the selected video camera in step 406.

[ 0073 ] The user selects an exposure value compensation parameter from the settings menu 502. Then, in step 410, the user is able to adjust the exposure value (EV)

compensation parameter. The exposure value compensation parameter is transmitted to the selected video camera in step 412 over the network 114. Next, the video camera 110 adjusts the camera control parameters based on the user adjusted parameter.

[ 0074 ] Fig. 4B is an example of the graphical user interface (GUI) 500 that enables users to manually adjust settings of the video cameras.

[ 0075 ] The GUI 500 is displayed on the workstation 122. Typically, the GUI 500 is overlaid on the image displayed on the workstation 122. In alternative embodiments, the GUI could be displayed on other devices such tablet computers, laptop computers, or mobile computing devices (e.g., smartphones), to list a few examples. [ 007 6 ] The user invokes the settings menu 502, which displays user selectable elements 504-518. In the illustrated example, the settings menu 502 includes user selectable elements for brightness 504, color 506, contrast 508, EV (exposure value) compensation 510, language 512, WDR range 514, and zoom 516. Additionally, the GUI 500 further includes a user selectable element 518 to reset any changes back to default settings.

[ 0077 ] After selecting one of the user selectable elements a slider or other user control is displayed that enables the user to manually adjust the selected setting. In the illustrated example, the user selectable element for the EV compensation 510 is invoked. In response, a slider bar 511 is displayed that enables the user to adjust the level of EV compensation in the image.

[ 007 8 ] Fig. 5 is a flow diagram illustrating alternative steps to adjust the exposure settings of the video camera by correlating bright pixels of an entire window histogram with any bright pixels of the window segments.

[ 007 9 ] In this example, after the DSP 156 analyzes the histograms of window segments to identify bright areas in the captured image based on brightness threshold in step 308, the DSP generates an entire window histogram for the captured image in new step 309.

[ 0080 ] Then, if the areas of the captured images are less than the predetermined percentage of the captured image (step 310), the DSP 156 correlates the bright pixels of the entire window histogram with any bright pixels of the window segments in step 360. Next, the window segments correlated with bright pixels are excluded from exposure setting calculations in step 362.

[ 0081 ] Fig. 6A illustrates histograms 602-608 for window segments 205-a, 205-b, 205- c, and 205 -d, respectively.

[ 0082 ] Window segments 205-a and 205-b illustrate areas that do not include bright areas. Conversely, windows segments 205-c and 205-d include spikes 607 and 609, respectively. These spikes 607, 609 are caused by bright areas in the scenes (e.g., headlights, sun, reflections, street lights).

[ 0083 ] Fig. 6B illustrates an example of a histogram 702 of a captured image, which includes spikes (or peaks) 704. These spikes 704 indicate at least one bright area in the captured image. The DSP of the video camera correlates the bright pixels 704 of entire window histogram 702 with any bright pixels of window segments. See pixels 607 and 609 of segments 205-c and 205-d, respectively, in Fig. 6A, leading to these segments being excluded from the exposure setting calculation.

[ 0084 ] While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.