FIELD OF THE INVENTION The present invention relates to a method for producing a background model based on the images acquired from a non-static camera.

BACKGROUND OF THE INVENTION In object-based video compression, as well as in other types of object-oriented video processing, the input video is separated into two streams.

The conventional video analytics is used to detect suspicious event using static camera. However, such detection requires background of the scene to be in a static condition as illustrated in Figure 1. This is due to the background pixels are stationary (variation of the intensity distribution for each background pixel is small) and any significant change in the intensity is corresponded to the foreground object. Common way of representing the background scene is using a single scalar value to represent each pixel. Scalar value represents a moving average of the pixel intensity over time. Despite of the simplicity of this model, it will not be able to correctly represent the dynamic background scene. Dynamic background scene also refers to the background scene that is having high variability of intensity over time, for. example waving trees and ocean waves. Using single scalar value for each pixel as the model, the variability of the background scene is not able to be captured. Thus, a dynamic background will be falsely identified as foreground object.

There are several solutions for modeling the dynamic background scene. Among others, early initiative is to model the background using normal distribution, multi models, mixture of Gaussian and kernel density. These models are able to robustly represent dynamic background scene. Unfortunately, these models are complex and are used to represent each pixel in the image. Thus, this has increased and contributed to high computational cost to maintain and update the model.

There is, thus, a need for a method for producing a background model based on the images which containing dynamic background. A method for modeling the dynamic background is presented in the present invention that improves the performance of the modeling approach using the adaptive region-based background model. Apart from dynamic background introduced by high variability of intensity distribution for certain dynamic background region, this invention also provides a solution to handle dynamic background due to changes of the camera view when a non-static camera is used. The present invention provides a considerable reduction of materials with even greater efficiency and economically during operation.

SUMMARY OF THE INVENTION

The present invention provides a method for producing a background model from images acquired from a camera comprising acquiring one or more images from the camera; providing a model for each current background scene according to the images acquired; storing the model of each background scene as a layer in a background model; determining a neighboring layer for each background scene based on the images acquired during the camera movement; storing linked indexes of each neighboring layer for each background scene; and forming a final background model based on the linked indexes of neighboring layers.

In one embodiment of the present invention, the step for providing a model for each current background scene according to the images acquired further comprising acquiring a time series intensity value for each pixel; updating a frequency for each intensity value for each pixel; computing an average intensity and an intensity variance for each pixel; determining whether the intensity variance for each pixel is more than predetermined threshold value; if the intensity variance of each pixel is more than a predetermined threshold value; grouping all connected dynamic pixels as a same region; merging a density distribution for each pixel to a same group; extracting background feature from the model for a current scene; storing background feature corresponding to the current scene as a model; if the intensity variance of each pixel is less than a predetermined threshold value; storing an average intensity as a model for the pixel; extracting background feature from the model for a current scene; and storing background feature corresponding to the current scene as a model.

In yet another embodiment of the present invention, a statistical model representation is used to determine whether average intensity belongs to background or foreground. In yet another embodiment of the present invention, a non-dynamic region consists of image pixels which correspond to static background pixels and a dynamic region corresponds to the dynamic background pixels.

In another embodiment of the present invention, adjacent dynamic background pixels having similar background models are grouped together.

A system for producing a background model for images acquired from a camera comprising a PTZ camera set to monitor wide area and capture images; an image processing unit to analyze the images and extract information from the images; a display unit to display the images captured and analyze output images; and a post event detection unit to trigger an alarm for post action. In one embodiment of the present invention, the camera view of PTZ camera is a multiple camera view.

In yet another embodiment of the present invention, the camera view of the PTZ at a current position is a current camera view and a surrounding scene as the other view.

In one embodiment of the present invention, the camera is a non-static camera.

In yet another embodiment of the present invention, the background model is a dynamic background model.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

Figure lillustratesof background of the scene to be in a static condition in a static camera and moving camera in a conventional video analytics. Figure 2 illustrates method for effectively model the dynamic scene within a camera view - adaptive region-based modeland method for modeling the dynamic scene due to camera movement (modeling multiple non-overlapping background scene) in accordance of an embodiment of the present invention.

Figure 3 illustrates a flow chart for a method for producing a dynamic background model based on the images acquired from a non-static camera in accordance of an embodiment of the present invention. Figure 4 illustrates a flow chart for a step of modeling current background scene in accordance of an embodiment of the present invention.

Figure 5 illustrates a modeling background scene using adaptive based region model in accordance of an embodiment of the present invention.

Figure 6 illustrates a flow chart for a step of modeling multiple non-overlapping background scenes in accordance of an embodiment of the present invention.

DETAILED DESCRIPTIONS OF THE INVENTION

Figure 2 illustrates an effectively model for the dynamic scene within a camera view - adaptive region-based modeland method for modeling the dynamic scene due to camera movement (modeling multiple non-overlapping background scene)of the present invention.

This present invention relates to a method for background modeling taking into consideration that the camera is moving. Background model is the best representation of the background scenes, which captures the changes happened in the background over time. The model is then be used to extract moving object in the scene by comparing the image captured by the camera at any time with the corresponding background model. The method of the present invention can be used in surveillance system as part of the processing unit to represent the background scenes. The surveillance system consists of a pan-tilt-zoom (PTZ)camera set to monitor wide area and capture images, an image processing unit to analyze the images and extract information from the images such as the presence of the moving objects or any events(e.g. intrusion, loitering etc.) happening in the scene, a display unit to display the images captured and analyzed output images, and a post event detection unit to trigger the alarm for post action when event is detected by the image processing unit. The alarm can be in any form of sound alarm or message send to the users' mobile phone or email.

In most occasions, the PTZ camera is set to monitor a wide area, which requires combination of multiple single camera view. For example, it has to cover areas from pan angle varies from 10 to 100 degrees. However, in the present invention, the area needs to be covered by multiple individual camera view. It is denoted that the camera view of the PTZ at current position as the current camera view and the surrounding scene as the overall view that is set to be covered by the camera. These notations are illustrated in Figure 2.

In some cases, the area to be covered by the PTZ might not be overlapped to each other. For example, the PTZ is used to monitor two different pre-set areas which are not overlapped. Thus, in this present invention, a method to model the background for this scenario is provided.

At any current camera view, there is a possibility that the background scene is dynamic. Generally, background has to be static, only foreground is moving in the image. However there are background areas that change dynamically such as the area corresponds to water fall, moving leaves etc. Dynamic scene refers to the background scene that is having high variability of intensity over time. In this case, this part of the background area has to be model effectively to capture this variability feature, so that it is not mistakenly assumed as a moving object.

Thus, the present invention provides a method to represent the dynamically changed background scenes using adaptive region-based model and a method to model multiple scenes without the scenes have to be overlapped to each other. The present invention relates to a method for modeling dynamic scene using region-based adaptive statistical learning to model dynamic background within one camera view scene- based modeling to model multiple non-overlapping regions of the background image scene as illustrated in Figure 3. To form a model for dynamic scene, all the images in sequence acquired from a non-static camera. These images are then undergoing a method for modeling current background scene. In the event some of these images are not covered by the step in modeling of current background scene, the current view of the camera also (known as non-overlapping view) will be changed until all the images are covered by the camera. A method for modeling for multiple non-overlapping background scenes is performed until an overall background model is formed.

Figure 4 illustrates a flow chart for a step of modeling current background scene in accordance of an embodiment of the present invention. A time series intensity value for each pixel is acquired from the images captured by the non-static camera. During this period, a frequency for each intensity value for each pixel for each image is updated. An average intensity and an intensity variance for each pixel for each image are computed. The intensity variance for each pixel is determined whether this value is more than predetermined threshold value or not. If the variance for any pixel is less than a threshold value, it is identified as a static pixel. This pixel is modeled as a single scalar value which is the average intensityof the pixel over a period of time.

However, if the intensity variance of each pixel is more than a predetermined threshold value, the pixel is a dynamic pixel. For each dynamic pixel, the neighbouring pixels are identified for any similar dynamic properties. All adjacent dynamic pixels which are having similar dynamic properties are grouped together and the region is represented using a single complex model. To determine whether two pixels are having the similar dynamic property, a density distribution is compared. If the density distribution is overlapped to certain conditions, then the two pixels are belonged to the same dynamic group and a density distribution for each pixel is merged in a same group. After the model has been constructed, the significant feature points are extracted from the background image. The feature points are stored as part of the background model.

Figure 5 illustrates a modeling background scene using adaptive based region model in accordance of an embodiment of the present invention. Figure 5 illustrates the output of the process flow shown in Figure 4. The output is the image pixels segmented into two different regions, either dynamic regions or non-dynamic region. The non-dynamic regions consist of image pixels which correspond to static background pixels. In this case, each of the pixels in this region is represented using a scalar value indicating the average intensity of the pixel over a period of time. The dynamic regions are corresponded to the dynamic background pixels. Adjacent dynamic background pixels having similar background models are grouped together. Since pixels in dynamic background regions exhibit high variability in intensity, it is represented using a statistical model representation. For instance, to use Gaussian mixture model distribution as illustrated in Figure 5a. From the graph, it shows that there are two components of intensity distributions, with average intensity for each component are τ and μ ₂ respectively. It means that if at any time, if the pixel in current frame is having value either iOr μ ₂ , then the pixel belongs to the background. On contrary, if the value of pixel in current frame is outside from these two values, then it concludes that the pixel belongs to foreground pixel. Figure 6 illustrates a flow chart for a step of modeling multiple non-overlapping background scenes in accordance of an embodiment of the present invention. First, each of the non - overlapping background scenes is modeled in accordance to the method as described in Figure 4. Each background scene is stored as a layer. Based on the movement of the camera, the link between one layer to the adjacent layers are constructed. This link defines and determines which background scene is adjacent to another background scene. The linked indexes of each neighboring layer for each background scene are stored and a final background model based on the linked indexes of neighboring layers is formed.

One of the advantages of the a method for modeling dynamic scene using region-based adaptive statistical learning to model dynamic background within one camera view and scene-based modeling to model multiple non-overlapping regions of the background image scene is that it provides better background representation as compared to background modeling using pixel-wise scalar value. Another advantage of the method of the present invention is that a less computational cost is used as compared to background modeling using pixel-wise statistical or kernel density. Furthermore, the method for the present invention has higher sensitive foreground detection as compared to privacy mask concept (masking out dynamic region from being modeled).

The foregoing embodiment and advantages are merely exemplary and are not to be construed as limiting the present invention. The description of the embodiments of the present invention is intended to be illustrative and not to limit the scope of the claims and many alternatives, modifications and variations will be apparent to those skilled in the art.