Method to Detect Abnormal Events for Wide View Video Images

Field of Invention

The present invention relates in general to a method to detect abnormal events in images captured by wide view camera through behavioral analysis; and more particularly in dealing with spatial distortion in the video images.

Background of Invention

Surveillance has evolved a long way since the days where videocassettes were used for video surveillance recording. The surveillance technology has grown rapidly and consumers are spoiled for choice with the many gadgets and methods of surveillance available. The common method being in use is the automated video surveillance system.

In automated video surveillance an event is classified as normal or abnormal based on predetermined condition(s) and is displayed automatically in real time. This would then enable the security personnel to take the necessary action. Therefore, there is a need for the surveillance camera to be able to circumscribe as wide an area as possible with the least number of cameras. In order to achieve this, wide view cameras i.e. video cameras with wide-angle lens are used. Wide view camera is able to increase the field of view to up to 225°. This is very useful in achieving a wide coverage, however it is at the expense of the quality of the images acquired. The images have barrel distortion, whereby the image looks like it has been formed around a sphere. Typically, the greater the field of view the greater the distortion, and the more curved straight lines appear.

Most of the surveillance systems cope with the problem of barrel distortion by including a pre-processing step for removing the distortion. This involves additional computational expense and slow execution of the resulting system.

Therefore there is a need for a method to cope with the image distortion and without adding any computational burden to the system.

Summary of the Invention

The object of this invention is to provide a method to detect abnormal event in systems using wide view images.

Another objective of the invention is to provide a system which incorporates image distortion handling strategies as opposed to image distortion correction pre-process.

Brief Description of Drawings

Figure 1 Conventional abnormal event detection system

Figure 2 Example of wide view images

Figure 3 (a) Example of wide - angle view prior to image processing

Figure 3 (b) Example of wide - angle view after flattening Figure 4 Abnormal event detection flow chart

Figure 5 Generation of abnormal events for storage

Figure 6 Abnormal event detection system

Figure 7 Multi classifier configuration for wide angle views

Figure 8 Multi classifier architecture for distorted image

Detailed Description

Figure 1 illustrates a flow chart of a conventional abnormal event system overview.

Live video images of an area of interest for example the area where surveillance is conducted is obtained through image acquisition devices such as surveillance camera(s) coupled wide view lens such as fish eye lens. The images from this type of camera are severely distorted as shown in Figure 1. In most conventional systems the images acquired are pre-processed to correct the spatial distortion caused by the lens. This result is then used as the input for the abnormal event analyzer.

Figure 3 illustrates a flow chart of an abnormal event detection system according to the present invention. Although the images for the present invention are also obtained from

a conventional wide view vide acquisition device, it does not require pre-processing step where the images are corrected of their spatial distortion.

The present invention successfully eliminates the image distortion correction by incorporating image distortion handling strategies at the level of a multi-classifier, namely: during training, classification and fusion. The main essence of this approach is to make the multi classifier savvy of the distortion geometry of the data that it is processing. To achieve this the invention incorporates distortion geometry prior knowledge in three different instances: a. During the training of the multi classifier b. During the execution of the multi classifier c. During the fusion of the multi classifier outputs

This enables the multi classifier to be robust to distortions of wide view images.

Figure 6 illustrates how a multiple classifier can be used in the context of a wide view image. The figure on the right indicates an actual wide view image. The image can be broken into two or more geometric bands. However, for the purpose of discussion the number of bands is limited to three as can be seen in Figure 6. The distortion parameters within a band are sufficiently similar, whereas the distortion parameters between the bands are sufficiently dissimilar. Each geometric band is assigned with a classifier and each classifier region overlaps with an adjacent classifier region in order to smoothen the output of the fusion process. The number and configurations of the classifiers are determined by the distortion characteristics of the image.

As stated earlier each classifier is assigned to a specific band or region with similar distortion parameters. Although the distortion parameters within a band are similar, there still exists a certain margin of variability. Therefore, during the training phase the classifier is trained with a distortion pattern that corresponds to a target object at a specific coordinate/position and also with a range of possible variables of that distortion pattern.

The variations of distortion parameters associated with various objects at different positions are the data with which the classifier will be trained on. These data may vary

depending on the various applications of the system. For example the data for training a classifier for a surveillance system in an airport would not be the same as for a classifier for a surveillance system in a bookstore.

During the execution phase the classifier, which has access to a target object's position is able to retrieve the distortion parameters which are associated with that particular ^~ position, from the data stored during the training phase. This allows the classifier to adapt its functions to be optimized for that set of parameters. In this phase the classifier compares the distorted images obtained with those stored in the database during the training phase. The result from this is used as input for the fusion phase.

The discussion below is directed to an abnormal event detection system that incorporates a multi classifier. Generally any automated surveillance is trained to compare the real time events to events in a database to in order to determine if an abnormal event has occurred.

In most systems it is much easier to create a database consisting of normal events, as abnormal events are widely variable and severely lacks consistency. Whereas, normal events consists of events that are expected to happen at a certain area of interest where the surveillance system is being employed.

Figure 4 indicates the process for generation of abnormal events for storage for the present invention. Live wide view video images are received from the acquisition device and the image data received is processed with data processor such as an ordinary computer. The image data is first processed in the illumination analysis component to recover the true pigmentation information of the images. The background and foreground of the area of interest is then identified through foreground detection. Once the key objects in the area of interest is known the various segments of these objects will be partitioned into the appropriate number of region so as to simplify and/or change the representation of the image into something that is more meaningful and easier to understand. The result of this step will be further processed to identify and select the key features of the object for event analysis and tracking. Upon obtaining the key features of the event, the behavioral characteristics as well as other important details will be

classified through the context sensitive multi classifier before being labeled and stored into the events database.

Once the events database is established, the system can be used in real time abnormal event detection. The abnormal event detection process is very much similar to that of the generation of abnormal event for storage.

The similarity however ends at the step where the key features of the object in the area of interest is obtained and processed. The data resulting from these steps is then compared to the pre-stored abnormal events in the event database, to determine if an abnormal event has occurred. -