FIELD OF INVENTION

The present invention relates to a system and a method for monitoring behaviour. More specifically, the invention relates to a system and a method for monitoring the behaviour of a person in a confined space.

BACKGROUND OF THE INVENTION

Camera surveillance and monitoring system is a crucial part of a security system. Although conventional camera surveillance and monitoring system allows a user to monitor multiple locations at once, usually the user can only evaluate any situation using one camera at a time. There is a need for a camera surveillance and monitoring system that can evaluate behaviour and alert the user about any detected behaviour that can be deemed as aggressive and dangerous, especially in an environment such as a prison. Therefore, many systems for monitoring behaviour have been invented that can perform behaviour analysis and alert the user about any detected behaviour.

A Chinese Patent Publication No. 206921109 U discloses a behaviour analysis system. The system comprises a video capture module, a video display module, an analysis module, a manual module, and an output alarm module. The analysis module performs behaviour analysis on a video stream to determine whether any abnormal behaviour can be detected in the video stream. Any detected abnormal behaviour is then further analysed by a human in the manual module. If the human confirmed that an abnormal behaviour has been detected, an alarm signal is transmitted by the system.

A United States Patent No. US 6678413 B1 discloses a system and a method for finding the position and shape of an object using video. The system uses a decision tree classification for behaviour identification. The system is able to analyse live video stream or recorded video stream.

A United States Patent Publication No. US 2004/0240542 A1 discloses a surveillance system and method for automatic detection of a potential alarm situation. Apart from analysing behaviour, the system is able to assign a parameter to a detected event or object which then can be searched by using the parameter to search for a specific event or object. Therefore, the user is able to retrieve either event associated with an object or objects associated with an event.

Although there are many systems for monitoring behaviour, most systems are designed to monitor open spaces. Therefore, the behaviour analysis in most systems for monitoring behaviour may not be suitable for conditions and parameters of a confined space. Thus, there is a need for a system for monitoring behaviour that is able to solve the aforementioned problems.

SUMMARY OF INVENTION

The invention relates to a system (1000) and a method for monitoring behaviour. The system (1000) comprising a video acquisition unit (300) configured to acquire at least one video stream from a plurality of capturing devices (100), a configuration unit (400) configured to produce configuration information relating to the plurality of capturing devices (100) and video analytics information relating to the plurality of capturing devices (100), a configuration database (401) configured to store configuration information for the plurality of capturing devices (100) and video analytics information, a video decoder (500) converts the at least one video stream into a sequence of still images, and a recording unit (200) configured to record the at least one video stream from the plurality of capturing devices (100). The configuration unit (400) is connected to the video acquisition unit (300). The configuration database (401) and the video decoder (500) are connected to the configuration unit (400). The system (1000) further comprising a video processing unit (600) configured to analyse the at least one video stream for behaviour detection and a smart client unit (700) configured to display and visualise active and inactive monitoring. The video processing unit (600) and the smart client unit (700) are each connected to the configuration database (401) and the video decoder (500).

Preferably, the smart client unit (700) further comprising an event module (701) configured to output the at least one video stream with an overlay of event metadata.

Preferably, the smart client unit (700) further comprising an integrated viewing module (702) configured to display multiple video streams from the plurality of capturing devices (100) with an overlay of event metadata. Preferably, the integrated viewing module (702) enables the user to communicate with a person seen in a live stream.

Preferably, the smart client unit (700) further comprising a forensic module (703) configured to enable a user to search for an event based on at least one keyword and retrieve at least one video stream depicting the event.

A method for monitoring behaviour comprising the steps of acquiring at least one video stream from a plurality of capturing devices (100) or a recording unit (200) by a video acquisition unit (300) and decoding the at least one video stream into a sequence of still images by a video decoder (500). The method further comprising the steps of detecting a plurality of behaviour displayed within the sequence of still images by the video processing unit (600), visualising active monitoring by a smart client unit (700), and visualising inactive monitoring by the smart client unit (700). The plurality of behaviours being detected relates to cell violation, perimeter violation, and presence detection. The active monitoring refers to monitoring at least one real-time live feed video stream from the plurality of capturing devices (100). The inactive monitoring refers to monitoring at least one video stream from a recording unit (200) or a media storage.

Preferably, the step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) further includes the substeps of identifying behaviour displayed within the sequence of still images by a video analytics agent, VAA, creating an alert trigger tag, ATT, for each behaviour that has been identified, recording each ATT in a configuration database (401), generating event clip and snapshot for each ATT, storing the event clip and snapshot for each ATT in the media storage, overlaying the event metadata on the at least one video stream, and transmitting the at least one video stream overlaid with the event metadata.

Preferably, the step of visualising active monitoring by the smart client unit (700) further comprising sub-steps of receiving configuration from the video processing unit (600), receiving an alert trigger tag, ATT, for each detected event, extracting information from the ATT, and displaying the at least one video stream overlaid with the event metadata based on the ATT. The information includes capturing device information and the event metadata

Preferably, the step of visualising inactive monitoring by the smart client unit (700) further comprising sub-steps of receiving at least one keyword for database search from a user, matching the at least one keyword with an alert trigger tag, ATT, in a configuration database (401), displaying search result of the ATT that matches the at least one keyword, selecting at least one event from the search result, retrieving the event clip and event snapshot of at least one selected event from the media storage based on the information from the ATT, displaying the event clip and snapshot of the at least one selected event, and compiling the search result, the event clip, and the event snapshot in a report.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

FIG. 1 illustrates a block diagram of a system (1000) for monitoring behaviour according to an embodiment of the present invention.

FIG. 2 illustrates a flow chart of a method for monitoring behaviour according to an embodiment of the present invention.

FIG. 3 illustrates a flowchart for a step of detecting a plurality of behaviours displayed within the sequence of still images by the video processing unit (600) of the method of

FIG. 2.

FIG. 4 illustrates a flow chart fora step of visualising active monitoring by a smart client unit (700) of the method of FIG. 2.

FIG. 5 illustrates a flow chart for a step of visualising inactive monitoring by the smart client unit (700) of the method of FIG. 2. DESCRIPTION OF THE PREFERRED EMBODIMENT

A preferred embodiment of the present invention will be described hereinbelow with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the description with unnecessary detail.

An initial reference is made to FIG. 1 which illustrates a block diagram of a system (1000) for monitoring behaviour according to an embodiment of the present invention. The system (1000) is configured to produce an alert signal if at least one person being monitored in a confined space exhibits a certain number of prohibited behaviours. The prohibited behaviours are predetermined by an operator of the system (1000). The system (1000) comprises a recording unit (200), a video acquisition unit (300), a configuration unit (400), a configuration database (401), a video processing unit (600), and a smart client unit (700).

The recording unit (200) is connected to a plurality of capturing devices (100). The capturing devices (100) may be in the form of an analogue, digital, or smart video camera. The recording unit (200) records at least one video stream from the capturing devices (100).

The video acquisition unit (300) is connected to the capturing devices (100). The video acquisition unit (300) acquires at least one video stream from the capturing devices (100). Alternatively, the video acquisition unit retrieves at least one recorded video stream from the recording unit (200). The video acquisition unit (300) is connected to the configuration unit (400).

The configuration unit (400) produces configuration information relating to the capturing devices (100) and video analytics for the capturing devices (100). The configuration information for the capturing devices (100) includes but not limited to the location of the capturing device, Internet Protocol address, and MAC address. The configuration information relating to the video analytics for the capturing devices (100) is produced by determining suitable video analytics types with respect to the capturing devices (100), monitoring area that includes a region of interest and line of interest, and video analytics parameter values. Examples of video analytic types are motion detection, object recognition, and video tracking. The configuration unit (400) is connected to the video decoder (500) and the configuration database (401). The configuration information for the capturing devices (100) and video analytics information is sent to the configuration database (401). The configuration database (401) stores configuration information for the plurality of capturing devices (100) and video analytics information.

The video decoder (500) converts the video stream into a sequence of still images. The video decoder (500) is connected to the video processing unit (600).

The video processing unit (600) is connected to the configuration database (401). The video processing unit (600) receives the still images from the video decoder (500) and video analytics information from the configuration database (401). The video processing unit (600) analyses the still images for behaviour detection based on the video analytic type assigned to the video stream by the configuration unit (400). The video processing unit (600) analyses the type of behaviour showed in the video stream and determined whether any detected behaviour is deemed to be against the security and safety policy of the building.

The video processing unit (600) further comprises a plurality of video analytics agents, VAA, configured to analyse the video stream for any behaviour that is considered as cell violation, perimeter violation, and presence detection. Each capturing devices (100) is assigned a VAA for detecting cell violation, a VAA for detecting perimeter violation, and a VAA for presence detection.

Cell violation is any safety violation that happens within a cell or confined area. Action that constitutes cell violation includes but not limited to climbing, loitering, aggressive behaviour or fighting and sudden stand-up. Loitering is considered as cell violation only if the loitering happens for longer than a predetermined period.

Perimeter violation is any safety violation that happens within a building except the cell or confined area. Perimeter violation includes but not limited to entering a predefined restricted area, moving against a valid direction, and crowding behaviours.

Presence detection constitutes direction detection and appearance detection. Direction detection involves observing any people entering and leaving a monitored area while appearance detection involves monitoring movement of people inside the monitored area. By detecting the presence of people at various location, a statistic of traffic flow of people is generated.

The video processing unit (600) outputs event metadata regarding detected behaviour in the video stream. Any behaviour that is considered as cell violation, perimeter violation, and presence detection is considered as an event. Examples of the event metadata include but not limited to a bounding box of a detected object or behaviour, timestamp of a detected event, and event alert message. The event metadata is stored in the configuration database (401). The video processing unit (600) also overlays the event metadata onto the video stream. The video processing unit (600) creates an event clip and event snapshot of detected behaviour and stores the event clip and event snapshot in a media storage. The event clip is a video clip that only shows the portion of the video stream that contains the detected event. The event snapshot is a sequence of still images that shows the detected event. The video processing unit (600) is connected to the smart client unit (700).

The smart client unit (700) displays and visualises active and inactive monitoring. Active monitoring refers to monitoring a real-time live feed video stream while inactive monitoring refers to monitoring a video stream from an archive such as the recording unit (200) or the media storage. The smart client unit (700) visualises the detected behaviour analysed by the video processing unit (600) by interpreting the event metadata assigned to the video stream by the video processing unit (600). The smart client unit (700) displays the video stream with an overlay of information. The smart client unit (700) also enables the user to receive an alert, communicate, and search a database for a specific event.

The smart client unit (700) further comprises three modules. The modules are an event module (701), integrated viewing module (702), and a forensic module (703). The event module (701) outputs the video stream with an overlay of the event metadata. The event metadata includes but not limited to alert notification message, visual alert alarm, audio alert alarm, and monitoring area. The event module (701) also displays the location of the capturing devices (100) that detect an event by showing the location of the camera within a 3-dimensional model of the building. The event module (701) also highlights the location of the cell which has the detected event. The user is able to obtain information about a detainee occupying each cell.

The event module (701) also summarises all detected events in a timeline sequence. Each detected event is summarised into an event detail. The event detail includes a snapshot of the detected event, date, time, location of the event, and the type of event. The event module (701) also enables live reporting and charting of statistical information of the event while depicting the distribution of detainees. The event module (701) is connected to the integrated viewing module (702).

The integrated viewing module (702) retrieves video stream overlaid with the event metadata and other information from the event module (701). The integrated viewing module (702) displays multiple video stream from multiple capturing devices (100) with an overlay of the event metadata. When an event is detected, the integrated viewing module (702) visualises and displays the location of the capturing device (100) on a 2-dimensional map and event details with event snapshot. The integrated viewing module (702) also enables the user to verbally communicate with a person seen in a live stream.

The forensic module (703) enables the user to use at least one keyword to search for an event and retrieve the video stream depicting that event. The forensic module (703) matches the keyword with the event metadata stored in the configuration database (401) and retrieves the event clip and event snapshot from the media storage. The forensic module (703) is connected to the configuration database (401).

Reference is now made to FIG. 2 which illustrates a flow chart of a method for monitoring behaviour according to an embodiment of the present invention. The method detects a plurality of behaviours that violates security and safety policy of a building. The behaviours are detected by tracking the action of at least one object. Initially, at least one video stream is acquired by the video acquisition unit (300) as in step 3100. The video stream may be acquired either directly from the capturing devices (100) or from the recording unit (200).

The video stream is then converted into a sequence of still images by the video decoder (500) as in step 3200. A plurality of behaviour displayed within the sequence of still images is then detected by the video processing unit (600) as in step 3300. The video processing unit (600) retrieves the configuration information and video analytics information from the configuration database (401). Based on the video analytics type assigned to the video stream, behaviour detection is performed by the video processing unit (600). Any behaviours detected in the video stream is analysed to determine whether the behaviours are against the security and safety policy of the building. The event metadata regarding any behaviours detected in the video stream is produced by the video processing unit (600). The step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) is further described in relation to FIG. 3.

The smart client unit (700) visualises active monitoring as in step 3400. Active monitoring refers to monitoring the video stream in real time. The event metadata regarding any behaviours detected in the video stream is interpreted by the smart client unit (700). The video stream is displayed by the smart client unit (700) with the overlay of the event metadata regarding detected behaviours. Preferably, a plurality of video stream is simultaneously displayed by the smart client unit (700). The step of visualising active monitoring by the smart client unit (700) is further described in relation to FIG. 4.

The smart client unit (700) visualises inactive monitoring as in step 3500. Inactive monitoring refers to monitoring the video stream from the recording unit (200) or the media storage. The media storage contains a video clip or snapshot of detected behaviours. Specific detected behaviour may then be searched by the keyword by the user. The step of visualising inactive monitoring by the smart client unit (700) is further described in relation to FIG. 5.

Reference is now made to FIG. 3 which illustrates a flowchart for the step of detecting the plurality of behaviour displayed within the sequence of still images by the video processing unit (600) of the method of FIG. 2. Initially, any behaviour that is considered as cell violation, perimeter violation, and presence detection displayed within the sequence of still images is identified by the VAA as in step 3310. If any behaviour that is considered as cell violation, perimeter violation, and presence detection is detected, the video processing unit (600) creates an alert trigger tag, ATT, as in step 3320. The ATT contains the information on the type of detected behaviour and the information of the capturing device (100) that produced the video stream that shows the detected behaviour. Each ATT is recorded in the configuration database (401) as in step 3330.

The event clip and snapshot of detected behaviour for each ATT is generated as in step 3340. The event clip is a video clip that only shows the portion of the video stream that contains detected behaviour. The event snapshot is a still image that shows the detected behaviour. The event clip and snapshot is stored in the media storage for record purposes as in step 3350. An address of the event clip and snapshot is tagged to the ATT. The address is a location of the event clip and snapshot within the media storage. The address enables the smart client unit (700) to retrieve the event clip and snapshot from the media storage.

All event metadata from detected behaviour are then overlaid on the video stream as in step 3360. The event metadata includes but not limited to the bounding box of detected object and behaviour, timestamp, and event alert message.

The video stream overlaid with the event metadata is then transmitted to the smart client unit (700) as in step 3370.

Reference is now made to FIG. 4 which illustrates a flow chart for a step of visualising active monitoring by the smart client unit (700) of the method of FIG. 2. Initially, the smart client unit (700) receives configuration information of the capturing devices (100), video analytics information, and the video stream overlaid with the event metadata from the video processing unit (600) as in step 3410.

The smart client unit (700) then receives the ATT for each event from the video processing unit (600) as in step 3420. If a behaviour is simultaneously detected in multiple video streams from multiple capturing devices (100), the smart client unit (700) receives multiple ATT from the video processing unit (600). The user is able to view any behaviour that has been detected from multiple points of view if the behaviour is simultaneously detected in multiple video streams from multiple capturing devices

(100).

The information within the ATT is then extracted as in step 3430. The information includes the event metadata and capturing device information.

The video stream overlaid with the event metadata is then displayed as in step 3440. The video stream overlaid with the event metadata is displayed in either an integrated viewing mode or a high-resolution mode. In the integrated viewing mode, a plurality of video stream overlaid with the event metadata is displayed simultaneously. In the high-resolution mode, only one video stream overlaid with the event metadata is displayed at a time.

Reference is now made to FIG. 5 which illustrates a flow chart for a step of visualising inactive monitoring by the smart client unit (700) of the method of FIG. 2. Initially, the keyword for database search is received from a user as in step 3510. The keyword may include the date, time, location of the capturing device (100), IP address of capturing device (100), and type of detected behaviour. The keyword is then matched with the ATTs in the configuration database (410) as in step 3520.

A search result is displayed as in step 3530. The search result is a list of ATTs that matches the keyword.

At least one event from the search result is selected by the user as in step 3531. The user may select from the search result any event that the user wants to investigate. The event clip and event snapshot of the selected event are retrieved from the media storage based on the information from the ATT as in step 3532.

The event clip and the event snapshot of the selected event are displayed as in step 3540. The event clip and event snapshot may be displayed in the integrated viewing mode, wherein all event clip and event snapshot from multiple capturing devices (100) that shows a similar detected event may be displayed simultaneously. Alternatively, only one event clip or event snapshot is displayed at a time.

The search result, the event clip, and the event snapshot are then compiled in a report as in step 3550.

While embodiments of the invention have been illustrated and described, it is not intended that these embodiments illustrate and describe all possible forms of the invention. Rather, the words used in the specifications are words of description rather than limitation and various changes may be made without departing from the scope of the invention.