TRAJECTORY INFORMATION

FIELD OF INVENTION

The present invention relates to a system and method to detect human loitering activity using trajectory information.

BACKGROUND ART

Loitering refers to the act of standing idly about or lingering aimlessly around a specific area. The present invention is based on measurement of the duration of loitering by human subjects in specific area, known as region-of-interest.

Presently, video surveillance systems are used in public areas for security or surveillance. The cameras are typically monitored by humans with the intent that the operators can recognize a suspicious event when it happens. However, as number of cameras increases, it becomes more challenging to monitor individual video streams. Further, research studies have shown that attention time of a human is limited to an average duration of 20 minutes, which further supports the ineffectiveness of human monitoring.

The present invention establishes a method and system for determining loitering event using variety of loitering thresholds. The present invention is directed to recording motion of object, deleting background scenes and triggers alert when loitering event is detected.

The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one exemplary technology area where some embodiments described herein may be practice. SUMMARY OF INVENTION

The present invention provides a system (800) for detecting loitering event in region of interest. The system comprises at least one background subtraction module (802), at least one motion tracking module (804), at least one time-span analysis module (806), at least one trajectory analysis module (808), at least one loitering detection module (810) and at least one decision making module (812). The said background subtraction module (802) determines presence of at least one object in region of interest by grouping motion pixels having similar properties and eliminating noise generated by motion pixels. The said motion tracking module (804) tags each object of interest with identity by comparing each object to at least one predetermined property including colours, spatial or geometric information and identifying each object based upon at least one predetermined property. The said time-span analysis module (806) computes total timestamp for each object of interest by classifying each object as normal or suspicious using a plurality of loitering thresholds, establishing first threshold value T _s to determine loitering event depending on speed and trajectory of each object, determining loitering event when total time span for object t in region of interest is greater than first threshold value T _s , establishing second threshold value T _L to determine loitering event depending on speed and trajectory status of each object and determining loitering event when a total time span for object t in region of interest is greater than second threshold value T _L .

Further, the said trajectory analysis module (808) determines time span of each object of interest based on trajectory information and speed profile by classifying each object as normal or suspicious using historical and/or current trajectory pattern. The said loitering detection module (810) detects loitering event based on combination of plurality of determining features and the said decision making module (812) triggers alert when loitering event is detected.

Another aspect of the present invention provides a method (900) for detecting loitering event in region of interest. The method comprising steps of determining presence of at least one object in region of interest (902), tagging each object of interest with identity (904), computing total time stamp for each object of interest (906), determining trajectory pattern and speed of each object of interest based on trajectory information and speed profile (908), detecting loitering event based on combination of plurality of determining features (910) and triggering alert when loitering event is detected (912).

Preferably, a method for determining presence of at least one object in region of interest further comprises grouping motion pixels having similar properties (1002) and eliminating noise generated by motion pixels (1004). A method for tagging each object of interest with identity further comprises comparing each object to at least one predetermined property including colours, spatial, or geometric information (1102) and identifying each object based upon at least one predetermined property (1104).

Further, a method for applying plurality of loitering thresholds further comprises establishing first threshold value T _s to determine loitering event depending on time span of each object (1202), determining loitering event when total time span for object t, in region of interest is greater than first threshold value T _s (1204), establishing second threshold value T _L to determine loitering event depending on time span of each object (1206) and determining loitering event when a total time span for object ¾ in region of interest is greater than second threshold value T _L and its trajectory status is abnormal (1208). The present invention consists of features and a combination of parts hereinafter fully described and illustrated in the accompanying drawings, it being understood that various changes in the details may be made without departing from the scope of the invention or sacrificing any of the advantages of the present invention.

BRIEF DESCRIPTION OF ACCOMPANYING DRAWINGS

To further clarify various aspects of some embodiments of the present invention, a more particular description of the invention will be rendered by references to specific embodiments thereof, which are illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail through the accompanying drawings in which:

Figure 1 illustrates sample region of interest in the present invention.

Figure 2 illustrates normal trajectory.

Figure 3 illustrates an example of detection of loitering event of a false positive event.

Figure 4 illustrates another example of detection of loitering event of false positive event.

Figure 5 illustrates example where a person is able to walk out of the region in a reasonable amount of time without detection.

Figure 6 illustrates an example of abnormal trajectory.

Figure 7 illustrates an example of false alarm.

Figure 8 illustrates architecture of a vision-based system to detect human loitering activity using trajectory information.

Figure 9 is a flowchart illustrating a method for detecting loitering event in region of interest.

Figure 10 is a flowchart illustrating a method for determining presence of at least one object in region of interest.

Figure 11 is a flowchart illustrating a method for tagging each object of interest with identity. Figure 12 is a flowchart illustrating a method for applying plurality of loitering thresholds.

Figure 13 is a flowchart illustrating a method for classifying each object as normal or suspicious using plurality of loitering thresholds.

Figure 14 is a flowchart illustrating a method for classifying each objet as normal or suspicious using historical and/or current trajectory information pattern. Figure 15 illustrates time-span analysis module.

Figure 16 illustrates trajectory patterns.

Figure 17 illustrates speed profiles.

Figure 18 illustrates combination of trajectory and speed profile information.

Figure 19 is a flowchart illustrating process flow of speed analysis in trajectory analysis module.

Figure 20 is a flowchart illustrating detection of trajectory pattern analysis in trajectory analysis module.

Figure 21 is a flowchart illustrates process flow of loitering detection module.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention provides a system and method to detect human loitering activity using trajectory information. Hereinafter, this specification will describe the present invention according to the preferred embodiments. It is to be understood that limiting the description to the preferred embodiments of the invention is merely to facilitate discussion of the present invention and it is envisioned without departing from the scope of the appended claims.

Reference is first being made to Figure 1 and Figure 2 respectively. Figure 1 illustrates sample region of interest in the present invention. Figure 2 illustrates normal trajectory pattern in Figure 2a wherein the person was able to walk out of the region of interest within predefined threshold time for loitering, T _L . For all of the examples, tj represents total time span for object of interest or person that is spent within the region of interest. In Figure 2b, t _s T _L Hence, no loitering was detected, resulting in true negative and normal trajectory.

Reference is now being made to Figures 3 and 4 respectively. Figure 3 illustrates detection of loitering wherein t, > T _L , resulting in false positive event detection. Loitering is detected because the person was unable to clear or exit the region of interest within the allocated or predetermined time. This may because he was pushing something heavy across the space due to slower walking speed or that he stopped to tie his shoe laces.

Reference is now being made to Figure 5 and 6 respectively. Figure 5 illustrates an example where a person was able to walk out of the region in a reasonable amount of time. However, from its trajectory pattern, it is clear that he had spent a considerable amount of time loitering in the region. However, the loitering alarm was not raised as T _L > tj. An example of an abnormal trajectory is shown in Figure 6 wherein loitering was detected as total time spent within the region was greater than T _L . Table 1 summarizes various combinations of time span with the trajectories, as well as the resultant system output which indicates loitering or no loitering, (c), (e) and (f) are not possible while (d) illustrates some of the shortcomings of the current systems that the present invention seeks to overcome.

Tabtel: Loitering Detection

A threshold value, T _L must be selected carefully in order to minimize number of false alarms. Having a large T _L may allow people to spend considerable amount of time loitering whereas, a short T _L produces false alarms. This phenomenon is illustrated in Figure 7. Further, real-time behaviour of humans in each scenario varies, making it difficult to obtain optimal T _L value and eliminate false alarms completely.

Reference is now being made to Figure 8. Figure 8 illustrates the architecture of a vision-based system that monitors the activities of the object of interest in the scene to detect abnormal events. The system (800) for detecting loitering event in region of interest, comprising at least one background subtraction module (802), at least one motion tracking module (804), at least one time-span analysis module (806), at least one trajectory analysis module (808), at least one loitering detection module (810) and at least one decision making module (812). The said background subtraction module (802) determines presence of at least one object in region of interest by grouping motion pixels having similar properties and eliminating noise generated by motion pixels while the said motion tracking module (804) tags each object of interest with identity by comparing each object to at least one predetermined property including colours, spatial, or geometric information and identifying each object based upon at least one predetermined property.

The said time-span analysis module (806) computes total timestamp for each object of interest by classifying each object as normal or suspicious using a plurality of loitering thresholds, establishing first threshold value T _s to determine loitering event depending on time span of each object, determining loitering event when total time span for object t, in region of interest is greater than first threshold value T _s , establishing second threshold value T _L to determine loitering event depending on time span of each object and determining loitering event when a total time span for object ¾ in region of interest is greater than second threshold value T _L and its trajectory status is abnormal.

The said trajectory analysis module (808) determines time span of each object of interest based on trajectory information and speed profile by classifying each object as normal or suspicious using historical and/or current trajectory pattern. The said loitering detection module (810) detects loitering event based on combination of plurality of determining features while the said decision making module (812) triggers alert when loitering event is detected.

Reference is now being made to Figure 9, Figure 10 and Figure 11 respectively. Figure 9 is a flowchart illustrating a method for detecting loitering event in region of interest while Figure 10 is a flowchart illustrating a method for determining presence of at least one object in region of interest and Figure 1 is a flowchart illustrating a method for tagging each object of interest with identity.

As illustrated in Figure 9, the presence of at least one object in region of interest is determined (902). As illustrated in Figure 10, the method for determining presence of at least one object in region of interest further comprises grouping motion pixels having similar properties (1002) and eliminating noise generated by motion pixels (1004). Thereafter, each object of interest is tagged with identity (904) and total time stamp for each object of interest is computed accordingly (906). As illustrated in Figure 11 , the method for tagging each object of interest with identity further comprises comparing each object to at least one predetermined property including colours, spatial, or geometric information (1102) and identifying each object based upon at least one predetermined property (1104).

Time span of each object of interest is determined based on trajectory information and speed profile (908). Determining time span of each object of interest based on trajectory information and speed profile further comprises classifying each object as normal or suspicious using historical and/or current trajectory pattern.

Subsequently, loitering event is detected based on combination of plurality of determining features (910) and alert is triggered when loitering event is detected (912). Detecting loitering event based on combination of plurality of determining features further comprises classifying each object as normal or suspicious using historical and/or current speed. Further, detecting loitering event comprises combination of responses of time stamp of each object, trajectory pattern and speed or velocity of each object.

Detecting loitering event based on combination of plurality of determining features further comprises applying plurality of loitering thresholds.

Reference is now being made to Figure 12 and Figure 13 respectively. Figure 12 is a flowchart illustrating a method for applying plurality of loitering thresholds while Figure 13 is a flowchart illustrating a method for classifying each object as normal or suspicious using plurality of loitering thresholds. Computing total timestamp for each object of interest further comprises classifying each object as normal or suspicious by applying plurality of loitering thresholds. As illustrated in Figure 12, the method for applying plurality of loitering thresholds further comprises establishing first threshold value T _s to determine loitering event depending on time span of each object (1202), determining loitering event when total time span for object t in region of interest is greater than first threshold value T _s (1204), establishing second threshold value T _L to determine loitering event depending on time span of each object (1206) and determining loitering event when a total time span for object t, in region of interest is greater than second threshold value T _L and its trajectory status is abnormal(1208). The first threshold value T _s and the second threshold value T _L are different wherein the first threshold value T _s is larger than the second threshold value T _L

As illustrated in Figure 13, the method for classifying each object as normal or suspicious using plurality of loitering thresholds further comprises validating each labeled motion pixel with region of interest wherein object having validated pixel will be tagged with time information known as total time span for the object t, (1302). Thereafter, total time span for object t, of validated motion pixel with first threshold value T _s and second threshold value T _L is compared (1304). Subsequently, occurrence of loitering event when total time span for object t, in region of interest is greater than first threshold value T _s is determined (1306) and occurrence of loitering event when total time span for object t, in region of interest is greater than second threshold value T _L wherein if total time span for the object ¾ is not greater than first threshold value T _s and when its trajectory status is abnormal is determined (1308).

Reference is now being made to Figure 14. Figure 14 is a flowchart illustrating a method for classifying each objet as normal or suspicious using historical and/or current trajectory information pattern. As illustrated in Figure 14, upon analyzing trajectory pattern of object (1402), trajectory pattern of object with plurality of normal pattern is compared with abnormal trajectory patterns (1404). Thereafter, occurrence of loitering event is determined when trajectory pattern of object matches at least one abnormal trajectory pattern (1406).

As illustrated in Figure 15, time information or stamp that is associated with each object is compared with two predefined loitering thresholds. Firstly, if the time information of object-of-interest exceeds predefined strict loitering threshold, T _s, object will directly considered as exhibiting loitering threshold. Otherwise, time information will go through second layer of time-span analysis; compared to loose threshold, T _L which is the second threshold value. Only object-of-interest with time-span that exceeds predefined loose loitering threshold will be fed into subsequent module which is trajectory analysis module. Strict loitering threshold is introduced as cut-off factor to directly determine loitering event regardless of time span information. Trajectory patterns, such as, normal and abnormal are illustrated in Figure 16. Similarly, speed profiles are illustrated in Figure 17. Figure 18 illustrates combination of trajectory and speed profile information.

Figure 19 illustrates process flow of speed analysis in trajectory analysis module. Similarly, process flow of trajectory pattern analysis in trajectory analysis module is illustrated in Figure 20. As illustrated in Figure 20, length of moving window may vary according to criticality of the applications. Object-of-interest will be considered suspicious if speed is inconsistent and counter to keep track of level of suspiciousness of object-of-interest will be increased. A plurality of defined rules (rule-based) can be applied to speed feature to determine if speed is suspicious. Otherwise, classification method with speed feature as the input can be applied. Accumulated level of suspiciousness will then be compared against defined speed threshold, Τ _5ρββ ο· If suspicious level exceeds speed threshold, T _speed , object is tagged as exhibiting suspicious speed (Status of object i, Speedj is suspicious).

Thereafter, trajectory pattern is analyzed. Trajectory pattern analysis is similar to the aforementioned speed analysis and is illustrated in Figure 21 wherein Figure 21 illustrates loitering detection module. If trajectory pattern is deemed suspicious according to set of defined rules, object-of-interest is given suspicious status (Status of object I, Trajectory is suspicious) and vice versa. If any of the speed status, Speedi or trajectory status, Trajectory is tagged suspicious; object-of-interest will have suspicious status, StatuSi. The said status is important as it will be analyzed in loitering detection module to infer an outcome.

In the present invention, an object of interested will not be deemed to be loitering event if the associated time stamp exceeds the predetermined threshold unless its trajectory pattern is abnormal. The final module is the decision making module which stores loitering event in database. This allows for post-event analysis on video to further assist with investigations.

The uniqueness of the present invention lies in a system and method for detecting loitering event in a region of interest which includes providing timestamp ¾ of each tracked subject in a predefined region of interest. The present invention determines trajectory pattern for each object. The present invention includes classifying each object as normal or suspicious based upon predetermined loitering threshold time T|_. Loitering event is determined by trajectory pattern, timestamp ti, and predetermined loitering thresholds T _L .

The present invention may be embodied in other specific forms without departing from its essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore indicated by the appended claims rather than by the foregoing description. All changes, which come within the meaning and range of equivalency of the claims, are to be embraced within their scope.