FIELD OF THE INVENTION

The present disclosure relates to the technical field of area monitoring systems. Using at least one sensor, a system and method of analyzing and interpreting the activity within an area is described.

BACKGROUND

The monitoring of a designated area is used routinely to identify activity by detecting certain criteria via motion, sound, and other various inputs. An important application of these methods is in the home security industry where such technology is utilized to detect human movement and activity.

Legacy monitoring systems consist of one or more sensors and a common control circuit that notifies a monitoring service or person of a change in status in the sensor. These systems often include contact-sensors to detect door and window openings and motion sensors to detect motion inside a premise. Additional sensors may be distributed throughout the premises. The outcome is a simple "on/off type functionality stating whether the door or window has been breached. These types of monitoring systems typically require human intervention to understand the meaning of the event that just occurred within the sensor network. More sophisticated monitoring systems may incorporate more advanced sensors such as sound and video surveillance to complement or substitute for the above traditional sensors and gain additional information over the monitored area; however these monitoring systems still rely on human intervention to determine the cause of the event.

The problems of legacy area monitoring systems can be narrowed to focus on three deficiencies: (1) the difficulty of installing sensors to ensure the appropriate coverage, (2) the ability to identify the meaning and the appropriate required action relating to any detected event, and (3) the large initial cost for the infrastructure. Speaking to the first point, the difficulty in traditional systems lies in installation where contact sensors require fixing at each potential entry point, including doors and windows. Other sensors such as passive infrared (PI ) motion sensors, sound sensors, or video sensors must be installed with a certain level of expertise to avoid dead spots and false alarms. Furthermore, in a wired arrangement, walls of the building will need to breached in certain cases to install the electrical wire throughout the premises.

With respect to the second point, traditional systems do not attempt to identify the activity tied to any event but rather provide binary information - whether a sensor has been tripped or a signal threshold has been exceeded. In such systems it is generally up to a person at the monitoring service to interpret this information; that is, the lack of intelligence in the system requires the application of human intelligence. False positives caused by animals, wind and many other factors are often identified as being problems using traditional area monitoring systems. Without appropriate information about an event, it is difficult to determine the proper response. In the case of a traditional security monitoring system, this can result in unnecessary calls to public emergency services (which often results in a charge to the home owner) due to false alarms.

The third problem with traditional systems is that for most systems requiring sensors on every window and door and motion detectors in several rooms, the amount of equipment can quickly accumulate. Furthermore if such systems are wired, the cost of wiring and installation of the same can cause a sizable increase in costs incurred. Depending on the size of the house, the cost will scale accordingly.

Therefore there is a need for an area monitoring system that provides for ease of installation, diagnosis of correct events captured by system, and a more cost-effective system at installation.

This background information is provided to reveal information believed by the applicant to be of possible relevance to the present invention. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present invention. SUMMARY OF THE INVENTION

An object of the present invention is to provide an Area Monitoring Method and System. In accordance with an aspect of the present invention, there is provided an area event monitoring system comprising one or more sensors configured to detect a signal relating to an environmental change in the area, and a sensor activity processor associated with each of the said one or more sensors, the sensor activity processors being configured to receive the signal regarding the environmental change from its respective sensor and correlate the signal to a hypothetical activity associated with the environmental change, and an area event processor configured to receive the hypothetical activity from each of the sensor activity processors to evaluate the input to determine the event occurrence.

In accordance with another aspect of the present invention, there is provided a method of monitoring events in an area, detecting a signal relating to an environmental change in the area using one or more sensors, and receiving signals regarding the environmental change from its respective sensor said one or more sensors, a sensor activity processor correlates the input to determine a hypothetical activity associated with the environmental change, and receiving the hypothetical activity from each of the sensor activity processors, an area event processor determines the probability of an event occurrence.

BRIEF DESCRIPTION OF THE FIGURES

Figure 1 illustrates a system for area event detection with sensors connected to one or more computing devices.

Figure 2 illustrates a system for area event detection with each Sensor Activity Processor capable of communicating with other Sensor Activity Processors.

Figure 3 illustrates a system configuration where the computing devices are distributed and output includes a legacy alarm system.

Figure 4 illustrates a system configuration where the computing devices are integrated. Figure 5 illustrates a system configuration where the computing devices utilize a cloud service.

Figure 6 illustrates components of a Sensor Activity Processor.

Figure 7 illustrates components of a Microwave Doppler Radar (MDR) Sensor Detector.

Figure 8A illustrates one example of identification of power ratio data output from the MDR Discriminator.

Figure 8B illustrates another example of identification of power ratio data output from the MDR Discriminator.

Figure 9 illustrates probability distribution for power ratio for common types of objects from data output from the MDR Discriminator.

DETAILED DESCRIPTION OF THE INVENTION

Definitions

As used herein, the term "about" refers to a +/-10% variation from the nominal value. It is to be understood that such a variation is always included in a given value provided herein, whether or not it is specifically referred to.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

System of Area Monitoring

Disclosed herein are a method and a system of area monitoring. The system collectively analyzes information from one or more sensors to more accurately determine the activity occurring within an area. The system is a platform, providing for the incorporation of multiple types of sensors and detection algorithms, each having the ability to contribute to the monitoring of area activity. One or more sensors, such as Microwave Doppler Radar (MDR), Infrasonic (IS) and Passive Infrared (PIR), are combined with enhanced signal processing and analytics to analyze monitored signals.

The enhanced signal processing occurs at two distinct levels within the system. Each sensor is attached to its own corresponding signal processing unit which analyzes the signal and provides data output for a second layer of signal processing. The second layer of signal processing is distinct from the first layer as the inputs include the aggregate data output from the first level of signal processing. Therefore, the second layer of signal processing performs analysis based on a variety of sensor data and can come to a more thorough conclusion for correct event detection.

Both levels of enhanced signal processing utilize various mathematical algorithms and techniques including, but not limited to, logic, fuzzy logic, linear, nonlinear, deterministic, probabilistic, static, dynamic, discrete, continuous, deductive, inductive or floating mathematical models.

The system processes information in real-time to provide immediate feedback to the end user for event detection. Once a sensor picks up a signal, the signal would undergo immediate analysis and be combined with analysis of any other sensor which may have picked up the same event. Once the outcome has been determined by the system, the event would be recorded, and the end user would be immediately notified if the event is judged to be of importance to the user.

The system operates in a "noisy" environment whereby there is no control mechanism to isolate external behaviour in order to gain additional clarity for result determination. The system functions in environments where many activities occur simultaneously (e.g. household). Therefore, through use of advanced algorithms and techniques, the system can distinguish between different activities and deliver correct feedback with respect to event detection.

The system can either be used as a stand-alone area monitoring system or integrated with components of a legacy monitoring system. In configurations with legacy system integration, the legacy system would either function as one of several distinct sensors connected to system as a whole, or the system as a whole could function as an input sensor for the legacy system. System Configurations

Figure 1 represents one configuration of the system wherein the information flows in an area monitoring system with one or more sensors. The system is composed of a series of sensors identified by the numbers 1 to N (1). Each sensor is sending sensor information (3) to a Sensor Activity Processor (5). This processed information (7) from each sensor is sent to the Area Event Processor (9) which evaluates the information from the N sensor(s) and the result is an Output (11) describing one or more events within the sensor network. The analysis process is done with one or more computing devices. A computing device refers to the component, or collection of components, in the system where the signal analysis is performed. The computing device consists of, at minimum, the Sensor Activity Processor. However, this criterion may be expanded to include all of the components in the system including various sensors and Area Event Processors. The information from the sensor to various processes can be carried out through a wire or wireless connection (including IP) and the transfer of information can be facilitated by a common communication protocol, for example the TCP IP protocol.

An alternative configuration of the system can be seen in Figure 2. The system may be configured such that a Sensor Activity Processor (16) may send information^ 5) to other Sensor Activity Processors (16). This may result in a more accurate finding as each Sensor Activity Processors has unique input data to perform signal analysis. The communication between Sensor Activity Processors is not restricted by Computing Device configurations. Therefore, whether all Sensor Activity Processors are within the same computing device, or are in distinct computing devices, the information can still be communicated by conventional means such as TCP/IP. The Area Event Processor (18) takes the aggregate output data from each Sensor Activity Processor to perform the final signal analysis to record and output event detection to the end user.

In an alternative configuration of the system, an implementation of the system as seen in Figure 3 can be realized. The system and method is applicable to any combination of sensors and computing devices. The only limitation is that the sensors and computing devices are connected with either wired or wireless connections. Figure 3 is an example of a system composed of Sensors (200). The information from each Sensor (Sensor 1, Sensor 2, Sensor N - series denoted Sensor^) is sent to its associated Sensor Discriminatori_ _N (220), Sensor Detectori_ _N (230) and Sensor Activity Analyzeri_ _N (240). For each sensor, the corresponding Sensor Discriminator, Sensor Detector, and Sensor Activity Analyzer are contained within the same Computing Devicei_N (210). The Sensor Activity Analyzer information (250) is sent to the Area Event Processor in the same or a separate Computing Device (270). The information from the Area Event Processor is sent via a connection (280) to a legacy alarm system (290). Examples of the application of systems in this configuration may include Home Security or Business Security. In such a system, the sensor and sensor detectors are modular and thus can be added to a monitored area on an as-needed basis. Secondly, if there is a need for a specific type of Sensor, the system is flexible enough to allow for the addition of one or more discrete Sensors and the corresponding systems for those Sensors. The Area Event Processor can be a separate unit or integrated with the legacy alarm system.

In an alternative configuration of the system, the computing device includes all discrete Sensor Detectors, Sensor Discriminators, and Area Event Processors. As seen in Figure 4, in this configuration, each Sensori_ _N (300) is connected (310) to the same computing device (320). The computing device can include a Sensor Discriminatori_ _N (330), Sensor Activity Detectorsi-N (340), Sensor Activity Analyzersi-N (350) as well as an Area Event Processor (350). After the appropriate processing, a decision process of the Area Event Processor (360) can trigger an output. This information can be sent (370) through a communication connection (380) such as the internet or wireless connection for further processing by an external system. An example of an application based on this type of system configuration is a portable, or remote, monitoring system. Since all the processing is carried out within one computing device, the system can be moved to any location. This portability is important for applications such as temporary installations where the monitoring system can be easily moved from one location to another. One example application of such a system configuration is for monitoring the activity in a parked car. For example, the combination of a MDR sensor, an IS sensor and a pressure sensor coupled with a computing device that runs the system or method can be used to monitor the activity within a car. A change in air pressure correlated with a detection from the IS sensor and a detection from the MDR sensor can indicate an event is occurring within with the car, such as unauthorized entry. Another example would be for monitoring activity in a hotel room, where the end-user is temporarily residing. This example exemplifies the portability and ease of installation of the system.

Figure 5 represents another alternative of an example of the system where the computing devices are in a remote location. Sensor information is carried via a wired or wireless connection (410) to the internet (600). The information is communicated using a standard protocol such as TCP/IP. In some embodiments, the information is subjected to a secure authentication process whereby the information relayed is in an encrypted and secure format. The Sensor Discriminatorsi_ _N (430), Sensor Activity Detectorsi-N (440), Sensor Activity Analyzersi-N and the Area Event Processor (460) reside on a computing device, physical or virtual computing service, on the Internet Cloud (480). The output from the Area Event Processor is transmitted (490) through a communication connection (500) to the end user or for further processing. An application for such a system configuration could be in the area of shopper traffic monitoring. In such applications, sensor information from a network of retail stores can be transferred via the internet for analysis using a cloud service. The system processes the information from each sensor to reach a decision about the nature of customer traffic within the stores.

Sensor Activity Processor

In Figure 6, the information flow among various components within the Sensor Activity Processor is summarized.

The Sensor Activity Processor (21) analyzes and interprets the signal from a sensor to detect one or more activities. The Sensor Activity Processor is composed of three components: the Signal Discriminator (24), one or more Activity Detectorsi-χ (28) and a Sensor Activity Analyzer (32).

In the system and method, sensor information (22) is passed to the Sensor Activity Processor (21). The Signal Discriminator (24) transforms the sensor signals to an appropriate form, which can then serve as inputs (26) to one or more Activity Detectors (28). The Activity Detectors process the Signal Discriminator outputs and identify the activity based on specific criteria, then outputs information (30) to the Sensor Activity Analyzer (32). The Sensor Activity Analyzer combines all the information (30) from each Activity Detector and based on one or more criteria to create an output (34) describing the activity for this sensor (36).

Signal Discriminator

A Signal Discriminator is the first component of the Sensor Activity Processor. The signal from each sensor is filtered through its respective Signal Discriminator to ensure that only pre-qualified signals are passed along to one or more respective Activity Detectors. The signal from a sensor can be analog or digital.

One or more algorithms or instructions on a computing device that identifies and modifies the sensor signal are implemented such that the signal meets one or more criteria. These algorithms can be implemented in software (as instructions for the computing device) or in hardware (in the form of a printed circuit). The role of the Signal Discriminator is to restrict the sensor information to only that which is pertinent to the activities which the Sensor Activity Processor is intended to detect; this is done to improve accuracy.

Activity Detector

An Activity Detector is the second component of the Sensor Activity Processor. There may be multiple Activity Detectors for each Sensor Activity Module whereby each Activity Detector is comprised of a set of instructions that analyze and characterize the sensor signals received from a Sensor Discriminator to detect specific activities.

For each sensor, the Sensor Discriminator can send its information to one or more Activity Detectors since a sensor signal can be pertinent to multiple activities. For example, for an MDR sensor, the MDR signal can be processed by an MDR Pet Detector and an MDR Door detector, among others. The Activity Detector uses information from the Sensor Discriminators to improve the measurement of probability and confidence of a hypothesis.

The hypothesis testing component combines the probability, confidence intervals and meta-data from each measurement module (ex. stride frequency, signal comparison, power ratio, and other modules) to calculate a probability measurement and confidence interval for human locomotion based on the information provided by the MDR sensor. The calculation can also be expanded by one or more criteria (ex. Dynamic weighting based on confidence measures, machine learning and the context of the measurements).

These measurements are sent to the Activity Analyzer which can then use this information to evaluate competing hypotheses.

Sensor Activity Analyzer

For each sensor, a Sensor Activity Analyzer processes results from multiple Activity Detectors to reach a description of the activity for that sensor. An example of the method or process that can be used to combine the information from different Activity Detectors is through hypothesis testing. The hypotheses presented by all the Activity detectors are evaluated by the Sensor Activity Analyzer using one or more methods of statistical analysis (for example, but not limited to, Bayesian analysis).

Optionally, signature libraries and machine learning methods can be used to choose the hypothesis which best fits the available evidence. This choice represents a decision being made by the analyzer. The system has the ability to determine the history of events and categorizations and utilize this data as input for probability and likelihood.

The utility of combining these diverse detectors is that in an uncontrolled, real-world application, at various times one or more of the detectors may give an erroneous probability measure or a low confidence to a certain hypothesis, for example the Human Locomotion hypothesis. Examples include when a human walks behind a piece of furniture or pauses. Detector analytics can discount one or more of the detector measurements by lowering its weighting, if the confidence measure is low. For example, in a MDR Sensor System, there may be a MDR Locomotion Detector, MDR Pet Locomotion Detector, a MDR Door Detector and a MDR Null Detector, among other possible detectors. Each MDR Detector sends a probability and confidence of an event to the MDR Activity Analyzer. The MDR Activity analyzer uses a statistical method to determine the most likely hypothesis of motion detected by the MDR sensor.

Select Type of Sensors Implemented

The following comprise a list of some of the prevalent sensors utilized in the described system and method. This list is not an exhaustive list wherein the system can utilize any type of sensor provided it is compatible with the system; this would include using a legacy security system as a sensor in some applications.

Microwave Doppler Radar (MDR) Sensor

Microwave Doppler Radar (MDR) is a sensor which utilizes a comparative analysis of frequency between the generated microwave signal and returning microwave signal to calculate velocity characteristics of objects within the area. There is a multitude of applications for such a sensor.

In one instance, the sensor can be utilized to characterize door activity by detecting a door open or close motion by the metrics of the velocity and time spectrum signature. Distinguishing features include: velocity of the door edge; spread of velocities from all parts of the door face. Additionally door metrics can be measured by identifying a particular door or door type by one or more of: duration of motion; distance (by amplitude); angle of door and hinge to sensor (velocity profile); amplitude profile over the motion duration. Swinging motion can also be detected by distinguishing doors from other hinged motions with similar signatures by one or more of: velocity spread (hinged vs. unhinged); duration; distance covered by the motion; presence of locomotion limb components.

In another instance, the sensor can be utilized to characterize gestures from humans and animals. Low-velocity signals, short-duration motion associated with a stationary (e.g. sitting) human or animal that is moving its head, limbs, etc. may be used for analysis. Stride classification may be utilized to classify the type of moving being. For example, children tend to have a faster stride and sudden accelerations while seniors typically have a slower stride.

In another instance, the sensor can be utilized to characterize locomotion location and type. Passers-by exterior to the monitored area are distinguished by their amplitude and mirror-image time profile of approach followed by departure. This identifies both vehicles and people, whether approaching the dwelling (e.g. mail delivery) or travelling past on a tangential trajectory.

Microwave Doppler Radar (MDR) Discriminator & Detector

In the case where the sensor is a Microwave Doppler Radar and the activity that the sensor is designed to detect is that of locomotion (such as human movement, gait analysis, animal movement, or the like), the sensor signal in this system and method is processed to extract information relating to locomotion. This can be achieved, for example, by having the MDR Discriminator eliminate excess information arising from noise and non- locomotion signal energy from machinery (such as fans), from inanimate objects (such as toys and thrown objects) or motion too low in velocity or amplitude to be relevant. These are accomplished by use of various signal filtering techniques and mathematical algorithms such as Power Ratio measurements.

As a result, the MDR Discriminator reduces the instance of false positive and false negatives when the sensor signal is processed by the MDR Activity Detectors.

Figure 7 illustrates the components of an MDR sensor detector. The following is an example of the type of processes and functions carried out by an MDR Activity Detector. In this example, the MDR sensor provides the following information when one or more criteria which are applied separately or in combination have been satisfied, such as: if the input voltage is greater than a pre-set threshold; or if the sensor signal pattern indicates activities such as limb oscillation or door motion; or if the signal-to-noise ratio (SNR) is greater than a preset threshold.

Depending on requirements of the MDR Activity Detectors, the MDR Discriminator can add additional criteria to focus the MDR sensor signal on the description of limb oscillations. Examples of such additional criteria include: application of a low-pass filter (up to about 0.5 km/h, as represented by the equivalent Doppler shift for the MDR base frequency being employed); or measuring the number of times for which the measured voltage crosses between positive and negative ("zero crossings") wherein if the number of zero-crossings is between about 1 and 5, then there is a positive signal for detection or limb oscillations (zero-crossing is a common metric used in the wavelet method of digital signal processing (DSP)).

Each criterion can be implemented using software (as a set of computer instructions) or through hardware (as a computer circuit that implements the same rules and restrictions). Once the sensor signal has been analyzed and modified by the MDR Discriminator, the information is sent to the appropriate Activity Detector.

Infrasonic (IS) Sensor

An infrasonic (IS) sensor detects very low frequency sounds which are inaudible to humans. Such sounds are typically building structure resonances stimulated by footsteps, door closings and other activity. The IS sensor can be constructed from a microphone with useable sensitivity down to 5 Hz or lower. This is achievable with most commercial microphone elements.

In one instance, footsteps can be characterized by identifying sequence of resonances whose periodicity is in the range associated with human or animal gait (stride frequency).

In another instance, locomotive direction can be characterized by identifying footstep resonances (per previous description) whose amplitude increase or decrease over time (positive or negative derivative), indicating whether the human or animal whose radial motion is toward, away or normal to the sensor.

In another instance, footsteps can be characterized by identifying sequence of resonances whose periodicity is in the range associated with human or animal gait (stride frequency). In another instance, variance in wind can be characterized as randomly-timed low- amplitude resonances that correlate with weather.

In another instance, objects falling can be characterized as high- amplitude resonance that is isolated in time from other resonances is correlated with a person or heavy object falling to the floor, or forced entry by an intruder.

In another instance, operations of doors can be characterized by interpreting resonance excited by a door when the latch is operated or when the door strikes the jamb stop.

Infrasonic (IS) Discriminator

The IS Discriminator removes unwanted audio signal energy with a bandpass filter for the IS band from 0 Hz up to about 40 Hz and tracking IS power in real-time. Human activity, due to its unique impact on the building structure, have time and frequency profiles that distinguish them from other IS signal energy. As a result, the IS Discriminator reduces the instance of false positives and false negatives by the IS Activity Detectors.

An example of an implementation of an IS Discriminator for an IS sensor is as follows. The IS Discriminator passes the IS signal to the IS Activity Detector based on multiple criteria separately or in combination, such as: Signal is above an amplitude threshold at or near a specific frequency (resonance); A temporal pattern of IS signal energy, such as a periodic pattern; The signal-to-noise ratio (SNR) is greater than a pre-set threshold;

Those criteria can be implemented using software (as a set of computer instructions) or through hardware (as a computer circuit that implements the same rules and restrictions).

Infrasonic (IS) Activity Detector

The Infrasonic (IS) Detector can be constructed using similar set of analytical techniques as the MDR Detectors. IS Detectors use time, frequency, periodicity and SNR measurements to evaluate competing hypotheses such as: human locomotion; interior or exterior door opening or closing; window opening or closing; other impact events and null activities. The hypothesis testing module within an IS Detector provides summary information based on different measurements to support a particular hypothesis. The hypothesis for different types of activity, as determined from different types of IS Detectors, are sent to a Sensor Activity Analyzer for further processing.

Video Camera Sensor

Video cameras are often used to monitor the activity over a large area. Under appropriate lighting conditions, or with sensitivity to infrared radiation, video cameras can provide a great deal of visual information that is useful for area monitoring.

In one instance, the video camera can be configured such that certain regions may be selected for analysis while non-selected regions are disregarded for analysis.

In other instances, the video camera can be configured for facial recognition to correctly identify individuals utilizing specific facial recognition software.

In other instances, video cameras can be configured to detect motion within the area being monitored.

In other instances, video cameras can be configured to detect changes, such as an object being removed from the area.

In other instances, video cameras can be can be configured to detect or confirm whether people or animals are present.

Video Camera Discriminator

The Video Discriminator filters the video information so that the Activity Detector can analyze the pertinent information. The Video Discriminator passes signals on to the Video Activity Detector after processing the video signal. Examples of the type of processing that can be applied to the information from a Video Camera are as follows: highlight particular shapes or regions of interest to observe changes within those regions; change contrast or brightness according to environmental conditions (e.g. low light).

The processing can be implemented using software (as a set of computer instructions) or through hardware (as a computer circuit that implements the same rules and restrictions).

Video Activity Detector

Video Detectors use analytical techniques to evaluate competing hypotheses such as: human motion; animal motion; face detection; and, null activity. Similar to other detectors, one component of a Video Detector is to calculate a probability measure and a confidence interval based on different measurements to support a particular hypothesis. For example, using standard face recognition techniques, the signal from the Video Discriminator can be analyzed to calculate the probability measurement and confidence interval of a particular person within the view of the Video camera.

Audible Sound(AS) Sensor

A conventional microphone that focuses on a higher range than infrasonic may be utilized to capture frequencies above 20 Hz for analysis. This is achievable through commercial microphone elements.

In one instance, the AS sensor could be used to characterize or identify the frequency, amplitude and frequency-time pattern whose signature fits that for a smoke, CO, water leakage, radon or similar locally-generated sonic alarm.

In another instance, the AS sensor could be utilized to characterize or identify the operation of motorized appliances, or their absence, from their unique motor or gear system that tends to be stable in frequency and has a long-period on-off cycle. In another instance, the AS sensor could be utilized to characterize or identify nearby traffic. Internal combustion engines have a sonic signature that identifies whether it is in constant motion, decelerating or accelerating. In particular a vehicle can be identified as approaching or stopping near the monitored area as distinguished from passing traffic.

In another instance, the AS sensor could be utilized to characterize or identify exterior sound as they may have distinct frequency profiles due to preferential attenuation of higher frequencies due to walls and corners.

In another instance, the AS sensor could be utilized to characterize or identify glass breaks. The high-frequency signature of breaking glass can be detected, whether as a prelude to intrusion, kitchen accident, weather event, or other events of interest.

In another instance, the AS sensor could be utilized to characterize or identify objects falling. Dropped or falling heavy objects and falling humans can be detected, whether during a period where there is no other activity or during other activity.

In another instance, the AS sensor could be utilized to characterize or identify vocalized sounds. These include barking dogs, entertainment devices, normal conversation, screams and code words (ex. "Help!").

Audible Sound (AS) Discriminator

The AS Discriminator removes unwanted audio signal energy with bandpass filters for the signals of interest and detection and removal of noise. Each AS Activity Detector can have unique discriminator requirements. For example, detections for sonic smoke and carbon monoxide alarms typically operate near 3,200 Hz, and therefore a bandpass filter can be employed to discriminate against signals outside of a moderately-narrow band surrounding this frequency. Similarly, bandpass filter for voice (approximately 300 to 3,000 Hz), various chimes and non-emergency sonic alarms (approximately 1,000 to 3,000 Hz), appliances (40 to 200 Hz), and so forth. Noise is a combination of thermal and electronically-generated spurious energy which can be recognized by its statistically-random or isolated spurs, respectively, and therefore reduced or eliminated. As a result, the AS Discriminator reduces the instance of false positives and false negatives by the IS Activity Detectors.

An example of an implementation of a AS Discriminator for a AS sensor is as follows. The AS Discriminator passes the microphone signal to the AS Activity Detector based on multiple criteria separately or in combination, such as: Signal is above an amplitude threshold within a specific frequency band; a temporal pattern of signal energy, such as a periodic pattern; the signal-to-noise ratio (SNR) is greater than a pre-set threshold.

Those criteria can be implemented using software (as a set of computer instructions) or through hardware (as a computer circuit that implements the same rules and restrictions).

Audible Sound (AS) Activity Detector

The AS Activity Detector can be constructed using similar set of analytical techniques as the MDR Detectors. AS Activity Detectors use time, frequency, periodicity and SNR measurements to evaluate competing hypotheses such as: sonic alarms, human conversation, entertainment devices, loud noises, impact events, glass breaking, and null activities. The hypothesis testing module within an AS Activity Detector provides summary information based on different measurements to support a particular hypothesis. The hypothesis for different types of activities, as determined from different types of AS Activity Detectors, is sent to a Sensor Activity Analyzer for further processing.

Temperature/ Air Pressure Sensor

Temperature and air pressure sensors can be utilized to measure changes in ambient temperature and air pressure. A change in these variables can be utilized to detect potential intrusions into sealed environments such as a vehicle.

In one instance, the temperature sensor could be used to characterize or identify the fire hazards, where sudden fluctuations in temperature could indicate a potential fire hazard. In another instance, in combination with weather data, a decrease or increase of temperature beyond a pre-set threshold can indicate that a furnace or air conditioner, respectively, is no longer operating effectively.

Area Event Processor

In this system and method, the Area Event Processor combines the information from each Sensor Activity Analyzer to reach a decision on the presence of one or more events within the area being monitored. The analysis process can be any decision process that is mathematical or rule based and subject to one or more criteria. The exact specification will be highly situational and context dependent since any Area Event Processor must take into account the type of sensor, the expected activities and the area being monitored.

Example of Area Event Processor Implementation

An example of the implementation of an Area Event Processor is described as follows. For this example the Area Event Processor is configured for sensors that are normally found in a typical area monitoring system. For example, the sensors can include one or more of MDR, IS, and other types of sensors. The expected activities include general household activities such as human and animal motion as well as the motion of objects such as doors. The area being monitored can be a typical domestic environment.

Under such conditions, the information from each Sensor Activity Detector can be analyzed based on various rules or criteria to improve on the inference about an activity within the area being monitored. For example:

• Grouping sensor information into classes based on their common sensing technology. For example, a class of MDR sensors or a class of IS sensors. This diversity of sensor information provides multiple lines of evidence to be used in calculating the probability and confidence interval for hypothesis detection. • Results from detectors of the same class that are in different locations (e.g. MDR sensors in different locations in the home) can be correlated to improve detection confidence by eliminating blind spots and by improved determination of the location of the motion.

There are other well-known methods to accomplish the above-mentioned correlation. For example:

• Most likely event wins: all of the detector results are compared and the one with the highest probability and/or confidence wins

• Weighted average: Different weightings can be applied to various classes of sensors. The weighting can be multiplied by the events probability and/or confidence, so that a weighted sum can be computed

• Bayesian inference: Bayesian inference allows the probability and confidence of an event to be computed over time, with increasing reliability as more information is gathered

• Rules-based systems

• Neural Networks

• Fuzzy Logic Inferences

Each of these methods can be used either as alternatives or applied together. The above list is not exhaustive but rather an example of different commonly known rules based algorithms which calculate weighted probability.

In such a process, the lack of correlation between the information from each Sensor Activity Analyzer does not necessarily indicate that a hypothesis has not occurred; that is, agreement between different Sensor Activity Analyzers information is not mandatory. For example, if the IS Activity Analyzer fails to detect footsteps, the MDR Activity Analyzer can be relied upon to indicate that there is a footstep if its confidence level is sufficiently high. This example illustrates the importance of using information from all sensors in the area; the disagreement between the IS Activity Analyzer and the MDR Activity Analyzer can be due to the lighter footsteps of a child which can be determined using MDR detector and a lower confidence from an IS detector. The confidence of detecting the activity of a child is improved by combining the information from both MDR and IS sensors.

In addition to detector correlation, the Analyzer may utilize other methods and criteria to increase the confidence of detection. Examples of the type of methods and criteria that are used include: learning about the normal activity occurring within the home based on time of day or the day of week, historical data, logical deduction and statistical inference.

Correlation with respect to Real-world Applications

The following are examples of what type of evidence is weighed by the Activity Event Processor to make a determination of an event for output (wherein the evidence is supplied by a number of Sensor Activity Processors).

In one instance, with respect to verifying door activity hypothesis, correlation of infrasonic resonances at the start and end of a door motion will confirm the door event, distinguish it from other hinged motions, and distinguish the type of door, whether it is being opened or closed and its distance. Correlation of human locomotion before and/or after the door motion, but not during the door motion, further confirms the door event.

In another instance, with respect to locomotion (human/animal) hypothesis, footstep resonances correlated with limb motion (MDR) confirms a locomotion event. Relative amplitude of footsteps and limb motion can distinguish: human from small animals (pets); and an individual human or animal.

In another instance, with respect to identifying an intruder hypothesis, doors, locomotion, glass break and other sensor events can be correlated, together with the state of the system arming, to determine if an intruder is suspected to be present, as opposed to an authorized occupant.

In another instance, with respect to a fire hazard hypothesis, correlation of sonic smoke alarms with a steady, unexpected temperature rise (temperature sensor) is indicative of a fire. The degree of temperature rise can determine a minor kitchen fire from one that is larger and catastrophic. For example, the correlated sound of air movement and "roaring" further confirms the presence of a catastrophic fire.

In another instance, with respect to an evacuation hypothesis, correlation of a smoke or CO sonic alarm with locomotion or its absence in the time immediately following the start of the sonic alarm indicates whether the area being monitored is being successfully evacuated.

In another instance, with respect to a falling object hypothesis, locomotion correlated with a fall (sound), with or without continuation of the locomotion is indicative of an accident or other adverse event.

Output

The Area Event Processor summarizes all the information from one or more Sensor Activity Detectors and outputs an event. This output can trigger other processes. In the case of a home security system, the output can act as an alarm trigger for a legacy alarm system or as a signal to notify, for example, users or a monitoring service that the monitored area requires human intervention.

The output of the system is delivered to one or more end-users. This may be accomplished by various means, such as but not limited to email, instant messaging, or phone.

The invention will now be described with reference to specific examples. It will be understood that the following examples are intended to describe embodiments of the invention and are not intended to limit the invention in any way. EXAMPLES

EXAMPLE 1:

The following illustrates the implementation of a specific MDR Activity Detector, namely a MDR Human Locomotion Detector.

The main function of the MDR Human Locomotion Detector is to test the hypothesis that the MDR sensor has detected human locomotion. This could be achieved with many different types of MDR signal analysis and measurement. Each type of analysis can be made using the information from the MDR Discriminator and the measurements from each analysis are compiled to calculate the probability of human locomotion.

Figure 7 illustrates an example of an MDR Human Locomotion Detector. In this example, the MDR Human Locomotion Detector (120) comprises four different modules for measurement and analysis of the Signal from the MDR Discriminator (100). The modules in this case are: stride frequency measurement; power ratio measurement; signal comparison measurement; and potentially other modules. Each measurement and analysis module provides a measurement for a particular metric.

A hypothesis-testing module (140) calculates a total measurement by combining the outputs from the preceding modules. The measurement of probability and confidence of the hypothesis is sent to the Sensor Activity Analyzer for further analysis. Each module in the MDR Human Locomotion Detector is described in greater detail below.

Stride Frequency Measurement

A Stride Frequency measurement module is a component of the MDR Human Locomotion Detector. This module uses an analysis of the stride frequency as the basis for identifying the type of movement occurring within the MDR field of coverage. Stride frequency can be determined in a number of ways. The MDR Human Location Detector can use a number of methods including one or more of the following methods to analyze the stride frequency: • Transform the signal into the time-frequency domain using FFTs (Fast Fourier Transforms) and then using the "Step Extraction" approach to count strides.

• "Step Extraction takes the time-frequency domain input signal and for all times that have frequencies above a certain power level threshold (which varies, based on velocity), creates a velocity profile by storing the maximum frequency that exceeds that power-level threshold in a time-indexed vector. The periodic elements of the original signal can be extracted by passing this vector through some commonly-practiced signal processing techniques. Each footstep is a period of the vector. For example, a footstep can be defined as a sinusoidal periodic element of the vector with two minima at the endpoints and a maximum defining the peak within those particular endpoints, or likewise two maxima as the endpoints and a minimum as a trough

• Perform a second FFT of the time-frequency domain signal from the above method to extract the dominant frequencies in the resultant waveform (typically, a dominant frequency in the 1-5 Hz range is the stride frequency).

• Filtering the time-domain signal with a 1-5 Hz bandpass filter, and then counting the zero crossings. Zero crossings are the number of times the time- domain signal crosses from positive to negative. The number of zero crossings will be double the stride frequency.

Someone knowledgeable in the art can implement the appropriate algorithms to describe the stride frequency based on the information available from the MDR discriminator. The output for this module is a probability measurement and confidence interval for the hypothesis of Human Locomotion based on the stride frequency measurements, supplemented with the data summarizing the measurements.

Signal Comparison Measurement

A signal comparison measurement module is a component of the MDR Human Locomotion Detector. This module matches the signal from the MDR Discriminator to pre-defined template signals in order to identify the type of locomotion that is indicated by the MDR Discriminator information. For example, signal comparison can be done according to the following steps:

• Extract a short time section of a signal, for example, one footstep's worth.

One way to do this is using the Step Extraction approach, as defined in the above section

• Normalize the extracted signal in the amplitude and length, so that it is on the same scale as the pre-defined template signal.

• Compare the extracted signal with the pre-defined template signal from a library of locomotion signatures and develop a result to indicate the degree of fit to each signature. One approach is to use the least mean squared method to correlate the extracted signal to the signature. This uses the average of the squares of the differences between the normalized measured signal and the template signal, as computed at each point in time.

• The probability and confidence of the fit is based on degree to which the signal components each match the template, or fall within or outside of a specified range of values. This step is called correlation, and is a common DSP method.

Someone knowledgeable in the art can implement the appropriate algorithms to implement the idea of signal comparison measures as a means of identifying locomotion based on the information available from the MDR Discriminator. The output for this module is a probability measurement and confidence interval for the hypothesis of Human Locomotion based on the signal comparison, supplemented with the data summarizing the measurements.

Power Ratio Measurement

A power ratio measurement module is a component of the MDR Human Locomotion Detector. This module compares the level of MDR signal reflecting from the limbs with the signal reflected from the torso of a subject. The power ratio can be developed using a series of steps, as follows: • Transform the signal into the time-frequency domain using FFTs (Fast Fourier Transforms).

• Extract the signal energy for the frequency range corresponding to the torso.

The exact frequency range will depend on the velocity of the subject. Human locomotion typically shows a signal power trough between the frequency bands associated with the torso and limb components. The torso components are of relatively high amplitude and have a low variation over time. The limb components are of relative low amplitude and have a periodic time variation in accordance with the stride frequency. This is illustrated graphically in Figure 8A and Figure 8B.

• Extract a second set of higher frequency ranges corresponding to the limbs.

• Divide the sum of the power in the torso frequency range by the sum of the power in the limbs frequency range.

• The probability and confidence of the fit is based on one or more of: frequency range of the power gap in comparison to average torso velocity and maximum limb velocity; total power, averaged over the time of the detection but at least one stride; and the measured power ratio.

Four- legged animals typically found in a residence have a power ratio signature that is distinct from that for humans, and it is therefore an indicator to identify and distinguish detections of human from animal locomotion.

Someone knowledgeable in the art can implement the appropriate algorithms to use power ratio measurements as a means of identifying locomotion based on the information available from the MDR discriminator.

The power ratio measurement can be used to determine a probability of Human Locomotion based on a model probability distribution. Figure 9 represents an example of such a probability distribution for a human (151) and a dog (153) as a function of the power ratio (torso/limb). The same approach is followed to determine the confidence interval for human locomotion. The output for this module is a probability measurement and confidence interval for the hypothesis of Human Locomotion based on the power ratio, supplemented with the data summarizing the measurements.

Other Techniques utilized for Analysis

Other measurement or analysis techniques can be used as modules in the MDR Human Locomotion Detector. Examples of other analysis techniques are:

• Hump Ratio: as can be seen in Figure 8B, there are "humps" in the signal, where the signal rises to 300-400 Hz from its baseline of 100-200 Hz, with these frequencies corresponding to a velocity of about 1-2 m/s (based on a MDR sensor operating at 10.5 GHz). As can be seen, there are gaps between these humps where there is very little signal power. The ratio of these gaps is different for humans and pets, and therefore can be used to determine the subject.

• Foot Tracing: The spectrogram shown in Figure 8A can also be used for the technique of foot tracing (179). The spectrogram shows the distinct pattern of a human foot swinging within each of the previously described "humps". This foot outline, which is shown in the enlarged portion of the figure, can be determined using commonly-practiced signal processing techniques. The foot outline is apparent for human subjects but not for animals which are typically pets.

Similar to other modules in the MDR Human Locomotion Detector, each analysis technique yields a probability measurement and confidence interval, supplemented with the data summarizing the measurements. These data are sent to the hypothesis testing module.

Hypothesis Testing

The hypothesis testing module within the MDR Human Locomotion Detector combines the probability, confidence intervals and meta-data from each measurement module (e.g. stride frequency, signal comparison, power ratio, and other modules) to calculate a probability measurement and confidence interval for human locomotion based on the information provided by the MDR sensor. The calculation can also be expanded by one or more criteria. For example:

• Dynamic weighting based on confidence measures, machine learning and the context of the measurements

• Signal-to-noise ratio (SNR) with highest confidence corresponding to highest SNR

The MDR Human Location Detector then provides an overall Human Locomotion probability and confidence measurements, supplemented with the data summarizing the measurements to the Sensor Activity Analyzer for further processing

These type of examples can be applied to other implementations will are briefly discussed in the following paragraphs:

MDR Pet Locomotion Detector

A Pet Locomotion Detector can be constructed from the same modules as the MDR Human Locomotion Detector but adjusted to detect the movement of pets or other animals. For each module, the MDR Pet Locomotion Detector uses distinct probability and confidence profiles that have been pre-determined for the intended subjects. The MDR Pet Detector provides overall probability and confidence measurements, supplemented with the data summarizing the measurements, to the Analyzer for further processing.

MDR Door Detector

The MDR Door Detector is example of a detector which can be constructed to determine specific events. A Door Detector can be constructed using commonly- practiced signal processing techniques similar to other MDR Activity Detectors. The motion of a door can be detected by using the MDR signal in the time domain and identifying periods where the signal gradually rises and falls in frequency and in amplitude. The velocity is at a maximum when the frequency profile is at a maximum, and the signal amplitude is strongest when the door face is perpendicular to the sensor. The MDR Door Detector will have distinct Probability and Confidence distributions and distinct weightings so that a probability measurement and confidence interval for a door action can be sent to the Sensor Activity Analyzer for further processing.

MDR Null Detector

The MDR Null Detector is example of a detector which can be constructed to test the null hypothesis. The null hypothesis is an important component in Bayesian inference that is used to guard against false positive detections.

The Null Detector independently determines that no significant motion activity has been detected within the monitored area. This is used in the evaluation of the alternative hypotheses to ensure that not only is one Detector dominant over the others, but also that it is dominant over a quantitative measure of no motion or locomotion event. That is, for a decision to be made with best confidence, not only must a Detector have a high relative probability and confidence over other Detectors, but also over motion and locomotion that does not fall within the filters of any Detector for specific events. The MDR Null Detector will have distinct Probability and Confidence distributions and distinct weightings associated with this hypothesis of no significant motion; this information is sent to the Sensor Activity Analyzer for further processing.

It is obvious that the foregoing embodiments of the invention are examples and can be varied in many ways. Such present or future variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims.