FIELD OF THE INVENTION

The invention relates to a system for detecting human and/or animal presence based on changes in received radio frequency signals.

The invention further relates to a method of detecting human and/or animal presence based on changes in received radio frequency signals.

The invention also relates to a computer program product enabling a computer system to perform such a method.

BACKGROUND OF THE INVENTION

The use of standalone presence detectors like the Philips Hue motion sensor is gaining popularity and may be used to automatically activate lighting, for example. However, better results may be obtained by using a wireless sensing network. A wireless sensing network consists of several nodes communicating with each other and exchanging information for a certain sensing application. For example, human presence may be detected based on received RF signals due to the absorption of RF energy by the human body and the reflections and scattering of the RF signals by the human body.

For instance, US 10,404,387 B1 discloses a network of wireless communication devices that transmit wireless signals to each other over wireless communication links. These wireless signals can be used as motion probes to detect motion of persons or objects in the signal paths between the devices in multiple detection zones. For example, each device may process received wireless signals to detect motion changes based on changes detected in the communication channel. Each device may send channel information to a central device or system that performs operations of the motion detection system.

Since human bodies comprise a lot of water, they are normally relatively easy to detect using wireless sensing, e.g. RF-based sensing. However, if a human is present in a room and an object in this room has a non-default physical state, e.g. a window is open, then it may be difficult to determine whether changes in RF signals are caused by the presence of the human or by the object having a non-default physical state. In an RF-based sensing system, the RF signal strength per directional link will typically fluctuate depending on its environment. Objects in the environment that change physical state will have effect on the RF signal strength distribution, due to potential blocking of an RF signal by an object and due to the multipath behavior of the signals being different.

It may be possible to improve the detection of environmental changes by using extra devices such as dedicated door/window sensors, but this also increases the cost and complexity of the sensing system.

US2018/292520 discloses methods for determining whether or not a monitored space is occupied by one or more humans and/or animals, e.g., based on one or more radiofrequency (RF) receivers monitoring one or more RF frequencies for changes in received signal strength that may be due to changes in occupancy of the space being monitored. The received signal strength is analyzed using nonparametric online change-point detection analysis to determine change-points in the received signal(s). One or more statistical measures of the received signal(s), such as mean and variance, are used in conjunction with the change-point detection to determine a probability that the occupancy of the monitored space has changed.

SUMMARY OF THE INVENTION

It is a first object of the invention to provide a system, which is able to detect a relatively accurate environmental state based on changes in RF signals, even in environments with objects that can change physical state, without requiring additional dedicated sensors.

It is a second object of the invention to provide a method, which can be used to detect a relatively accurate environmental state based on changes in RF signals, even in environments with objects that can change physical state, without requiring additional dedicated sensors.

In a first aspect of the invention, a system for detecting human and/or animal presence based on changes in received radio frequency signals comprises at least one input interface, at least one output interface, and at least one processor configured to cause, via said at least one output interface, a first set of one or more radio frequency signals to be transmitted by at least one node of a first set of nodes and received by at least one other node of said first set of nodes, determine, via said at least one input interface, a characteristic of said received first set of radio frequency signals, cause, via said at least one output interface, a second set of one or more radio frequency signals to be transmitted by at least one node of a second set of nodes and received by at least one other node of said second set of nodes, determine, via said at least one input interface, a characteristic of said received second set of radio frequency signals, and compare said characteristic of said received first set of radio frequency signals with at least a first baseline, said first baseline being associated with said first set of nodes.

The at least one processor is further configured to select said second set of nodes in dependence on a result of said comparison, a first zone covered by said first set of nodes overlapping partially with or being proximate in location to a second zone covered by said second set of nodes, said second zone comprising an object with a changeable physical state, said characteristic of said received second set of radio frequency signals depending at least on said physical state of said object, compare said characteristic of said received second set of radio frequency signals with at least a second baseline in dependence on said result of said comparison of said characteristic of said received first set of radio frequency signals with at least said first baseline, said second baseline being associated with said second set of nodes, and determine a human and/or animal presence in said first zone based on a result of at least one of said comparisons.

With this system, there is no need for extra devices (such as dedicated door/window sensors) to detect the state of the environment relatively accurately. The nodes in the wireless sensing network provide characteristics of the received RF signals, e.g. RF sensing signal strength and/or CSI (Channel State Information) information, and by analyzing changes in two partially overlapping or proximate detection zones, of which one zone comprises an object that can change physical state, the physical state (changes) of doors and windows can be inferred by the impact they have on RF signals. The physical state of the object may be communicated to a user or may be used to detect human and/or animal presence, for example.

Thus, the state of the environment, such as a door being open and/or the presence of a person in a room or other area, may be detected relatively accurately based on changes in RF signals without requiring additional dedicated sensors. Apart from the device and installation cost of such additional devices, this also saves on maintenance (window/door sensors typically are battery powered so the battery level needs to be monitored, and batteries need to be replaced after some time). In many situations, this detection may be carried out regardless of the activity being carried out by the human(s) and/or animal(s) in the area.

Said object may comprise a door, a window, a drawer, furniture, a hospital crash cart, machinery, a loading dock, a loaded pallet, or an autonomous robot like a vacuum cleaner, for example. As a first example, if RF -based sensing is performed in an industry environment, a piece of machinery may or may not be present in a certain sensing zone or the metal door of the machinery may be open or closed. As a second example, in a storage warehouse, the RF-based sensing will likely differ whether a wooden pallet with rice bags is standing there or not.

Said at least one processor may be configured to determine whether a human is present, whether an animal is present or whether a human and/or animal is present. As animals are smaller, presence detection may require more sensitive RF-based sensing threshold settings for an animal than for a human. Alternatively, a higher sensing frequency may be applied, as smaller wavelength RF sensing signals can resolve smaller objects better.

The physical state of a door, window or drawer may represent whether it is open or closed, for example. The physical state may additionally represent intermediate states. For example, it may be interesting to know whether a door or window is partially open, as this could indicate that someone is trying to ventilate a space. The physical state of a hospital crash cart, an autonomous robot, or furniture may represent whether it is present or not, for example. Said first baseline is typically recorded when no human, and in many situations no animal, was present in said first zone and said second baseline is typically recorded when no human, and in many situations no animal, was present in said second zone.

Said at least one processor may be configured to select said second set of nodes in dependence on a difference between said characteristic of said received first set of radio frequency signals and said first baseline exceeding a threshold, and determine said human and/or animal presence in said first zone based on said result of said comparison of said characteristic of said received first set of radio frequency signals with at least said first baseline and a result of said comparison of said characteristic of said received second set of radio frequency signals with said second baseline. Thus, the RF-based sensing in the second zone may be used to improve the human and/or animal presence detection in the first zone by determining the impact of the physical state of the object in the second zone on the RF-based sensing in the first zone.

Additionally, said at least one processor may be configured to select said second set of nodes in dependence on whether the frequency patterns of the received first set of radio frequency signals are within a certain frequency range or not (to be able to separate changes due to noise from changes due to human and/or animal activity). The above- mentioned threshold may depend on whether these frequency patterns are taken into account or not. Said at least one processor may be configured to determine a physical state of said object in said second zone based on a result of said comparisons. The physical state of the object may explicitly be determined in order to communicate it to a user, e.g. as an alert that a window has been left open or a door has been opened.

Said at least one processor may be configured to determine said human and/or animal presence in said first zone based on said result of said comparison of said characteristic of said received first set of radio frequency signals with said first baseline, select said second set of nodes from at least a first potential set of nodes and a second potential set of nodes based on said human and/or animal presence being detected in said first zone, and determine said physical state of said object in said second zone based on said result of said comparison of said characteristic of said received second set of radio frequency signals with at least said second baseline. Thus, human and/or animal presence detection in the first zone may be used to reliably determine the physical state of the object in the second zone.

Said at least one processor may be configured to compare said characteristic of said received second set of radio frequency signals with multiple baselines, said multiple baselines being associated with said second set of nodes and each of said multiple baselines being associated with a different physical state of said object. The physical state of the object in the second zone may be detected by using multiple baselines for the second zone, one per possible physical state of the object. Alternatively or additionally, the physical state of the object may be detected using movement signatures recorded for the object. For example, the closing of a door or window is typically characterized by acceleration, turning around a hinge and rapid deceleration (of the RF signal characteristics for nodes on different sides of the door) when the door hits the frame. This creates a distinct movement pattern which can for instance be detected by the time variation in the Wi-Fi CSI multipath signals.

Said at least one processor may be configured to cause said second set of one or more radio frequency signals to be transmitted by said at least one node of said second set of nodes and received by said at least one other node of said second set of nodes in dependence on said result of said comparison of said characteristic of said received first set of radio frequency signals with at least said first baseline. If always the same nodes are transmitting RF signals and the other nodes receive all RF signals from transmitting nodes in their vicinity, the data applicable to the second zone only need to be selected from the data collected by all the receiving nodes. However, if this is not the case, the nodes in the second zone may be instructed to start scanning the selected second zone after the nodes for the second zone have been selected.

Said at least one processor may be configured to control, via said at least one output interface, a lighting device, a building automation device, a smart audio speaker, and/or a security system based on said human and/or animal presence being detected. Alternatively, the detection of the physical state of the object may be improved based on the human and/or animal presence detection, for example.

Said at least one processor may be configured to determine human and/or animal activity and/or human and/or animal vital signs and/or recognize one or more gestures in said first zone based on a result of at least one of said comparisons. As a first example, by determining with Wi-Fi sensing at which location a human is breathing vertically, the height of the human may be detected. As a second example, if chest movement is determined to be close to the floor, the system may conclude that the human has fallen on the floor and may then raise an alarm. Breathing rate and heartbeat may be determined with 60GHz Wi-Fi sensing, for instance. As a third example, Wi-Fi sensing may be used to determine body posture, e.g. whether a human is lying on the couch or reading. If human activity is detected and/or human vital signs are detected, a human may be assumed (i.e. determined) to be present.

As described above, a higher sensing frequency may be applied to detect animals, e.g. pets, as smaller wavelength RF sensing signals can resolve smaller objects better. If vital signs of a dog or cat are to be detected (e.g. breathing monitoring), higher RF sensing frequencies (e.g. 60GHz Wi-Fi) are preferred/required to detect the chest movements, as the chest movements of a dog or cat or smaller than those of a human. Determining animal activity and/or animal vital signals may be important for certain pet owners. For example, certain pet owners want to track the activity level of the pet when they are away from home e.g. to make sure that their pet is not depressed. If animal activity is detected and/or animal vital signs are detected, an animal may be assumed (i.e. determined) to be present.

In a second aspect of the invention, a method of detecting human and/or animal presence based on changes in received radio frequency signals comprises causing a first set of one or more radio frequency signals to be transmitted by at least one node of a first set of nodes and received by at least one other node of said first set of nodes, determining a characteristic of said received first set of radio frequency signals, causing a second set of one or more radio frequency signals to be transmitted by at least one node of a second set of nodes and received by at least one other node of said second set of nodes, determining a characteristic of said received second set of radio frequency signals, and comparing said characteristic of said received first set of radio frequency signals with at least a first baseline, said first baseline being associated with said first set of nodes.

Said method further comprises selecting said second set of nodes in dependence on a result of said comparison, a first zone covered by said first set of nodes overlapping partially with or being proximate in location to a second zone covered by said second set of nodes, said second zone comprising an object with a changeable physical state, said characteristic of said received second set of radio frequency signals depending at least on said physical state of said object, comparing said characteristic of said received second set of radio frequency signals with at least a second baseline in dependence on said result of said comparison of said characteristic of said received first set of radio frequency signals with at least said first baseline, said second baseline being associated with said second set of nodes, and determining a human and/or animal presence in said first zone based on a result of at least one of said comparisons. Said method may be performed by software running on a programmable device. This software may be provided as a computer program product.

Moreover, a computer program for carrying out the methods described herein, as well as a non-transitory computer readable storage-medium storing the computer program are provided. A computer program may, for example, be downloaded by or uploaded to an existing device or be stored upon manufacturing of these systems.

A non-transitory computer-readable storage medium stores at least one software code portion, the software code portion, when executed or processed by a computer, being configured to perform executable operations for detecting human and/or animal presence based on changes in received radio frequency signals.

The executable operations comprise causing a first set of one or more radio frequency signals to be transmitted by at least one node of a first set of nodes and received by at least one other node of said first set of nodes, determining a characteristic of said received first set of radio frequency signals, causing a second set of one or more radio frequency signals to be transmitted by at least one node of a second set of nodes and received by at least one other node of said second set of nodes, determining a characteristic of said received second set of radio frequency signals, and comparing said characteristic of said received first set of radio frequency signals with at least a first baseline, said first baseline being associated with said first set of nodes.

The executable operations further comprise selecting said second set of nodes in dependence on a result of said comparison, a first zone covered by said first set of nodes overlapping partially with or being proximate in location to a second zone covered by said second set of nodes, said second zone comprising an object with a changeable physical state, said characteristic of said received second set of radio frequency signals depending at least on said physical state of said object, comparing said characteristic of said received second set of radio frequency signals with at least a second baseline in dependence on said result of said comparison of said characteristic of said received first set of radio frequency signals with at least said first baseline, said second baseline being associated with said second set of nodes, and determining a human and/or animal presence in said first zone based on a result of at least one of said comparisons.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a device, a method or a computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system." Functions described in this disclosure may be implemented as an algorithm executed by a processor/microprocessor of a computer. Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied, e.g., stored, thereon.

Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a computer readable storage medium may include, but are not limited to, the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of the present invention, a computer readable storage medium may be any tangible medium that can contain, or store, a program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber, cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java(TM), Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor, in particular a microprocessor or a central processing unit (CPU), of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer, other programmable data processing apparatus, or other devices create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of devices, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention are apparent from and will be further elucidated, by way of example, with reference to the drawings, in which:

Fig. l is a block diagram of an embodiment of the system and connected RF sensing nodes;

Fig. 2 shows an example of an office floor with the RF sensing nodes of Fig. i;

Fig. 3 is a flow diagram of a first embodiment of the method; Fig. 4 is a flow diagram of a second embodiment of the method; Fig. 5 shows an example of two zones that could be created with RF sensing nodes of Fig. 2, to illustrate the methods of Figs. 3 and 4;

Fig. 6 shows an alternative zone for detecting the physical state of a door;

Fig. 7 is a flow diagram of a third embodiment of the method;

Fig. 8 is a flow diagram of a fourth embodiment of the method;

Fig. 9 shows an example of three zones that could be created with RF sensing nodes of Fig. 2, to illustrate the methods of Figs. 7 and 8;

Fig. 10 is a flow diagram of a fifth embodiment of the method;

Fig. 11 shows an example of an object state associated with a first baseline; Fig. 12 shows an example of an object state associated with a second baseline;

Fig. 13 is a block diagram of an exemplary data processing system for performing the method of the invention.

Corresponding elements in the drawings are denoted by the same reference numeral.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Fig. 1 shows an embodiment of the system for detecting human and/or animal presence based on changes in received radio frequency signals: a controller 1. The controller 1 communicates with nodes 20-39 of an RF-based sensing network, e.g. using Zigbee technology. In the example of Fig. 1, the nodes 20-39 are lighting devices, e.g. Philips Hue lamps. However, the RF-based sensing network may comprise other nodes than lighting devices.

In the example of Fig. 1, the RF-based sensing network is a mesh network and the controller 1 communicates directly with multiple nodes. Alternatively, the RF-based sensing network has a different topology, controller 1 communicates with only one of the nodes directly and/or controller 1 does not communicate with any of the nodes directly.

The controller 1 comprises a receiver 3, a transmitter 4, a processor 5 and memory 7. The processor 5 is configured to cause, via the transmitter 4, a first set of one or more radio frequency signals to be transmitted by at least one node of a first subset of the nodes 20-39 and received by at least one other node of the first subset and determine, via the receiver 3, one or more characteristics of the received first set of radio frequency signals.

The processor 5 is further configured to cause, via the transmitter 4, a second set of one or more radio frequency signals to be transmitted by at least one node of a second subset of the nodes 20-39 and received by at least one other node of the second subset and determine, via the receiver 3, one or more characteristics of the received second set of radio frequency signals.

The processor 5 is further configured to compare the one or more characteristics of the received first set of radio frequency signals with at least a first baseline and select the second subset of nodes in dependence on a result of the comparison such that a first zone covered by the first subset of nodes overlaps partially with or is proximate in location to a second zone covered by the second subset of nodes.

The first baseline is associated with the first subset of nodes. The second zone comprises an object with a changeable physical state. The object may comprise a door, a window, a drawer, furniture, a hospital crash cart, machinery, a loading dock, a loaded pallet, or an autonomous robot, for example. The one or more characteristics of the received second set of radio frequency signals depend at least on the physical state of the object.

The processor 5 is further configured to compare the one or more characteristics of the received second set of radio frequency signals with at least a second baseline in dependence on the result of the first comparison, i.e. the comparison of the characteristic of the received first set of radio frequency signals with at least the first baseline, and determine a human and/or animal presence in the first zone based on a result of at least one of the comparisons. The second baseline is associated with the second set of nodes.

In other words, the state of an area, e.g. room, may be determined by using multiple (partially overlapping or proximate) zones in that area, wherein each of the zones uses a baseline associated with that zone and the combination of multiple zones leads to potential improved performance of the detection. During commissioning of an area, a user (or the system, possibly guided by the user) can establish baselines for each of the zones. During detection, the characteristic(s) of the received RF signals of each zone within the area is compared with a baseline, to check for environmental changes.

Dividing an area in specific zones, in which each zone has a specific detection scope, improves the reliability of certain detection features, within the selected area. This will reduce or eliminate “false positive” indications, thereby saving time and money. Also “false negatives” will be reduced or eliminated, e.g. no human and/or animal presence is detected in an area while someone is present in the area.

By determining the state of an RF -based sensing zone, more detailed information of an area may be obtained than by using a single sensor (e.g. PIR), because RF- based sensing has a more detailed grid to monitor changes in the environment. This may be used to prevent a false alarm, as it would not be possible for someone to enter a room (e.g. in a house) via an adjacent zone without first opening a previously closed door or a window. Especially the use of different zones in the sensing area makes a more reliable detection possible.

The processor 5 may be configured to control, via the transmitter 4, one or more lighting devices (e.g. one or more of nodes 20-39) based on the human presence being detected. Additionally or alternatively, the processor 5 may be configured to control a building automation device, a smart audio speaker, and/or a security system based on the human and/or animal presence being detected.

In the example of Fig. 1, the controller 1 determines the characteristics of received radio frequency signals from data received from multiple of the nodes 20-39. The nodes 20-39 each comprise a radio frequency sensor. The radio frequency signals are received by these radio frequency sensors. Signal strength data (RSSI, Received Signal Strength Indication) or Wi-Fi CSI of at least some of the node-to-node connections is collected.

A node determines at least one characteristic from the radio frequency signal(s) received by its radio frequency sensor, after which the node includes the at least one characteristic in data transmitted to the controller 1. The characteristics may comprise power levels/signal strengths (e.g. RSSI) and/or Channel State Information (CSI), for example. Changes in received signal strength or Wi-Fi CSI can be measured at each of the nodes when wireless packets are sent around in the network.

The controller may process the RSSI or CSI data from each node per zone to look for specific changes and transitions on the collected signal strength data. By collecting signal strength variations in different zones in an area, a detection algorithm can conclude that there was a change in the environment, based on information from each of the zones.

In the embodiment of the controller 1 shown in Fig. 1, the controller 1 comprises one processor 5. In an alternative embodiment, the controller 1 comprises multiple processors. The processor 5 of the controller 1 may be a general-purpose processor, e.g. ARM-based, or an application-specific processor. The processor 5 of the controller 1 may run a Unix-based operating system for example. The memory 7 may comprise one or more memory units. The memory 7 may comprise one or more hard disks and/or solid-state memory, for example. The receiver 3 and the transmitter 4 may use one or more wired and/or wireless communication technologies, e.g. Zigbee, to communicate with the nodes 20-39, for example. In an alternative embodiment, multiple receivers and/or multiple transmitters are used instead of a single receiver and a single transmitter. In the embodiment shown in Fig. 1, a separate receiver and a separate transmitter are used. In an alternative embodiment, the receiver 3 and the transmitter 4 are combined into a transceiver. The controller 1 may comprise other components typical for a controller such as a power connector.

The invention may be implemented using a computer program running on one or more processors. In the embodiment of Fig. 1, the system comprises a single device. In an alternative embodiment, the system comprises multiple devices. For example, the system may comprise nodes 20-39.

Fig. 2 shows an example of an office floor with the RF sensing nodes of Fig.

1. The office floor comprises four offices 41, 42, 44 and 45 and a hallway 43. The nodes 33- 34 and 38-39 are located in office 45. The nodes 32 and 37 are located in the hallway 43 between office 45 and office 42. A door 49 is located between the office 45 and the hallway 43. In the example of Fig. 2, the door 49 is open. There are also doors between the other offices 41, 42 and 44 and the hallway 43, but these doors are not shown for the sake of simplicity.

In the example of Fig. 2, the nodes of the wireless sensing network are lighting devices. If many lighting devices are present and all of them are capable of providing RSSI/CSI data, it may be possible, and even beneficial, to use only a subset of them.

A first embodiment of the method of detecting human and/or animal presence based on changes in received radio frequency signals is shown in Fig. 3. A step 101 comprises causing a first set of one or more radio frequency signals to be transmitted by at least one node of a first set of nodes and received by at least one other node of the first set of nodes. A step 103 comprises determining a characteristic of the received first set of radio frequency signals.

A step 105 comprises causing a second set of one or more radio frequency signals to be transmitted by at least one node of a second set of nodes and received by at least one other node of the second set of nodes. A step 107 comprises determining a characteristic of the received second set of radio frequency signals. Further sets of one or more radio frequency signals may be caused to be transmitted by one or more nodes of further sets of nodes at the same time and/or at a different time. This is not shown in Fig. 3. A step 109 comprises comparing the characteristic of the received first set of radio frequency signals, as determined in step 103, with at least a first baseline. The first baseline is associated with the first set of nodes. If the characteristic is equal to the first baseline, e.g. the differences are negligible or the difference between said characteristic and the first baseline does not exceeds a threshold, a human and/or animal is not considered to be present in the first zone and step 115 is performed directly after step 109. If the characteristic is not equal to the first baseline, e.g., when the difference between said characteristic and the first baseline exceeds a threshold, a step 111 is performed after step 109.

Step 111 comprises selecting the second set of nodes in dependence on a result of the comparison such that a first zone covered by the first set of nodes overlaps partially with or is proximate in location to a second zone covered by the second set of nodes. The second zone comprises an object with a changeable physical state. The characteristic of the received second set of radio frequency signals depends at least on the physical state of the object.

Next, a step 113 comprises comparing the characteristic of the received second set of radio frequency signals with at least a second baseline. The second baseline is associated with the second set of nodes. In the embodiment of Fig. 3, the first baseline has been recorded when no human was present in the first zone and the second baseline has been recorded when no human was present in the second zone. Step 115 is performed after step 113.

Step 115 comprises determining a human presence in the first zone based on a result of steps 109 and 113. If the result of the comparison of step 109 was that the characteristic determined in step 103 is equal to the first baseline, e.g., when the difference between said characteristic and the first baseline does not exceed a threshold, the result of step 115 is that no human and/or animal presence has been detected. If the result of the comparison of step 109 was that the characteristic determined in step 103 is not equal to the first baseline, e.g., when the difference between said characteristic and the first baseline exceeds a threshold, and the result of the comparison of step 113 was that the object has a first physical state, e.g. the door is closed, then the result of step 115 is that human and/or animal presence has been detected.

If the result of the comparison of step 109 was that the characteristic determined in step 103 is not equal to the first baseline, e.g., when the difference between said characteristic and the first baseline exceeds a threshold, and the result of the comparison of step 113 was that the object has a second physical state, e.g. the door is open, then the result of step 115 is that no human and/or animal presence has been detected, i.e. the deviation from the first baseline was caused by the object having the second physical state. The first baseline may have been recorded when the object had the first physical state, for example.

A second embodiment of the method of detecting human and/or animal presence based on changes in received radio frequency signals is shown in Fig. 4. In the second embodiment, the second set of one or more radio frequency signals are caused to be transmitted by the at least one node of the second set of nodes and received by the at least one other node of the second set of nodes in step 105 in dependence on the result of the comparison of step 109.

Steps 105 and 107 are only performed if it is determined in step 109 that the characteristic determined in step 103 is not equal to the first baseline. Steps 105 and 107 are performed after the second set of nodes have been selected in step 111. Steps 113 and 115 are performed after 107 has been performed in the manner described in relation to Fig. 3.

Fig. 5 shows an example of two zones created with RF sensing nodes 32-34 and 37-39 of Fig. 2, to illustrate the methods of Figs. 3 and 4. In the example of Fig. 5, the first zone 61 comprises a first set of nodes 33, 34, 38 and 39. When the difference between the characteristic of the received first set of radio frequency signals and the first baseline, i.e. between the characteristic of the RF signals received in the first zone 61 and the baseline for the first zone 61, exceeds a threshold (this threshold may be configured by a user during commissioning, for example), it might not be possible to determine from this difference whether this difference is caused by human presence or by the door 49 being open.

In this case, it is checked whether the door 49 is open or closed. A second zone is used for this purpose. The second zone 62 comprises a second set of nodes 32, 33, 37 and 38 and door 49 is located within this second zone 62. The characteristic of the received second set of radio frequency signals is compared with a second baseline. In other words, the characteristic of the RF signals received in the second zone 62 is compared with at least one baseline for the second zone 62.

Based on this comparison, it is detected whether the door 49 is open or closed. A human is determined to be present in the first zone 61 if the difference between the characteristic of the received first set of radio frequency signals and the first baseline exceeds the threshold and the door 49 is detected to be closed. The first baseline is typically recorded when no one is in the room 45 and the door 49 is closed. In the example of Fig. 5, the door 49 is located in the center of the second zone 62. Fig. 6 shows an alternative second zone for detecting the physical state of door 49. In the example of Fig. 6, a second zone 64 comprising nodes 83 and 88 is used to detect whether the door 49 is open or closed. Nodes 83 and 88 of Fig. 6 are located a bit closer to the door 49 than nodes 33 and 38 of Fig. 5.

A third embodiment of the method of detecting human and/or animal presence based on changes in received radio frequency signals is shown in Fig. 7. Step 101 comprises causing a first set of one or more radio frequency signals to be transmitted by at least one node of a first set of nodes and received by at least one other node of the first set of nodes. Step 103 comprises determining a characteristic of the received first set of radio frequency signals.

Step 105 comprises causing a second set of one or more radio frequency signals to be transmitted by at least one node of a second set of nodes and received by at least one other node of the second set of nodes. Step 107 comprises determining a characteristic of the received second set of radio frequency signals.

Step 109 comprises comparing the characteristic of the received first set of radio frequency signals, as determined in step 103, with at least a first baseline. The first baseline is associated with the first set of nodes. Next, a step 131 comprises determining a human presence in the first zone based on a result of step 109.

A step 133 is performed after step 131. Step 133 comprises selecting the second set of nodes in dependence on a result of the comparison such that a first zone covered by the first set of nodes overlaps partially with or is proximate in location to a second zone covered by the second set of nodes. The second zone comprises an object with a changeable physical state. The characteristic of the received second set of radio frequency signals depends at least on the physical state of the object.

In the embodiment of Fig. 3, it depends on the result of step 109 whether a second set of nodes needs to be selected of which a characteristic of received radio frequency signals is to be compared with at least a second baseline (in step 113). In other words, it depends on the result of step 109 whether steps 111 and 113 need to be performed. In the embodiment of Fig. 7, it depends on the result of step 103 which second set of nodes is selected of which a characteristic of radio frequency signals is to be compared with at least a second baseline. Steps 131 and 133 are therefore always performed. In step 133, the second set of nodes is selected from at least a first potential set of nodes and a second potential set of nodes based on the human presence being detected in the first zone. Thus, a different set of nodes is selected if human presence is detected in step 131 than if no human presence is detected in step 131.

Next, a step 135 comprises comparing the characteristic of the received second set of radio frequency signals with at least a second baseline. The second baseline is associated with the second set of nodes. In the embodiment of Fig. 7, the first baseline has been recorded when no human was present in the first zone and the second baseline has been recorded when no human was present in the second zone.

A step 137 is performed after step 113. Step 137 comprises detecting the physical state of the object in the second zone based on a result of step 135. For example, a first physical state of the object may be detected if the characteristic determined in step 107 is equal to the second baseline, e.g. if the differences are negligible.

A fourth embodiment of the method of detecting human and/or animal presence based on changes in received radio frequency signals is shown in Fig. 8. In the fourth embodiment, the second set of one or more radio frequency signals are caused to be transmitted by the at least one node of the second set of nodes and received by the at least one other node of the second set of nodes in step 105 in dependence on the result of the comparison of step 109.

If the first potential set of nodes is selected as second set of nodes in step 133, then the second set of one or more radio frequency signals is not transmitted by the second potential set of nodes. On the other hand, if the second potential set of nodes is selected as second set of nodes in step 133, then the second set of one or more radio frequency signals is not transmitted by the first potential set of nodes. For this reason, steps 105 and 107 are performed after step 133 has been performed. Steps 135 and 137 are performed after 107 has been performed in the manner described in relation to Fig. 7.

Fig. 9 shows an example of three zones that could be created with RF sensing nodes 32-34 and 37-39 of Fig. 2, to illustrate the methods of Figs. 7 and 8. In this example, the goal is to detect the physical state of door 49. The physical state of door 49 could be detected with nodes 32 and 37 or with nodes 32, 33, 37 and 38. Since nodes 32, 33, 37 and 38 provide better detection results when no one is present in office 45 and nodes 32 and 37 provide better detection results when someone is present in office 45, it is first checked whether a human is present in office 45.

In the example of Fig. 9, the first zone 61 comprises a first set of nodes 33, 34, 38 and 39. This first zone 61 is used to detect human presence in office 45. If human presence is detected in zone 61, a second zone 63 comprising nodes 32 and 37 is formed. If not, a second zone 62 comprising nodes 32, 33, 37 and 38 is formed. The selected second zone, second zone 62 or second zone 63, is then used to detect whether the door 49 is open or closed. If necessary, the nodes in the selected second zone are instructed to start scanning the selected second zone, as described in relation to Fig. 8. If always the same nodes are transmitting RF signals and the other nodes receive all RF signals from transmitting nodes in their vicinity, the applicable data only needs to be selected from the data collected by all the receiving nodes, as described in relation to Fig. 7.

In the embodiments of Figs. 7 and 8, the physical state of the object is determined. For example, it may be interesting to know whether a specific door in an area is open or closed for the following reasons:

• A door to the outside being opened could be an intruder coming in. A security alarm could be triggered to notify a security service;

• Opening of an emergency exit could be a sign of an emergency situation or could be a person leaving the building in an “illegal” way or could make illegal entry possible. A security organization may be warned in case emergency exits are used.

The system is typically trained beforehand by the user to determine what the measured signals in the room look like, e.g. when a door is open or closed, and based on this, determines dynamically by comparison in which state the room is.

A fifth embodiment of the method of detecting human and/or animal presence based on changes in received radio frequency signals is shown in Fig. 10. The embodiment of Fig. 10 is a variation on the embodiment of Fig. 8. In the embodiment of Fig. 10, steps 151- 159 replace steps 135 and 137 of Fig. 8.

Step 151 is performed after the second set of nodes has been selected in step 133, the second set of radio frequency signals has been caused to be transmitted in step 105, and the characteristic of the received second set of radio frequency signals has been determined in step 107. Step 151 comprises comparing the characteristic of the received second set of radio frequency signals, as determined in step 107, with a first one of multiple (second) baselines. The multiple baselines are associated with the second set of nodes and each of the multiple baselines are associated with a different physical state of the object.

If the characteristic determined in step 107 is equal to the first one of the multiple baselines, e.g. the differences are negligible, a step 153 is performed after step 151. In step 153, the object is detected to have a first physical state. If the characteristic determined in step 107 is not equal to the first one of the multiple baselines, a step 155 is performed after step 151. Step 155 comprises comparing the characteristic of the received second set of radio frequency signals, as determined in step 107, with a second one of the multiple (second) baselines. If the characteristic determined in step 107 is equal to the second one of the multiple baselines, e.g. the differences are negligible, a step 159 is performed after step 155. In step 159, the object is detected to have a second physical state. If the characteristic determined in step 107 is not equal to the second one of the multiple baselines, a step 157 is performed after step 155. In step 157, the object is detected to have an unknown physical state.

Fig. 11 shows an example of an object state associated with a first baseline. In the example of Fig. 11, the object 49 in zone 62 is a door 49 whose physical state is open.

Fig. 12 shows an example of an object state associated with a second baseline. In the example of Fig. 12, the object 49 in zone 62 is a door 49 whose physical state is closed.

With these multiple baselines, specific baselines can be established per zone, which can be used to detect different types of changes in the environment. A deviation relative to the baseline of the current physical state may trigger detection of a physical state change. For each zone in an area, a different baseline is typically used, as each zone has its own RSSI/CSI characteristics (e.g. because of furniture or texture of the walls). Any node in the network may be able to act as a detector, or in case RSSI/CSI data is forwarded into some other device (e.g. local controller or Internet server), it may be processed in that other device.

Multiple baselines may be used in the example of Fig. 9, for example. Fig. 9 shows three zones: zone 61, zone 62 and zone 63. Zone 61 may have a baseline- 1 as a reference. Zone 62 may have baseline-2 and baseline-3 as a reference. Zone 63 may have baseline-4 and baseline-5 as a reference. All nodes in a zone are within reach of each other. The controller or application may be able to decide whether the collected data gets used or not.

The baselines may be defined as follows:

• Baseline-1: All nodes in zone 61 (33, 34, 38 and 39) are sending and receiving wireless packets. At commissioning time or dynamically at a later stage, the baseline is established by measuring the signal strength/parameter values during a certain period (e.g. 30 seconds) with no persons in the room.

• Baseline-2: nodes 33 and 38 are inside office 45 and nodes 32 and 37 are part of the hallway 43. Nodes 33 and 38 are sending traffic in zone 62, which will be received by nodes 32 and 37. Nodes 32 and 37 only listen and collect their packets within their zone. At commissioning time, this baseline is established by measuring the signal strength/parameter values during a certain period (e.g. 30 seconds) when the door 49 is closed and with no persons in the office 45.

• Baseline-3 : see baseline-2, but now with the door 49 open.

• Baseline-4: nodes 32 and 37 are outside the office 45, in hallway 43 only, and keep an “eye” on the door 49 to office 45. Nodes 32 and 37 are sending traffic in zone 63. At commissioning time, the baseline is established by measuring the signal strength values during a certain period (e.g. 30 seconds) when the door 49 is closed and with no persons in the office 45.

• Baseline-5: see baseline-4, but now with door 49 open.

In one embodiment, nodes receive packets from all other nodes that are in reach of the node. Packets may contain a zone identification to distinguish the zones from each other. The nodes 32 and 37 are in the hallway 43 adjacent to offices 42 and 45. They cannot only be used for detection of the physical state of door 49 to office 45, but also for the same kind of detection of the physical state of a door to office 42 (not depicted in Fig. 9), as these nodes are also in reach of the nodes in office 42. Same zoning and baselining may be applied with respect to office 42. During configuration, these zones may be defined and established. Multiple baselines are needed when there are two doors in zone 63 to detect the physical states of both doors, e.g. one baseline for each combination of open/closed state of the two doors.

The system has knowledge of these multiple zones and their baselines and detects (a) which baseline is most likely with the current and past measurements (and the corresponding environmental state, e.g. including whether the door is open or closed) as well as (b) deviations from either baseline.

If a human body is present in office 45 while the state of the door 49 is to be detected, the following strategy can be used for detection of the state of the door 49 to the office 45 (which may be a private office, for example):

1. Result zone 61 : Analyze the sensing data in office 45, covered by the nodes in the zone 61 (33, 34, 38, 39), and compare the actual collected sensing data with the reference of baseline-1 sensing data and confirm that a human body mass is present in this zone.

2. Result zone 62: Analyze the sensing data in the hallway 43, covered by the nodes in zone 62 (nodes 33, 34, 38, 39), and compare the actual collected sensing data with the reference of baselines 2&3. 3. Result zone 63 : Analyze the sensing data in the hallway 43, covered by the nodes in zone 63 (32 and 37), and compare the actual collected sensing data with the reference of baselines 4&5.

4. If a human body mass is detected in the office 45, the algorithm mostly relies on the results of zone 63 to determine the open/close status of the door.

5. If no human body mass is detected in the office 45, the algorithm uses use the results of zone 62 to determine the open/close status of the door.

The user typically has to teach the system what the "known" baselines are and may be able to apply semantics to them (e.g. which one is "good" (door/window closed) and which one is cause for alert (door/window open)). More than two baselines could be used. If the results are different from either known baseline, the system may alert the staff to look at the situation as the layout of furniture (or lights) may have changed, which may result in corrective action (putting the furniture back in place), or re-baselining (accepting the changed situation).

In the above example, the physical state of a door was detected. This may not only be used to send an alarm, but alternatively or additionally to perform path tracking, i.e. to give an indication of possible paths followed by a potential intruder within a home or other building. For example, if by the time of leaving their home, a family closes the doors leading to each individual rooms and later a security system determines that someone broke in, the detection results might be used to help insurance companies evaluate whether objects have been stolen from specific rooms, as it is possible to analyze whether these rooms where entered or not by checking which doors have been opened.

The system may not only be able to detect the physical state of a wooden door or PVC door, but also to detect the physical state of a glass door, e.g. in office meeting rooms and huddle rooms. Security cameras are known to have difficulties to determine the open/closed state of a glass door. As the wireless signal is also attenuated when passing through glass, it will be possible to detect the door status of a glass door based on analyzing the RSSI or CSI between two nodes. If required, the signal strength of the CSI Wi-Fi sensing can be reduced to maximize the impact of the glass door in the wireless multi-path signals and hence enable reliable detection of the orientation of the door. The same principle may be applied to a glass window.

A baseline may be recorded per physical state of the object, e.g. door. Alternatively or additionally, a signature/pattern may be recorded based on subsequent changes in received RF signals to detect the physical state of the object being changed, e.g. a door being opened or closed. For example, the acceleration and deceleration of a door may be recorded by the RF sensing system (e.g. via Wi-Fi CSI), This acceleration and deceleration signature/pattern may be used to distinguish whether the door is getting opened or closed (door hitting the doorframe and then locking has a different deceleration pattern from a door being opened and bouncing back from door stopper).

In addition to or instead of using the system to detect the physical state of a door or window, it may be possible to use the system to detect the physical state of a drawers. For example, RF sensing may be used determine the drawer-status of office furniture such as file cabinet drawers and tables with adjustable desk height. File cabinet drawers, which are left open, both impede the traffic in corridors (safety risk) as well as raising confidentiality concerns (e.g. an open drawer in the evening indicates that the drawer in the law firm has not been properly locked).

In addition, the RF -based sensing system may detect that drawers have been opened in the evening by the security guard (or other persons) raising concerns about theft. Similar to path tracking, the opening/close of drawers additionally will indicate which drawers may have been compromised by the intruder. This also applies to (metal) doors of office filing cabinets.

Baselines may be recorded for the drawer being open and closed. Alternatively or additionally, the acceleration and deceleration of the office drawer may be recorded by the RF-based sensing system (e.g. via Wi-Fi CSI). This acceleration & deceleration signature/pattern may be used to distinguish whether the drawer is getting opened or closed (a drawer hitting the back and then locking into position has a different deceleration signature/pattern than a drawer being opened).

The embodiments of Figs. 3, 4, 7, 8, and 10 differ from each other in multiple aspects, i.e. multiple steps have been added or replaced. In variations on these embodiments, only a subset of these steps is added or replaced and/or one or more steps are omitted. For example, not only the embodiment of Fig. 8, but also the embodiments of Figs. 3, 4 and 7 may be extended to support multiple second baselines.

Fig. 13 depicts a block diagram illustrating an exemplary data processing system that may perform the method as described with reference to Figs. 3, 4, 7, 8, and 10.

As shown in Fig. 13, the data processing system 300 may include at least one processor 302 coupled to memory elements 304 through a system bus 306. As such, the data processing system may store program code within memory elements 304. Further, the processor 302 may execute the program code accessed from the memory elements 304 via a system bus 306. In one aspect, the data processing system may be implemented as a computer that is suitable for storing and/or executing program code. It should be appreciated, however, that the data processing system 300 may be implemented in the form of any system including a processor and a memory that is capable of performing the functions described within this specification. The data processing system may be an Intemet/cloud server, for example.

The memory elements 304 may include one or more physical memory devices such as, for example, local memory 308 and one or more bulk storage devices 310. The local memory may refer to random access memory or other non-persistent memory device(s) generally used during actual execution of the program code. A bulk storage device may be implemented as a hard drive or other persistent data storage device. The processing system 300 may also include one or more cache memories (not shown) that provide temporary storage of at least some program code in order to reduce the quantity of times program code must be retrieved from the bulk storage device 310 during execution. The processing system 300 may also be able to use memory elements of another processing system, e.g. if the processing system 300 is part of a cloud-computing platform.

Input/output (I/O) devices depicted as an input device 312 and an output device 314 optionally can be coupled to the data processing system. Examples of input devices may include, but are not limited to, a keyboard, a pointing device such as a mouse, a microphone (e.g. for voice and/or speech recognition), or the like. Examples of output devices may include, but are not limited to, a monitor or a display, speakers, or the like.

Input and/or output devices may be coupled to the data processing system either directly or through intervening EO controllers.

In an embodiment, the input and the output devices may be implemented as a combined input/output device (illustrated in Fig. 13 with a dashed line surrounding the input device 312 and the output device 314). An example of such a combined device is a touch sensitive display, also sometimes referred to as a “touch screen display” or simply “touch screen”. In such an embodiment, input to the device may be provided by a movement of a physical object, such as e.g. a stylus or a finger of a user, on or near the touch screen display.

A network adapter 316 may also be coupled to the data processing system to enable it to become coupled to other systems, computer systems, remote network devices, and/or remote storage devices through intervening private or public networks. The network adapter may comprise a data receiver for receiving data that is transmitted by said systems, devices and/or networks to the data processing system 300, and a data transmitter for transmitting data from the data processing system 300 to said systems, devices and/or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapter that may be used with the data processing system 300.

As pictured in Fig. 13, the memory elements 304 may store an application 318. In various embodiments, the application 318 may be stored in the local memory 308, the one or more bulk storage devices 310, or separate from the local memory and the bulk storage devices. It should be appreciated that the data processing system 300 may further execute an operating system (not shown in Fig. 13) that can facilitate execution of the application 318. The application 318, being implemented in the form of executable program code, can be executed by the data processing system 300, e.g., by the processor 302. Responsive to executing the application, the data processing system 300 may be configured to perform one or more operations or method steps described herein.

Various embodiments of the invention may be implemented as a program product for use with a computer system, where the program(s) of the program product define functions of the embodiments (including the methods described herein). In one embodiment, the program(s) can be contained on a variety of non-transitory computer-readable storage media, where, as used herein, the expression “non-transitory computer readable storage media” comprises all computer-readable media, with the sole exception being a transitory, propagating signal. In another embodiment, the program(s) can be contained on a variety of transitory computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., flash memory, floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory) on which alterable information is stored. The computer program may be run on the processor 302 described herein.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of embodiments of the present invention has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the implementations in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present invention. The embodiments were chosen and described in order to best explain the principles and some practical applications of the present invention, and to enable others of ordinary skill in the art to understand the present invention for various embodiments with various modifications as are suited to the particular use contemplated.