Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
SYSTEM AND METHOD FOR COUNTING AND LOCALIZING OCCUPANTS USING THERMAL IMAGES
Document Type and Number:
WIPO Patent Application WO/2018/177973
Kind Code:
A1
Abstract:
A method (300) to characterize one or more occupants within an environment (200) includes the steps of: obtaining (320), by a thermopile array (14) of each of a plurality of thermal sensors (12) of a sensor system (10), one or more thermal images from within the environment, wherein at least one of the thermal sensors is positioned along a first surface of the environment and at least one of the thermal sensors is positioned along a second surface of the environment; fusing (340), by the controller, a thermal image obtained by a thermal sensor positioned along the first surface with a thermal image obtained by a thermal sensor positioned along the second surface into a fused image; characterizing (360), from the fused image, a number of occupants in the environment; characterizing (360), from the fused image, a location of each of the occupants in the environment; and communicating (380) the characterized number of occupants and the characterized location of each of the occupants to an environmental control system (30).

Inventors:
GOPAL SAMY MATHAN (NL)
HEGAZY TAMIR (NL)
Application Number:
PCT/EP2018/057583
Publication Date:
October 04, 2018
Filing Date:
March 26, 2018
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
PHILIPS LIGHTING HOLDING BV (NL)
International Classes:
G06T3/40; G06V10/143
Foreign References:
US20120106782A12012-05-03
Other References:
GADE RIKKE ET AL: "Long-Term Occupancy Analysis Using Graph-Based Optimisation in Thermal Imagery", IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION. PROCEEDINGS, IEEE COMPUTER SOCIETY, US, 23 June 2013 (2013-06-23), pages 3698 - 3705, XP032492931, ISSN: 1063-6919, [retrieved on 20131002], DOI: 10.1109/CVPR.2013.474
T. AKIN: "CMOS-based Thermal Sensors", ADVANCED MICRO AND NANOSYSTEMS, 1 January 2005 (2005-01-01), pages 479 - 512, XP055407124, ISBN: 3-527-31080-0, Retrieved from the Internet [retrieved on 20170915]
Attorney, Agent or Firm:
VAN EEUWIJK, Alexander, Henricus, Walterus et al. (NL)
Download PDF:
Claims:
CLAIMS:

1. A method (300) to characterize one or more occupants within an environment (200) using thermal imaging, the method comprising the steps of:

obtaining (320), by a thermopile array (14) of each of a plurality of thermal sensors (12) of a sensor system (10), one or more thermal images from within the

environment, wherein at least one of the plurality of thermal sensors is positioned along a first surface of the environment, and further wherein at least one of the plurality of thermal sensors is positioned along a second surface of the environment;

fusing (340), by the controller, a thermal image obtained by a thermopile array of the thermal sensor positioned along the first surface with a thermal image obtained by a thermopile array of the thermal sensor positioned along the second surface into a fused image, wherein the fused image contains information from thermopile arrays that obtain thermal information about the environment from at least two different angles;

characterizing (360), from the fused image, a number of occupants in the environment;

characterizing (360), from the fused image, a location of each of the occupants in the environment; and

communicating (380) the characterized number of occupants and the characterized location of each of the occupants to an environmental control system (30).

2. The method of claim 1 , further comprising the step of fusing (330), by a controller (20) of the sensor system, thermal images obtained from the thermopile arrays of at least two thermal sensors positioned along the first surface or second surface into a single image.

3. The method of claim 1 , further comprising the step of removing (350), by the controller, non-human thermal information from the fused image.

4. The method of claim 1 , further comprising the step of characterizing (370) an activity of one or more of the occupants in the environment.

5. The method of claim 4, wherein the activity comprises sitting, standing, or lying down. 6. The method of claim 4, wherein the step of characterizing an activity of one or more of the occupants in the environment comprises the step of extracting, from the fused image, one or more parameters of an identified occupant.

7. The method of claim 1, wherein the sensor system comprises commissioning information about the environment.

8. The method of claim 1, the step of characterizing a location of each of the occupants in the environment comprises the steps of: (i) identifying, from the fused image, a feature characterizing an occupant in the environment; and (ii) identifying coordinates for the identified feature.

9. The method of claim 1, wherein the environmental control system is a lighting system, an HVAC system, or a sound system. 10. A networked sensor system (10) for characterizing one or more occupants within an environment (200) using thermal imaging, the system comprising:

two or more thermal sensors (12) each comprising a thermophile array (14) configured to obtain one or thermal images from within the environment, wherein at least one of the thermal sensors is positioned along a first surface of the environment, and further wherein at least one of the thermal sensors is positioned along a second surface of the environment; and

a central unit (28) in communication with each of the two or more thermal sensors and comprising a communications module (18) and a controller (20), the controller configured to: (i) fuse a thermal image obtained by a thermal sensor positioned along the first surface with a thermal image obtained by a thermal sensor positioned along the second surface into a fused image, the fused image containing infromation from thermopile arrays that obtain thermal information about the environment from at least two different angles; (ii) characterize, from the fused image, a number of occupants in the environment; and (iii) characterize, from the fused image, a location of each of the occupants in the environment; wherein the communications module is configured to communicate the characterized number of occupants and the characterized location of each of the occupants to an environmental control system (30). 11. The system of claim 10, wherein the controller is further configured to fuse thermal images obtained from the thermopile arrays of at least two thermal sensors positioned along the first surface or second surface into a single image.

12. The system of claim 10, wherein the controller is further configured to characterize an activity of one or more of the occupants in the environment.

13. The system of claim 10, wherein characterizing a location of each of the occupants in the environment comprises: (i) identifying, from the fused image, a feature characterizing an occupant in the environment; and (ii) identifying coordinates for the identified feature.

14. A system (10) for characterizing one or more occupants within an environment (200) using thermal imaging, the system comprising:

two or more thermal sensors (12) each comprising a thermophile array (14) configured to obtain one or thermal images from within the environment, wherein at least one of the thermal sensors is positioned along a first surface of the environment, and further wherein at least one of the thermal sensors is positioned along a second surface of the environment;

a central unit (28) in communication with each of the two or more thermal sensors and comprising a controller (20) configured to: (i) fuse a thermal image obtained by a thermal sensor positioned along the first surface with a thermal image obtained by a thermal sensor positioned along the second surface into a fused image, the fused image containing infromation from thermopile arrays that obtain thermal information about the environment from at least two different angles; (ii) characterize, from the fused image, a number of occupants in the environment; and (iii) characterize, from the fused image, a location of each of the occupants in the environment; and

an environmental control system (30) configured to adjust a parameter of the environment based at least in part on the characterized number of occupants or the characterized location of each of the occupants.

15. The system of claim 14, wherein the environmental control system is a lighting system, an HVAC system, or a sound system.

Description:
SYSTEM AND METHOD FOR COUNTING AND LOCALIZING OCCUPANTS USING THERMAL IMAGES

FIELD OF THE INVENTION

The present disclosure is directed generally to systems and methods for occupancy detection, counting, and location. BACKGROUND

Occupancy-based control (OBC) is a popular energy saving strategy applied in modern spaces. These systems typically employ one or more passive infrared (PIR) sensors to detect occupants within the space. With an OBC system, the space can control electricity and HVAC energy consumption by turning off lights or lowering the temperature of the space when it is unoccupied.

It has recently been shown that occupancy sensors capable of counting the number of occupants within a space could conserve even more energy when compared to conventional PIR-based sensors and control. For example, energy consumption by HVAC systems can be optimized by adapting to the system to the actual number of occupants within the space.

However, counting the number of occupants within a space is challenging. Traditional PIR-based sensor systems are reasonably good at detecting the presence of occupants within a space, although even these systems typically require at least periodic movement by the occupant(s) to avoid an unintended lowering of energy consumption. Traditional sensor and control systems are unable to count the number of occupants within a space.

One method of counting the number of occupants within a space includes video sensors. Indeed, video-based occupant counting solutions are popular in retail spaces where video cameras are employed to recognize and count occupants. However, video-based occupant counting solutions are not practical in most buildings, including residential buildings and office buildings due to privacy concerns. Accordingly, there is a continued need in the art for sensor and control methods and systems that enable accurate detection and counting of occupants within a space.

SUMMARY OF THE INVENTION

The present disclosure is directed to inventive methods and apparatus for detecting and counting occupants within a space using multiple thermal sensors. Various embodiments and implementations herein are directed to a networked system of two or more sensors each comprising an array of thermopiles. The sensors are situated in a room or other space to generate sensor data representative of the occupants within the space. The system analyzes the sensor data to identify the presence of occupants, count the number of occupants, and potentially localize the occupants within the space. Using this information, the system can control, adjust or regulate one or more connected systems, including lighting, HVAC, and other systems.

Generally, in one aspect, a method for characterizing one or more occupants within an environment using thermal imaging is provided. The method includes the steps of: (i) obtaining, by a thermopile array of each of a plurality of thermal sensors of a sensor system, one or more thermal images from within the environment, wherein at least one of the thermal sensors is positioned along a first surface of the environment, and further wherein at least one of the thermal sensors is positioned along a second surface of the environment; (ii) fusing, by the controller, a thermal image obtained by a thermal sensor positioned along the first surface with a thermal image obtained by a thermal sensor positioned along the second surface into a fused image; (iii) characterizing, from the fused image, a number of occupants in the environment; (iv) characterizing, from the fused image, a location of each of the occupants in the environment; and (v) communicating the characterized number of occupants and the characterized location of each of the occupants to an environmental control system.

According to an embodiment, the method further includes the step of fusing, by a controller of the sensor system, thermal images obtained from the thermopile arrays of at least two thermal sensors positioned along the first surface or second surface into a single image.

According to an embodiment, the method further includes the step of removing, by the controller, non-human thermal information from the fused image.

According to an embodiment, the method further includes the step of characterizing an activity of one or more of the occupants in the environment. According to an embodiment, the activity comprises sitting, standing, or lying down. According to an embodiment, the step of characterizing an activity of one or more of the occupants in the environment comprises the step of extracting, from the fused image, one or more parameters of an identified occupant.

According to an embodiment, the sensor system comprises commissioning information about the environment.

According to an embodiment, the step of characterizing a location of each of the occupants in the environment comprises the steps of: (i) identifying, from the fused image, a feature characterizing an occupant in the environment; and (ii) identifying coordinates for the identified feature.

According to an embodiment, the environmental control system is a lighting system, an HVAC system, or a sound system.

According to an embodiment, at least one of the two or more thermal sensors are positioned on a ceiling within the environment, and at least one of the two or more thermal sensors are positioned on a wall within the environment.

According to an aspect is a networked sensor system configured to characterize one or more occupants within an environment using thermal imaging. The system includes: two or more thermal sensors each comprising a thermophile array configured to obtain one or thermal images from within the environment, wherein at least one of the thermal sensors is positioned along a first surface of the environment, and further wherein at least one of the thermal sensors is positioned along a second surface of the environment; a central unit in communication with each of the two or more thermal sensors and comprising a communications module and a controller, the controller configured to: (i) fuse a thermal image obtained by a thermal sensor positioned along the first surface with a thermal image obtained by a thermal sensor positioned along the second surface into a fused image; (ii) characterize, from the fused image, a number of occupants in the environment; and (iii) characterize, from the fused image, a location of each of the occupants in the environment; wherein the communications module is configured to communicate the characterized number of occupants and the characterized location of each of the occupants to an environmental control system.

According to an embodiment, the controller is further configured to fuse thermal images obtained from the thermopile arrays of at least two thermal sensors positioned along the first surface or second surface into a single image. According to an aspect is a system configured to characterize one or more occupants within an environment using thermal imaging. The system includes: two or more thermal sensors each comprising a thermophile array configured to obtain one or thermal images from within the environment, wherein at least one of the thermal sensors is positioned along a first surface of the environment, and further wherein at least one of the thermal sensors is positioned along a second surface of the environment; a central unit in

communication with each of the two or more thermal sensors and comprising a controller configured to: (i) fuse a thermal image obtained by a thermal sensor positioned along the first surface with a thermal image obtained by a thermal sensor positioned along the second surface into a fused image; (ii) characterize, from the fused image, a number of occupants in the environment; and (iii) characterize, from the fused image, a location of each of the occupants in the environment; and an environmental control system configured to adjust a parameter of the environment based at least in part on the characterized number of occupants or the characterized location of each of the occupants.

In various implementations, a processor or controller may be associated with one or more storage media (generically referred to herein as "memory," e.g., volatile and non-volatile computer memory such as RAM, PROM, EPROM, and EEPROM, floppy disks, compact disks, optical disks, magnetic tape, etc.). In some implementations, the storage media may be encoded with one or more programs that, when executed on one or more processors and/or controllers, perform at least some of the functions discussed herein.

Various storage media may be fixed within a processor or controller or may be transportable, such that the one or more programs stored thereon can be loaded into a processor or controller so as to implement various aspects of the present invention discussed herein. The terms "program" or "computer program" are used herein in a generic sense to refer to any type of computer code (e.g., software or microcode) that can be employed to program one or more processors or controllers.

In one network implementation, one or more devices coupled to a network may serve as a controller for one or more other devices coupled to the network (e.g., in a master/slave relationship). In another implementation, a networked environment may include one or more dedicated controllers that are configured to control one or more of the devices coupled to the network. Generally, multiple devices coupled to the network each may have access to data that is present on the communications medium or media; however, a given device may be "addressable" in that it is configured to selectively exchange data with (i.e., receive data from and/or transmit data to) the network, based, for example, on one or more particular identifiers (e.g., "addresses") assigned to it.

The term "network" as used herein refers to any interconnection of two or more devices (including controllers or processors) that facilitates the transport of information (e.g. for device control, data storage, data exchange, etc.) between any two or more devices and/or among multiple devices coupled to the network. As should be readily appreciated, various implementations of networks suitable for interconnecting multiple devices may include any of a variety of network topologies and employ any of a variety of communication protocols. Additionally, in various networks according to the present disclosure, any one connection between two devices may represent a dedicated connection between the two systems, or alternatively a non-dedicated connection. In addition to carrying information intended for the two devices, such a non-dedicated connection may carry information not necessarily intended for either of the two devices (e.g., an open network connection).

Furthermore, it should be readily appreciated that various networks of devices as discussed herein may employ one or more wireless, wire/cable, and/or fiber optic links to facilitate information transport throughout the network.

It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein. It should also be appreciated that terminology explicitly employed herein that also may appear in any disclosure incorporated by reference should be accorded a meaning most consistent with the particular concepts disclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.

FIG. 1 is a schematic representation of a networked sensor system, in accordance with an embodiment.

FIG. 2 is a schematic representation of a networked sensor system within an environment, in accordance with an embodiment. FIG. 3 is a flowchart of a method for determining the number of occupants within an environment, in accordance with an embodiment.

FIG. 4 is a schematic representation of thermal information obtained by a networked sensor system within an environment, in accordance with an embodiment.

FIG. 5 is a schematic representation of a networked sensor system within an environment, in accordance with an embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS

The present disclosure describes various embodiments of a networked sensor system configured to detect and count occupants within a space. More generally, Applicant has recognized and appreciated that it would be beneficial to provide a networked sensor system with two or more sensors each comprising an array of thermopiles. A particular goal of utilization of certain embodiments of the present disclosure is to characterize occupancy of the space using the thermal imaging information.

In view of the foregoing, various embodiments and implementations are directed to a networked sensor system with a plurality of thermopile array sensors. A processor of the networked sensor system receives and analyzes the thermal image information from the thermopile array sensors to determine the occupancy of the space. The system may also determine or estimate the location of the occupants within the space, and patterns of occupancy such as sitting, standing, and other occupant characteristics. The system optionally uses that occupancy information to inform and/or control lighting, HVAC, and other systems.

Referring to FIG. 1, in one embodiment, is a networked sensor system 10. The sensor system comprises two or more thermal sensors 12a, 12b, a central unit 28 in communication with the two or more thermal sensors, and an environmental control system 30. Many other configurations and embodiments are possible.

Thermal sensors 12a, 12b each comprise at least one thermopile sensor array 14a, 14b. A thermopile is a non-contact sensor that measures absolute temperature using infrared radiation emitted by the heat source. A thermopile is different from a pyroelectric (PIR) sensor, which only measures temperature gradient. PIRs are insensitive to stationary occupants while thermopiles can detect non-moving individuals, as the measured absolute temperature in the viewing region will be higher than that of environment. An 8x8 thermopile array sensor, for example, divides a viewing area into 8x8 cells and provides 64 absolute temperatures per measurement, one for each cell. The thermopile array may be any size, which may optionally be determined by cost, the size of the space, the intended use of the sensor system, and/or other parameters.

The thermopile sensor array of each thermal sensor obtains thermal information, such as absolute temperatures, continuously or periodically. For example, the system may be configured or programmed to obtain thermal information at the same frequency at all times. The thermal sensors may be configured to obtain thermal information at any frequency, such as once a second, once a minute, once every five minutes, one every 15 minutes, and any other interval. The frequency is not necessarily a fixed value, and can optionally be modifiable by a user or installer of the system. For example, the system may be configured or programmed to obtain thermal information at different frequencies throughout the day, week, or other timeframes. As an example, the thermal sensors can obtain thermal information at a higher frequency during the day when a space is more likely to be occupied and at a lower frequency during the night when a space is less likely to be occupied. The expected use of the space may also influence the frequency with which thermal information is obtained by the thermal sensors. For example, if the space is only used for meetings on Wednesdays, the system may be configured or programmed to obtain thermal information, and/or obtain thermal information at a higher frequency, on Wednesdays around the time the meetings are held. The system may be integrated with a scheduling calendar to increase the thermal sensor activity during times when the space is scheduled to be utilized, and to limit or cease thermal sensor activity when the space is not scheduled to be utilized.

The thermal sensors 12a, 12b each comprise a wired or wireless communications module 16a, 16b configured to communicate the thermal information obtained from the thermopiles with the central unit 28. The thermal sensors may be in wired connection to the central unit 28, or may communicate with the central unit via a wireless protocol such as Wi-Fi, Bluetooth, IR, radio, near field communication, and/or any other protocol. The thermal sensors may be configured or programmed to continuously or periodically communicate information to the central unit, and the frequency may be adjusted based on a variety of factors including those discussed herein. The thermal sensors may also be configured or programmed to provide thermal information or other information to the central unit in response to a query for information from the central unit.

Referring to FIG. 2, in one embodiment, is an environment 200 which includes a networked sensor system 10. The networked sensor system comprises two thermal sensors 12a, 12b, although the system may include many more thermal sensors as needed. In this embodiment, one of the thermal sensors 12a is positioned on or in the ceiling and the other thermal sensor 12b is positioned on or in one wall of the environment 200. Both the thermal sensors communicate with a central unit 28, which may be located within the space, or may be located remotely. For example, the central unit may be located in a single location within a building, and can communicate via wired and/or wireless communication to sensors or sensor systems throughout the building. Additionally, for a network of buildings the central unit may be located in a single location, such as within one of the buildings or within a location remote from the buildings, and can receive information via wired and/or wireless

communication to the sensors or sensor systems located throughout the network of buildings.

The thermal sensors 12a, 12b may be placed anywhere within the environment 200. There may be an optimal placement of the sensors, for example, that depends upon numerous factors such as the size and shape of the room. For example, the sensors may be positioned to allow for the entire room to be analyzed by one or more of the sensors, and this will be informed at least in part on the size and/or shape of the room. The sensors may also be positioned to only analyze a portion of the room. This configuration can be predetermined using maps, blueprints, or other information about the rooms, or can be determined during installation and/or testing of the networked sensor system. The configuration can later be modified or adjusted, for example, if the original placement is determined to be less than optimal, or if the use of the room changes over time.

According to an embodiment, existing structures within the room may be adapted, modified, or otherwise utilized to host a thermal sensor. For example, switches, outlets, thermostats, luminaires, and other structures can comprise one or more thermal sensors. In FIG. 2, for example, thermal sensor 12a may be housed within and facing outwardly from a luminaire, while thermal sensor 12b may be housed within and facing outwardly from a thermostat. Accordingly, although the thermal sensors 12a, 12b in FIG. 2 are depicted as protruding into the environment 200 for purposes of illustration, the sensors may in fact be unidentifiable or nearly unidentifiable by occupants of the room.

Referring again to FIG. 1, the networked sensor array 10 further comprises a central unit 28 configured to receive thermal information from the two or more thermal sensors. The central unit comprises a wired or wireless communications module 18 configured to communicate with the two or more thermal sensors, including receiving obtained thermal information from the sensors. The communications module may facilitate wired communications via a LAN or any other wired network. The communications module may also or alternatively facilitate wireless communications and may communicate via a wireless protocol such as Wi-Fi, Bluetooth, IR, radio, near field communication, and/or any other protocol. The wired or wireless communications module 18 can also be configured to communicate with an environmental control system 30, as described herein.

The central unit 28 comprises a controller 20 configured or programmed to control or facilitate functionality of the networked sensor system. Controller 20 comprises a processor 22 programmed using software to perform one or more of the various functions discussed herein, and can be utilized in combination with a memory 24. Memory 24 can store data, including one or more commands or software programs for execution by processor 22, as well as various types of data including but not limited to thermal information obtained by the thermal sensors. For example, the memory 24 may be a non-transitory computer readable storage medium that includes a set of instructions that are executable by processor 22, and which cause the system to execute one or more of the steps of the methods described herein.

The networked sensor system also includes a source of power 26, most typically AC power, although other power sources are possible including DC power sources, solar-based power sources, or mechanical-based power sources, among others. The power source may be in operable communication with a power source converter that converts power received from an external power source to a form that is usable by the lighting unit. In order to provide power to the various components of the system, it can also include an AC/DC converter (e.g., rectifying circuit) that receives AC power from an external AC power source 26 and converts it into direct current for purposes of powering the light unit's components. Additionally, the system can include an energy storage device, such as a rechargeable battery or capacitor, that is recharged via a connection to the AC/DC converter and can provide power to controller 20 and/or the thermal sensors 12a, 12b when the circuit to AC power source 26 is opened.

According to an embodiment, the networked sensor array 10 further comprises or is in communication with an environmental control system 30. The sensor system may send information or commands to the environmental control system 30 based at least in part on the thermal information received from one or more of the thermal sensors 12a, 12b. For example, the sensor system may determine that the room is unoccupied because the thermal sensors are not detecting any thermal signature identified as human, and thus send a command to the environmental control system 30. If the environmental control system is a lighting system, the sensor system informs the system that the room is unoccupied and the lighting system dims the lights, turns off the lights, or otherwise modifies the lights. If the environmental control system is an HVAC system, the sensor system informs the system that the room is unoccupied and the HVAC system lowers, raises, or modifies the heat or AC accordingly.

According to another embodiment, the environmental control system is a monitoring system configured to obtain, track, and/or analyze information about the environment 200. For example, the monitoring system may be configured to determine occupancy levels for the space over time, and provide recommendations about space usage or environmental controls. Many other environmental control systems and environmental modifications are possible.

Referring to FIG. 3, in one embodiment, is a flowchart illustrating a method 300 for using thermal imaging to determine or estimate the number of occupants within the environment, localize the occupants within the environment, detect occupant activities within the environment, and/or differentiate between human and non-human heat sources within the environment.

At step 310 of the method, a networked sensor system 10 is provided.

Networked sensor system 10 can be any of the embodiments described herein or otherwise envisioned, and can include any of the components of the systems described in conjunction with FIGS. 1 and 2, such as two or more thermal sensors 12a, 12b, a central unit 28, and an environmental control system 30, among other elements.

At step 320 of the method, the thermal sensors 12a, 12b of sensor system 10 obtain thermal information from the environment 200 in which they are installed. The thermopile array of each sensor receives thermopile measurements, more generically called thermal information, from the environment, and the thermal sensor communicates the thermal information to the controller 20, where the information can be analyzed and/or can be stored within memory 24. According to one embodiment, the thermal sensors obtain thermal information continuously. According to another embodiment, the thermal sensors obtain thermal information periodically, such as one every minute or multiple times per minute, among many other periods of time.

Referring to FIG. 4, in one embodiment, is a schematic representation of wall- mounted thermopile measurements obtained in an environment comprising one occupant 40a and a non-human hot object 50, such as a mug of coffee, CPU, window, or other object within the space. The information is two-dimensional as it is obtained from a single thermopile array 14a.

The processor 22 and/or controller 20 then analyzes the received thermal information and determines whether a human is occupying the space. The system may utilize many different methods to determine whether one or more humans are within environment 200, and where the one or more humans are located. According to one embodiment, the system is programmed with or otherwise obtains commissioning information about the environment 200, such as information about sensor locations, angle and/or tilt of the sensors, fields of view for the sensors including obstructions, ceiling height, shape of the room, 2D and/or 3D floorplan, and/or other information. This information is utilized to determine the boundaries of the area covered by each thermal sensor.

At step 330 of the method, the system fuses the thermal images from two or more thermopile arrays from a first region, area, or surface to generate a fused thermal image (S). For example, referring to FIG. 5, in one embodiment, is an environment 500 which includes a networked sensor system. The networked sensor system comprises two thermal sensors 12a and 12a' on a ceiling surface or region, and two thermal sensors 12b and 12b' on a wall surface or region. At step 330 of the method, the networked sensor system fuses thermal images from 12a and 12a' to generate a fused thermal image (S) that presents overall thermal information from the viewpoint of the ceiling surface or region. Similarly, the networked sensor system fuses thermal images from 12b and 12b' to generate a fused thermal image (S) that presents overall thermal information from the viewpoint of the wall surface or region.

The system receives thermal images from the thermopile arrays of each of the two or more thermal sensors and stitches the images of one thermopile array into a single image S such that any duplication occurring due to overlapping fields of view is removed or otherwise identified and/or labeled. The stitched image S is optionally thresholded to generate a binary image B. The stitched image can be thresholded using any method sufficient to produce a binary image B.

According to an embodiment, binary image B is optionally processed. For example, the image can undergo one or more binary morphological operations such as image opening (image erosion followed by image dilation) and image closing (image dilation, followed by image erosion), among other operations. According to an embodiment, the size of the structuring elements is designed such that they eliminate small spurious blobs and fill any holes in the blobs. The processed image is labeled as image C and is ready for additional analysis.

At step 340 of the method, the system fuses the fused images (S, B, and/or C) into a single fused image (E). For example, binary image B or processed image C from the ceiling-mounted thermopile arrays is combined with binary image B2 or processed image C2 from the wall-mounted thermopile arrays to produce a fused image (E). This provides a single fused image (E) that contains information from thermopile arrays that obtain thermal information about the environment from at least two different angles.

At step 350 of the method, the system analyzes image E to remove any thermal heat signatures with a profile that does not match a human profile. The system can utilize spatial and/or temporal clues to identify non-human profiles. For example, the system can compare heat signatures image E to a database of human heat signatures to determine which heat signatures are similar to human heat signatures, and can remove any heat signatures that do not match the human heat signature. Additionally and/or alternatively, the system can compare heat signatures image E to a database of non-human heat signatures to determine which heat signatures are similar to the non-human heat signatures, and can remove any heat signatures that match or are sufficiently similar to the non-human heat signatures. This method can be based on, for example, a machine learning approach with training data. According to an embodiment, a filtered version F of the stitched image S above is obtained by the equation:

F = E . S (Eq. 1)

where '.' is the element-by-element multiplication operator. As a result only those pixels of S that represent a human will appear.

According to another embodiment, the system can analyze images obtained over time to identify non-human profiles. For example, the system can analyzes images obtained from the same sensor(s) at different times, and/or can analyze images obtained from different sensors at different times. By comparing an image obtained at time point 1 to an image obtained an amount of time x later at time point 1+x, the system can identify cues that identify non- human profiles. Among those cues is temporal decay of a heat source within the environment. Since humans are a relatively stable heat source, an object with significant heat decay is most likely a non-human object. Additionally, the parameters or characteristics of the decay may be utilized to distinguish between non-human objects.

At step 360 of the method, the system characterizes the occupants within the environment using an image such as image F, including the number of occupants of the environment and an approximate location for each of the occupants within the environment. For example, the controller 20 may comprise an algorithm configured to count the humanlike heat signatures remaining in image F.

The system also characterizes an approximate location for each of the occupants within the environment, such as an x,y location of that occupant. According to an embodiment, the system identifies a feature of the measured image, such as the center of mass of each identified occupant in image F, and provides coordinates for that occupant. The system can also track the feature, such as an identified center of mass, over time and/or space using multiple images obtained over time.

According to one embodiment, thermal images can be obtained and analyzed over time. For example, the number and/or location of occupants within the environment may be detected or obtained at a first time Tl, and can be compared to the number and/or location of occupants within the environment at a second time T2. Differences between the Tl thermal image and the T2 thermal image may provide additional information for the system and/or the environmental control systems.

At optional step 370 of the method, the system characterizes an activity of the one or more identified occupants by analyzing the human heat signatures in the modified thermal image. For example, image F may be processed with a segmentation algorithm to separate the human heat signature, and the system can then determine one or more parameters of that human heat signature, including but not limited to spatial volume, major length, minor length, eccentricity, and other features.

According to an embodiment, the system uses one or more of the determined features to classify the human heat signature as standing, sitting, lying down, and/or any of a wide variety of other activities, postures, and positions. The system may utilize a supervised learning algorithm that trains a classifier configured to label these human activities.

At step 380 of the method, the system communicates the characterized number of occupants and/or characterized activity to one or more environmental control systems 30. For example, if the environmental control system is a lighting system, the sensor system informs the lighting system that three people are in the room. The lighting system may then keep the lights stable, or may raise or lower the lights depending on the predetermined thresholds for the space. The sensor system may inform the lighting system that there are four people in the room, and that all of them are lying down. The lighting system may then lower the lights. According to another embodiment, the lighting system may utilize the thermal information to adjust or otherwise adapt the light profile emitted by the lighting unit or system. According to an embodiment, the lighting system can adjust the beam width, angle, and/or intensity of one or more light sources.

A sound or music control system may utilize the information from the characterized number of occupants and/or characterized activities to raise or lower the sound within an environment. For example, a sound system may raise the speaker levels within the environment as the number of occupants in the room increases, and vice versa.

An HVAC system may lower, raise, or otherwise modify the heat or AC based on the number of occupants in the room and/or their activity. For example, an HVAC system may lower heat levels (or AC levels, depending on the time of year) if the occupants are standing and exerting energy, and may raise heat levels if the occupants are sitting. Many other control systems and settings are possible.

According to another embodiment, the networked sensor system 10 is an integral component of the environmental control system 30, and controller 20 of the networked sensor system sends control commands to one or more components within the system to control lighting, HVAC, sound, music, and/or any other environmental factors or parameters.

While several inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein, and each of such variations and/or modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the inventive teachings is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, if such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the inventive scope of the present disclosure.

All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and/or ordinary meanings of the defined terms. The indefinite articles "a" and "an," as used herein in the specification and in the claims, unless clearly indicated to the contrary, should be understood to mean "at least one."

The phrase "and/or," as used herein in the specification and in the claims, should be understood to mean "either or both" of the elements so conjoined, i.e., elements that are conjunctively present in some cases and disjunctively present in other cases.

Multiple elements listed with "and/or" should be construed in the same fashion, i.e., "one or more" of the elements so conjoined. Other elements may optionally be present other than the elements specifically identified by the "and/or" clause, whether related or unrelated to those elements specifically identified. Thus, as a non- limiting example, a reference to "A and/or B", when used in conjunction with open-ended language such as "comprising" can refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

As used herein in the specification and in the claims, "or" should be understood to have the same meaning as "and/or" as defined above. For example, when separating items in a list, "or" or "and/or" shall be interpreted as being inclusive, i.e., the inclusion of at least one, but also including more than one, of a number or list of elements, and, optionally, additional unlisted items. Only terms clearly indicated to the contrary, such as "only one of or "exactly one of," or, when used in the claims, "consisting of," will refer to the inclusion of exactly one element of a number or list of elements. In general, the term "or" as used herein shall only be interpreted as indicating exclusive alternatives (i.e. "one or the other but not both") when preceded by terms of exclusivity, such as "either," "one of," "only one of," or "exactly one of." "Consisting essentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.

As used herein in the specification and in the claims, the phrase "at least one," in reference to a list of one or more elements, should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements and not excluding any combinations of elements in the list of elements. This definition also allows that elements may optionally be present other than the elements specifically identified within the list of elements to which the phrase "at least one" refers, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B," or, equivalently "at least one of A and/or B") can refer, in one embodiment, to at least one, optionally including more than one, A, with no B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, with no A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.

It should also be understood that, unless clearly indicated to the contrary, in any methods claimed herein that include more than one step or act, the order of the steps or acts of the method is not necessarily limited to the order in which the steps or acts of the method are recited.

In the claims, as well as in the specification above, all transitional phrases such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," "composed of," and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases "consisting of and "consisting essentially of shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures.