Technical field

The invention relates to an arrangement and a method for monitoring objects in monitored area, such as a building and/or people conveyor system.

Background

The usage of buildings and different parts of the buildings tends to vary during different times of day and also based on different locations of people conveyor systems, such as elevators, escalators and moving walkways, in the building. Typically for example elevators are often crowded at certain times a day, such as morning rush hour and afternoon rush hour or weekends, depending on the location of the people conveyor systems. This can be problematic in some situations and lead to reduced comfort and/or reduced safety or health for the people in the related areas. In order to be able to provide optimal usage for buildings and for people conveyor systems in the buildings, people flow data and/or people conveyor usage data are collected.

Solutions have been presented in prior art for monitoring people in the elevators. In one prior art solution sensors are arranged to elevator cars and used to observe people in the proximity of elevator doors for controlling the elevator doors. In another prior art solution camera or sensor-based load estimation for elevators is used with which the number of people in the elevator car and elevator waiting area can be monitored.

The solutions of the prior art are not able to provide adequately detailed and reliable people flow information for certain circumstances, such as COVID19- type of situations where social distancing should be ensured in all areas of the community. Also, some of the systems of the prior art are complex and expensive to implement in buildings and/or people conveyor systems. Summary

An object of the invention is to present an arrangement and a method for monitoring objects in monitored area which arrangement and method is able to collect and provide people flow information in a reliable, cost-effective and detailed way so that it can be utilized e.g. for social distance monitoring. With the solution of the invention more detailed information about for people flow in buildings and people conveyor systems, such as elevators, escalators and moving walkways, can be provided and in some embodiments of the invention this can be achieved with the existing sensors of the building and/or the people conveyor systems installed in the building. Based on the determined information also devices or systems, such as people conveyor systems, doors, gates, access control systems, building automation, building management system, can be controlled in order to change the people flow.

According to a first aspect, the invention relates to a method for determining distance between the monitored objects in monitored area with an arrangement comprising at least one sensor configured to observe objects, e.g. people, in the monitored area. The method comprises receiving observation data from the at least one sensor indicating location of different objects in the monitored area. The method further comprises determining distances between the monitored objects in the monitored area based on the observation data and/or the determined location of the objects.

In one embodiment of the invention the method further comprises determining locations in the monitored area, such as a building, where the distance between the observed objects is the smallest and/or below certain threshold-level.

In one embodiment of the invention the method further comprises sending the observation data and/or the determined distance data between the measured objects to an external unit or computing device, e.g. for determining how the observed distances have changed when comparing to earlier determined values.

In one embodiment of the invention the method further comprises determining distance between the objects based on coordinate transformation transforming the measured coordinates to real-world coordinates. In one embodiment of the invention the method further comprises tracking objects present in the monitored area by using the received observation data, wherein determining distances between the monitored objects in the monitored area is at least in part based on the tracked position of the objects.

In one embodiment of the invention the method further comprises performing an alert in the monitored area and/or sending an alert or notification to an external system, such as building automation or people conveyor system, if the determined distance between the observed objects at certain location is the smallest and/or below certain threshold-level.

In one embodiment of the invention the method further comprises controlling building automation and/or apparatus and/or system in the monitored area, such as a door, a gate, a guiding system, a people conveyor system, in the monitored area at least in part based on the determined distance between the observed objects.

In one embodiment of the invention the at least one sensor comprises at least one of the following: depth-sensing sensor, camera, image sensor, structured light measurement, phase shift measurement, time of flight measurement, stereo triangulation device, sheet of light triangulation device, light field cameras, coded aperture cameras, computational imaging techniques, simultaneous localization and mapping (SLAM), imaging radar, imaging sonar, scanning LIDAR, flash LIDAR, Passive Infrared (PIR) sensor, and small Focal Plane Array (FPA), 3D sensor comprising stereo vision arrangement.

According to a second aspect, the invention relates to an arrangement for monitoring distances between objects in monitored area, the arrangement comprising at least one sensor configured to observe objects, e.g. people, in the monitored area and control means configured to process and/or to transfer data related to sensor observations. The arrangement is configured to receive observation data from the at least one sensor indicating location of different objects in the monitored area and to determine distances between the monitored objects in the monitored area based on the observation data and/or the determined location of the objects. According to a third aspect, the invention relates to a people conveyor system, wherein the people conveyor system comprises the arrangement according to an embodiment of the invention.

In one embodiment of the invention, the people conveyor system comprises an elevator system comprising at least an elevator car, a motor configured to move the elevator car and an elevator controller configured to control the elevator and/or the motor. In one embodiment of the invention, the people conveyor system comprises an escalator system comprising at least steps, step chain, a motor configured to move the step chain and an escalator controller configured to control the escalator and/or the motor. In one embodiment of the invention, the people conveyor system comprises a moving walkway system comprising at least pallets, pallet chain, a motor configured to move the pallet chain and a moving walkway controller configured to control the moving walkway and/or the motor.

According to a fourth aspect, the invention relates to a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method according to an embodiment of the invention.

According to a fifth aspect, the invention relates to a computer-readable medium comprising the computer program according to the solution of the invention.

With the solution of the invention, it is possible to obtain reliably people flow data and e.g. to track and compare factual social distancing in the building and in people conveyor systems. With the solution of the invention, it’s possible to determine e.g. how well the distancing materializes, how it evolves over time and/or how it compares to another monitored area, e.g. in people conveyor system and/or in a certain building. It is also possible to pinpoint the areas and/or situations where the social distancing tends to fail.

The expression "a number of” refers herein to any positive integer starting from one, e.g. to one, two, or three.

The expression "a plurality of” refers herein to any positive integer starting from two, e.g. to two, three, or four. Various exemplifying and non-limiting embodiments of the invention both as to constructions and to methods of operation, together with additional objects and advantages thereof, will be best understood from the following description of specific exemplifying and non-limiting embodiments when read in connection with the accompanying drawings.

The verbs “to comprise” and “to include” are used in this document as open limitations that neither exclude nor require the existence of unrecited features. The features recited in dependent claims are mutually freely combinable unless otherwise explicitly stated. Furthermore, it is to be understood that the use of “a” or “an”, i.e. a singular form, throughout this document does not exclude a plurality.

Brief description of the Figures

The embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which:

Fig. 1A illustrates monitored area viewed from above according to one embodiment of the invention,

Fig. 1 B illustrates monitored area as a side view according to one embodiment of the invention, Fig. 2 illustrates an elevator according to one embodiment of the invention,

Fig. 3 illustrates an elevator car according to one embodiment of the invention,

Fig. 4 illustrates an escalator according to one embodiment of the invention, and

Fig. 5 presents a flow chart illustrating a method according to one embodiment of the invention. Description of the exemplifying embodiments

The specific examples provided in the description given below should not be construed as limiting the scope and/or the applicability of the appended claims. Lists and groups of examples provided in the description given below are not exhaustive unless otherwise explicitly stated.

Reference will now be made in detail to the embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

With the solution of the invention, it is possible to monitor and/or control people flow in monitored area, such as an office, shop, retail, shopping mall, museum, or other corresponding areas. The data required for this purpose can be collected by at least one sensor i.e. means to determine location of objects in the monitored area. In one embodiment existing sensors, e.g. in a building, can be utilized for this purpose to collect the required information. The determined people flow data can be used in connection with people conveyor system, access control systems, doors, gates, building automation and/or building management system. These systems can also be controlled based on the determined data.

Figures 1A and 1B illustrate one example of monitored area and how a sensor used in the solution of the invention can be arranged to the monitored area. Figure 1A presents the monitored area viewed from above. Figure 1B presents the monitored area of Figure 1A as a side view. In this example embodiment a sensor 111 for observing and monitoring objects, e.g. people, is arranged to a hallway 140 and to the ceiling of the hallway. The measuring area 109 of the sensor 111 is illustrated in the Figures 1A and 1B with dashed lines. The measuring area in this example is arranged next to elevator doors 107 which are arranged to enable entry to an elevator car travelling in an elevator shaft 101. The sensor 111 for monitoring objects can be arranged to identify objects in the measurement area 109 of the sensor and in this example case two persons 113, 114 in the monitored area can be recognized and/or measured and the sensor and/or a control unit arranged in connection with the sensor can submit measurement results or data related to measurement results of the sensor to a control unit, a computing device and/or to a server or service, such as a cloud service. Based on the measurement results the sensor and/or computing device can determine distance d between the persons 113, 114. Figure 2 schematically illustrates an embodiment according to the present invention in which the solution of the invention is used in the connection with a people conveyor system. In this case the people conveyor system is an elevator. The elevator comprises an elevator shaft 201 in which an elevator car 202 moves to serve different floors. In Figure 2 the elevator car 102 can stop in a first floor 203, second floor 204, third floor 205 and fourth floor 206. The floors may be any floor in a building and not necessarily the first and second floor of the building. The first floor 203 may be, for example, a garage and the second floor 104 the ground level. A landing door 207 can be arranged in each floor in front of the elevator car 202. In Figure 2 the elevator comprises a motor 208 configured to move the elevator car via the hoisting rope, wherein the motor 208 is controlled by an elevator control unit 210. This arrangement is, however, only an example.

A sensor 211 for monitoring objects can be arranged to the elevator car or in connection with the elevator car and/or the elevator waiting area. The sensor 211 can be communicatively connected to elevator control unit 210 and/or to a remote system 212. The elevator control unit 210 can be communicatively connected to a remote system and/or can control the operation of the sensor 211. The sensor can be used to observe the objects in the monitored area according to the solution of the invention.

Figure 3 schematically illustrates an embodiment according to the present invention in which an example elevator car is presented. In Figure 3 one example implementation of the sensor is illustrated in more detail. In this example the sensor 211 is arranged in top part of the elevator car 202 so that it is configured to measure and observe objects, such as people, in the elevator car. The sensor can be arranged e.g. to the inner side wall and/or the inner ceiling of the elevator car. The measurement area (indicated with dashed line) of the sensor can be arranged e.g. so that it essentially covers the area of the elevator car for example so that the sensor is able to recognize objects in the elevator car and determine their location in the elevator car. The sensor can process at least part of the data itself and/or the control unit 210 can process at least a part of the data. This measurement related data can be sent forward to a remote system 212 and/or a server, such as a cloud server. The service can also take care of at least part of the data processing. Figure 4 schematically illustrates an embodiment according to the present invention in which the solution of the invention is also used in the connection with a people conveyor system. In this case the people conveyor system is an escalator 400. The escalator may comprise a step-chain coupled to a motor 406 via a transmission 402 comprising at least a chain or belt or similar. The motor 406 may generate a rotational force via the transmission causing the step-chain to move in an intended travelling direction. A brake may be arranged to the conveyor system so that when de-energized it is configured to meet the rotating axis of the transmission and, in that manner, to brake movement of the step chain or keep the step chain standstill when the escalator system is idle. When energized, the brake opens, allowing movement of step-chain. The transmission may comprise, in the context of escalator system, a gearbox with the mentioned entities. Furthermore, the escalator system may comprise an escalator control unit 410 which may e.g. be configured to control the movement of step-chain through a control of a power supply to the motor 402 and to the escalator brake. In one embodiment of the invention the conveyor system can comprise a processing unit 404. In some embodiments a second motor 407 with a second transmission 403 may be provided, for example, at the opposite end of the step- chain. Then a second processing unit 405 may be mounted to the in connection with the second motor. The sensor 411 for observing objects in the monitored area can be arranged in connection with the escalator, e.g. at either end of the escalator. The sensor 411 can be communicatively connected to the conveyor control unit.

The sensor and/or control unit can send data relating to sensor measurements to a remote system or service. At least one communication channel can be arranged between the sensor and/or the control unit and a remote system. In the solution of the invention the sensor and/or the control unit provides data related to sensor measurements via the at least one communication channel to the remote system. In one embodiment of the invention the remote system can request data relating to sensor measurements from the sensor and/or the control unit. In one embodiment of the invention the sensor and/or control unit can provide, e.g. continuously and/or periodically, data relating to sensor measurements from the sensor and/or the control unit to the remote system also without the remote system requesting the data.

In the solution of the invention the area or at least part of the area at or around the presented structures of Figures 1A, 1 B, 2, 3 and 4 can be monitored according to the solution of the invention. With the solution of the invention distance between the monitored objects in monitored area can be determined with an arrangement comprising at least one sensor configured to observe objects, e.g. people, in the monitored area. Observation data from the at least one sensor indicating location of different objects in the monitored area is received and distances between the monitored objects in the monitored area can be determined based on the observation data and/or the determined location of the objects. This way it is possible to monitor and/or to control people flow in monitored area. The determined data can be used for controlling people flow, e.g. by closing and/or opening doors, controlling people conveyor systems, building automation systems and/or by changing operation of guiding systems. For example, if certain areas or locations are crowded, change in guidance system can be implemented and/or door(s) can be closed or opened so that people are directed away from the location which has been determined to be too crowded at the present time or usually at that time. The data collected by the system can for example be utilized by a dynamic guidance or routing system which takes into account the determined (real time or historical) distances between people to avoid the situations in which too many people are at certain area of the monitored area at the same time.

The solution of the invention can also provide information for the users of the building and/or for the manager of the building. This way for example the building manager can see the change to the monitored distances between people when changes are made in the building, such as changes to doors, furniture or guidance. The arrangement or system of the invention can for example provide a characteristic value relating to social distance in the monitored area determined from the monitored distances in the area, which value can be used to monitor the effectiveness of the change. In one embodiment of the invention also data from the sensors relating to quality of the observed distances can be provided.

In one embodiment of the invention the locations in the monitored area, such as a building, are determined where the distance between the observed objects is the smallest and/or below certain threshold-level. This way risky places and/or locations can be indicated for people in the area. The information can be displayed in a screen, e.g. via building automation, or sent to devices, such as mobile devices, of the people in the monitored area.

In one embodiment of the invention the observation data and/or the determined distance data between the measured objects is sent to an external unit or computing device, e.g. for determining how the observed distances have changed when comparing to earlier determined values.

In one embodiment of the invention the determination of distance between the objects is based on coordinate transformation transforming the measured coordinates to real-world coordinates. The determined location of an object, e.g. a person, can in one embodiment comprises x-, y- and/or z-coordinate. In one embodiment of the invention the sensor and/or system can be calibrated first so that sensors are able to provide locations of the identified objects in the monitored area, e.g. the coordinates of the monitored people.

In one embodiment of the invention the objects are tracked by the at least one sensor or multiple sensors by using the received observation data and the determination of distances between the monitored objects in the monitored area is at least in part based on the tracked position of the objects. Tracking can be implemented with a single sensor or with multiple sensors. The person who is tracked can be notified or alerted, e.g. through a device of the person, if he is or has been in a situation in which the monitored distances of this specific person to other people in the area have been too small, e.g. under a certain threshold value, and/or for over a certain duration.

In one embodiment of the invention an alert or notification is performed in the monitored area and/or an alert or notification is sent to an external system, such as building automation or people conveyor system, if the determined distance between the observed objects at certain location is the smallest and/or below certain threshold-level. An alert or notification, e.g. in certain area, can be given if an area has been usually crowded at certain time because safety of the people in that area can be decreased at that time, e.g. because of congestion or risk of infection.

In one embodiment of the invention building automation and/or apparatus and/or system in the monitored area, such as a door, a gate, a guiding system, a people conveyor system, are controlled in the monitored area at least in part based on the determined distance between the observed objects. This way the people flow can be altered for example so that people are more evenly distributed to the area and thus the distances between the objects can be increased. The effect of the controlling can also be monitored with the solution of the invention and further adjustments or control in the above listed systems and components can be implemented.

The control of the systems or devices can be implemented in many ways. For example, the sensor or sensor system can control a single device, such as door, directly or via local computing unit such as a house server. The control of the systems or devices can also be implemented via an external server or service, e.g. a cloud server. In one embodiment of the invention previously defined different ways of controlling the devices and systems can be combined.

In one embodiment of the invention the arrangement can comprise a processing module in communication with the sensor to receive the data, the processing module configured to sense, measure and/or track an object and calculate data associated with the sensed object. The sensor, the processing module and/or the control unit can calculate at least one of the following object parameters with respect to the sensed object, comprising: location, size, direction and/or distance to other determined objects. Determination or at least part of the determination of location, size, direction and/or distance to other determined objects can be also be carried out at the remote system based on the received data relating to sensor measurements.

The means to determine location of objects in the monitored area, i.e. the sensor, can be configured to provide data relating to location of the objects in the monitored area or data which is processed to determine location of the objects in the monitored area. At least one sensor is used for this purpose in the monitored area. In the case where multiple sensors are used for determining locations of objects in the monitored area, different types of sensors and data collected by different kinds of sensors can be utilized by the system. The sensors can be for example sensors of the people conveyor system, e.g. arranged in connection with the people conveyor systems and/or sensors of the building.

In one embodiment of the invention the at least one sensor comprises at least one of the following: depth-sensing sensor, camera, image sensor, structured light measurement, phase shift measurement, time of flight measurement, stereo triangulation device, sheet of light triangulation device, light field cameras, coded aperture cameras, computational imaging techniques, simultaneous localization and mapping (SLAM), imaging radar, imaging sonar, scanning LIDAR, flash LIDAR, Passive Infrared (PIR) sensor, and small Focal Plane Array (FPA), 3D sensor comprising stereo vision arrangement.

In one embodiment of the invention the sensor can be e.g. a depth sensing sensor, such as a 3D sensor. The sensor can be mounted or arranged to ceiling or wall of the monitored area and configured to monitor the area. The sensor can be configured to measure the area using stereo vision. This can be implemented e.g. by providing two wide-angle lenses observing the area from different perspectives (e.g. left image, right image). The sensor can determine the optical disparity for every pixel of the image. This results in a precise depth map or 3D image of the area. Subsequent processing can be used for person recognition and tracking, for example to allow analyzing individual paths of each person and/or distances between the observed persons.

In the following paragraph, one example implementation embodiment is described which utilizes the solution of the invention. In this implementation example depth sensing sensors, which are used in elevators for simple people counting and/or people flow measurement purposes, can be used to push and/or stream the coordinate time series of every identified object (i.e. person) in the sensor’s view. Also other types of sensors can be used. The coordinate time series from the sensor can comprise several coordinate points per second for every object. This coordinate push and/or stream is sent and stored. In one example sensor is configured to periodically push the coordinate collections to a server, e.g. over HTTP POST. A HTTP server and an event handler may be arranged to process, extract and store the coordinate data. In one example the pushed coordinates can be “pixel coordinates” that do not have constant relationship with real-world coordinates and/or distances. For this reason, the necessary coordinate transformation matrices that transform the “pixel coordinates” to real-world coordinates can in one embodiment be arranged to transform the measurement results to real world coordinates and distances. This can be obtained thru, e.g. sensor’s REST API and constructed from the sensor’s configuration data. The sensor configuration data can be a result of the sensor’s calibration procedure that can be carried out when the sensor is installed into a new location. The received measurement related data can be processed in order to determine pairwise distances between the identified objects and to identify and determine close encounters.

In Figure 5 a method according to an example embodiment is disclosed. The method is initiated by receiving observation data from the at least one sensor indicating location of different objects in the monitored area. Based on the observation data and/or the determined location of the objects, the method determines distances between the monitored objects in the monitored area.

The transmissions between the units and/or components, e.g. sending requests, responses and/or sensor observation data, may be performed by using a wireless or wired transmitter. The transmitter may be an independent transmitter using mobile communication networks, Wi-Fi or similar. The components can be communicatively connected to each other, e.g. via a serial data bus, such as CAN bus, LON bus or ethernet. The sensor and/or the control unit can communicate, e.g. by using a transfer protocol accepted between, for example TCP/IP. However, it is also possible to use the transmission channels that are dedicated particularly for a certain unit and/or part on the people conveyor, e.g. in the case of an elevator, a specific elevator car. The transmission may be encrypted. The transmission needs not to be a direct transmission but may involve any number of network elements in between. Thus, using an ordinary wireless internet connection is acceptable provided that the required units are able to receive the transmission.

The sensor and/or the control unit may be a separate processing unit, or it may be a functionality added to some existing conveyor component, such as an elevator control unit and/or elevator group controller. A sensor or controller of a conveyor system in which the solution of the invention can be used may comprise at least one processor connected to at least one memory. The at least one memory may comprise at least one computer program which, when executed by the processor or processors, causes the controller to perform the programmed functionality. In another embodiment, the at least one memory may be an internal memory of the at least one processor. The controller may also comprise an input/output interface. Via the input/output interface, the control apparatus may be connected to the required devices or units. The controller may be a control entity configured to implement only the above disclosed operating features, or it may be part of a larger elevator control entity, for example, a conveyor controller, such as an elevator controller, an escalator controller or a moving walkway controller.

As stated above, the components or other parts of the exemplary embodiments can include computer readable medium or memories for holding instructions programmed according to the teachings of the present embodiments and for holding data structures, tables, records, and/or other data described herein. Computer readable medium can include any suitable medium that participates in providing instructions to a processor for execution. Common forms of computer-readable media can include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other suitable magnetic medium, a CD-ROM, CD±R, CD±RW, DVD, DVD-RAM, DVD1 RW, DVD±R, HD DVD, HD DVD-R, HD DVD-RW, HD DVD- RAM, Blu-ray Disc, any other suitable optical medium, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other suitable memory chip or cartridge or any other suitable medium from which a computer can read.

The embodiments of the invention described herein before in association with the figures presented and the summary of the invention may be used in any combination with each other. At least two of the embodiments may be combined together to form a further embodiment of the invention.

The specific examples provided in the description given above should not be construed as limiting the applicability and/or the interpretation of the appended claims. Lists and groups of examples provided in the description given above are not exhaustive unless otherwise explicitly stated.