METHOD THEREOF

TECHNICAL FIELD

[0001] The present invention relates to the field of human flow estimation, and more specifically, to a human flow estimation system and a failure processing method therefor.

BACKGROUND ART

[0002] A human flow estimation system plays a critical role in a variety of construction applications. Generally speaking, solutions based on sensor fusion are extensively studied and applied to estimation of human flow within a construction. However, because human flow estimation of a construction level requires a large number of sensors of different types to acquire human flow data, there is a high risk of sensor failures. It is obvious that if some sensors fail, the accuracy of the human flow estimation system will be greatly reduced because these failed sensors are still configured in the sensor network model even if they are disconnected. At present, some of the existing technologies have paid attention to the sensor failure problem in a human flow estimation system. However, these technologies mainly focus on how to detect these failed sensors and the failures that occur thereon. At present, the problem to be further solved urgently is how to process these failed sensors and ensure that the estimation result of the human flow estimation system is still reliable.

SUMMARY OF THE INVENTION

[0003] The objective of the present invention is to provide a stable and reliable human flow estimation system.

[0004] Another objective of the present invention is to provide a failure processing method for improving the stability of a human flow estimation system.

[0005] In order to realize the objective of the present invention, according to one aspect of the present invention, a human flow estimation system is provided, comprising: a sensor network comprising a plurality of sensors arranged in a to-be-estimated region for detecting the human flow; a model building configured to build a human flow state model based on arrangement positions of the sensors, and build a sensor network model based on data of the sensors; and a human flow estimation module configured to estimate the human flow and provide a data weight of the estimated human flow based on the human flow state model and the sensor network model, wherein the human flow estimation system further comprises a failure detection module configured to detect whether each sensor in the sensor network is abnormal, and the model building module is further configured to adjust the human flow state model and the sensor network model when an exception exists on the sensor.

[0006] In order to realize another objective of the present invention, according to another aspect of the present invention, a failure processing method for a human flow estimation system is further provided, wherein the human flow estimation system comprises a sensor network, a model building module, a human flow estimation module, and a failure detection module; and the failure processing method comprises: a failure detection step SI 00, for detecting in real time whether each sensor for detecting the human flow in the sensor network is abnormal; a model adjustment step S200, for adjusting a human flow state model and a sensor network model involved in the human flow estimation system when an exception is detected on a sensor, and not adjusting the human flow state model and the sensor network model when no exception is detected on the sensor; and a human flow estimation step S300, for estimating the human flow and providing a data weight of the estimated human flow based on the human flow state model and the sensor network model in step S200; wherein the human flow state model is built based on arrangement positions of the sensors, and the sensor network model is built based on data of the sensors.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Fig. 1 is a schematic layout diagram of an embodiment where the human flow estimation system of the present invention is applied in a to-be-estimated region.

[0008] Fig. 2 is a schematic layout diagram of an embodiment where the human flow estimation system of the present invention is applied in another to-be-estimated region.

[0009] Fig. 3 is a schematic layout diagram of an embodiment where the human flow estimation system of the present invention is applied in another to-be-estimated region when a specific sensor failure exists.

DETAILED DESCRIPTION

[0010] The present invention shows a human flow estimation system. The human flow estimation system comprises: a sensor network comprising a plurality of sensors arranged in a to-be-estimated region for detecting the human flow; a model building module configured to build a human flow state model based on arrangement positions of the sensors, and build a sensor network model based on data of the sensors; and a human flow estimation module configured to estimate the human flow and provide a data weight of the estimated human flow based on the human flow state model and the sensor network model, wherein the human flow estimation system further comprises a failure detection module configured to detect whether each sensor in the sensor network is abnormal, and the model building module is further configured to adjust the human flow state model and the sensor network model when an exception exists on the sensor. By adjusting the human flow state model and the sensor network model in real time, the highly flexible and stable human flow estimation system proposed in the present invention can effectively process sensor failures such that its estimation is kept in a high-precision level around the clock.

[0011] More specifically, the mode of adjustment to the human flow state model and the sensor network model when an exception exists on a sensor at least comprises two types, which will be described below one by one.

[0012] With regard to the first adjustment mode, the model building module will reduce a data weight involving a specific sensor in the human flow state model and the sensor network model when the failure detection module determines that an exception exists on the specific sensor. Alternatively, the human flow estimation module will automatically reduce a data weight of the estimated human flow of a specific sensor when an exception exists on the specific sensor. It is noteworthy that the exception mentioned in the latter is not obtained through intuitive detection, and it might be a parameter which indicates that an exception exists on a specific sensor according to a calculation result in the estimation process, and then makes an adverse inference. As a result, when the data weight of the specific sensor decreases, the contribution degree thereof in the analysis of the human flow state model and the sensor network model will become smaller, and therefore the impact thereof on the analysis result will also become smaller, which will avoid excessive impact of the failure on the accuracy of the result.

[0013] Optionally, to more intuitively present to a user the data weight corresponding to each piece of data, the human flow estimation system further comprises a credibility evaluation module configured to evaluate credibility of data from each sensor according to the data weight of the estimated human flow of each sensor provided by the human flow estimation module. Based on the credibility evaluation module, the user can monitor the state of the human flow estimation system and learn the impact of the sensor failure on the entire system, thereby deciding, according to the human flow accuracy requirements in different application scenarios, whether or when to change the sensor failure.

[0014] More specifically, when the model building module is configured to adjust the human flow state model and the sensor network model based on the Kalman filter algorithm, or when the human flow estimation module is configured to analyze the human flow state model and the sensor network model based on the Kalman filter algorithm, the data weight corresponds to a covariance value of an equation involving a specific sensor in the Kalman filter algorithm; and when the data weight of the specific sensor decreases, the covariance value in the corresponding equation increases, while the credibility decreases accordingly.

[0015] Optionally, the credibility of the estimation result can also be used in the early discovery of the possibility that a potential sensor failure might happen. For example, when the credibility of a specific sensor provided by the credibility evaluation module within a second preset time period gradually decreases, the human flow estimation system sends out a failure early warning signal for the specific sensor. As such, it is also possible to take failure processing measures in advance, and further improve the estimation accuracy.

[0016] With regard to the second adjustment mode, it is also possible to remove from the sensor network model a specific sensor that fails, and perform human flow estimation based on the adjusted sensor network model and the corresponding human flow state model, thereby providing a very flexible way of re-configuring the models in the human flow estimation system when the sensor failure occurs.

[0017] Specifically, the plurality of sensors in the sensor network model are arranged at each entrance and exit of a to-be-detected space, to divide the to-be-estimated region into a plurality of to-be-estimated subregions; and the sensor network model is configured to reflect human flow data detected by each sensor, while the human flow state model is configured to reflect the number of people within each of the to-be-estimated subregions and the human flow entering and leaving each of the to-be-estimated subregions. When the failure detection module determines that an exception exists on a specific sensor, two adjacent to-be-estimated subregions divided by the specific sensor are combined into a new to-be-estimated subregion. In this setting, although the in and out human flow data of a detection point of the failed sensor can no longer be estimated, it can be ensured that other detected data has a relatively high accuracy, thereby enabling the human flow estimation of the entire system to have a relatively high accuracy.

[0018] In addition, more exquisite improvements are also made to other components in the human flow estimation system, which will be explained one by one below.

[0019] For example, the model building module can, more elaborately, comprise: a self- adaptive configuration submodule configured to adjust the human flow state model, and a dynamic sensor fusion submodule configured to build or adjusting the sensor network model based on data of the plurality of sensors. [0020] For another example, the model building module processes the human flow state model and the sensor network model based on the Kalman filter algorithm, the Bayesian Networks algorithm or the Dempster Shafer (DS) evidence algorithm. Alternatively, the human flow estimation module estimates the human flow based on the Kalman filter algorithm, the Bayesian Networks algorithm or the DS evidence algorithm.

[0021] In addition, as several specific examples of failure detection, the failure detection module can also determine that an exception exists on a specific sensor when data from the specific sensor is not received within a first preset time period; and/or the failure detection module can determine that an exception exists on a specific sensor when quality of data received from the specific sensor is lower than a first preset interval.

[0022] Optionally, specific types of the sensors can comprise a light beam sensor and/or a passive infrared red (PIR) sensor and/or a video camera.

[0023] An application embodiment of the human flow estimation system will be provided here in conjunction with the previously stated solutions and Fig. l, which estimates the human flow in a to-be-estimated region in real time based on a dynamic human flow state model and through the Kalman filter algorithm. For a set of linear system, the human flow state model can be expressed as:

x(t+l)=Ax(t)+w(t+l);

y(t)=Cx(t)+v(t);

where x represents a state vector, and y represents a measured value vector; and w and v are vectors respectively representing the uncertainties of the human flow state model and the measured value model.

[0024] Referring to Fig. 1 again, an example of a simple to-be-estimated region where the human flow estimation system is applied is shown. The example reflects the first adjustment mode for the human flow state model and the sensor network model when an exception exists on a sensor. In this example, there are 2 to-be-estimated subregions: one room and one corridor. In the to-be-estimated regions, 6 states are defined:

xi: the number of people entering the corridor;

x ₂ : the number of people in the corridor;

x ₃ : the number of people leaving the corridor;

x ₄ : the number of people entering the room;

X5: the number of people leaving the room; and

x ₆ : the number of people in the room. [0025] The human flow state model for the example is:

xi(t+l)=xi(t)+wi(t+l);

x ₂ (t+ 1 )=x ₂ (t)+x i (t)-x ₃ (t)-x ₄ (t)+x ₅ (t)+w ₂ (t+ 1 );

x ₃ (t+l)=aix ₂ (t)+w ₃ (t+l);

x ₄ (t+ 1 )=a ₂ x ₂ (t)+w ₄ (t+ 1 );

x ₅ (t+l)=a ₃ x ₆ (t)+w ₅ (t+l);

x ₆ (t+ 1 )=x ₆ (t)+x ₄ (t)-x ₅ (t)+w ₆ (t+ 1 );

where ai is an unknown parameter which can be taken as the ratio of people entering another region from a specific region within one period of sample time. For example, ai is the ratio of people leaving the corridor within one period of sample time; a ₂ is the ratio of people entering the room within one period of sample time; and a ₃ is the ratio of people leaving the room within one period of sample time.

[0026] As above, 6 equations are built through the mutual relation between the number of people in each to-be-estimated subregion and the number of people entering and leaving the to-be-estimated subregion, and therefore, the number of people in each to-be-estimated subregion and the human flow entering and leaving same can be obtained, by way of analysis and processing, through a plurality of sensors that divide these two to-be-estimated subregions.

[0027] Specifically, in the example, four sensors including two light beam sensors and two video cameras 1 and 2 are used to acquire population data and human flow data. The sensor network model here is:

yi(t)=xi(t)+vi(t);

y ₂ (t)=x ₃ (t)+v ₂ (t);

y (t)=x ₄ (t)+v ₃ (t); and

y ₄ (t)=x ₅ (t)+v ₄ (t);

where yi and y ₂ are measured values from the video cameras, while y ₃ and y ₄ are measured values from the light beam sensors. Vi is the uncertainty of the measured values.

[0028] As above, 4 equations are built through the mutual relation between the detected data of each sensor and the human flow related to the detected data, and therefore the human flow data of the part related to these sensors can be directly obtained.

[0029] Continuing with the previously stated embodiment, suppose that video camera 2 in Fig. 1 fails, and the failure type may be that the data from the camera cannot be received or the quality of the received video is rather poor. At this point, we can easily find out this type of problem and rebuild a human flow estimation system afterwards. It can be known from the previously stated sensor network model that the human flow variable corresponding to video camera 2 is x ₃ , and at this point, a relatively large error value v ₂ can be set for the sensor model corresponding thereto. If a failure occurs, the system only needs to adjust the corresponding sensor model used in estimation, instead of changing the entire state vector space model.

[0030] To return to the previously stated model equation, it only needs to set a relatively large error value for v ₂ in the equation y ₂ (t)=x ₃ (t)+v ₂ (t), because only this equation involves x ₃ (i.e., video camera 2). In the estimation method based on a Kalman filter algorithm, we set a relatively large covariance value for the model, and in the subsequent calculation of the Kalman filter algorithm, the data weight of the sensor data will be automatically reduced. In this setting, we can likewise obtain a sufficiently stable estimation result even under the circumstance that some sensors fail.

[0031] The output value of the Kalman filter comprises two parts: estimated human flow data (the human flow at the location of each sensor, for example, region A and region B shown in the figure, and the number of people in each region) and a covariance of the estimated human flow data. The covariance of the human flow data represents the accuracy of the estimated human flow data, and can be regarded as the credibility of the estimated human flow data of the human flow estimation system.

[0032] In addition, an application embodiment of another human flow estimation system is provided in conjunction with Fig.2 and Fig. 3. The example reflects the second adjustment mode for the human flow state model and the sensor network model when an exception exists on a sensor. The system likewise estimates the human flow in a to-be-estimated region in real time based on the dynamic human flow state model and through the Kalman filter algorithm. For the system, the human flow state model can be expressed as:

x(t+l)=Ax(t)+w(t);

y(t)=Cx(t)+v(t);

referring to Fig. 2 and Fig. 3 again, the example has two rooms and one T-shaped corridor; and a PIR motion sensor and a light beam sensor are arranged in each room, while 3 video cameras and 1 light beam sensor are arranged in the corridor. The arrangement of these sensors divides the to-be-estimated region into 7 to-be-estimated subregions (i.e., regions A-G shown in the figure). For the human flow and the number of people in these regions, 19 states in total are defined:

xi: the number of people walking from region A to region B;

x ₂ : the number of people walking from region B to region A;

x ₃ : the number of people walking from region B to region C;

x ₄ : the number of people walking from region C to region B; X5: the number of people walking from region C to region G;

x ₆ : the number of people walking from region G to region C;

x ₇ : the number of people walking from region B to region D;

x ₈ : the number of people walking from region D to region B;

X9: the number of people walking from region D to region E;

xio: the number of people walking from region E to region D;

xii : the number of people walking from region E to region F;

X12: the number of people walking from region F to region E;

XA: the number of people in region A;

XB: the number of people in region B;

xc: the number of people in region C;

XD: the number of people in region D;

XE: the number of people in region E;

XF: the number of people in region F; and

XG: the number of people in region G;

[0033] The human flow state model for the example is:

x _A (t+l)=XA(t)-xi(t)+x ₂ (t)+w _A (t);

X _B (t+l)=XB(t)+Xl(t)-X2(t)+X ₄ (t)-X3(t)+X ₈ (t)-X ₇ (t)+WB(t);

xc(t+l)=xc(t)+x3(t)-x ₄ (t)+x ₆ (t)-x5(t)+wc(t);

X _D (t+ 1 )=X _D (t)+X7(t)-X ₈ (t)+X lo(t)-X9(t)+W _D (t);

x _E (t+ 1 )=x _E (t)+x ₉ (t)-x io(t)+x i ₂ (t)-x 11 (t)+w _E (t);

x _F (t+l)=x _F (t)+xii(t)-xi ₂ (t)+WF(t); (if PIR sensor 1 is expressed as right);

XF(t+l)=0; (if PIR sensor 1 is expressed as wrong);

XG(t+l)=XG(t)+X5(t)-X6(t)+WG(t); (if PIR sensor 2 is expressed as right); and

XG(t+l)=0; (if PIR sensor 2 is expressed as wrong).

[0034] Continuing to refer to the previously stated embodiment, when the failure detection module locates an exception in the data sent by light beam sensor 3 in the T-shaped corridor, the data detected by light beam sensor 3 at this time will bring enormous misleading to human flow estimation. Subsequently, the failure detection module will send out a prompt to the model building module, notifying that light beam sensor 3 fails. The model building module will automatically remove light beam sensor 3 from the sensor network model and re- divide the to-be-estimated region based on the remaining sensors. The re-divided regions are as shown in Fig. 3. Specifically, because light beam sensor 3 has already been removed, region B and region D previously divided by light beam sensor 3 will be combined into a new to-be- estimated subregion, i.e., region BD.

[0035] Then the human flow state model will be re-expressed as:

x _A (t+l)=XA(t)-xi(t)+X2(t)+w _A (t);

XBD(t+l)=XBD(t)+Xl(t)-X2(t)+X ₄ (t)-X3(t)+Xlo(t)-X9(t)+W _B D(t);

xc(t+l)=xc(t)+x3(t)-x ₄ (t)+x ₆ (t)-x5(t)+wc(t);

x _E (t+ 1 )=x _E (t)+x ₉ (t)-x io(t)+x i ₂ (t)-x 11 (t)+w _E (t);

x _F (t+l)=x _F (t)+xii(t)-xi2(t)+WF(t); (if PIR sensor 1 is expressed as right);

XF(t+l)=0; (if PIR sensor 1 is expressed as wrong);

XG(t+l)=XG(t)+X5(t)-X6(t)+WG(t); (if PIR sensor 2 is expressed as right); and

XG(t+l)=0; (if PIR sensor 2 is expressed as wrong).

[0036] After that, the human flow estimation system will re-estimate the human flow and the number of people in each to-be-estimated subregion based on updated sensor network model and human flow state model. The previously stated embodiment realizes that a relatively high human flow estimation accuracy is still maintained when some sensors fail.

[0037] It is without doubt that the two adjustment modes can also be used in combination, for example respectively providing a set of estimated human flow data with credibility (no failed sensor is removed from the models) and a set of estimated human flow data obtained by using the updated models for analysis and processing (failed sensors have already been removed from the models), thereby realizing the complementation of the two modes. On one hand, the processed data with unchanged models can still be used so that there are relatively less alterations and the credibility of the data is clearly known, and on the other hand, the models can be slightly adjusted to acquire models of a higher accuracy and analyzed data thereof.

[0038] According to another aspect of the present invention, also provided is a failure processing method for the human flow estimation system in the previously stated embodiment. The failure processing method at least comprises:

a failure detection step S 100, for detecting in real time whether each sensor for detecting the human flow in the sensor network is abnormal;

a model adjustment step S200, for adjusting a human flow state model and a sensor network model involved in the human flow estimation system when an exception is detected on a sensor; and not adjusting the human flow state model and the sensor network model when no exception is detected on the sensor; and

a human flow estimation step S300, for estimating the human flow and providing a data weight of the estimated human flow based on the human flow state model and the sensor network model in step S200.

[0039] Based on the previously stated failure processing method, it is possible to correspondingly adjust the involved human flow state model and the sensor network model when a sensor in the sensor network fails, thereby reducing the impact brought by the detected data of the failed sensor, or to remove the failed sensor from the models, such that the estimation thereof is kept in a high-precision level around the clock; and it is also possible to automatically calculate the data weight of the estimation result and evaluate the credibility thereof when no sensor failure is detected, thereby making a contribution to the improvement of estimation accuracy as well.

[0040] More specifically, in the present method, the mode of adjustment to the human flow state model and the sensor network model when an exception exists on a sensor comprises two types, which will be described one by one below.

[0041] With regard to the first adjustment mode, step S200 further comprises: an exception value assignment step S210, for reducing a data weight involving a specific sensor in the human flow state model and the sensor network model when it is determined that an exception exists on the specific sensor. Alternatively, step S300 further comprises: automatically reducing a data weight of the estimated human flow of a specific sensor when an exception exists on the sensor. When the data weight of the specific sensor decreases, the contribution degree thereof in the analysis of the human flow state model and the sensor network model will become smaller, and therefore the impact on the analysis result will also become smaller, which will avoid excessive impact of the failure on the accuracy of the result.

[0042] Optionally, to more intuitively present to the user the data weight corresponding to each piece of data, the failure processing method further comprises a credibility evaluation step S400 for evaluating credibility of data from each sensor according to the data weight of each sensor provided in step S300. Based on the credibility evaluation step S400, the user can more intuitively learn the credibility of these pieces of data and make a judgment or processing.

[0043] More specifically, when in step S300 the human flow state model and the sensor network model are analyzed based on a Kalman filter algorithm, the data weight corresponds to a covariance value of an equation involving the specific sensor in the Kalman filter algorithm; and when the data weight decreases, the covariance value increases, and the credibility decreases accordingly.

[0044] Optionally, the credibility of the estimation result can also be used in the early discovery of the possibility that a potential sensor failure might occur. For example, the method can further comprise a failure early warning step S500 for sending out a failure early warning signal for a specific sensor when the credibility of the specific sensor provided within a second preset time period in step S400 gradually decreases. As such, it is also possible to take failure processing measures in advance, and further improve the estimation accuracy.

[0045] With regard to the second adjustment mode, it is also possible to remove from the sensor network model a specific sensor that fails, and to perform human flow estimation based on the adjusted sensor network model and the corresponding human flow state model, in order to further improve the estimation accuracy.

[0046] Specifically, the plurality of sensors in the sensor network model are arranged at each entrance and exit of a to-be-detected space, to divide the to-be-estimated region into a plurality of to-be-estimated subregions; and the sensor network model is configured to reflect human flow data detected by each sensor, while the human flow state model is configured to reflect the number of people within each of the to-be-estimated subregions and the human flow entering and leaving each of the to-be-estimated subregions, and step S200 further comprises: an exception processing step S220, for removing a specific sensor from the sensor network model when an exception is detected on the specific sensor, and combing two adjacent to-be- estimated subregions divided by the specific sensor into a new to-be-estimated subregion.

[0047] In addition, several more exquisite steps are also arranged for the failure processing method, which will be explained one by one below.

[0048] For example, step S200 comprises a dynamic sensor fusion sub-step for building the sensor network model based on data of the plurality of sensors. For another example, step S200 comprises a self-adaptive configuration sub-step for adjusting the human flow state model, so as to reduce a data weight involving a specific sensor in the human flow state model and the sensor network model when the failure detection module determines that an exception exists on the specific sensor.

[0049] For another example, in step S200, the human flow state model and the sensor network model are processed based on the Kalman filter algorithm, the Bayesian Networks algorithm or the DS evidence algorithm. Alternatively, in step S300, the human flow is estimated based on the Kalman filter algorithm, the Bayesian Networks algorithm or the DS evidence algorithm.

[0050] In addition, as several specific means for failure detection, step S 100 comprises: determining that an exception exists on a specific sensor when data from the specific sensor is not received within a first preset time period; and/or determining that an exception exists on a specific sensor when quality of data received from the specific sensor is lower than a first preset interval.

[0051] The above examples mainly explain the human flow estimation system and the failure processing method therefor of the present invention. Although only some embodiments of the present invention are described, a person of ordinary skill in the art should understand that the present invention can be implemented in many other forms without deviating from the concept and scope thereof. Therefore, the illustrated examples and implementations are considered illustrative but not restrictive, and the invention can cover various modifications and replacements without departing from the spirit and scope of invention defined by the appended claims.