The present invention refers to an interactive door system, comprising a new intelligent interface, which is designed to control entering the place with restricted entry, as is the case with entering an apartment and the like.

The interactive door system of the aforementioned kind is known through a number of documents, as for example WO 2011127739, WO 9966467, CN101882335, etc.; however, none of the known solutions simultaneously meets safety, functionality and reliability requirements. Moreover, all the solutions known are demanding for the user in terms of both usage and learning. It is the object of the present invention to create a new interactive door system, which remedies the drawbacks of the known systems.

According to the invention the object as set above is solved by the characteristics, set in the characteristics clause of the first patent claim. The details of the invention are further disclosed in sub-claims. It is provided for according to the present invention that by means of recognizing the persons entering/exiting the place, the automatic persons presence list management is enabled, thus setting the basis for offering services said persons need when entering/exiting. Therefore, the system actually performs the activities, which are commonly carried out by the human doorkeeper. The advantage of the system is applying automation and motivation of the users to cooperate, thus ensuring a high reliability level of managing the presence list and recognizing the persons entering/exiting. Consequently, a reliable presence list register and a person's identification upon entering/exiting the place are ensured. The invention is further described in detail by way of non-limiting preferred embodiment, and with a reference to the accompanying drawing, where

Fig. 1 shows a schematic view of an interactive door system according to the invention, Fig. 2 shows a schematic view of the operation of a system of Fig. 1. According to the invention, an interactive door system 1 comprises a multi-media device, as shown in Fig. 1 arranged in the area of the door with restricted entrance and is as well connected to at least one external group. Said device comprises a set of components, e.g. a management component 2, a processing component 3 and a user interface component 4. The first component of door system 1 has been selected as the management component 2, managing at least one said external group. According to the present invention it is provided that said external group comprises a sensor group 5, preferably represented by at least one sensor, at least one locking group 6, preferably represented by at least one lock, particularly the electromechanical lock, and a group 7 for sending the alarms and warnings. Each of the mentioned groups 5, 6, 7, capable of communicating with said device, can be an integral part thereof or can be completely separated from it. The door system requires a variety of sensors, comprised in sensor group 5 for its operation. In order for the door system according to the invention to operate reliably, at least the following sensors are required:

An identification sensor, e.g. a finger-print scanner, a RFID card reader (Radio- Frequency Identification), a code-entering dial pad and the like,

A walk-through sensor, e.g. a door open sensor, an ultrasonic distance sensor, laser beam interruption sensor, placed onto the door jamb and the like,

And optionally a door moving sensor, e.g. an accelerometer, a gyroscope, magnetometer and the like.

In addition, sensors measuring the data, typical of the entrance of a particular person are provided as well. This particular sensor is for example a camera with a face-recognition ability or a camera, estimating physical parameters of the person, an ultrasonic sensor for measuring physical parameters of a person walking through the door and sensors, detecting the presence of people, as for example, a motion sensor or a camera with motion-detection ability. Said locking group 6 comprises at least one electromechanical lock, providing electronically controlled door locking and unlocking. Said group 7 for sending alarms and warnings enables redirecting the alarms and warnings from the process component 3 via the management component 2 to the users 8. Said group 7 can be a regular alarm system, connected to the call centre or a security station, which can be connected to the door system 1 according to the invention. It is provided for, according to the preferred embodiment of the invention that the said group 7 be selected as a module, providing transmission of at least the textual data, e.g. a GSM module capable of sending SMS messages to users, and preferably it is selected as a module, enabling picture and/or sound transmission, e.g. an internet server, capable of sending picture and/or sound to the users. The second component of the mentioned device is selected as a central processing unit 3, receiving the collected information from said management component 2 and orders of the users 8; it processes them and according to the results it then issues commands to manage said locking group 6, the group 7 for sending alarms and warnings, and display group on the user interface component 4. The third component of said device is represented by the user interface component 4, connected to the central processing unit 3 and provides communication between the user 8 and the mentioned door system 1. As the user interface component 4, a touch-screen or any other device or a set of devices, capable of displaying information and receiving orders issued by the users may be used. According to the present invention it is provided that the management component 2 is designed by means of a microcontroller with an appropriate number of inputs and outputs and the software, capable of receiving data from a variety of sensors, sending data to the processing component 3 by wire or wireless, and controlling the mentioned locking group 6 and group 7 for sending alarms and warning. Said components 2, 3, 4 may be formed either as a single integrated system or may consist of a number of parts.

In the continuation the system operation is disclosed in more detail represented by means of a process, as shown on Fig. 2. Said process consists of following essential elements:

a) detecting a person on the outside of the door and operating accordingly upon entry, b) detecting a person on the inside of the door and operating accordingly upon exit, c) detecting security threats and acting accordingly,

d) learning of the typical way of passing through, i.e. entering/exiting the door of the place being monitored, learning about the users, identifying and verifying them by comparing the collected sensor information of the walk-through events, with the models, based on the past walk-through events.

When starting 8a an interactive door system according to the invention, a system initializations 9 of the separate external groups of the system 1 are primarily performed, e.g. the sensor group 5, locking group 6 and group 7 for sending alarms and warnings. Afterwards, processes 10 and 13a, detecting the presence of people on both sides of the door and security process 14a, detecting security threats are starting to run in parallel, launching the corresponding processes, described in the continuation. On basis of the selected sensor information upon every entry or exit, the user modelling process 12 is started. The latter builds a model 17 of user behaviour by applying the methods of machine learning, consisting of the physical characteristics of the person, e.g. the size, model of the face and the like, the characteristics of the way someone enters, e.g. the speed of opening the door, the time during the identification with the ID sensor and the door opening, the time interval of the door opened and the like; the characteristics of entry times, e.g. a particular time in the day, day in the week, working day/holiday, the presence of other people, the time of the previous entry and the like. In phase one, the system collects information to model the user. When enough information is collected to build a reliable user-classification model, either by identification or verification, phase two may begin. In the second phase, the built models are used for user identification or verification and these models are updated and upgraded based on the new information, making it possible for the system to automatically adjust to behaviour changes and users' needs.

A process 11 upon entry and detection of the person on the outside, respectively, is launched every time the sensor detects the possibility of a person entering a place being monitored. To this end, the process 10 is constantly running, periodically checking whether or not a person outside of the door of a place being monitored is present. The presence of a person is detected by means of certain information, collected from an appropriate sensor of the sensor group 5, e.g. the PIR motion sensor, directed straight to the front side of the door and is set to detect people. If the system does not include an appropriate sensor for entry detection process 10, the information is constantly being stored and then used upon entry. First step collects sensor information. If a person is detected in front of the door, the process 11 for the entry starts, and process 10 for detection of people in front of the door stops. All the relevant sensor information is saved in temporary storage. Afterwards, one of the three options may follow: (i) the person does not enter, (ii) the person enters without identification, (iii) the person enters with identification.

If after detecting the presence of a person and later detected person is no longer present in front of the door, the time set in advance passes, this is considered as option (i). In this case, storing information is discontinued. Of all the information stored only the time of event and the picture or pictures of the event in front of the door, are stored (if the system includes the image-capturing sensor: video camera or infrared camera), the rest of the temporary information is deleted. The picture and time, get listed for example, on the events taking place in front of the door; this is shown to the user, when he/she enters the door, thus providing an overview of the visitors. If after detecting a person, an entry is detected without preliminary identification, the event is considered as option (ii). The entry is detected via an appropriate sensor from the sensor group 5 (i.e. the open-door sensor or a laser beam interruption sensor, placed in the door jamb). In this case an appropriate security procedure 14 is launched, described in the continuation.

Otherwise, option (iii) is considered, i.e. entry with identification, recognized by first detecting the person in front of the door and afterwards triggering one of the identification sensors (i.e. fingerprint scanner, magnetic card reader, RFID card reader, code-entering dial pad). The next step checks whether the identified person has entry clearance. If he/she does not have it, the picture and time of event are stored on the event list, followed by running the previously mentioned security procedure 14; otherwise the user presence list 16 is checked. If the identified person is marked as present on the presence list 16, the previously mentioned security procedure 14 is launched, otherwise the procedure continues by following the next steps. If the user presence list 16 contains no present users, security threat detection 14a and., i.e. an alarm system, are shut down, as it is known that a person with a required identification is making an entrance and therefore, there is no need for the alarm system to be active. In the following step the processing component 3 sends an order to unlock the door via sensor group 5 managing the sensors and the locking group 6, and starts a timer . If after unlocking the door, the time limit set for the entry passes without detecting an entrance, the processing component 3 sends an order for locking the door, therefore completing the procedure otherwise the system, however, detects the person's entry with at least one of the already mentioned entry detection sensors. Upon entry the sensor information is no longer stored and is sent together with the person's identity, the entry information and entry time, the information is sent to be processed further by user modelling procedure 12. The user presence list 16 registers the presence of an identified person; afterwards, the verification procedure 15 of the person identified is carried out on the basis of collected information. If the verification procedure 15 confirms the identity of the person identified, the door-locking order is sent (locking group 6 triggers the locking only after the door is closed); after that a set of services is launched, related to the entry of the person. This includes displaying the events in front of the door (i.e. the visitors list) on the mentioned user interface component 4; voice greeting; forwarding messages for the identified person; activation or operation mode change of certain devices and systems in a smart home: heating, lighting, stereo, and the like. If the verification procedure 15 does not confirm the identity of the person, the mentioned security procedure 14 is launched. Exiting, procedure 13 upon exit is launched every time the system detects the possibility of the person exiting the place. To this end, the exit detection process 13a is constantly running, periodically checking whether or not a person is present at the inside side of the door. The presence of a person is detected on the basis of information from sensor group 5 (i.e. detecting the presence of people with a camera and the motion detection algorithm or a PIR motion sensor). If there is a person detected in front of the door, procedure 13 upon exit is launched and an exit detection process 13a for detecting people in front of the door is broken off. In the first phase collecting sensor information is launched: All the relevant sensor information is saved in temporary storage. If the system does not include the appropriate sensor for entry detection process 10, the information is constantly being stored and used upon entry. Afterwards, one of the three options may follow: (i) the person does not exit, (ii) the person exits without identification, (iii) the person exits with identification.

If after detecting a presence of a person and later detecting person absence, the time set in advance passes, this is considered as option (i). In this case, storing information is stopped and all the information collected so far is deleted from the temporary storage.

If after detecting a person, the exit without preliminary identification is detected, the event is considered as option (ii). The exit is, as already described, detected via the appropriate sensor. If the list of present users does not show any person present, the alarm system is activated, as it can be assumed that the last person left the place, therefore, security threat detection 14a for detecting a breaking an entry and i.e. an alarm system, need to be activated. Afterwards, collecting sensor information is cut off and the information is used for identifying the person via identification procedure 15. If the identification is successful, the collected information, together with the exit tag, time and identity of the person exiting, is sent to be further processed by user modelling procedure 12. It is registered in the user presence list 16 that the identified person is absent and that perhaps the presence list does not in fact match the actual condition per se (the identification was carried out automatically, without the user's explicit identification). If the door is unlocked, the central processing unit 3 sends an order for door locking. If the identification procedure 15 determines that a person can not be reliably identified, security procedure 14 is launched and, the door is locked, following the order of the processing component 3, if necessary.

If the person in front of the door identifies himself/herself after the detection, the event is considered as option (iii). In order to exit, the person may identify with one of the identification sensors (the same sensors as are applicable for entry identification) or by pressing the button, reserved for this particular purpose, located on the user interface component 4. If the person identifies with the identification sensor, the compliance of the user presence list 16 is verified and if necessary, corrected. If the only way for identification when exiting is to use the interface, the latter must provide managing the presence list of users, should it fail to comply with the actual condition. The following step launches the services, related to the exit of the person identified. Afterwards, the order to unlock the door is sent and a timer is started. If after unlocking the door the time limit set for exiting passes without detecting an exit, the central processing component 3 sends an order for locking the door, therefore completing the procedure. Otherwise the system detects an exit of the person with at least one of the already mentioned entry/exit detection sensors. The following step checks - in the user presence list 16 - whether the person exiting the place is the only present person: if this is true, the security threat detection 14a or, for example, an alarm system, is activated. Afterwards, collecting sensor information discontinues; together with the person's identity, time and exit tag, it is sent to be processed further by user modelling procedure 12. Furthermore, the user presence list 16 of the users is updated by marking the identified person as absent. Finally, on the basis of collected information, verification procedure 15 for verification of the identity of the person exiting the place is activated. If the verification procedure 15 confirms the identity, the door locks, therefore completing the procedure; otherwise, the appropriate security procedure 14 is launched besides door locking.

User modelling procedure 12 for modelling the behaviour of users is activated at every entry or exit of the user, who identifies himself/herself. The aim of the procedure is to build a model of the behaviour of the users on the basis of the collected sensor information using machine learning methods. The model can afterwards be used for identification and verification of people. As is the case of the user modelling procedure 12, the verification is also launched at every entry or exit of the user, who identifies himself/herself. The purpose of the verification and/or identification procedure 15 is to check whether the behaviour of the person, who properly identifies himself/herself, matches the expected behaviour of the person, whom the identification actually belongs to, i.e. to determine whether the person, who identified himself/herself, is in fact the person, whom the mentioned identification belongs to. The second purpose is to recognize an unusual behaviour of the familiar person upon entering or exiting, and based on the results, launch the appropriate security procedure 14, if necessary. Identification is used at every entry or exit of a user, who does not identify himself/herself. The purpose of the identification procedure 15 is to recognize the identity of the user, who failed to make a proper identification, on the basis of sensor information; if the procedure is confident about the results, the recognized identity is used in the entry and exit procedures as the identity that would explicitly be given by the user. Otherwise, appropriate security procedure 14 is launched, therefore making sure the user correctly identifies himself/herself and therefore ensuring that the user presence list 16 complies with the actual condition per se.

As stated, procedures 11, 12, 13 are launched at the person's entry or exit. Procedure 11 upon entry, or procedure 13 upon exit, makes sure that all the relevant sensor information is collected, information about time and information whether an entry or exit took place. After phase one of collecting necessary information, the user modelling procedure 12 goes into phase two, in which the so called features or attributes relevant for an individual's entry are extracted. First, time aggregation of collected information is performed. Time relations between the responses of individual sensors and the detected durations of individual sensors states are presented in a form that is appropriate for further processing. For example, it can be determined from the sensor information that first a motion sensor in front of the door was triggered, 3 seconds after that an identification sensor was triggered, 2 seconds after that the person touched the doorknob and then the person was opening the door for 1.3 seconds and closing it for 1.6 seconds. In parallel to that, the information fusion is carried out, making it possible to acquire additional data by combining information from a variety of sensors. For example, the time of opening the door is determined on the basis of the moment the doorknob was touched and the accelerometer, placed in the door. Sensor information fusion also serves to increase the detection reliability: by combining the information, arising from multiple similar sensors or the sensors, which enable detecting a similar characteristic, the quantitative estimate of the given characteristic of passing through the door can be determined in a more accurate and more reliable way. The complexity of the time aggregation and data fusion depends on the number and type of sensors from the mentioned sensor group 5; all are connected into the system scheme and must be specifically adapted to the actual sensors. In the following step, the machine learning features, i.e. the attributes, are calculated from the acquired information, whenever the user behaviour modelling or the classification (used for identification or verification) is required. The features are presented in the form of attribute values description, where the values of an attributes may be nominal or numerical. Thereafter, the procedure is independent of the type and number of the sensors used. In case of the user modelling procedure 12, the calculated features (attributes) are arranged into the attributes vector and are labelled with a class label which equals the identity of the user. After that the vector is used as learning set in the machine learning procedure. The machine learning algorithm may be chosen among the supervised machine learning algorithms, which provide sufficient classification accuracy and have appropriate time efficiency of the classification process (i.e. decision trees, decision rules, k-nearest neighbours algorithm, naive Bayes classifier, support vector machine, artificial neural network etc.). The built model 17 of user behaviour, which is updated every time a new learning vector is available, is stored in the form, which can be used to update the model and classify the user.

In case of identification and/or verification procedure 15, the first thing done after the calculation of the attribute vector, is checking whether sufficiently large learning set has been collected in order to build a sufficiently reliable model 17 of the user behaviour. If the collected information is not sufficient, both procedures reply that the person can not be recognized (identified or verified). Otherwise, the attribute vector and the model 17 of the user behaviour are used to classify the user, the result of which is the identity of a person (Y) and classification confidence. In case of verification, the returned result is the estimated probability that explicitly expressed identity of the user (X) equals the recognized identity (Y) on the basis of collected sensor information and behavioural patterns of the person X in the past. In case of identification procedure 15, the output is the identity of the person and the probability that the person, passing through the door, is truly recognized; this is estimated by the classification reliability of the model 17 of user behaviour, referring to that particular person's behaviour. If the probability estimation are below a particular threshold, the identification or verification is considered invalid, otherwise they are considered valid.

As stated above, the procedure 11 upon entry and procedure 13 upon exit may launch security procedure 14 in certain cases. Based on the reason of the security procedure launch, a proper action sequence is carried out. Procedure at entry may launch a security procedure due to one of the following four reasons: (i) because after entry, the person's verification failed, (ii) because the person entered without identifying himself/herself, (iii) because the user presence list 16 does not fit the actual state which is recognized by the entry identification sensor or (iv) because the person, identifying himself/herself does not have an entry clearance. If the verification fails (i), the users are sent a warning immediately after an entry, notifying them about the event. Via the user interface component 4, the system then requires an additional identification from the person making an entry. If additional identification is successful, the system sends a warning cancellation; otherwise it sets off the alarm. Either way, the entry procedure continues after that and locks the door. The services, related to the entry of person X are launched only, if the identification is successful.

If the person enters without identification (ii), identification procedure 15 is first launched. If the identification procedure is successful, the recognized user's identity is considered as the identity given by the one making an entry and procedure continues in the same way as is the case of failed verification (i) (send a warning, request additional verification and cancel the warning or set-off an alarm). If the identification fails, the system requires an identification of the person making an entry via the user interface component 4. If this fails, an alarm is set-off, otherwise the security procedure immediately finishes. If the result of the security procedure 14 is setting off the alarm, the procedure 11 of the entry, which launched a security procedure, is stopped. Otherwise procedure 11 of the entry continues as if the person making an entry would identify himself/herself with an identification sensor: if the user presence list 16 shows no user marked as present, security threat detection 14a and e.g., an alarm, is stopped; after that, the presence of the person is registered in the user presence list 16; the door is locked and services, related to the entry of the identified person, are launched.

If the user presence list 16 does not match the actual condition at entry (iii), i.e. the user presence list 16 of users states that the person with the given identity is present, it is first checked whether the user presence list 16 is marked as trustworthy (if it was corrected by automatic person identification and has not been confirmed by the users yet). If the user presence list 16 is marked as not trustworthy, the procedure continues in the way it runs when verification is unsuccessful (i); after the end of the security procedure 14 the procedure 11 upon entry only continues if the alarm was not set off. If the user presence list 16 is trustworthy, additional identification request is immediately launched. If it succeeds, the procedure 11 upon entry continues normally, otherwise the alarm is set off and the procedure 11 upon entry stops.

If the person, who identifies himself/herself for the entry has no entry clearance, the users are sent a warning about an entry attempt and the entry procedure is stopped without allowing the person to enter.

When a person exits security procedure 14 can be launched for two reasons. The first reason is that the verification procedure 15 of the person exiting failed. In this case, the user presence list 16 is marked as potentially not trustworthy and afterwards, the selected users are sent a warning about a failed verification upon exiting, which could indicate an unusual exit. The warning serves as a notification as well; it signifies that the user presence list 16 must be updated manually. The second reason for security procedure 14 to be launched upon exit is when the person exiting the place did not identify him/herself. In this case the identification procedure 15 of the person exiting is launched. If the identification fails, security procedure 14 continues in the way it runs when verification upon exiting fails. If the identification is successful, it is noted that the presence list 16 is potentially not trustworthy.

Security procedure 14 may as well be launched on the basis of security threat detection 14a, which operates on the basis of one of the security sensors (i.e. the motion detection sensor on the inside of the door, the sensor detecting a physical attack on the door or a dedicated alarm system), when the user presence list 16 registers no present users. In this case a security procedure 14 sets off the alarm.