METHODOLOGIES

This application claims priority of U.S. provisional patent application 61/915, 175 filed December 12, 2013. Technical Field

This invention relates to facility security, and in particular to systems and methods for augmented reality enhanced advanced security authentication.

Background

In the field of facility security management, there is a continuing need for enhancing access security.

With reference to Figures 1A and 1 B, current intelligent access control systems are generally composed of door lock hardware and associated personal computer (PC) software. The lock hardware represents physical devices mounted on doors or other types of access points (such as gates, barriers, lockers, etc.) to limit access to that physical location and/or facility to authorized personnel only. The PC software provides the intelligent interface to the access control system which allows an administrator to input configuration and authorization rights for a population of users. Traditionally, these access control systems also include a "programming" device, for example a Personal Data Assistant (PDA) or palm computer which ferries physical lock configuration information from the host PC software to battery operated locks and vice versa.

In recent years, with reference to Figure 2, advancements in radio-frequency technologies integrated into the lock hardware have made locks more intelligent and are now communicating wirelessly to the PC software. This has created "Access Control Networks" which now provide real-time features such as remote access, instant revocation and real-time activity notifications. Software technology has also been evolving and the new concept of "Software as a Service" (SaaS) is taking root in the Access Control field. SaaS in the form of Cloud computing is a shift from the traditional PC installed software, Figures 1A/1 B and 2, to software hosted on a remote sever managed by a third party (Figure 3). These facility security products are now offered as a service versus the traditional shrink- wrapped CD that is sold to a facilities administrator (end user).

Summary

Developments in Access Control Systems (ACS) have now created ACS clouds via which end users subscribe to their security software and can scale their usage based on evolving requirements for example as illustrated in Figure 3. Along with the ACS Cloud, Figure 3 illustrates components for a full online cloud-based access control system. The main components include:

Tokens - RFID card, FOB, smartphone, tablet, etc. that are used to authenticate to the lock hardware on the customers door either directly to the device via Near Field Communications (NFC) or indirectly to the device through the ACS Cloud;

Wi-Fi / 3G Enabled Device - laptop / smartphone / tablet with web access used to administrate the customers locks via the ACS Cloud remotely or via Wi-Fi at the facility/residence location;

Router - the customer's premises wireless router;

Gateway Device - a dedicated device connected to the router which can be paired with multiple locks installed in range as well with range extending infrastructure; · Lock Hardware - lock hardware with a wireless connection back to the

ACS gateway, NFC/RFID, and key override; and ACS Cloud - ACS software which includes the customer-facing web portal, private web service accessed by customer's gateway hardware, and/or a public API to be used for third-party applications or other ACS systems.

The ACS cloud is accessible via the Web using any web enabled device with a browser anywhere in the World. It is noted that in this deployment scenario the door lock may not have a physical keypad human-machine interface. The human- machine interface is on the web enabled device, typically via an app.

In accordance with the proposed solution, instead of using a company ID card or a PIN code to gain access to a door, an augmented reality device is employed to provide a more natural and secure authentication process. In particular the human- machine interface cannot be snooped as it is only presented to the heads-up display wearer.

In accordance with an aspect of the proposed solution there is provided an access control system for granting access to a facility, the system comprising: a controlled lock configured to communicate with a remote access control server via data network infrastructure, said controlled lock providing lock identification information; and a heads-up display device configured to provide a human-machine interface enabling a user to input credentials or authenticate himself by other means to unlock said controlled lock, said heads-up display device being further optionally configured to communicate wirelessly with said remote access control server via said data network infrastructure, said heads-up display device including a forward facing camera configured to obtain lock identification information from said lock.

In accordance with another aspect of the proposed solution there is provided a method for granting access to a facility, the method comprising: identifying a lock by obtaining lock identification information; determining whether said lock is accessible via a particular heads-up display device authenticating a user wearing said heads- up display device with a remote access control system; server executing logic instruction implementing facility access control; requesting opening of said lock; and informing the user wearing said heads-up display to proceed with actuating a knob of said lock.

Brief Description of the Drawings

The invention will be better understood by way of the following detailed description of embodiments of the invention with reference to the appended drawings, in which:

Figure 1 is a schematic diagram showing a prior art facilities security deployment;

Figure 2 is another schematic diagram showing a prior art wireless facilities security deployment; Figure 3 is a schematic diagram illustrating a cloud based facilities security deployment in accordance with the proposed solution;

Figure 4 is schematic diagram illustrating, in accordance with the proposed solution, trust relationships and communications channels via which the trust relationships are established in providing access control; Figure 5 is a schematic message passing diagram illustrating primitives employed in establishing a new access control service in accordance with an embodiment of the proposed solution;

Figure 6 is a schematic diagram illustrating a heads-up display interface for inputting credentials in accordance with the proposed solution; Figure 7 is another schematic diagram illustrating a heads up display interface for inputting biometric credentials in accordance with the proposed solution;

Figure 8 is a schematic message passing diagram illustrating a message flow in granting access to a facility to an enrolled user in accordance with the proposed solution; Figure 9 is a schematic message passing diagram illustrating a message flow for an enrolled user granting access for an unenrolled person to enter a facility in accordance with the proposed solution;

Figure 10 is a schematic diagram illustrating a QR code for conveying enrolled lock particulars in accordance with the proposed solution;

Figure 11 is a schematic message passing diagram illustrating a message exchange in opening a lock in accordance with a preferred embodiment of the proposed solution; and

Figure 12 is another schematic message passing diagram illustrating another message exchange in opening a lock in accordance with another preferred embodiment of the proposed solution, wherein similar features bear similar labels throughout the drawings.

Detailed Description

Methods and apparatus for actuating a controlled lock to access a facility are described. The apparatus includes an electronic lock enrolled with an access control cloud service. The lock is associated, in programming, with at least one user also enrolled with the access control cloud service. The methods include sequences of messages and commands for requesting and granting access to the facility.

Figure 4 schematically illustrates trust relationships and lists possible communications channels via which the trust relationships are established in providing access control in accordance with the proposed solution.

In accordance with the proposed solution, because facilities access control is provided as a remote service, a number of trust relationships are broadened and a number of new trust relationships are possible. The core remains the same as in the prior art deployments: Access to a facility such as, but not limited to: restricted grounds, a room, a locker, etc. is controlled through a door/gate having an intelligent lock also referred to as an Electronic Access Point (EAP). The grounds or room have a fixed geographical location defining inherent trust relationships. However, a locker can change geographical location, for example when implemented in a secure truck or secure container. This new trust relationship is enabled by the cloud based ACS which removes the restriction that the lock be collocated with the ACS Gateway device.

The type of access control is not limited to doors. These methods can further extend to other lock form factors such as: o Padlock form factors for use around the home, garage, fencing, etc. o Gun safe locks o Kid's room locks o Game/entertainment system access control by parents o Child safety locks to secure cabinets containing dangerous chemicals

In accordance with the proposed solution, each person actor is equipped with an Artificial Reality (AR) device participating in the ACS deployment. AR devices include, while not being limited to: a wearable Heads-Up Display (HUD), a smartphone, a tablet, a laptop, etc. Examples of wearable heads-up displays include but are not limited to: Google Glass by Google Inc., Meta Spaceglasses by www.spaceglasses.com, etc.

In particular heads-up displays can access the ACS cloud through the customer's associated network-connected device (smartphone, tablet, laptop, etc.) or a network connection internal to the HUD itself. The ACS could authenticate the user and administrate the customer's system. Voice activated HUDs, such as Google Glass, enable voice operated control of the customers lock's from anywhere in the World. Google Glass allows for different authentication scenarios such as voice entered PIN codes to grant hands-free access to the lock, image-based authentication, etc. Eventually a Google Glass style headset may be able to allow the user to look at their hand, read gestures or read their fingerprints or other biometric data, such as pulse, vasculature, etc., and grant access

New Access Control Service Subscription

In accordance with an embodiment of the proposed solution Figure 5 is a schematic message passing diagram illustrating primitives employed in establishing a new access control service. Assuming an authentication between the administrator user and an AR device and an authentication between the lock and the cloud-based ACS, registration entails adding a new user of type "administrator". Assuming a single administrator scenario, this message also defines a new configuration. The response to the new user message is a request for credentials. Depending on the AR device used, the credential response includes an AR device type and the corresponding credential. For example, if the device is a "smartphone" the credential can be "PIN", Codeword, Un/LockPattern, etc. If the device is a tablet the credential can be "PIN", Codeword, Un/LockPattern, Un/LockGesture, etc. Figure 6 is an example of a pin unlock code input screen, whereas Figure 7 is an example of a fingerprint reader input screen such as provided by AIRPrint incorporating the use of the HUD-mounted camera and AR application to get the intuitive biometric input with assistive feedback shown in Figure 7.

Following the registration procedure, the Admin user logs in by requesting admin privileges via the AR device and providing the corresponding credentials. The following description makes an abstraction of other primitives such as are necessary in changing credentials on a particular device, etc.

A logged in admin can enroll a new lock and a new user. In enrolling a new lock, the Admin user provides an admin credential for the lock via an "Add Lock ()" command. This credential can be different depending on the device employed. In some deployment scenarios, the credential is the admin credential employed by the Admin user to authenticate with the ACS cloud. The ACS cloud responds with requesting "New Lock ()" particulars. The Admin enters the LockID, and the AR device provides the ACS cloud service the entered LockID and the current location of the AR device. The ACS cloud service respond with "Stored ()" message. In this way a number of locks can be enrolled with the ACS cloud service. Enrolling locks can be terminated by a "Logout()" command, for example.

In enrolling a new user, the Admin user provides an admin credential for administering a new user's profile via an "Add User ()" command. This credential can be different depending on the device employed. In some deployment scenarios, the credential is the admin credential employed by the Admin user to authenticate with the ACS cloud. The ACS cloud responds with requesting "New User ()" particulars. The Admin enters the user name and the device the user is allowed to employ to actuate managed locks into the Admin's AR heads-up display device. The AR device provides the ACS cloud service the new user's name and the user's device. The user's device unique device identifier which can be a smartphone IMEI, a Media Access Control (MAC) address, etc. The ACS cloud service contacts the device based on the unique device identifier. Alternately the user could connect to the ACS cloud and enter a code provided by the Administrator if contacting the user's device is not practical or possible. The user's device displays a credential entry screen and user enters the credentials, for example a Codeword or an Un/LockPattern. The ACS cloud service responds to the Admin's AR heads-up device with "Stored ()" message. In this way a number of new users can be enrolled with the ACS cloud service. Enrolling users can be terminated by a "LogoutO" command, for example.

Once at least one lock and at least one user are enrolled with the ACS service, users and locks can be associated thus granting users access to locks and therefore access to specific facilities. The Admin user provides an admin credential for associating user(s) profiles with LocklDs via an "Add Association ()" command. This credential can be different depending on the device employed; the credential is the admin credential employed by the Admin user to authenticate with the ACS cloud. The ACS cloud responds with requesting "New Pair ()" particulars. The Admin enters the user name and LockID the user is allowed actuate into the Admin's AR heads-up display device. The AR device provides the ACS cloud service the user's name and LockID. The ACS cloud service responds with "Stored ()" message. The ACS cloud service also sends a "Grant ( LockID ) message to the user's device based on the user's profile, and the user's device stores the LockID. In this way a number of users can be associated with locks. Associating users with locks can be terminated by a "Logout()" command, for example. It is noted that compatibility between locks and users' devices is not required as the locks are actuated and authenticated by the ACS cloud service.

Enrolled User Operating an Enrolled Lock

In accordance with the proposed solution, Figure 8 illustrates a method granting access to a facility.

Accessing a facility may be initiated, for example by a user actuating an EAP of an enrolled lock. The lock can broadcast its LockID only in response to the actuation; this can reduce power requirements for a battery operated lock. However the enrolled lock can periodically emit in-the-blind a signal identifying itself. For example, the lock can blink an LED in an identification pattern. Optionally the lock can inform the ACS cloud service of a temper incident by providing its LockID and optionally the local time. The time of the tamper incident can also be generated by the ACS cloud service at the time of the receipt of the "Tamper ()" message.

The heads-up display AR device of the user reads the LockID, for example via an app executing on the AR device. Alternatively, and without limiting the invention thereto, as illustrated in Figure 10 the LockID can be in the form of a QR-code which can be read by the AR device via a QR code reader app. While references to QR codes are made throughout this application, it is understood that other marker types can be employed and are more generically referred to as AR markers, bar codes (both 1 D and 2D), etc. The AR device performs a lookup based on the LockID to determine whether the LockID corresponds to a known lock thereto. If the LockID is unknown the AR device displays "restricted access" indicia. If the LockID is known, the AR device displays at least open indicia signifying that the lock can be opened for example displays a particular icon. The user interacts with the open indicia to request access. The AR device responds by logging in to the ACS cloud service providing the user name and location. An open request can be implicit or explicit as illustrated providing the LockID broadcast locally by the lock. The ACS cloud service requests the login credentials from the heads-up display AR device and the AR device presents the user with a credential input screen. Once the unlock gesture is input, the AR device provides the ACS cloud service with the credential.

With the removal of the limitation of a fixed controlled facility and collocation between the lock and geographical location, the location becomes an actor. The ACS cloud service performs a lookup based on the LockID provided by the lock and determines if the user location and the location of the lock match. Inside buildings location can be provided by location markers, essentially a sort range broadcasting milestone device. A positive match with respect to the location results in a "Success ()" message being sent to the heads-up display AR device and an "Unlock ()" message being sent to the lock itself. An actuation of the lock EAP results in an open door. While the ACS cloud service clears the tamper indication. In some implementations, the tamper flag is only cleared if the "Success ()" message was generated within a predetermined period of time since the "Tamper ()" message. In other implementations the "Unlock ()" message carries a predetermined period of time within which the lock is to remain in the unlocked state. Enrolled User Grants Access to an Enrolled Lock

Along with authentication the person, the person's location can also add to the process with location based services available today on all smartphones. For example it becomes possible to remotely allow entry to a visiting grandmother. That is the person actuating the lock's EAP need not necessarily be the owner/registered user.

Figure 9 illustrates a message passing diagram implementing remote lock opening by an enrolled user. For example the process can be initiated by an unenrolled person by actuating the EAP of the enrolled lock. In response, the lock emits its LockiD which is received by the unenrolled user's AR device. Alternatively, the lock can emit its LockiD continuously. Yet another way, the lock can bear a unique QR code identifying the lock, QR code which is read by the unenrolled person's AR device via a QR code reader. (The LockiD can be displayed to the unenrolled person.) Optionally the AR device of the unenrolled person provides the location of the AR device to the ACS cloud service informing the ACS cloud service that the AR device is in front of the door.

With the LockiD, the unenrolled person places a voice call to a user associated with the lock. The LockiD is provided to either the enrolled user as a voice communication or to the enrolled user's AR device via inter device communications (including an SMS message, an iMessage, a BBM message, etc.) The enrolled user authenticates with his/her heads-up display AR device and provides the LockiD for opening the door. The heads-up display AR device performs a lookup to determine if the LockiD corresponds to a lock with which it is associated (and/or administers if the enrolled user is the administrator). If the lock is known, then the heads-up display AR device logs in the enrolled user with the ACS cloud service. In response, the ACS cloud service can provide a lock tamper report with a LockiD and the time of the lock's EAP actuation. The head-up display AR device compares the LockID provided by the user with the LockID from the tamper message and reassures the enrolled user that indeed a person is in front of the door. The time of the tamper incident can provide further reassurance that the voice call and the tamper are happening in real time. Optionally the enrolled user can ask the unenrolled user to actuate the lock's EAP which results in the ACS cloud service providing another tamper report as a further reassurance to the enrolled user.

When sufficiently reassured that the unenrolled person is in front of the correct door, the enrolled user can chose to unlock the door by interacting with unlock indicia on his heads-up display AR device. An "Open( LockID )" message is send from the enrolled user's heads-up display AR device to the ACS cloud service, the LockID being provided to ensure that there is no ambiguity as to which lock is being actuated. As a further optional sanity check, the ACS cloud service can compare the location associated with the LockID and the location reported by the unenrolled user are the same before proceeding. Next, the ACS cloud service requests credentials from the enrolled user to unlock the lock. As before, the heads-up display AR device displays a credential input screen, for example but not limited to as shown in Figures 6 or 7. After the enrolled user enters the unlock gesture, the credentials are provided by the heads-up display device to the ACS cloud service which responds with a "Success ()" message. The ACS cloud service also sends an "Unlock ( timeDuration )" command to the lock and clears the tamper condition for the LockID. The timeDuration parameter is optional, the lock can have a present time duration for which it unlocks when instructed to do so.

The success of the remote opening is displayed to the enrolled user via the heads- up display AR device and the enrolled user can instruct the unenrolled user to try opening the door again. If the actuation of the EAP takes place within the timeDuration of the "UnLock ()" command the door opens.

Having described the above facility access granting processes, a variety of advanced access granting processes are possible. For example: Advanced Facility Access Granting

Current advances are being made in authentication, for example:

^* TouchGo, by Kaba, is a non-invasive wireless technology allowing users to keep a physical token on their person for accessing a door lock. TouchGo uses intra-body low frequency communication to convey authentication information (RFID or Bluetooth 4.0). Such tokens can be in the form of a ring worn on a finger, incorporated into a smart watch, worn as a jewelry pendant.

^* Another non-invasive technology is provided by Eye Lock which uses a commodity smart phone/tablet camera and a cloud service to provide iris scanning in providing authentication.

^* Electronic tattoos, by The Rogers Research Group at the University of Illinois at Urbana-Champaign, are printed directly on skin and generally include the components of an RFID card and a stretchable battery. Such an electronic tattoo can be employed to authenticate the person wearing it for an extended period of time, about two weeks, it takes for the skin to naturally exfoliate the tattoo.

* Ingestible tokens, by Proteus Digital Health, have been FDA-approved for medical applications to identify a medicine pill type, which when ingested emits an 18-bit signal like an electrocardiogram identifying the medicine pill type. Such technology can be extended to provide a unique signal identifying a particular pill, which can be employed to authenticate the person who swallowed it for an extended period of time it takes for the pill to pass through the digestive tract. The authentication can be activated by touch, since the human body conducts electricity.

* Currently available injectable RFID tokens, typically employed in elder care and assisted living scenarios, can provide surgically removable long term authentication.

In accordance with a preferred embodiment of the proposed solution, the enrolled user's AR device is a heads-up display device configured to detect when the heads- up display device is taken off and put on. For example, such functionality can be implemented by: detecting electrical continuity through the users' skin between two electrodes, detecting a continuous heart beat in a heads-up display device configured to also measure heart rate, detecting repeated identification beacon messages from body worn/ingested/tattooed/implanted tokens (mentioned above), etc. In this way an initial authentication between the enrolled user and the heads-up display device can be employed as persistent authentication to automatically authenticate the enrolled user with the ACS cloud service.

Similarly in the preferred embodiment the lock provides information identifying the lock either: via a repeatedly broadcast blinking pattern, via a Bluetooth beacon message, via a WiFi beacon message, via NFC messaging, etc. Alternatively the lock has displayed thereon a QR-code identifying at least the lock and perhaps the ACS service with which the lock is enrolled and the heads-up display AR device executes a QR code reader app. For certainty, broadcasting LockID information upon actuating the lock EAP is not excluded.

For example, a modified lock opening procedure is illustrated in Figure 11 . A single authentication with the heads-up display device is performed once the heads-up display device is put on. The heads-up display device automatically displays the credential input screen, for example as illustrated in Figures 6 or 7, if a body worn/ingested/tattooed/implanted tokens (mentioned above), etc. identification beacon message is not detected. One of the advantages of displaying a credential input screen on a heads-up display device is that the credential entry is private. In order for the credential not to be discerned by others from user's gestures, the virtual keypad displayed as illustrated in Figure 6 can be scrambled and/or can be rescrambled with each combination digit entry.

Referring to the description of the messages exchanged as illustrated in Figure 8, a more automated and expedient unlock method can be achieved by removing the authentication step for the enrolled user to actuate the lock EAP. As well, the unlocked state the lock can be displayed to the enrolled user privately via the heads-up display device for the predetermined timeDuration. In accordance with yet another embodiment of the proposed solution, the procedure illustrated in Figure 1 1 is slightly modified as illustrated in Figure 12.

Namely, the timeDuration the lock is to be unlocked for is set to a very short duration such as a few seconds, for example 2sec at a particular time provided by the ACS cloud service to both the lock and the heads-up display device. The heads-up display device is then configured to countdown to the unlock time specified.

Customized functionality based of the location of devices linked to your account (i.e. the mother in-law cannot open the front door unless someone is home and then she can come in on her own, etc.)

While the invention has been shown and described with referenced to preferred embodiments thereof, it will be recognized by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.