CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the priority and benefit of U.S. Provisional Application No. 63/309,133, filed on February 11, 2022, the entirety of which is incorporated herein by reference.

BACKGROUND

[0002] Checking-in to hotels can be a long process involving in-person verification of identification, hotel staff confirming reservation record, checking room availability, distributing physical room keys and the like. During the in-personal verification, guests may have to provide physical identification documents (e.g., ID card, driver license, passport, etc.) at the check-in desks for the hotel staff to scan/upload into their system which further increases the waiting time meanwhile the human verification can be inaccurate (e.g., lack of capability to detect a fake ID). Additionally, physical room key may burden the guests and hotel staff for their losses of physical room key cards or wrong keys being issued to the room guests.

SUMMARY

[0003] Recognized herein is a need for a check-in system with improved efficiency, accuracy for identity verification while reducing the requirement for resources or extra set up. Provided herein are methods and systems for managing contactless check-in to premises (e.g., hotels, workplace, etc.) by transmitting digital keys to users (e.g., digital room keys to guests). The methods and systems herein may allow guests to be able to check-in in a totally remote and contactless manner. For example, guests may be able to provide their identification document without the need to hand over to the hotel staff, and verify themselves when being issued/usage of their digital room keys via a guest mobile device. The provided systems and method may also allow hotels and property managers to minimize the check-in time and simplify the digital room key management system. [0004] In an aspect of the present disclosure, a computer-implemented method is provided for providing a digital room key to a user. The method comprises: extracting personal identification data and a first facial data of the user by processing an image of a physical element with aid of a machine learning algorithm trained model; receiving a request for requesting the digital room key or a request for accessing a premise associated with a reservation record; capturing a selfie image of the user using the mobile device, the selfie image comprises at least one image frame such as a static image or a video clip; verifying an identity of the user based on a comparison of a second facial data extracted from the selfie image and the first facial data; verifying the at least portion of the personal identification data against the reservation record; and receiving at the mobile device the digital room key.

[0005] In a related yet separate aspect, the present disclosure provides a non-transitory computer-readable storage medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations. The operation comprise: extracting personal identification data and a first facial data of the user by processing an image of a physical element with aid of a machine learning algorithm trained model; receiving a request for requesting the digital room key or a request for accessing a premise associated with a reservation record; receiving a selfie image of the user from the mobile device, the selfie image comprises at least one image frame such as a static image or a video clip; verifying an identity of the user based on a comparison of a second facial data extracted from the selfie image and the first facial data; verifying the at least portion of the personal identification data against the reservation record; and transmitting the digital room key to the mobile device.

[0006] In some embodiments, the physical element is a photo ID. In some cases, the photo ID comprises an image of the user and text data. In some instances, extracting the personal identification data comprises using an optical character recognition model trained for detecting and localizing the text data. In some instances, the first facial data extracted from the physical element comprises facial landmarks and wherein the first facial data is extracted from the image of the user on the photo ID.

[0007] In some embodiments, the second facial data is extracted from the selfie image using the machine learning algorithm trained model. In some cases, the machine learning algorithm trained model comprises cascaded convolutional neural networks. In some embodiments, the digital room key is used to lock or unlock a door via a peer-to-peer networking protocols comprising Near Field Communication (NFC) technology, Bluetooth, BLE, Wi-Fi Direct, or RFID.

[0008] In some embodiments the method or operations further comprise providing a user interface on the mobile device for the user to request the digital room key. In some cases, the user interface prompts the user to take the selfie image and notifies the user when the selfie image is not valid.

[0009] A “selfie image” as described herein may comprise one or more images frames. A selfie image may comprise one static image frame, an image sequence or dynamic image data (e.g., video).

[0010] Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in this art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the disclosure. Accordingly, the drawings and descriptions are to be regarded as illustrative in nature, and not as restrictive. INCORPORATION BY REFERENCE

[0011] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings (also “figure” and “FIG.” herein) of which:

[0013] FIG. 1 illustrates an exemplary system in which the contactless check-in and identity verification method described herein may be implemented;

[0014] FIGs. 2-4 illustrates an exemplary method for registration;

[0015] FIGS. 5 shows an example of photo ID and OCR result;

[0016] FIG. 6 shows an example of a network architecture that detects and recognizes text from an image;

[0017] FIG. 7 schematically illustrates an exemplary architecture of a deep learning framework for face detection;

[0018] FIG. 8 shows an example of an OCR system configured to extract the personal data from an ID image;

[0019] FIG. 9 schematically illustrates an exemplary architecture of a deep learning framework for face detection system;

[0020] FIGs. 10-13 illustrate an example of the process for contactless check-in and checkout at a hotel;

[0021] FIG. 14 shows examples of graphical user interfaces (GUI) for registration; [0022] FIG. 15 shows an example of a graphical user interface (GUI) for a user to perform authentication by taking a selfie image;

[0023] FIG. 16 shows an example GUI for locking/unlocking a door using a digital key; [0024] FIGs. 17-19 show an example of a contactless check-in process; and

[0025] FIGs. 20-24 shows exemplary flow for obtaining a digital key.

DETAILED DESCRIPTION

[0026] While various embodiments of the invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed.

[0027] An aspect of the present disclosure provides systems and methods for providing digital room keys. The digital keys (e.g., digital room key, digital access/door key, an entrance passcode, etc.) may be provided to a user (e.g., guest) upon authentication of the guest. The identity verification or authentication may be performed using a mobile device of the user using facial recognition techniques. The identity verification or authentication may be performed in order to issue the digital keys. The digital key may be possessed by the verified guest for a period of time (one day, two days, three days, 1 week, 2 weeks, 3 weeks, 1 month, 2 months, etc.) until expiration. In some cases, the digital key may expire upon a check-out by the user. In some cases, the identity verification (e.g., facial recognition) may be performed only once in order to issue the digital key to the user and once the digital key is retrieved, the user may access the place (e.g., unlock a hotel door) using the digital key without going through the identification verification (e.g., facial recognition) again.

[0028] The term “guest,” as used herein, generally refers to an individual who is seeking to gain access to a premise or a resource with a digital key. For example, a guest (or user) may be a guest to check in at a hotel or an individual to gain an access to a premise, (e.g., building, workplace, etc.) and may be issued a digital key upon authentication or user identity verification.

[0029] In some embodiments, the process for a guest to obtain a digital key to enter a premise until check-out may comprise multiple phases including, for example, making reservation, registration, check-in or digital key request/issuance and check-out. In some cases, at least a portion of or the entire process may be performed without in-person verification, without physical check-in or without user intervention from hotel staff. It should be noted that although the methods and processes are described with respect to a hotel check-in and check-out process, the methods and systems herein can be utilized in various other scenarios where a key is required to enter a premise.

[0030] As an example, during the registration phase, a user may submit a registration request to gain access to a premise via a user interface or an application running on a user device. The user may be guided to take an image of an identification document of the user and upload the image via the user interface. The identification document may contain identity information about the user such as name, gender, date of birth, address, nationality and the like that describe identity of a user. The identity document may be a card (e.g., ID card, driver license, payment card, library card, login card, etc.), documents (passport, legal document, healthcare record, etc.), and the like that are issued by an authority entity such as government, DMV, federal agency and the like in the form of credentials. The identification document may be a person’s civil credential such as social security card, passport, driver license, e-passport, birth certificates, employee identity cards and the like to establish identity of a user. In some cases, the user interface may provide information about the type of identification document (e.g., passport, driver license, ID card, etc.) that is acceptable or may inform the user when the image of the identification document is processed and determined that the type of the ID document is not an acceptable type based on the hotel policy. [0031] In some embodiments, the identification document may comprise a photo of the user

(i.e., photo ID). The image data of the identification document may be processed by a backend system to extract identification data including both text data about, the user’s identity and facial data (e.g., facial landmarks data) from the 2D photo image (e.g., passport photo image) contained in the image data.

[0032] For instance, the backend system may comprise an optical character recognition (OCR) system using a trained model to produce the output of data fields prior to registration submission. The output of data fields may include, for example, name, date of birth, nationality, or other personal information about the user, information about the ID document such as issuance date, expiration date, issuer, type of ID, and facial data. In some cases, the OCR system may further authenticate or verify the physical ID such as by identifying that ID is fake or altered based on visual information. For instance, the OCR system may be trained to verify whether certain visual features (e.g., color code pattern, machine readable zone, etc.) presented in the image of the ID document is in compliance with a type/format of ID document. The OCR system may be trained to be able to identify a variety types/formats of ID documents. Details about the registration method are described later herein.

[0033] After a check-in phase, a user may submit a request for a digital key (e.g., Digital Door Key to a premise's room) via a user interface provided on a user device. The user interface and/or user device needs not be the same device used for the registration process. The user may be guided to take an image data of themselves (selfie) using the user device. The image data may be received by the backend system and processed by a Facial Recognition (FR) system running on the backend system. For example, the FR system may extract the facial data (e.g., facial landmarks) from the selfie image and compare the facial data with the facial data extracted from the passport image to verify an identity of the user. The system may determine if the facial data extracted from the selfie image matches the facial data extracted from the user ID image (e.g., passport image). Upon the determination of a match, the user may be authenticated or verified to complete the check-in process.

[0034] The user may be approved for the issuance of a digital door key. The digital door key may be generated and delivered to the user based on a reservation record and information provided during the check-in phase. In some cases, once check-in is completed, the guest may have the option to pick up a physical key card at the front desk, and/or download a hotel- branded mobile app to receive the digital key directly in their smartphone.

[0035] The hotel digital key system (property management system) may be integrated with the system herein so that the user verification and reservation information is seamlessly transmitted to the hotel property management system. The digital key may be issued to the use for a predetermined period of time such as based on the reservation record. In some cases, the digital key may expire upon a user check-out. During check out, the guest can settle bill payments as well as check-out the room using the web portal or mobile application same as those used for check-in. Details about the facial recognition-based verification method are described later herein.

[0036] FIG. 1 illustrates an exemplary system 121 in which the contactless check-in and identity verification method described herein may be implemented. The system 121 may be implemented in a network environment 100 comprising one or more user devices 101-1, 101-2, 101-3, a server 120, the system 121, third-party systems 130, and one or more databases 111, 123, 132. Each of the components 101-1, 101-2, 101-3, 111, 123, 120, 130, 132 may be operatively connected to one another via network 110 or any type of communication links that allows transmission of data from one component to another. The third-party system 130 may be a system for managing a premise (e.g., hotel, workplace, building, etc.). For example, the third- party system 130 may be a hotel data center including a property management system 131 and/or local storage resources 132. In some cases, the hotel data center 130 may comprise one or more of a property management system (PMS) 131, communications network, backend database

132, and/or a digital door key server.

[0037] The system 121 may be configured to process input data (e.g., image data) from the user device to extract identification data during a registration phase, perform identity verification by processing an image of the user (e.g., selfie image) during a check-in phase and may communicate with the third-party system 130 to facilitate reservation, check-in and check-out. In some cases, the system 121 may also receive user information from external data sources such as local government database e.g., EVA (E- Visitor Authentication) for further verifying the authenticity of an identification document or an identity of the user.

[0038] The system 121 may be implemented anywhere within the network environment 100. In other embodiments, a portion of the system 121 may be implemented on the user device. Additionally, a portion of the system may be implemented on the server or cloud 120. Alternatively, the system may be implemented in one or more databases. The system may be implemented using software, hardware, or a combination of software and hardware in one or more of the above-mentioned components within the network environment.

[0039] The user device 101-1, 101-2, 101-3 may comprise an imaging sensor 105-1, 105-2, 105-3 serves as imaging device. The imaging device may be on-board the user device. The imaging device can include hardware and/or software elements. In some embodiments, the imaging device may be a camera or imaging sensor operably coupled to the user device. In some alternative embodiments, the imaging device may be located external to the user device, and image data of at least a body part of the user may be transmitted to the user device via communication means as described elsewhere herein. The imaging device can be controlled by an application/software configured to take image or video of the user. In some cases, the camera may be configured to take an image of at least face of the user. In some embodiments, the software and/or applications may be configured to control the camera on the user device to take image or video (e.g., live images). [0040] The imaging device 105-1, 105-2, 105-3 may be a fixed lens or auto focus lens camera.

A camera can be a movie or video camera that captures dynamic image data (e.g., video). A camera can be a still camera that captures static images (e.g., photographs). A camera may capture both dynamic image data and static images. A camera may switch between capturing dynamic image data and static images. Although certain embodiments provided herein are described in the context of cameras, it shall be understood that the present disclosure can be applied to any suitable imaging device, and any description herein relating to cameras can also be applied to any suitable imaging device, and any description herein relating to cameras can also be applied to other types of imaging devices. The camera may comprise optical elements (e.g., lens, mirrors, filters, etc.). The camera may capture color images (e.g., RGB images), greyscale image, infrared images, depth image, and the like.

[0041] The imaging device 105-1, 105-2, 105-3 may be a camera used to capture visual images of at least part of the human body (e.g., face). In some cases, the camera may also be used to take an image of a physical document possessed by the user (e.g., ID card, passport, etc.). Alternatively, the image of the physical document may be scanned, stored or uploaded using any other suitable devices. Any other type of sensor may be used, such as an infra-red sensor that may be used to capture thermal images of the human body. The imaging sensor may collect information anywhere along the electromagnetic spectrum, and may generate corresponding images accordingly.

[0042] In some embodiments, the imaging device 105-1, 105-2, 105-3 may be capable of operation at a fairly high resolution. The imaging sensor may have a resolution of greater than or equal to about 100 pm, 50 pm, 10 pm, 5 pm, 2 pm, 1 pm, 0.5 pm, 0.1 pm, 0.05 pm, 0.01 pm, 0.005 pm, 0.001 pm, 0.0005 pm, or 0.0001 pm. The image sensor may be capable of collecting 4K (3840 x 2160 pixels) or higher resolution images. The imaging device 105-1, 105-2, 105-3 may capture an image frame or a sequence of image frames at a specific image resolution. In some embodiments, the image frame resolution may be defined by the number of pixels in a frame. In some embodiments, the image resolution may be greater than or equal to about 352x420 pixels, 480x320 pixels, 720x480 pixels, 1280x720 pixels, 1440x1080 pixels, 1920x1080 pixels, 2048x1080 pixels, 3840x2160 pixels, 4096x2160 pixels, 7680x4320 pixels, or 15360x8640 pixels.

[0043] A user device used to capture the selfie image may or may not be the same device used to upload the image of the physical identification document. For example, a user may be associated with multiple user devices and may use a first user device for registration (e.g., submitting a registration request, reservation and image of a passport) and use a second user device to take a selfie for requesting a digital key or check-in. The submitted image data of the photo ID may then be processed by the system 121 to extract identification data and the selfie image data may be processed by the system 121 for verifying the user. In some cases, the image data may be 2D/3D image data or video data. The image data may be color (e.g., RGB) images or greyscale images. In some cases, the image data may be raw data captured by a user device camera without extra setup or cost. Details about using computer vision and machine learning techniques for extracting the personal data and facial recognition are described later herein. [0044] In some cases, the user device may comprise a biometric reader or sensor which can collect biometric identifiers, and may have built-in biometric authentication functionalities. In some cases, authentication may be performed for a user to activate an application or for unlocking the device. The user may or may not be required to input login information (e.g., user name, passcode, password) or other credentials in order to use a digital room key that has been issued to the user. In some cases, a user may not be required to provide credentials in order to lock/unlock a door using the digital room key. Alternatively, credentials or authentication may be performed to use the digital room key. In some cases, the authentication may be performed using a built-in biometric authentication module of the user device. Biometric identifiers include, but are not limited to, fingerprint, face recognition, DNA, palm print, palm veins, hand geometry, iris recognition, retina, or odor. In some embodiments, the biometric reader or sensor may be part of a built-in biometric fingerprint authentication technology. The biometric fingerprint authentication technology enables a fingerprint to be converted into a formula, encrypted, and carried over a hardware channel to a secure component within the mobile computing device. In some cases, the biometric identifier may be facial data using face recognition techniques. In some cases, the authentication or credentials required for using the digital room key may not be provided by the system 121. For example, such authentication or verification for using the digital room key may be conducted by the third-party system 130 or PMS 131.

[0045] User device 101-1, 101-2, 101-3 may be a computing device configured to perform one or more operations consistent with the disclosed embodiments. Examples of user devices may include, but are not limited to, mobile devices, smartphones/cellphones, tablets, personal digital assistants (PDAs), laptop or notebook computers, desktop computers, media content players, television sets, video gaming station/system, virtual reality systems, augmented reality systems, microphones, or any electronic device capable of analyzing, receiving, providing or displaying certain types of data to a user. The user device may be a handheld object. The user device may be portable. The user device may be carried by a human user. In some cases, the user device may be located remotely from a human user, and the user can control the user device using wireless and/or wired communications.

[0046] User device 101-1, 101-2, 101-3 may include one or more processors that are capable of executing non-transitory computer readable media that may provide instructions for one or more operations consistent with the disclosed embodiments. The user device may include one or more memory storage devices comprising non-transitory computer readable media including code, logic, or instructions for performing the one or more operations. The user device may include software applications that allow the user device to communicate with and transfer data between server/cloud 120, the system 121, the third-party system such as PMS 131, and/or database 111, 123. The user device may include a communication unit, which may permit the communications with one or more other components in or outside of the network 100. In some instances, the communication unit may include a single communication module, or multiple communication modules. In some instances, the user device may be capable of interacting with one or more components in the network 100 using a single communication link or multiple different types of communication links.

[0047] In some cases, after verifying the identity of the user, the user device of a guest may communicate with the PMS 131 for retrieving a digital room key and/or communicate with a door locking system or a room door lock for locking/unlocking a door. In some cases, the communication with the door locking system or the room door lock may be over a conventional Bluetooth, RFID, BLE, near field communication (NFC), in the form of a peer-to-peer (P2P) networking or Wi-Fi network. Depending on the door locking system, suitable communication techniques e.g., Wi-Fi P2P (i.e., Wi-Fi Direct), beacon technology (e.g., iBeacon), Bluetooth may allow mobile devices with the appropriate hardware to connect to each other directly without the need for an intermediate access point (AP). In some cases, the communication may be direct communication between the mobile device of the guest and the door lock device, even when the guest mobile device lacks access to the Internet or the cellular network. For instance, NFC technology may be implemented to establish connection between the guest mobile device and the door lock device. In some cases, the digital key may be provided in a form (e.g., one time passcode) that does not require a communication between the user device and a door lock system.

[0048] User device 101-1, 101-2, 101-3 may include a display. The display may be a screen. The display may or may not be a touchscreen. The display may be a light-emitting diode (LED) screen, OLED screen, liquid crystal display (LCD) screen, plasma screen, or any other type of screen. The display may be configured to show a user interface (UI) or a graphical user interface (GUI) rendered through an application (e.g., via an application programming interface (API) executed on the user device). The GUI may allow the user to make reservation, upload image data of a physical document, capture selfie using a camera at check-in, conduct transaction (e.g., pay bill), check out or various other functions. The user device may also be configured to display webpages and/or websites on the Internet. One or more of the webpages/web sites may be hosted by server 120 and/or rendered by the system 121.

[0049] In some cases, users may utilize the user devices to interact with the system 121 by way of one or more software applications (i.e., client software) running on and/or accessed by the user devices, wherein the user devices and the system 121 may form a client-server relationship. For example, the user devices may run dedicated mobile applications or software applications for registration, check-in, requesting a digital key, conducting transaction (e.g., pay bill) and check out.

[0050] In some cases, the client software (i.e., software applications installed on the user devices 101-1, 101-2, 101-3) may be available either as downloadable software or mobile applications for various types of computer devices. Alternatively, the client software can be implemented in a combination of one or more programming languages and markup languages for execution by various web browsers. For example, the client software can be executed in web browsers that support JavaScript and HTML rendering, such as Chrome, Mozilla Firefox, Internet Explorer, Safari, and any other compatible web browsers. The various embodiments of client software applications may be compiled for various devices, across multiple platforms, and may be optimized for their respective native platforms.

[0051] Server 120 may be one or more server computers configured to perform one or more operations consistent with the disclosed embodiments. In one aspect, the server may be implemented as a single computer, through which user device are able to communicate with the system and database. In some embodiments, the user device may communicate with the system directly through the network. In some embodiments, the server may embody the functionality of one or more of the systems 121. In some embodiments, one or more systems 121 may be implemented inside and/or outside of the server. For example, the systems 121 may be software and/or hardware components included with the server or remote from the server. A server may include a web server, an enterprise server, or any other type of computer server, and can be computer programmed to accept requests (e.g., HTTP, or other protocols that can initiate data transmission) from a computing device (e.g., user device and/or wearable device) and to serve the computing device with requested data. In addition, a server can be a broadcasting facility, such as free-to-air, cable, satellite, and other broadcasting facility, for distributing data. A server may also be a server in a data network (e.g., a cloud computing network).

[0052] A server may include known computing components, such as one or more processors (e.g., central processing units (CPUs), general purpose graphics processing units (GPUs), Tensor Processing Unit (TPU), etc.), one or more memory devices storing software instructions executed by the processor(s), and data. A server can have one or more processors and at least one memory for storing program instructions. The processor(s) can be a single or multiple microprocessors, field programmable gate arrays (FPGAs), or digital signal processors (DSPs) capable of executing particular sets of instructions. Computer-readable instructions can be stored on a tangible non-transitory computer-readable medium, such as a hard disk, a CD-ROM (compact disk-read only memory), and MO (magneto-optical), a DVD-ROM (digital versatile disk-read only memory), a DVD RAM (digital versatile disk-random access memory), or a semiconductor memory. Alternatively, the methods can be implemented in hardware components or combinations of hardware and software such as, for example, ASICs, special purpose computers, or general purpose computers. In some cases, the server or computing system may be GPU-powered servers that may each include a plurality of GPUs, PCIe switches, and/or CPUs, interconnected with high-speed interconnects such as NVLink and PCIe connections.

[0053] A server may access and execute system(s) to perform one or more processes consistent with the disclosed embodiments. In certain configurations, the system(s) may be software stored in memory accessible by a server (e.g., in memory local to the server or remote memory accessible over a communication link, such as the network). Thus, in certain aspects, the system(s) may be implemented as one or more computers, as software stored on a memory device accessible by the server, or a combination thereof. For example, one system(s) may be a computer training, building, developing one or more predictive models, and another system(s) may be software that, when executed by a server, performs inferences using the trained models.

[0054] The system 121 though is shown to be hosted on the server or cloud 120. The system may be implemented as a hardware accelerator, software executable by a processor, cloud applications, and various others. In some embodiments, one or more systems 121, methods or components of the present disclosure are implemented as a containerized application (e.g., application container or service containers). The application container provides tooling for applications and batch processing such as web servers with Python or Ruby, JVMs, or even Hadoop or HPC tooling. Application containers are what developers are trying to move into production or onto a cluster to meet the needs of the business. The methods and systems can be implemented in application provided by any type of systems (e.g., containerized application, unikernel adapted application, operating-system-levei virtualization or machine level virtualization).

[0055] The platform 100 may comprise one or more third-party systems 130. The one or more third-party systems 130 may be hotel, workplace or any premises where a guest or user is permitted for entrance for a period of time. As an example, the third-party systems 130 may include hotel servers, or PMS 131, the door key server, and/or storage device 132. The hotel servers and the check-in devices 211-214 are connected via a LAN, WLAN, Wi-Fi, or any other type of communications networks that are referred to herein. The hotel server or PMS may be in communication with the system 121 via the network 110.

[0056] In some cases, the system 121 may have access to the data stored in the hotel servers or data managed by the third-party system, and also may be able to request information from, or send information to, the hotel servers. Optionally, the system 121 may store information on each hotels and hotel chains. This information may include check-in policies for each hotel (e.g., type of ID document required, data fields required for reservation, etc.), hotel chains, or check-in policies based on a region or country. Alternatively, such information may be retrieved on the fly through API or communication gateway. For example, the platform 100 may utilize API requests to interact with various hotel service providers to retrieve hotel policy information associate with a reservation or interact with the hotel payment gateway or third-party credit institution to perform transactions (e.g., e-commerce transactions, electronic payment or online transactions).

[0057] In some cases, when a holder of a mobile device checks in at a first hotel, and later in time, the same guest attempts to check in at a second hotel, the backend system may be configured to keep track of the check-in record of the mobile device and inform the second hotel of such information. Depending on the check-in policy for hotel chain — or any regulations or requirements of the specific country or region — guests who have confirmed their physical ID or related documents at one hotel, may be able to skip the in-person confirmation process (i.e., confirming physical ID) at other hotels under the same management umbrella.

[0058] The PMS 131 or the hotel server 130 may be in communication with the system 121 during registration, check-in, check-out or other processes. For example, during a check-in process, the PMS may receive a status confirmation from the system 121, and the PMS may request the digital door key server to issue a digital room key to the guest mobile device. In some cases, the digital door key may be provided to the user via an application running on the hotel server. For example, the user may be required to download an application hosted on a hotel server (e.g., digital door key server) to retrieve and use the digital key. The digital door key server may then transmit a digital room key to the user device via the communications network (e.g., LAN). Alternatively, the digital door key (e.g., one time passcode) may be provided to the user via the application running on the system 121 or via a graphical user interface (GUI), webhooks that can be integrated into the application provided by the system 121. For example, a hotel user interfaces or APIs may be integrated to the mobile application and integrated in the front-end user interface (e.g., within the GUI) so the user can lock/unlock a door using the same mobile application. The digital key can be provided in various forms. For example, a digital key such as a passcode may be delivered to the user via email, text message or other communications. The passcode may expire within a predetermine period of time.

[0059] In some cases, a digital door key server may store information on digital room keys (i.e., digital door keys) and may be configured to manage the issuance and retrieval of digital room keys. Digital room keys may be in a digital form (i.e., software) as opposed to a physical form (e.g., physical keys, plastic key cards). Digital room keys may work in conjunction with a keyless-entry door system, which may enable unlocking rooms without a plastic keycard or a physical key. One or more of the PMS, backend database of the hotel, digital door key server, may be cloud-based, which is accessible online via the communications network 110, a local area network (LAN), wireless local area network (WLAN), wide area network (WAN), Ethernet or the Internet, wireless links, or any combination of the aforementioned types of networks. [0060] Network 110 may be a network that is configured to provide communication between the various components illustrated in FIG. 1. The network may be implemented, in some embodiments, as one or more networks that connect devices and/or components in the network layout for allowing communication between them. For example, user device 101-1, 101-2, 101- 3, the system 121, the PMS 131 or third-party server 130 may be in operable communication with one another over network 110. Direct communications may be provided between two or more of the above components. The direct communications may occur without requiring any intermediary device or network. Indirect communications may be provided between two or more of the above components. The indirect communications may occur with aid of one or more intermediary device or network. For instance, indirect communications may utilize a telecommunications network. Indirect communications may be performed with aid of one or more router, communication tower, satellite, or any other intermediary device or network. Examples of types of communications may include, but are not limited to: communications via the Internet, Local Area Networks (LANs), Wide Area Networks (WANs), Bluetooth, Near Field Communication (NFC) technologies, networks based on mobile data protocols such as General Packet Radio Services (GPRS), GSM, Enhanced Data GSM Environment (EDGE), 3G, 4G, 5G or Long Term Evolution (LTE) protocols, Infra-Red (IR) communication technologies, and/or Wi-Fi, and may be wireless, wired, or a combination thereof. In some embodiments, the network may be implemented using cell and/or pager networks, satellite, licensed radio, or a combination of licensed and unlicensed radio. The network may be wireless, wired, or a combination thereof.

[0061] User device 101-1, 101-2, 101-3, server 120, and/or the system 121 may be connected or interconnected to one or more databases 111, 123. The databases may be one or more memory devices configured to store data. Additionally, the databases may also, in some embodiments, be implemented as a computer system with a storage device. In one aspect, the databases may be used by components of the network layout to perform one or more operations consistent with the disclosed embodiments. One or more local databases, and cloud databases of the platform may utilize any suitable database techniques. For instance, structured query language (SQL) or “NoSQL” database may be utilized for storing the image data, user data, historical data, predictive model or algorithms. Some of the databases may be implemented using various standard data-structures, such as an array, hash, (linked) list, struct, structured text file (e.g., XML), table, JavaScript Object Notation (JSON), NOSQL and/or the like. Such data- structures may be stored in memory and/or in (structured) files. In another alternative, an object- oriented database may be used. Object databases can include a number of object collections that are grouped and/or linked together by common attributes; they may be related to other object collections by some common attributes. Object-oriented databases perform similarly to relational databases with the exception that objects are not just pieces of data but may have other types of functionality encapsulated within a given object. In some embodiments, the database may include a graph database that uses graph structures for semantic queries with nodes, edges and properties to represent and store data. If the database of the present invention is implemented as a data-structure, the use of the database of the present invention may be integrated into another component such as the component of the present invention. Also, the database may be implemented as a mix of data structures, objects, and relational structures. Databases may be consolidated and/or distributed in variations through standard data processing techniques. Portions of databases, e.g., tables, may be exported and/or imported and thus decentralized and/or integrated.

[0062] In one embodiment, the databases may comprise storage containing a variety of data consistent with disclosed embodiments. For example, the databases may store, for example, raw data collected by the imaging device located on user device. The databases may also store user personal data, ID verification record, historical data (e.g., payment transaction, hotel reservation, etc.), data relating to hotel or premise policies, predictive models (e.g., parameters, hyperparameters, model architecture, threshold, rules, etc), data generated by a predictive model (e.g., intermediary results, output of a model, latent features, input and output of a component of the model system, etc.), algorithms, training datasets (e.g., image, video clips), and the like.

[0063] FIGs. 2-4 illustrates an exemplary method 200 for registration. In some embodiments, at least a portion of the method or process 200 is implemented by the system 121 in FIG. 1. In some cases, prior to registration, a user may make a reservation such as via a hotel website. Alternatively, the user may make the reservation via the application provided by the system 121. The reservation information may then be transmitted to the hotel server or PMS for creating a reservation record. A reservation record may be created and maintained by the hotel server or PMS. In some cases, a reservation code or other identifier may be created by the PMS and provided to the user for check-in.

[0064] During registration, a user may be prompted to scan or upload an image of an identification document of the user via a user interface. The user interface may be a registration website or on a mobile application provided by the system herein (e.g., system 121). In some cases, a user (i.e., hotel guest) may provide his or her personal or identification information via a frontend (e.g., GUI rendered on a mobile device or user device, a web portal). In some embodiments, this registration process may be executed via a mobile application or within a web portal which may be configured to run or execute on a user device. The user or hotel guest personal information may be completed prior to the arrival of the guest at the hotel. In some cases, the user personal information may be provided during a mobile application sign up process which may happen before arriving at the hotel. If the user is setting up the mobile application for the first time, the user may be prompted to provide the personal information during the check-in process.

[0065] The personal information or identification data may include passport information, driver’s license, or any other type of identification (ID) information that can be potentially relevant for the purposes of hotel check-ins and identity verification. In some cases, the system and method herein may simplify the registration process by reducing the amount of personal information inputted by a user manually. For example, the user may provide passport information by uploading an image of the passport where the system may process the image data to extract the identification data/personal information and facial features of the user automatically. In some cases, the guest name may be automatically filled in and the user may be prompted to confirm for accuracy. Alternatively, a user may manually input the information required for making a reservation whereas the personal data extracted from the photo ID image may be verified against the user input (e.g., verify the name of the user) or to supplement the user input information.

[0066] As described above, personal data or identification data may be automatically extracted from an image of the passport or other forms of a physical element that is issued by an authority. In some embodiments, the physical element may contain at least an image of a face of the user. The physical element may comprise other information related to an identity of the user which may be required for check-in a hotel or entering a property. The physical element may be a card (e.g., ID card, driver license, library card, login card, etc.), documents (e.g., passport, legal document, healthcare record, etc.), and the like that are issued or provided by an entity or service provider. In some cases, the physical element may be a card, a paper document, or other form of credentials issued by an authority entity such as government, DMV, federal agency and the like. In some embodiments, the physical element may be a person’s civil credential such as social security card, passport, driver license, e-passport, birth certificates, employee identity cards and the like.

[0067] During the registration or sign up process 200, the system may receive an ID check request initiated by the frontend 201. The ID check request may include information related to a reservation. For example, as shown in FIG. 14, a user may enter a booking number and check-in date information via a user interface 1410. The ID check request information (e.g., booking number and/or check-in date) may be used to verify the reservation with the PMS. Upon receiving the ID check request, the system may verify if the ID check request is in a valid format (e.g., contains correct booking number and/or check-in date with a hotel) 203, if yes, the user may be prompted to scan and/or upload the passport or other forms of ID via a user interface 1420.

[0068] In some cases, the user may use a user device to capture an image of the physical document or photo ID. The user device may comprise an imaging sensor serves as imaging device. The imaging device may be on-board the user device. The imaging device can include hardware and/or software element. In some embodiments, the imaging device may be a camera operably coupled to the user device. In some alternative embodiments, the imaging device may be located external to the user device, and image data of the passport or other official ID document may be transmitted to the user device via communication means as described elsewhere herein. In some embodiments, the software and/or applications may be configured to activate the camera on the user device to scan the code. In other embodiments, the camera can be controlled by a processor natively embedded in the user device. [0069] Upon receiving the image data of the photo ID, the image data may be checked to see if it is in a required format 205 or is valid for further processing. For example, the image data captured via the mobile application may be automatically encrypted such that an encryption key and encryption algorithm may be required in order to decrypt the image data at the backend. In some cases, only the backend system or the hotel PMS may possess the encryption key and method. The key and method may be standard. For example, data can be encrypted using a 1024 bit polymorphic cipher, or, depending on the export controls, an AES 256 bit encryption method. Furthermore, encryption can be performed using remote key (seeds) or local keys (seeds). Alternative encryption methods can be used as would be understood by those skilled in the art, for example, SHA256, AES, Blowfish, RSA and the like. This beneficially ensures a secure transmission of the image data.

[0070] If the image data is in the correct format (e.g., encoded or encrypted correctly), the system (e.g., system 121 in FIG. 1) may use an optical character recognition (OCR) system to extract personal data or identification data of the user from the image data 207. In some embodiments, personal data (identity data) or identification data of the user may be extracted from the image of the physical element (e.g., photo ID). Identification data may contain information used to authenticate or verify identity of a user. The identification data may contain personal information such as name, date of birth, address, nationality and the like that describe identity of a user. In some embodiments, the identification data or personal data may be extracted from image of the physical element using an optical character recognition (OCR) system which will be described with respect to FIGs. 6-8.

[0071] Referring back to FIG. 2, the method may comprise processing the image data to determine whether the image size meets a requirement 209 (e.g., predetermined range of image size). If yes, the method may detect visual features associated with a type of ID document (e.g., passport) such as machine readable zone (MRZ) in the image data 301. For example, if the MRZ is not detected in the image data, the method may determine that the physical document is not a passport and may continue to identify other likely type of the ID document 303.

[0072] The OCR system may comprise an ID reader 305 to identify a likely ID type of the ID document. For instance, the ID reader may be a machine learning algorithm trained model that predict an ID type with a confidence score or recognition rate. A classifier may be trained to predict a most likely ID type along with a recognition rate. If the recognition rate for the most likely ID type is lower than a threshold (e.g., designated pass rate), the ID type may be determined to be not identified and the process may exit with an output as “invalid document type, recognition failed.” If the ID type prediction has a sufficient recognition rate (e.g., recognition rate is at or above a predetermined threshold), the process may proceed to determine whether face check is required for the current ID document 307. Face check may be performed to determine whether a person’s face is contained in the image data and/or whether the location of the face within the image or with respect to other features in the image is in compliance with a predetermined format. In some cases, face check may be passed/successful if a face is detected in the image data and the location of the face is determined to be at an expected/correct location within the photo ID. For instance, face check may be performed to segment the user photo from the image data and if a face is detected in the image data, the OCR system may extract facial features (e.g., facial landmarks) from the image data 401. In some cases, the segmented image of the photo and/or the facial features may be stored into the system for Facial Recognition process at later stages during Check-in and Digital Key Request.

[0073] Referring back to FIG. 3, if the MRZ is detected to exist in the image, the process may proceed to determine that the document is a passport 311 and continue with face check 313. If the face check is passed (e.g., if a face is detected in the image data and the location of the face is determined to be at an expected/correct location within the photo ID, or facial features of a user is correctly extracted or segmented), the process may proceed to parse MRZ details 315 from the image of the photo ID. The method may comprise identifying the passport type 403 (e.g., passport from country A), date of issuance/expiration, and various other data fields based on the text extracted from the image, parsing information encoded in optical character recognition format in the MRZ (e.g., document type, name, document number, nationality, date of birth, sex, and document expiration date), the color code pattern recognition result, the face segment and various other features a shown in FIG. 5.

[0074] In some cases, the OCR system may further authenticate or verify the physical ID such as by identifying that the ID is fake or altered based on the visual information. For instance, the OCR system may be trained to verify whether certain visual features (e.g., color code pattern, format of the machine readable zone, etc.) presented in the image of the ID document is in compliance with a type/format of ID document. In some cases, the OCR system may be in communication with external data sources such as local government database e.g., EVA (E- Visitor Authentication) for further verifying the authenticity of the identification document.

[0075] The OCR system may comprise a machine learning algorithm trained model to accurately localize text lines in natural image. For instance, an output of the model may comprise text, lines recognized from the image of the passport. In some cases, the model may be a deep learning network trained to detect a text line in a sequence of fine-scale text proposals directly in convolutional feature maps. An end-to-end trained model of the OCR system may be capable of detecting texts of any language or scale without further post-processing. The OCR system may provide accurate localization and recognition of texts from an image without knowing a format of the text fields and location in the image.

[0076] In some embodiments, the model may include an artificial neural network that can employ any type of neural network model, such as a feedforward neural network, radial basis function network, recurrent neural network, convolutional neural network, deep residual learning network and the like. In some embodiments, the machine learning algorithm may comprise a deep learning algorithm such as convolutional neural network (CNN). Examples of machine learning algorithms may include a support vector machine (SVM), a naive Bayes classification, a random forest, a deep learning model such as neural network, or other supervised learning algorithm or unsupervised learning algorithm. The model network may be a deep learning network such as CNN that may comprise multiple layers. For example, the CNN model may comprise at least an input layer, a number of hidden layers and an output layer. A CNN model may comprise any total number of layers, and any number of hidden layers. The simplest architecture of a neural network starts with an input layer followed by a sequence of intermediate or hidden layers, and ends with output layer. The hidden or intermediate layers may act as learnable feature extractors, while the output layer in this example provides the location of the texts and/or the recognized texts. Each layer of the neural network may comprise a number of neurons (or nodes). A neuron receives input that comes either directly from the input data (e.g., low quality image data, fast-scanned PET data, etc.} or the output of other neurons, and performs a specific operation, e.g., summation. In some cases, a connection from an input to a neuron is associated with a weight (or weighting factor). In some cases, the neuron may sum up the products of all pairs of inputs and their associated weights. In some cases, the weighted sum is offset with a bias. In some cases, the output of a neuron may be gated using a threshold or activation function. The activation function may be linear or non-linear. The activation function may be, for example, a rectified linear unit (ReLU) activation function or other functions such as saturating hyperbolic tangent, identity, binary step, logistic, arcTan, softsign, parameteric rectified linear unit, exponential linear unit, softPlus, bent identity, softExponential, Sinusoid, Sine, Gaussian, sigmoid functions, or any combination thereof.

[0077] FIG. 6 shows an example of a network architecture that detects and recognizes text from an image. The architecture may be fully convolutional network that allows an input image of arbitrary size. It may detect a text line by densely sliding a small window in the convolutional feature maps, and outputs a sequence of fine-scale text proposals. For example, as show in FIG. 6, a 3x3 spatial window slides through the last convolutional maps of the VGG16 model. The RNN layer is connected to a fully-connected layer, followed by the output layer. The architecture may further improve the localization accuracy by jointly predicting location and text/non-text score of each proposal. For example, as shown in FIG. 6, fully-connected layer and output layer jointly predict text/non-text scores, y-axis coordinates and side-refinement offsets of k anchors. It should be noted that the example is for illustration purpose only and people of skill in the art would recognize that the sliding- window method or the vertical anchor method can be applied to other deep models. The extracted texts may then be used to form at least part of the identification information of the user. Such identification information may be used to make reservation at a hotel, verify the user for check-in at the hotel or for receiving a digital room key.

[0078] The OCR system may also comprise a network for extracting facial landmark features from the photo contained in the image data. FIG. 7 schematically illustrates an exemplary architecture of a deep learning framework for face detection. As shown in the example, the framework may comprise cascaded CNNs for multi-task learning. The framework may comprise a first CNN (P-net) for obtaining candidate windows and the bounding box regression vectors, a second CNN (R-Net) which rejects a large number of false candidates, and performs calibration with bounding box regression and non-maximum suppression merge, and a third CNN to refine the results and output facial landmark positions. In some cases, the same method may also be applied to processing the image of the passport or other ID to extract the facial data (e.g., facial landmarks). It should be noted that the illustrated framework is for illustration purpose only and person of skill in the art would recognize other deep learning framework may be used for facial recognition. For example, a liveness check model may be employed as part of the FR system or as a separate model used concurrently with facial recognition. FIG. 9 shows an example of a network framework employed by the facial recognition system 900 for processing live image or selfie image at a check-in stage. The network framework in FIG. 9 may be used for liveness check (e.g., determine whether the image data is fraud usage of photo images from others to pass through the process). As illustrated in the example, the system 900 may comprise a facial transformation generator, a Mini filter-and- sum network (FaSNet) trained using Soft Loss and FT loss to extract facial features from the live image or selfie image. The liveness check model may or may not be part of the FR system. In some cases, the liveness check model may be a component separate from the FR system. In some cases, the liveness check may be performed prior to, concurrently with or after the facial recognition process.

[0079] FIG. 8 shows an example of an OCR system 800 which is configured to extract the personal data from an ID image 801. The ID image may contain text characters, a facial photo of a user and various other visual features. The OCR system 800 may implement the network architecture described in FIG. 6 and FIG. 7 for extracting the identification data from the texts contained in the image and the photo in the image, and/or other visual features from the image (e.g., MRZ). The output of the facial recognition (face classification) may be processed to determine whether the segmented face meets a predetermined size requirement. Text characters are also recognized to further extract the user identity data (e.g., name, date of birth, country, passport, number, issue date/location, etc.).

[0080] In some embodiments, the personal data and/or identification data such as the facial data (e.g., facial landmarks identified from the photo of the ID) and data related to identity of the user such as data fields extracted from the ID image (e.g., name, gender, nationality, etc.) may be stored by the system or stored using a session framework. In some cases, the session framework may store and retrieve data on a per-site-visitor basis. For example, the personal data may be stored locally within a web browser. In some cases, the system may set up rules determining expiration of the personal data. For instance, the stored data may expire when it exceeds a predetermined expiry time (e.g., one week, two weeks, one month, etc.). In some cases, the stored data may expire when new personal data is obtained, or when the user switches to private browsing (e.g., browser creates a temporary session that is isolated from the browser's main session and user data so the browser history is not saved). [0081] In some embodiments, the personal data may not be transmitted to the backend server or shared with a third-party system (e.g., hotel, premise) until receiving a request from a user. For instance, upon receiving a hotel reservation request from the user device, the system may submit the personal data to the relevant third-party system for check-in or issuing a digital room key. In some cases, the personal data may be used to automatically fill-in a reservation request or a request to enter a premise. In some cases, the personal data stored or transmitted to the backend may be based on the authentication requirement or policy of the premise (e.g., hotel). For instance, the personal data requested for authenticating the user in order to obtain a digital room key from a hotel may include user name, passport number, date of birth and the like. In another example, the personal data for entering another premise may require user address, contact information, ID number and the like.

[0082] During a check-in process, or upon user's request for a Digital Room Key to a premise or hotel's room, the user may be prompted to take a photo (selfie) of themselves. In some cases, the selfie may be a live photo. For instance, the user may be required to go through a Liveness Check process as a preventive measure against fraud usage of photo images from others to pass through the process. In some cases, the Liveness Check model can be the same the one illustrated in FIG. 9. The liveness check model may or may not be part of the FR system. In some cases, the liveness check model may be a component separate from the FR system. In some cases, the liveness check may be performed prior to, concurrently with or after the facial recognition process. A selfie image or video-clip may be taken for verifying a user at check-in. The system or method herein may comprise a Facial Recognition (FR) system for verifying the user by comparing the selfie image to the facial data extracted from the image of the photo ID, such as the passport or other form of photo ID as described above.

[0083] In some cases, the FR system may comprise a deep learning model for liveness check and face detection. FIG. 9 schematically illustrates an exemplary architecture of a deep learning framework as a component of the face recognition (FR) system 900. The facial recognition system may comprise one or more classifiers, models or components same as those in the OCR system for extracting facial features. For example, the FR system may comprise cascaded CNNs for multi-task learning. The selfie image may be processed to extract facial data (e.g., facial landmarks) which are compared against the stored facial landmarks obtained from the photo ID. It should be noted that the framework is for illustration purpose only and person of skill in the art would recognize other deep learning framework may be used for liveness check and facial recognition.

[0084] Upon obtaining the facial data (e.g., facial landmarks), the system may compare the facial data with the facial data extracted from the passport image to verify the user. The system may determine if the facial data extracted from the selfie image matches the facial data extracted from the user ID image (e.g., passport image). Upon the determination of a match, a user may be authenticated or verified. FIG. 15 shows an example of a graphical user interface (GUI) for a user to perform authentication by taking a selfie image. As shown in the example, a user may be prompted to take a selfie image prior to obtaining a digital room key or prior to entering a place/property. In some cases, the GUI 1500 may be triggered upon receiving a user request for a digital door key to a premise’s room. In some cases, the GUI may notify the user if the selfie image is invalid (such as insufficient image quality for performing facial recognition or failure of authentication). For instance, a user may be prompted to retake the selfie image if the image is not valid (e.g., for poor quality, failure of authentication, etc.) or be prompted to continue to the next step if the image is valid/authenticated 1510.

[0085] FIGs. 10-13 illustrate an example of the process for contactless check-in and checkout at a hotel. FIG. 10 illustrates an example of guest registration process. The process can be the same as the process as described above. The registration process may be conducted prior to check-in process. For example, a user may book a hotel room via a hotel reservation website or hotel application using a user device 1010. A reservation record may be created and maintained by the hotel PMS 1020. In some cases, prior to the check-in date (e.g., one day, two days, three days, etc. prior to the check-in date), a reminder may be sent to the user via a suitable communication channel (e.g., email, text message, etc.). The user may be required to submit information to complete the registration process via the user device 1010. For instance, as described above, the user may trigger an ID check request by submitting booking number, check-in date and image of a photo ID via the user device 1010. The user provided data may be transmitted to the system 1030 and may be processed by the OCR system to extract the identification data as described above. Next, if the identification data is successfully extracted, the user personal information may be updated in the registration record identified by the booking number. The PMS may also update the reservation information (e.g., update or supplement the record) with the user personal information.

[0086] FIG. 11 illustrates an example process for check-in. When the user arrives at the hotel, the user may perform contactless check-in using a user device 1110. The user device 1110 needs not be the same as the user device used for the registration. In some cases, the user may be prompted to provide payment method such as credit card information via a mobile application or a website. The payment information may be communicated to the PMS or payment gateway to pre-authenticate the payment method. Next, the user may be prompted to submit a selfie image using the user device 1110. The system 1030 may receive the check-in request from the user device, validate the check-in record and perform facial recognition to verify the user. As described above, a facial recognition engine of the system 1030 may compare the facial features from the selfie image to the facial features extracted from the photo ID image to determine a match. If the facial recognition passes, the status may be communicated to the PMS and the PMS may update the check-in status of the reservation associated with the user. The user may be notified as check-in success on the GUI rendered on the user device. In some cases, if the facial recognition fails, the user personal data, original selfie image, reservation information and/or other information about the user or the reservation may be directed to a hotel webpage (e.g., Access Management Portal) for manual approval of the check-in. For instance, a hotel staff or admins may be able to view, manage the user details via the Access Management Portal (AMP) to manually approve the check-in. In some cases, the AMP may display the selfie image, the guest details, the image of the photo ID submitted during registration, and may prompt the admin to click on a ‘approve’ or ‘reject’ button upon verifying the identity of the guest. In some cases, the manual check-in approval may only be performed (e.g., via the AMP by the Admins) when auto-approval process failed. The user may also have the option to conduct physical check-in at a reception of the hotel.

[0087] FIGs. 17-19 show an example of a contactless check-in process. As shown in the example, a selfie instruction may be generated and delivered to a user on the GUI, the user may take a selfie and submit the selfie image via the application. The selfie image may be pre- processed to determine whether it is valid. If invalid, the user may be prompted to retake the image such as by clicking on a retake button. If valid, the user may be prompted to click on the “next” button to continue. The user may then be prompted to submit a check-in request such as by inputting reservation number or other requested data (e.g., check-in date) via the GUI. The backend system may receive the check-in request including the associated parameters (e.g., pms reservation no) and retrieve the reservation record from the PMS. For example, the backend system may request the reservation information via API call and the returned reservation information may be dependent on the specific PMS record format. For example, the returned reservation information may include OPERA parameters such as reservation status, guest list, room number, room type, arrival date, departure date, or ETD. If no reservation is found based on the check-in request, a notification may be generated (e.g., “pre-arrival record is not found”) and the process may return to the beginning i.e., prompting the user to make a reservation first.

[0088] If the reservation record is found and the reservation information is returned, the GUI may display the reservation information for the user to verify. The backend system may continue to check whether the check-in request is submitted within a valid time window. For example, assuming Check-in time is 14:00 hrs and the check-in day is June, check-in can only occur from 20 June, 14:00 hrs onwards. The backend system may then check whether an active check-in request is already existed and active for the same reservation record and whether the status of the reservation is check-in, if no, the process may proceed with updating the check-in session to “processing.”

[0089] The backend system may verify whether a Liveness Check is passed. Regardless passing or failing the Liveness Check, the process may proceed with facial recognition to compare facial features from the selfie image to the facial data extracted from the photo ID for verification. In some cases, if the FR is failed (e.g., FR mismatch), an error message “facial authentication failed, please retry” may be generated and the system may update the record of the failure count. The process may return to selfie-instruction and the user may repeat the process. In some cases, the system may have a configurable number of failure threshold. For example, if the count of FR failure is above a threshold (e.g., 1 time, 2 times, 3 times, etc.), the process may proceed to manual approval. If the FR is passed, but the Liveness check is failed, the process may proceed to manual approval. If both the Liveness check and the FR are passed, the status may be updated to “auto approved” if no error occurred or “staff _pending” if error occurred.

[0090] Continue the process as shown in FIG. 19, if staff _pending status, the system may call the PMS API to get remaining reservation information. Upon retrieving the reservation information, the process may check whether there is more than one reservation available during check-in, if yes, the GUI may display a “Next Check-in button” for the user to confirm and may automatically send an email to the guest for confirmation. After the check-in approved via AMP, a second email may be sent to the guest with room number and other details about the reservation. If the status is “auto approved,” the subsequent process may be similar to the above except that the Check-in Approval via AMP is not required. As described above, the manual check-in process may allow a hotel staff or admins to view or manage the user details via the Access Management Portal (AMP) such as to manually approve or deny the check-in. In some cases, the AMP may display the selfie image, the guest details, the image of the photo ID submitted during registration, and may prompt the admin to click on a ‘approve’ or ‘reject’ button upon verifying the identity of the guest. In some cases, the manual check-in approval may only be performed (e.g., via the AMP by the Admins) when auto-approval process failed.

[0091] FIG. 12 shows an example process for requesting a digital key. In some cases, the user may request a digital key via the mobile application running on the user device 1210. The user may be prompted to take a selfie image. The selfie image may be transmitted to the system 1030 which comprising a facial recognition engine configured to validate the selfie image by comparing the facial features extracted from the selfie to the facial features extracted from the photo ID image. The system 1030 may approve the key request upon a determination of a match. The approval status may be transmitted to the PMS or a door lock provider at the hotel system which may then issue a digital key. The system 1030 may receive the key information from the door lock provider then provide the digital key to the user via the mobile application. In some cases, the key information may comprise the room and the expiration date associated with the key. The user may then be able to use the digital key for the issued period of time. In some cases, if the facial recognition fails, the user may be directed to the hotel mobile page to continue with the key request approval. For example, the user may be required to provide other credentials to verify the identity of the user.

[0092] The key may be provided to the user in various forms. For instance, the key may be a digital room key that may be used to unlock a door such as using NFC techniques, or a one-time passcode that a user may enter at a smart door lock. Alternatively, a user may have the option to obtain a physical key upon verification. FIG. 16 shows an example GUI for locking/unlocking a door using a digital key. In some cases, a user may possess one or more digital keys for entering one or more rooms or premises simultaneously. As described above, the digital door key (e.g., one time passcode) may be provided to the user via the application running on the system 1030 or via a graphical user interface (GUI), webhooks that can be integrated into the application provided by the system 1030. For example, hotel user interfaces or APIs may be integrated to the mobile application and integrated in the front-end user interface (e.g., within the GUI) so the user can lock/unlock a door using the same mobile application. Alternatively, the user may be prompted to download an application provided by the hotel system 1040 to obtain and/or use the digital door key.

[0093] FIGs. 20-24 shows exemplary flow for obtaining a digital key. The process may begin with detecting whether a guest is on premise. For example, a geo-location component of the backend system may retrieve geolocation data from the guest’s mobile device and determine if the guest is within a predetermined proximity of a premise (e.g., geo-fence). The proximity or geofence may be preset via a dashboard. As shown in the process, a guest may trigger a request which is then used to select the guest information a Guest List that is maintained by PMS. If the location sensor of the guest device is on (e.g., GPS turned on) then the location data may be processed by the system to determine if the guest is within a geo-fencing predetermined via a dashboard. Upon determining a geo-location match (e.g., guest’s current geo-location matches the premise geo-fencing and the gesut is on the Guest List), the process may proceed to taking a selfie for FR.

[0094] Continue to FIG. 21, the taking selfie and Liveness check process is similar to the process described in check-in process. As shown in the example, if the FR fails, a message may be generated indicting the selfie does not match the selected guest’s passport. In some cases, when a PMS exists or is provided by the Hotel, upon passing both the liveness check and the FR match, a key may be requested from the key server such as by calling postKey Request API in the backend and a digital key may be issued. Alternatively, the process may go through the approval flow. For instance, the status maybe set to “RequestKey Pending.” Next, the approval may go through an automated process where an email may be automatically sent to the guest for approval. Upon the guest approval, a key request may be sent to the Key Sever for key issuance. Alternatively, the approval may go through a manual process. FIG. 22 shows more operations in the approval process. For instance, an AMP manual approval may be performed. A staff may manually approve the key request or reject the key request and the guest may receive an email indicating the key request has been approved or rejected.

[0095] FIG. 23 continues with key issuance. If the key issue is successful, the mobile key information may be saved. The information may include, for example, room number, key code, endpointID, expiry date, Expiry time and the like. The status of the key request record may be updated to Staff Approved and a success response may be generated and delivered to the guest via a GUI either on a web page or via a mobile application. If the key issue is not successful, the status may be updated to Staff_pending, and AMP manual approval may be performed. FIG. 24 shows an exemplary process when the gest is in the PMS Guest List but not in the list of the system database. In this case, the guest details may be retrieved from the PMS and displayed on the GUI. The guest may be prompted to upload image of a photo ID such as passport. The OCR system may process the photo ID as described elsewhere herein to extract the identification data and validate the photo ID. the guest profile with the system may be updated with the identification data and saved as a record in the system database. The PMS may also update the record for the guest based on the extracted personal data.

[0096] FIG. 13 shows an example process for check-out. In some cases, a user may receive a check-out reminder or key expiry reminder such as via a mobile application, email or other communication channels. The system 1030 may store the reservation information, update the check in status and may generate the reminder at a time point that is close to the expiration date (e.g., 12 hours, 24 hours, two days, three days, etc. prior to the check-out date). The user may navigate to a web portal hosted by the hotel system 1040 or the application hosted by the system 1030 to pay the bill. In some cases, the system 1030 may communicate with the PMS to retrieve the bill information and may facilitate the bill transaction with the payment gateway. Upon completion of the transaction, the PMS may update the status of the user to check-out. [0097] In some cases, the payments or transaction history of a customer may be tracked by a transaction/payment component of the system 1030 when the guest performs transactions via a payment network of the system 1030 or the PMS 1020. For instance, once a guest decides to proceed with the payment, the payment component may capture the guest’ s credit card account information and submit it with the transaction details as an authorization request to a payment processor of the third-party service provider (e.g., hotel, PMS 1020). The payment processor may submit the authorization request to a payment network, which may then pass the authorization request to the bank or credit card company. Upon verifying the account, an authorization response is returned to the payment network. Next, the payment network may pass the authorization response to the hotel payment processor and/or to the PMS.

[0098] As used herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise by context. Therefore, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context.

[0099] Whilst it is described above that the invention is used for providing methods and systems for hotel check-ins, it will be appreciated that the invention has applications in other areas of property management and the management of check-ins at venues, including, but not limited to schools, offices, or other facilities requiring such services.

[0100] While preferred embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the invention be limited by the specific examples provided within the specification. While the invention has been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. Furthermore, it shall be understood that all aspects of the invention are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention. It is therefore contemplated that the invention shall also cover any such alternatives, modifications, variations or equivalents. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.