CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This patent document claims the benefits and priority of U.S. Provisional Patent Application No. 62/245,942, filed on October 23, 2015. The entire content of the before- mentioned patent application is incorporated by reference as part of the disclosure of this document.

TECHNICAL FIELD

[0002] This patent document generally relates to fingerprint recognition and its applications for securely accessing an electronic device or an information system. BACKGROUND

[0003] Fingerprints can be used to authenticate users for accessing electronic devices, computer-controlled systems, electronic databases or information systems, either used as a standalone authentication method or in combination with one or more other authentication methods such as a password authentication method. For example, electronic devices including portable or mobile computing devices, such as laptops, tablets, smartphones, and gaming systems can employ user authentication mechanisms to protect personal data and prevent unauthorized access. In another example, a computer or a computer-controlled device or system for an organization or enterprise should be secured to allow only authorized personnel to access in order to protect the information or the use of the device or system for the organization or enterprise. The information stored in portable devices and computer-controlled databases, devices or systems, may be personal in nature, such as personal contacts or phonebook, personal photos, personal health information or other personal information, or confidential information for proprietary use by an organization or enterprise, such as business financial information, employee data, trade secrets and other proprietary information. If the security of the access to the electronic device or system is compromised, these data may be accessed by others, causing loss of privacy of individuals or loss of valuable confidential information. Beyond security of information, securing access to computers and computer-controlled devices or systems also allow safeguard the use of devices or systems that are controlled by computers or computer processors such as computer-controlled automobiles and other systems such as ATMs,.

[0004] Security access to a device such as a mobile device or a system such as an electronic database and a computer-controlled system can be achieved in different ways such as use of user passwords. A password, however, may be easily to be spread or obtained and this nature of passwords can reduce the level of the security. Moreover, a user needs to remember a password to use electronic devices or systems, and, if the user forgets the password, the user needs to undertake certain password recovery procedures to get authenticated or otherwise regain the access to the device and such processes may be burdensome to users and have various practical limitations and inconveniences. The personal fingerprint identification can be utilized to achieve the user authentication for enhancing the data security while mitigating certain undesired effects associated with passwords.

[0005] Electronic devices or systems, including portable or mobile computing devices, may employ user authentication mechanisms to protect personal or other confidential data and prevent unauthorized access. User authentication on an electronic device or system may be carried out through one or multiple forms of biometric identifiers, which can be used alone or in addition to conventional password authentication methods. One form of biometric identifiers is a person's fingerprint pattern. A fingerprint sensor can be built into an electronic device or an information system to read a user's fingerprint pattern so that the device can only be unlocked by an authorized user of the device through authentication of the authorized user's fingerprint pattern.

SUMMARY

[0006] The examples of implementations described in this patent document provide fingerprint sensor designs that use optical sensors for sensing fingerprints. The described fingerprint sensor designs can be used in various devices, systems or applications, including mobile applications, and various wearable or portable devices (e.g., smartphones, tablet computers, wrist-worn devices), larger electronic devices or systems.

[0007] In one aspect, an electronic device having an optical fingerprint sensing function is provided to include a touch receiving surface including a touch area for receives a contact input; an optical sensor module that detects a presence of a received contact input associated with a fingerprint on the touch receiving surface to generate a first signal indicative of an image of a spatial pattern of the fingerprint and a second signal indicative of a biometric marker that is different from the spatial pattern of the fingerprint and represents a property of a live person. The optical sensor module includes: a light source to produce probe light projected onto the touch receiving surface, and an optical sensor array positioned to receive probe light from the touch receiving surface that carries information of the received contact input and to produce an optical sensor signal. The device may further include processing circuitry that is

communicatively coupled to receive the optical sensor signal to process the first signal to determine whether the detected image matches a fingerprint pattern of an authorized user and to process the second signal to determine whether the biometric marker indicates that the contact input associated with the fingerprint is from a finger of a live person.

[0008] In another aspect, the optical fingerprint sensor of the disclosed technology can be implemented to provide one or more of the following features. The optical fingerprint sensor includes a light source, coupler, spacer, photo diode array, and cover glass. The spacer may be made of glass material, adhesive material, or even air gap or vacuum. The coupler may be made of glass material, adhesive material, or even air or vacuum. The cover glass may be partial of the display cover glass, or separate cover glass. Each of the mentioned coupler, spacer, and cover glass may be of multiple layers.

[0009] The disclosed technology provides flexibilities to control the signal contrast by matching the materials shapes and refractive indexes. By matching the probe light beam incident angle, divergent angle, and the materials of the involved coupler, spacer and cover glass, the probe light beam may be controlled to be totally reflected or partially reflected at the sensing surface for different touching materials.

[0010] The disclosed optical fingerprint sensor also provides a water-free effect. A typical smartphone cover glass has a refractive index of about 1.50. One design is to use low refractive index material (MgF ₂ , CaF ₂ , Polymer etc.) to form the coupler. The disclosed technology can be used to control the local probe light beam incident angle at the sensing surface to be about 68.5°. The total reflection angle is about 62.46° when water touches the sensing surface of the optical fingerprint sensor, and the total reflection angle is about 73.74° when the ridges of a fingerprint touch the sensing surface. The total reflection angle is about 41.81° when nothing touches the sensing surface. In this design, at the water soaking area, the probe light is totally reflected to the photo diode array; at the fingerprint ridges touching positions, less than 5% of the probe light is reflected to the photo diode array; and at the dry fingerprint valleys positions, the probe light beam is also totally reflected to the photo diode array. This means that only the fingerprint ridges generate signals that are detected.

[0011] Sweat has a refractive index that is lower than the finger's skin. The disclosed technology provides a solution to distinguish the sweat pores in the fingerprint.

[0012] When air gap is used to form the coupler, total reflection at the sensing surface does not occur. The reflectance difference among different touching materials (the fingerprint ridges, fingerprint valleys, and other contaminations) can be used to detect the fingerprint image.

[0013] Due to the light path compression effect, the sensing area size may be greater than the photo diode array size.

[0014] The light source may be a point light source installed at proper distance.

[0015] The probe light beam may be collimated by spherical lenses, cylinder lenses, or aspheric lenses.

[0016] The probe light beam may be of proper divergent angle. The probe light beam may also be divergent or convergent.

[0017] Due to the light path compression effect, the coupler may be very thin. For example, less than 1mm thickness CaF ₂ coupler can be used to realize even 10mm sensing area size. In this example, the image compression ratio is 1 : 10. This helps to reduce the sensor cost.

[0018] The photo diode array is installed on one end of the coupler instead of under the coupler. This design leaves the flexibility to apply color paint, illumination light etc. to compensate the color or decorate the sensor area.

[0019] The probe light source may be modulated to help reduce the influence of the background light. The photo diode array is designed to work well in any illumination

environments.

[0020] The cover glass thickness does not limit the fingerprint sensing.

[0021] The principle can be used to build optical touch panel.

[0022] In another aspect, the optical fingerprint sensor of the disclosed technology can be implemented to perform live-finger detection including the following:

[0023] The optical fingerprint sensor can detect whether the touching material is a live-finger and can improve the security of the sensor. [0024] Specified light sources and detectors can be used to detect whether the object touching the sensing area is a live-finger or a nonliving material.

[0025] When single wavelength is used, the heartbeat detection provides a reliable criterion to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger.

[0026] When two or more wavelengths are used, the extinction ratio of the wavelengths are compared to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger.

[0027] The fingerprint sensor light sources and photo diode array can be used to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger.

[0028] The dynamic fingerprint images can be used to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger. The dynamic fingerprint images can also be used to detect the press force when a live finger is touching the sensing area.

[0029] Multiple security level can be set up for different security requirement tasks.

[0030] In yet another aspect, the optical fingerprint sensor can be implemented to enable various decorative elements including the following:

[0031] The bottom surface of the coupler can be painted with same color or pattern layers to match with the platform surface color.

[0032] The bottom surface of the coupler can be painted with different color or pattern layers to show new styles out-looking.

[0033] Color light sources can be installed around the coupler to decorate the sensor area.

[0034] In yet another aspect, the optical fingerprint sensor packaged as a separate button can perform the same fingerprint detection and live-finger detection as described above. In addition, the optical fingerprint sensor package as a separate button can be implemented to perform the following features:

[0035] The cover glass and related spacer material feature flexibility in the thickness according to the applications.

[0036] Especially, it is a practical package not to use cover glass and spacer material. [0037] Another practical design is to use a thin layer of cover glass to protect the coupler. The cover glass may be of high hardness.

[0038] To use colored glass or other optical materials to build the cover is also practical.

[0039] The package method provides a solution to build a compact button that can detect the fingerprint with improved security.

[0040] Other mechanical parts may be integrated to make the module strong.

[0041] In yet another aspect, an electronic device having an optical fingerprint sensing module is provided to include a touch sensing display panel including a touch sensing and displaying area for displaying images and contents and for receiving user contact inputs; a top transparent cover formed on top of the touch sensing display panel and operable as a top touch sensing surface for a user to provide user contact input to the touch sensing display panel and to transmit light for a user to view images displayed by the touch sensing display panel; and an optical sensor module placed underneath the top transparent cover and displaced from the touch sensing display panel. The optical sensor module is configured to detect a presence of a received contact input associated with a fingerprint on the top touch sensing surface to generate a first signal indicative of an image of a spatial pattern of the fingerprint and a second signal indicative of a biometric marker that is different from the spatial pattern of the fingerprint and represents a property of a live person. The optical sensor module includes a probe light source to produce probe light projected onto the top touch sensing surface, and an optical sensor array positioned to receive reflected probe light from the top touch sensing surface that carries information of the received contact input and to produce an optical sensor signal. The probe light source produces probe light of two different wavelengths with different optical absorptions by blood. The first signal indicative of the image of the spatial pattern of the fingerprint is captured by the optical sensor array by sensing the reflected probe light at each of the two different wavelengths, and the second signal indicative of the biometric marker is carried by differences in the reflected probe light at the two different wavelength.

[0042] The above and other aspects and features are described in greater detail in the attached drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] FIG. 1 A is a block diagram of an example of an optical sensing based fingerprint user authentication system that controls the access to a computer processor controlled device or system.

[0044] FIG. IB is a block diagram showing an exemplary fingerprint sensor device implementing in a mobile device such as a smartphone based on the design in FIG. 1 A.

[0045] FIG. 2 is a diagram showing an exemplary optical fingerprint sensor packaged under a screen cover glass of a platform, such as a smart phone.

[0046] FIG. 3 is a diagram showing an exemplary fingerprint sensing light path.

[0047] FIG. 4 is a diagram of an exemplary optical fingerprint sensor with an air or vacuum coupler.

[0048] FIG. 5 is a block diagram showing an exemplary optical fingerprint sensor for fingerprint sensing.

[0049] FIG. 6 is a diagram illustrating exemplary live-fingerprint detection.

[0050] FIG. 7 shows exemplary extension coefficients of materials being monitored.

[0051] FIG. 8 shows blood flow in different parts of a tissue.

[0052] FIG. 9 shows a comparison between a nonliving material (e.g., a fake finger) and a live-finger.

[0053] FIG. 10 shows a process flow diagram of an exemplary process 1000 for setting up different security levels for authenticating a live finger.

[0054] FIG. 11 is a diagram showing an exemplary optical fingerprint sensor for sensor area decorating.

[0055] FIG. 12 is a diagram showing an exemplary optical fingerprint sensor packaged as a separate button.

[0056] FIG. 13 is a diagram showing exemplary fingerprint and live-finger detection using the optical fingerprint sensor packaged as a separate button.

DETAILED DESCRIPTION

[0057] The fingerprint sensing described in this patent document includes optical sensing of a fingerprint pattern. [0058] FIG. 1 A is a block diagram of an example of an optical sensing based fingerprint user authentication system that controls the access to a computer processor controlled device or system. The system uses an optical fingerprint sensor with an array of optical detectors to capture an optical image of received light that carries the fingerprint pattern from a finger that is touched on the optical fingerprint sensor sensing surface that is illuminated by an illumination light beam. The system includes a fingerprint sensor control circuit that receives the outputs from the optical detectors in the optical fingerprint sensor, and a digital fingerprint processing processor which may include one or more processors for processing fingerprint patterns and determining whether an input fingerprint pattern is one for an authorized user. The fingerprint sensing system may compare a captured fingerprint to a stored fingerprint to enable or disable functionality in a device or system that is secured by the fingerprint user authentication system. For example, the fingerprint user authentication system at an ATM may determine the fingerprint of a customer requesting to access funds. Based on a comparison of the customer's fingerprint to one or more stored fingerprints, the fingerprint user authentication system may cause the ATM system to allow access to funds and may identify the customer in order to associate an appropriate account to credit or deduct the requested funds. A wide range of devices or systems may be used in connection with the disclosed optical fingerprint sensors, including mobile applications, and various wearable or portable devices (e.g., smartphones, tablet computers, wrist-worn devices), larger electronic devices or systems, e.g., personal computers in portable forms or desktop forms, ATMs, various terminals to various electronic systems, databases, or information systems for commercial or governmental uses, motorized transportation systems including automobiles, boats, trains, aircraft and others. FIG. IB illustrates an example for a smartphone or a portable device where the fingerprint user authentication system is a module integrated to the smart phone.

[0059] Fingerprint sensing is useful in mobile applications and other applications that use secure access. For example, fingerprint sensing can be used to provide secure access to a mobile device and secure financial transactions including online purchases. It is desirable to include robust and reliable fingerprint sensors features suitable for mobile devices. For example, it is desirable for fingerprint sensors in mobile devices to have a small footprint and thin to fit into the highly limited space in mobile devices; it is also desirable to include a protective cover to protect such a fingerprint sensor from various contaminants. [0060] The optical sensing technology described in this patent document for fingerprint sensing can be implemented to provide high performance fingerprint sensing and can be packaged in compact sizes to fit into mobile and other small device packaging. In capacitive fingerprint sensors, the sensing is based on measuring the capacitance between the sensing electrode and a finger surface due to their capacitive coupling. As the protective cover over the capacitive sensor pixels becomes thicker, the electrical field sensed by each capacitive sensor pixel disperses quickly in space leading to a steep reduction in the spatial resolution of the sensor. In connection with this reduction of the sensing spatial resolution, the sensor signal strength received at each sensor pixel also reduces significantly with the increase in thickness of the protective cover. Thus, when the protective cover thickness exceeds a certain threshold (e.g., 300 μπι), it can become more difficult for such capacitive sensors to provide a desired high spatial resolution in sensing fingerprint patterns and to reliably resolve a sensed fingerprint pattern with an acceptable fidelity.

[0061] In one aspect, the disclosed technology provides optical fingerprint sensor designs in thin optical fingerprint sensor packages for easy integration into a mobile device or other compact devices. In some implementations, the optical fingerprint sensors of the disclosed technology use matched light coupling solutions to provide optical fingerprint sensing at low cost, high performance, and flexible package structures. The disclosed optical fingerprint sensors may also be configured to provide live-finger detection to improve the security. In addition, disclosed optical fingerprint sensor solutions may include various decorative options that provide customized appearance of the platforms that integrates the sensor.

[0062] Examples of implementations of the disclosed technology can be used to introduce an optical technology for sensing finger properties including fingerprint detection. The optical technology can be used for a wide range of devices and systems including those with a display structure. The optical fingerprint sensor technology can be integrated under the same cover of a display such as a touch sensing display device or be packaged in a discrete device.

[0063] The performance of the optical fingerprint sensors based on the disclosed technology is not limited by the package cover thickness that may hinder capacitive fingerprint sensors. In this regard, an optical fingerprint sensor based on the disclosed technology can be implemented into a thin package by using suitable optical imaging capture configurations, including

configurations that are free of imaging lenses or prisms that tend to render the optical imaging modules bulky. Implementations of optical fingerprint sensors based on the disclosed technology can be provide color matching design features to allow the colors of the optical fingerprint sensing areas to be in certain desired colors, e.g., matching colors of the surrounding structures.

[0064] In some implementations, the optical fingerprint sensors of the disclosed technology can be packaged under the platform screen cover glass without modifying the cover thickness and color. The optical fingerprint sensor can include an optical sensor array, e.g., a photo diode array, or a CMOS sensor array, and the optical sensor array can be dimensioned to a compact size due to the contribution of the compressed light path structure. Moreover, the design provides flexibility to decorate the sensor area, for example, with color light illumination.

[0065] In some implementations, in addition to the optical sensing of a fingerprint, optical sensing of a biometric indication is provided to indicate whether an input of the fingerprint pattern is from a live person. This additional optical sensing feature can be used to meet the needs for defeating various ways that may compromise the secured or authorized access to fingerprint-protected devices or systems. For example, a fingerprint sensor may be hacked by malicious individuals who can obtain the authorized user's fingerprint, and copy the stolen fingerprint pattern on a carrier object that resembles a human finger. Such unauthorized fingerprint patterns may be used on the fingerprint sensor to unlock the targeted device or system. Hence, a fingerprint pattern, although a unique biometric identifier, may not be by itself a completely reliable or secure identification. The techniques, devices and systems described in this document supplement the disclosed optical sensing based fingerprint authentication technology further improve the security level by using an optical sensing technique to determine whether the input fingerprint is from a live person.

[0066] Fingerprint Sensor Circuitry and Live Finger Detection

[0067] FIG. IB is a block diagram showing an exemplary fingerprint sensor device 23 implementing in a mobile device such as a smartphone, a tablet or a portable computing device 1 with a touch sensing display screen or touch panel 10 for both touch sensing user inputs and display images and functions of the device 1. This is specific implementation example of the general optical fingerprint sensing controlled system in FIG. 1 A. The touch panel or sensing display screen 10 can be implemented based on various touch sensing display designs, including, a display screen having light emitting display pixels without using backlight where each individual pixel generates light for forming a display image on the screen such as an organic light emitting diode (OLED) display screens or electroluminescent display screens or other display screens such as LCD-based touch sensing display screens. The touch sensing display panel includes a touch sensing and displaying area for both displaying images and contents and for receiving contact inputs from a user.

[0068] A fingerprint sensor device marker 21 is shown in FIG. IB to illustrate an exemplary position of the fingerprint sensor device 23 with respect to the mobile device 1. The fingerprint sensor device 23 includes a sensing unit or circuitry 2 that performs fingerprint scanning, live- fingerprint detection, and sensing area decorative functions. The sensing unit 2 is

communicatively coupled to processing circuitry 5 that handles signal flows from the sensing unit 2 and to process the signals associated with fingerprint scanning and live-fingerprint judgment, etc.

[0069] An interface 6 bridges a signal flow between the fingerprint sensor device 23 and an application platform or a host device 7, which is the smartphone 1 in this example. Examples of the application platform 7 include the smart phone 1, a tablet computer, a laptop computer, a wearable device, and other electronic device where a secure access is desired. For example, the interface 6 can communicate with a central processor (either directly or through other components, such as a bus or an interface) of the smartphone 1 to provide sensor data from the fingerprint sensor device 21 including fingerprint image data and information indicative of whether the detected fingerprint making the contact input belongs to a live fingerprint.

[0070] In the illustrated example in FIG. IB, the sensing unit 2 includes a fingerprint sensor 3, a live-fingerprint detector 4, and a light coupling and illumination unit 8. The fingerprint sensor 3 captures a fingerprint pattern and can be implemented using one or more optical techniques. The live-fingerprint sensor 4 can include circuitry for analyzing fingerprint image dynamics. The live finger sensor 4 can include circuitry, such as optical sensors, for sensing additional biometric markers, such as heartbeat or heart rate from the scanned fingerprint.

[0071] The live finger sensor 4 is designed to detect whether a fingerprint is from a finger of a live person and this live finger detection or judgment is based on the fact that a finger of a live person may exhibit certain motions or physical traits that are typically associated with a live person, e.g., a pulsing signal due to blood flows through the user's vessels. For example, blood cells manifest different optical absorption spectral signatures at visible wavelengths (e.g., a higher optical absorption) and near IR wavelengths (e.g., a lower optical absorption than that iat a visible wavelength). Such different optical absorption signatures by blood can be optically captured by the liver finger sensor 4. Other signatures of blood flows may be reflected by pressure variations in blood vessels. In some implementations, the live finger sensor 4 can include a pressure sensor, an optical sensor, or other sensors that can detect the moving, stretching, or pulsing of a live finger. For example, an optical sensor can include a light source, such as a light emitting diode (LED) or a laser diode (LD) to emit light and a light detector, such as a photodiode to detect scattered light scattered from the finger responsive to the emitted light. When the light propagates through the finger tissues or the blood cells, the light is partially absorbed and partially scattered. The live finger movement or the blood flow causes a change in the light absorption cross-section. The photodiode detects this kind of change and the detected signal can be used to indicate whether a fingerprint that is being presented to the device is from a live person.

[0072] The light coupling and illumination unit 8 creates a probe light beam at the fingerprint sensing surface which generates a reflected probe light beam into an optical sensor array (e.g., a photo diode array or CMOS sensor array) of the sensing unit. The fingerprint signals are generated when the probe light beam meets with the finger skin that touches the sensing surface. The fingerprint sensor 3 acquires the fingerprint signals by detecting the reflection differences of the probing light beam at the sensing surface across a fingerprint pattern where locations of the skin of fingerprint ridges in a finger in contact with the sensing surface creates a lower optical reflection than the optical reflections at locations of fingerprint valleys in the finger where the finger skin does not contact the sensing surface. The spatial distribution the above reflection differences across the touched sensing surface by the finger is carried by the reflected optical probe light beam as an optical image that is detected by the array of optical detectors in the fingerprint sensor 3.

[0073] The disclosed technology provides for two fingerprint sensor packaging techniques to implement fingerprint detection and live-finger detection. The first packaging technique is to package the fingerprint sensor under the screen cover glass of the platform, such as a

smartphone. The second packaging technique is to package the fingerprint sensor as a separate fingerprint sensing button.

[0074] Fingerprint Sensor Packaged Under the Screen Cover Glass [0075] FIG. 2 is a diagram showing an exemplary optical fingerprint sensor packaged under a screen cover glass of a platform, such as a smart phone, a tablet or a portable electronic device. FIG. 3 further illustrates an exemplary fingerprint sensing light paths of the device in FIG. 2.

[0076] In FIG. 2, the exemplary optical fingerprint sensor 23 is packaged under a screen cover glass, such as an enhanced cover glass 50 of a platform 1, such as a smartphone. The location of the optical fingerprint sensor 23 is shown by a fingerprint sensor mark 21 in the top- down view in the upper left side of FIG. 2. The smartphone platform 1 includes a touch panel assembly 10, other sensors 12, such as a camera, and physical buttons 12 and 16 on the side. Under the cover glass 50 can include a color material layer 52, display layers 54 (e.g., OLED layers or LCD layers) as part of the display screen in the touch panel assembly 10, and bottom layers 56 of the display screen in the touch panel assembly 10. Also, the touching sensing layers may also be placed to overlay the display layers 54 to provide the touching sensing functions.

[0077] In the optical fingerprint sensor design in FIG. 2, the packaging design is different from some other fingerprint sensor designs using a separate fingerprint sensor structure from the display screen with a physical demarcation between the display screen and the fingerprint sensor (e.g., a button like structure in an opening of the top glass cover in some mobile phone designs) on the surface of the mobile device. Under the illustrated design in FIG. IB, the fingerprint sensor 23 formed in the area 21 for optical fingerprint is located under the top cover glass or layer 50 so that the top surface of the cover glass or layer 50 serves as the top surface of the device as a contiguous and uniform glass surface across both the display screen of the touch display assembly 10 and the optical detector sensor module 23. In the examples shown in FIGS. 1-6, the optical sensor module is located on one side of the transparent substrate 50 such as a glass cover that is contiguous without any opening at or near the optical sensor module. This design is different various smartphones with a fingerprint sensor and provides unique features and benefits. This design for integrating optical fingerprint sensing and the touch sensitive display screen under a common and uniform surface provides benefits, including improved device integration, enhanced device packaging, enhanced device resistance to failure and wear and tear, and enhanced user experience. In some implementations of the optical sensing of fingerprints and other sensing operations, such as the design in FIG. 12, the optical sensor module may be packaged in a discrete device configuration in which the optical sensor module is a distinct structure that has a structural border or demarcation with the display screen, e.g., a button like fingerprint sensor structure in an opening of the top glass cover in some mobile phone designs based on all optical sensing or a hybrid sensing with both capacitive sensing and optical sensing.

[0078] The optical fingerprint sensor 23 disposed under the cover glass 50 can include an optical coupler 31 disposed over a matched color material layer 25 and a probe light source 29. The matched coupler 31, the matched color material layer 25, and the probe light source 29 are disposed over a circuit 27, such as a flexible printed circuit (FPC) with desired circuit elements. Also disposed on the FPC 27 are light sources for liveness detection 33, photo diodes for liveness detection 34, light sources for decorating illumination 35, and a photo diode array 37.

[0079] The light coupler 31 is fixed onto the cover glass 50 and an underlying spacer material 39 as shown in FIG. 3. The probe light source 29 is fixed at a proper position so that the probe light beam or a portion of the probe light beam may be projected into the coupler 31 at desired angles. The coupler 31, the spacer material 39, and the cover glass 50 can each be made of multiple layers. The photo diode array 37 is fixed at a proper position to receive the reflected probe light beam A'B' for capturing the optical image of the fingerprint pattern carried by the reflected probe light beam A'B' .

[0080] Probe light source 29 projects probe light beam AB into coupler 31 which further directs the probe light beam AB through the opening of the optional color material layer 52 onto the fingerprint sensing surface 45 on the top of the cover glass 50 to illuminate the finger in contact. The light beam AB is coupled into cover glass 50 with the help of the spacer material 39 placed underneath the cover glass 50. When nothing is placed on the top sensing surface 45 of the cover glass 50, a portion or all of the probe light beam power is reflected into the spacer 39, and this reflected light enters into coupler 31 and forms the reflected probe light beam A'B' . The reflected probe light beam A'B' is received by the matched optical sensor array 37 (e.g., a photo diode array) which converts the optical image carried by the reflected probe light beam A'B' into an array of detector signals for further processing.

[0081] When a finger 43 touches the sensing surface 45 of the cover glass 50, the fingerprint ridges 73 change the local surface reflectance as shown by the right figure in FIG. 3. A portion 61 of the probe light incident on the finger ridge is refracted as light 65 that is scattered in the finger 43, the rest is reflected as light 67 that is reflected by the finger ridge. The fingerprint valleys are separate from the sensing surface 45 and generally do not significantly change the local surface reflection at the sensing surface 45. The incident light 63 that is incident on the fingerprint valleys is reflected as light 69 that is reflected by the sensing surface 45. The reflected probe light beam A'B' carries the fingerprint signals. Similarly, when something other than a finger skin touches the sensing surface 45 of the cover glass 50, the reflected probe light beam A'B' carries the touching material information, which is different from a live fingerprint.

[0082] In the example of FIGS. 2 and 3, the materials of the coupler 31, spacer 39, and cover glass 50 are of a proper level of optical transparency so that the probe light beam can be transmitted in and through them. The refractive index of the coupler 31 is nc, the refractive index of the spacer material 39 is ns, the refractive index of the cover glass 50 is nd, and the refractive index of the touching material is nf. The probe light beam's propagating directions are decided by these materials' refractive indexes.

[0083] The desired probe light beam angles may be realized by proper design of the light source 29 and the end surface tilting angle of the coupler 31. The divergent angle of the probe light beam is controlled by the structures of the light source 29 and the shape of the coupler 31 's end surface.

[0084] To obtain a clear fingerprint image without an optical lens, normally the light source 29' s emitting area should be small as a point light source, or the probe light beam should be collimated. A small LED light source can be installed far away from the coupler 31 to achieve this in the optical system shown in FIG. 3.

[0085] By matching proper refractive indexes (nc, ns, nd, nf) of the materials in the optical fingerprint sensor and initiating the probe light beam incident angles, the probe light beam can be designed to be totally reflected or partially reflected at the sensing surface 45. For example, such an optical sensor can be designed so that the probe light beam is totally reflected when the touch material is water having a refractive index of about 1.33 at 589 nm, and partially reflected when the touch material is finger skin having a refractive index of about 1.44 at 589 nm.

[0086] The probe beam AB size is defined as H at the incident end of the coupler 31. The probe beam size may be W at the sensing surface 45. By matching the refractive indexes of all of the materials and the shape of the coupler 31 and spacer 39, W may be set to be greater than H. Namely, the received probe light beam A'B' may be smaller than the probe light beam at the sensing surface 45. The compression ratio is typically decided by refractive indexes nc and nd. This is an effective method to image a large area with a small detector array without using an imaging lens. In addition, by adjusting the probe light beam divergent angle and the photo diode array tilting angle, the compression ratio can be further adjusted at all dimensions. The reflection from the coupler-spacer interface and from the spacer-cover interface constitutes optical noise and can be removed in the processing of the outputs of the optical detectors in the optical sensor array 37.

[0087] In some implementations, the probe light source 29 may be modulated. The matched photo diode array should be designed to be high efficiency and to work in all optical illumination environments.

[0088] Fingerprint Sensing-Air or Vacuum Coupler

[0089] FIG. 4 is a diagram of an exemplary optical fingerprint sensor 23a with an air or vacuum coupler. The optical fingerprint sensor 23a of FIG. 4 is similar to the optical fingerprint sensor 23 shown in FIGS. 2 and 3 in certain aspects. In the optical fingerprint sensor 23a, a coupler 32 made of air or vacuum is implemented rather than coupler 31 of FIGS. 2 and 3. Also, a light path window may be implemented to direct the probe light to the finger 43.

[0090] The probe light source 29 and a matched prism 101 cooperate to couple the probe light beam AB towards the sensing surface 45. The spacer material 39 may include anti- reflection coatings. The prism 103 helps to direct the reflected probe light beam A'B' into the photo diode array 37. The matched color layers 25 are painted on a substrate 105.

[0091] In the optical fingerprint sensor 23a, the optical configuration of the cover glass 50 in receiving the probe light is configured so that the total internal reflection does not happen in the cover glass 50. Due to differences of the optical interfacing conditions of the cover glass 50 with respect to fingerprint ridge positons and fingerprint valley positions, when a finger 43 touches the sensing surface 45, the reflectance at the fingerprint ridge positions differs from the reflectance at the fingerprint valley positions. This difference represents the fingerprint signals that are carried by the reflected probe beam A'B' .

[0092] Because the air or vacuum coupler is relatively low cost and can be of any size, this design can be used to develop optical touch panel for any size display.

[0093] Fingerprint Sensing-A Sample Design

[0094] FIG. 5 shows an exemplary optical fingerprint sensor 23b for fingerprint sensing. The optical fingerprint sensor 23b is substantially similar to the optical fingerprint sensor 23 of FIGS. 2 and 3 with some variations in the coupler 31. In the exemplary optical fingerprint sensor 23b shown in FIG. 5, one surface 111 of the coupler 31 on the left side has a curved (spherical or aspheric surface) mirror shape for imaging. The probe light source 30 is placed at the focus point of the curved mirror surface 111. A pinhole can be used on the probe light source 30 to spatially confine the probe light so that a modified light source 30a only projects a portion of the light beam to the curved mirror surface 111, and the influence of the scattered light is reduced or eliminated. The coupler 31 is set to be off center with proper distance D when the curved surface 1 11 is fabricated. Therefore, the curved mirror surface 111 is tilted properly so that the collimated light beam is incident into the spacer material 39 and the cover glass 50 with desired angles. For example, divergent light beam ASB is collimated and projected to the sensing surface 45. The reflected probe light beam A'B' is detected by the photo diode array 37. correspondingly, the central light SC is reflected back to the photo diode 37 center C .

[0095] In the example shown in FIG. 5, the light beams are propagated mostly in the coupler 31. The structure can be made compact and robust. In the example shown in FIG. 5, the material of the coupler 31 can be of single material, or multiple material compounds.

[0096] The optical fingerprint sensor of the disclosed technology can be implemented to provide one or more of the following features. The optical fingerprint sensor includes a light source, coupler, spacer, photo diode array, and cover glass. The spacer may be made of glass material, adhesive material, or even air gap or vacuum. The coupler may be made of glass material, adhesive material, or even air or vacuum. The cover glass may be partial of the display cover glass, or separate cover glass. Each of the mentioned coupler, spacer, and cover glass may be of multiple layers.

[0097] The disclosed technology provides flexibilities to control the signal contrast by matching the materials shapes and refractive indexes. By matching the probe light beam incident angle, divergent angle, and the materials of the involved coupler, spacer and cover glass, the probe light beam may be controlled to be totally reflected or partially reflected at the sensing surface for different touching materials.

[0098] The disclosed optical fingerprint sensor also provides a water-free effect. A typical smartphone cover glass has a refractive index of about 1.50. One design is to use a low refractive index material (MgF ₂ , CaF ₂ , Polymer etc.) to form the coupler 31. For example, the disclosed technology can be used to control the local probe light beam incident angle at the sensing surface 45 of the cover glass 50 to be about 68.5°. The total reflection angle is about 62.46° when water touches the sensing surface 45 of the optical fingerprint sensor, and the total reflection angle is about 73.74° when the ridges of a fingerprint touch the sensing surface 45. The total reflection angle is about 41.81° when nothing touches the sensing surface 45. In this design, at the water soaking area, the probe light is totally reflected to the photo diode array 37; at the fingerprint ridges touching positions, less than 5% of the probe light is reflected to the photo diode array; and at the dry fingerprint valleys positions, the probe light beam is also totally reflected to the photo diode array. Under this design, the optical reflection varies from the ridges to valleys of the finger and reflection caused by the fingerprint ridges generates stronger optical signals that are detected to create a high contrast optical image of the fingerprint pattern at the photo diode array 37.

[0099] Human sweat has a refractive index that is lower than the finger's skin. Therefore, based on the differences in optical reflection in the above design, the disclosed technology provides a solution to distinguish the sweat pores in the fingerprint. When air gap is used to form the coupler, the total reflection at the sensing surface does not occur. The reflectance difference among different touching materials (the fingerprint ridges, fingerprint valleys, and other contaminations) can be used to detect the fingerprint image.

[00100] Due to the light path compression effect in the above optical design, the sensing area size at the sensing surface 45 on the cover glass 50 may be greater than the photo diode array size of the photo diode array 37.

[00101] In implementations, the light source 29 may be a point light source installed at proper distance. In some implementations, the probe light beam may be collimated by spherical lenses, cylinder lenses, or aspheric lenses, or just put the light source far away. The probe light beam may be of proper divergent angle. The probe light beam may also be divergent or convergent.

[00102] Due to the light path compression effect, the coupler 31 may be very thin. For example, less than 1mm thickness CaF ₂ coupler can be used to realize even 10mm sensing area size. In this example, the image compression ratio is 1 : 10. This helps to reduce sensor thickness and the sensor cost. The photo diode array 37 is installed on one end of the coupler instead of under the coupler. This design leaves the flexibility to apply color paint, illumination light etc. to compensate the color or decorate the sensor area. [00103] The probe light source may be modulated to help reduce the influence of the background light. The photo diode array is designed to work well in any illumination

environments. Under the above optical design, the cover glass thickness does not limit the optical fingerprint sensing. The principle can be used to build optical touch panel. [00104] Live-Fingerprint Detection

[00105] FIG. 6 is a diagram illustrating exemplary live-fingerprint detection. The live- fingerprint detection can be implemented by a designed optical system such as the light source 33 and optical detector 34 in the example in FIG. 2 that are separate from the light source 29 and the optical detector array 37 for fingerprint sensing. This is shown in FIG. 6. Alternatively, the live-fingerprint detection can be performed by the same the light source 29 and the optical detector array 37 for fingerprint sensing without using a separate optical sensing as shown in FIG. 2. The live fingerprint detection in FIG. 6 can be performed by a finger print sensor, such as one of the optical fingerprint sensors 23 in FIG. 3, 23a in FIG. 4, or 23b in FIG. 5, in a way similar to what is now described below in the specific example in FIG. 6.

[00106] In FIG. 6, the light sources 33 and the receiving photodetector (PD) array 34 are isolated by the matched coupler 31 so that the emitting light beams cannot directly reach the photodetector (PD) 34 for sensing whether a fingerprint is from a live finger. The light beams propagate through the light path window 41 and transmit into the touching material, for example, a finger 43. For a live-fingerprint of a live-person, the blood flow 81 varies with the heartbeat, the pressing force against the sensor, the breathing etc. When the light beams 83 enter the material being monitored, the tissues in the material scatter a portion of light 85 into the receiving PD array 34. By analyzing the signals received, a sequence of signals can be obtained.

[00107] The fingerprint sensor photo diode array 37 may also be used to detect the scattered light from the touching materials. The fingerprint sensing light source 29 may also be used for live-fingerprint detection. The micro movement of the fingerprint can be used to indicate whether the fingerprint is from a live-finger. A sequence of fingerprint images are used to recover the signal amplitude and bright spots distribution change with time. A fake, non-live- finger manifests different dynamics from a live-finger.

[00108] FIG. 7 shows exemplary optical extinction coefficients of materials being monitored in blood where the optical absorptions are different between the visible spectral range e.g., red light at 660 nm and the infrared range, e.g., IR light at 940 nm. By using probe light to illuminate a finger at a visible wavelength and an IR wavelength, the differences in the optical absorption can be captured determine whether the touched object is a finger from a live person. FIG. 8 shows the blood flow in different parts of a tissue. When a person' heart beats, the pulse pressure pumps the blood to flow in the arteries, so the extinction ratio of the materials being monitored in the blood changes with the pulse. The received signal carries the pulse signals. These properties of the blood can be used to detect whether the monitored material is a live- fingerprint or a fake fingerprint.

[00109] FIG. 9 shows a comparison between a nonliving material (e.g., a fake finger) and a live-finger. Referring to FIG. 6, the light source 33The optical fingerprint sensor can also operate as a heartbeat sensor to monitor a living organism. One or multiple light wavelengths are used. When two or more wavelengths of light are used, the extinction ratio difference can be used to quickly determine whether the monitored material is a living organism, such as live fingerprint. In the example shown in FIG. 9, two light sources are used to emit probe light at different wavelengths, one at a visible wavelength and another an IR wavelength as illustrated in FIG. 7.

[00110] When a nonliving material touches the optical fingerprint sensor, the received signal reveals strength levels that are correlated to the surface pattern of the nonliving material and the received signal does not contain signal components associated with a finger of a living person. However, when a finger of a living person touches the optical fingerprint sensor, the received signal reveals signal characteristics associated with a living person, including obviously different strength levels because the extinction ratios are different for different wavelengths. This method does not take long time to know whether the touching material is a part of a living person. In FIG. 9, the pulse-shaped signal reflects multiple touches instead of blood pulse. Similar multiple touches with a nonliving material does not show the difference caused by a living finger.

[00111] In an implementation where the live-fingerprint detection can be implemented by a designed optical system such as the light source 33 and optical detector 34 in the example in FIG. 2 that are separate from the light source 29 and the optical detector array 37 for fingerprint sensing, the designated light source 33 is operated to emit probe light at the selected visible wavelength and IR wavelength, e.g., at different times, and the reflected probe light at the two different wavelengths is captured by the designated optical detector 34 to determine whether touched object is a live finger based on the above operations shown in FIGS. 7 and 9. [00112] Alternatively, in an implementation, live-fingerprint detection can be performed by the same the light source 29 and the optical detector array 37 for fingerprint sensing without using a separate optical sensing. Under this design using the light source 29 and the optical detector array 37 for both fingerprint sensing and the live-fingerprint detection, the light source 29 is operated to emit probe light at the selected visible wavelength and IR wavelength at different times and the reflected probe light at the two different wavelengths is captured by the designated optical detector 34 to determine whether touched object is a live finger based on the above operations shown in FIGS. 7 and 9. Notably, although the reflected probe light at the selected visible wavelength and IR wavelength at different times may reflect different optical absorption properties of the blood, the fingerprint image is always captured by both the probe light the selected visible wavelength and the probe light at the IR wavelength at different times. Therefore, the fingerprint sensing can be made at both the visible wavelength and IR wavelength.

[00113] Security Level Set Up

[00114] FIG. 10 shows a process flow diagram of an exemplary process 1000 for setting up different security levels for authenticating a live finger based on the disclosed optical sensing technology for fingerprint sensing. Different security level criterions may be set up based on the type of action requested. For example, a regular action request is required to pass security level 1 check. A request for a financial transaction for an amount below a threshold, such as under $100 payment needs to pass security level 2. A financial transaction for an amount over the threshold may require a higher security level clearance. Different security level action is triggered after different safety level evaluation. The safety levels corresponding to different security levels can be set up by combining different live-finger signatures. For example, single light source signals can be used to set up safety level 1 gate, two light source signals can be combined to set up safety level 2 gate, and so on.

[00115] The method 1000 can begin when an action is requested (1002). The requested action is analyzed to determine an appropriate security level (1004). When determined that that security level 1 (the lowest security level) is required (1006), the safety trigger level 1 is required to be passed (1014). When the fingerprint analysis passes the safety trigger level 1, the requested action is performed (1024). However, when the fingerprint analysis fails the safety trigger level 1, the requested action is denied (1022). [00116] Similarly, when determined that that security level 2 is required (1008), the safety trigger level 1 is required to be passed (1016). When the fingerprint analysis passes the safety trigger level 1, the requested action is performed (1024). When the fingerprint analysis fails the safety trigger level 1, the requested action is denied (1022).

[00117] When determined that that security level 3 is required (1010), the safety trigger level 1 is required to be passed (1018). If the fingerprint analysis passes the safety trigger level 1, the requested action is performed (1024). If, however, the fingerprint analysis fails the safety trigger level 1, the requested action is denied (1022).

[00118] When determined that that security level N is required (1012), the safety trigger level 1 is required to be passed (1020). If the fingerprint analysis passes the safety trigger level 1, the requested action is performed (1024). If, however, the fingerprint analysis fails the safety trigger level 1, the requested action is denied (1022).

[00119] The optical fingerprint sensor of the disclosed technology can be implemented to perform live-finger detection including the following. The optical fingerprint sensor can detect whether the touching material is a live-finger and can improve the security of the sensor.

Specified light sources and detectors can be used to detect whether the object touching the sensing area is a live-finger or a nonliving material. When single wavelength is used, the heartbeat detection provides a reliable criterion to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger. When two or more wavelengths are used, the extinction ratio of the wavelengths are compared to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger. The fingerprint sensor light sources and photo diode array can be used to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger. The dynamic fingerprint images can be used to detect whether the object touching the sensing area is a live-finger or a nonliving material, including the fingerprint of a live-finger. Multiple security level can be set up for different security requirement tasks.

[00120] Sensor Area Decorating

[00121] FIG. 11 is a diagram showing an exemplary optical fingerprint sensor for sensor area decorating. When the optical fingerprint sensor (e.g., optical fingerprint sensor 23) is installed under the cover glass 50, an optical window should be opened for the light path. Specifically, a portion of the cover glass' color coating is removed. Because the fingerprint sensor detector is arranged to be at one end of the coupler 31, the bottom of the coupler 31 may be painted with color layers 25. The painted color layers 25 can be selected to match with the platform surface color. For example, to use same color or pattern under the coupler so that the sensor becomes invisible. In some implementations, the matched coupler 31 may also be painted with a desired or different color or pattern to achieve certain or different decorative effects or styles. The matched coupler 31 may also be painted with certain patterns or signs, such as homing button sign.

[00122] The design provides an attractive option to further decorate the sensor area. For example, different colored light waves can be used to illuminate the sensor area. This can be useful in dark environments when the bell rings on the smartphone to indicate where the fingerprint sensing area is located.

[00123] The optical fingerprint sensor can be implemented to enable various decorative elements including the following: the bottom surface of the coupler can be painted with same color or pattern layers to match with the platform surface color; the bottom surface of the coupler can be painted with different color or pattern layers to show new styles out-looking; and color light sources can be installed around the coupler to decorate the sensor area.

[00124] Fingerprint Sensor Packaged As A Separate Button

[00125] As an alternative implementation, the optical fingerprint sensors 23 in FIG. 3, 23a in FIG. 4, and 23b in FIG. 5 placed under a contiguous cover glass 50 can be packaged as a separate physical fingerprint sensor button with a physical demarcation with other parts of the cover glass 50.

[00126] FIG. 12 is a diagram showing an exemplary optical fingerprint sensor packaged as a separate button. FIG. 13 is a diagram showing exemplary fingerprint and live-finger detection using the optical fingerprint sensor packaged as a separate button. The optical fingerprint sensor of FIGS. 12 and 13 can be implemented as the optical fingerprint sensors 23 in FIG. 3, 23a in Fig. 4, and 23b in FIG. 5 but packaged as a separate button. Thus, the fingerprint sensing and live- finger detecting is also the same as those described above. A matched coupler 31 is used to set up the photo diode array 37 position and provide package flexibility to the visible area. All the aforementioned description regarding the different components of the optical fingerprint sensor in FIGS. 12 and 13 are substantially the same as the optical fingerprint sensors 23 in FIG. 3, 23a in FIG. 4, and 23b in FIG. 5 including the light sources. However, to implement the optical fingerprint sensor as a separate button, the rigidity or the strength of the material for the cover glass 51 may be required at a higher level than the designs in FIGS. 3-5 under the contiguous cover glass 50.

[00127] The spacer material 39 and the cover glass 51 add a position shift of D to the probe light beam AB. When the thickness of the cover glass 51 and the spacer material 19 is reduced to zero, specifically by excluding the cover glass and spacer, the probe light beam shift D is eliminated. For example, a 10mm sensing size can be realized with less than 1mm thickness CaF ₂ . Also, the photo diode array 37 should match with the light path to realize proper resolution and guarantee the performance in all illumination environments.

[00128] The optical fingerprint sensor packaged as a separate button can perform the same fingerprint detection and live-finger detection as the optical fingerprint sensor of FIGS. 2-11. In addition, the optical fingerprint sensor package as a separate button can be implemented to perform the following features:

[00129] The cover glass and related spacer material feature flexibility in the thickness according to the applications. Especially, it is a practical package not to use cover glass and spacer material. Another practical design is to use a thin layer of cover glass to protect the coupler. The cover glass may be of high hardness. To use colored glass or other optical materials to build the cover is also practical. The package method provides a solution to build a compact button that can detect the fingerprint with improved security. Other mechanical parts may be integrated to enhance the rigidity or strength of the module.

[00130] The optical fingerprint sensor packaged as a separate button can be implemented to integrate the functions of fingerprint detection with live-finger detection and sensor decoration.

[00131]

[00132] While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple

embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[00133] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all

embodiments.

[00134] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.