BIOMETRIC SIGNATURE AND IDENTIFICATION THROUGH THE USE OF

PROJECTIVE INVARIANTS

COPYRIGHT NOTICE

[0001] Contained herein is material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction of the patent disclosure by any person as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all rights to the copyright whatsoever.

STATEMENTREGARDING FEDERALLY SPONSORED RESEARCH OR

DEVELOPMENT

[0002] The work of the inventors was supported in part by Air Force Research Lab under agreement number F49620-03- 1-0207 through Network Information and Space Security Center (NISSC), U. Colorado at Colorado Springs. It was also funded in part by DARPA HID program under contract number N00014-00-1-0929, and by the Cojorado Institute for Technology Transfer and Implementation

BACKGROUND Field

[0003] Embodiments of the present invention generally relate to biometric signatures and identification. More specifically, embodiments of the present invention describe a system and method for accurate biometric signatures and identification using projective invariant features of a subregion of the human body.

Description of the Related Art

[0004] Biometrics generally are methods of identifying a person based on a physiological characteristic. Among the features measured are: face, fingerprints, hand geometry, palmprints, iris, retinal, vein, and voice comparison. Biometric technologies are becoming the foundation of an extensive array of highly secure identification and personal verification solutions.

[0005] The first modern biometric device, called Identimat, was introduced on a commercial basis in 1976. It measured the shape of the hand and looked particularly at finger length. Shortly thereafter, fingerprint identification systems were widely used in law enforcement. Because of the rapid development of hardware, including computing processors and image capture devices, other biometric-based techniques began to thrive. As the technologies become more stable and trustworthy, biometric identification is expected to substitute the existing personal IDs, e.g. passports.

10006] FFiinnggeerrpprriinntt iiddeennttiiffiiccaattion has been utilized as a positive human identifier for p f ^> : ηr / \ \\ c 3j j Dn fc κ| _, y,.•" pp cc:|ι q ent i pptι ^■ ' ^■ 3;;» over 100 yyeeaarrss,, aanndd ^' i iss tthhee ^' mmobsstt wwiiddeelry) used application of biometric technologies. However, it suffers from extracting some small unique features from the fingerprints of elderly people as well as manual laborers whose fingerprints are worn out. In addition, since fingerprint identification has been widely employed by law enforcement agencies, it becomes intrusive to individual's privacy. Furthermore, fingerprints acquisition generally require contact further increasing its intrusiveness.

[0007] For many applications, other less intrusive techniques that can serve a larger percentage of the population would be preferable. Alternative techniques that identify people based on unique geometric characteristics of subregions of the human body hold promise in addressing these concerns. One area where there has been much research is the hand geometry identification technique. As the name implies, it identifies a person by the geometric structure of hand. Hand geometry identification technique is based on the fact that nearly every person has reasonably different hand shape that does not change after certain age. The shape of hand is composed of certain measurements of the hand such as the length and the width of fingers, the thickness of fingers, the width of palm, the angle of some special points. In existing approaches, the shape of hand is looking at (Euclidean) geometric sizes of various hand features. Various methods are used to measure the size of the hand. These methods are most commonly based either on mechanical or optical principle. The latter ones are much more commonly used today. For instance, optical scanners and digital cameras are typical devices to capture the image of the hand. Constraining the hand allows measurements to be computed from these images and converted to actual sizes.

[0008] The biometric community has studied a variety of hand geometry techniques, with varying degrees of success. One approach uses a mirror to allow a side view, various features including widths of the fingers, lengths of the fingers, widths of the palm, as well as heights of the fingers and the palm to be measured. To provide for consistent positions of a hand to be measured, five pegs were used to guide the placement of user's hand on a flat surface of the imaging device.

[0009] A major limitation of the prior art is the need for contact, and often the need of pegs. This introduced several problems. First, pegs can deform the shape of the hand. Second, the dependence on alignment means improper placement of a hand can still happen due to the relatively complicated instruction, which can reduce the reliability of the system. Finally, the

prior art requires contact, which can be objectionable as it requiring users to place hands

P ₁ C T. ■ ^■• " O S D 5 / U ^! 3 ψB 3 _λ where many strangers just put their hands .

[0010] Thus, existing biometrics, especially for hand geometry and palm or finger prints, or iris, generally require strongly controlled imaging, usually requiring direct contact or constrained very close proximity to the sensor. In addition, measurements used for the biometric signature described in many previous inventions are Euclidian metrics and hence are distorted under projection and can be effectively used only for identification under nearly identical sensing geometries.

[0011] All biometric identification systems seek to find features such the intra-subject variations of the feature measurements are small and do not significantly overlap inter-subject distribution of those features. Features that do not change at all can be called invariant features, e.g. the number of fingers on a subject's hand. More commonly biometric systems consider feature measurements that may vary slightly during measurement but such that over a range of measurement conditions, as constrained by the system design, the measurements vary only slightly. Hence these slightly varying features may be considered quasi-invariant. When considering if a measurement is invariant or quasi-invariant, one must specify what is the range of measurements conditions or transforms under which the invariant property is expected to hold. For example the number of fingers visible on the hand is invariant to general pose variations and movement of the hand, but only if all the fingers are seen. However, while the 3D length of a finger is a general quasi-invariant, the visible "length" of a finger in an image is strongly impacted by the viewing angle of that finger and the pose of that finger. Generally speaking the broader the class of transforms allowed, the fewer invariants that will exist, the less discriminating they will be and the more intra-subject variations can be expected, i.e. the quasi-invariant property holds only over a smaller region of the general transform space. At the other extreme some prior art systems uses "Euclidian invariant", which restrict the sensing system so as preserve Euclidean distance between feature points, e.g. US 5,956,671 teaches an approach to speech recognition that is shift invariant and hence ID Euclidian invariant. If the sensing constraints are not exactly met, then the measurements will vary producing only Euclidian quasi-invariant features. There has been limited work addressing features that are more general than Euclidian invariants but still formal invariants. US Patent 6,178,261 teach an approach to image-based recognition for extracting features that are scale and translation invariant, which is a subset of affine invariant features.

[0012] Considerable prior art exists for object recognition through affine invariants, i.e. properties that are invariant to changes in rotation, translation or scale. US patent 6,362,875 describes a system with scale-invariant and rotation-invariant pattern recognition application that retrieves stored images representative of the object being viewed that includes data

representative of a fiducial within the object image. For example US patent 6,243,493 tpeaccheτs a/nu apsprooacsh t/op hanqdg wrsitin3g recognition us ^* ing features that are rotation, translation and scale invariant, i.e. affine invariant features defined as ratio of tangents, and a novel application of the normalized curvature feature. US patent 6,694,054 introduces a pattern recognition process wherein the pattern characterization step is used to obtain [affine] invariant descriptors of the pattern with a Fourier-Mellin transform. A projective invariant, well known to those skilled in the art, is a property or measurement that is retained when an object is subject to perspective projection. It is stronger than a general invariant, potentially providing improved discrimination, but requires fewer assumptions than an affine or Euclidian invariant. A pure invariant would not change at all, more commonly people consider quasi- invariants where the feature chances very little over a wide range of projective angles.

»

[0013] A key issue when considering projective invariants for biometrics is the actual feature points used to derive the invariant. The features themselves must be stable and be able to be re-identified if true projective invariance is to be computed. A simple example is cross-ratios of feature points - if they cannot be consistently labeled then the cross-ratio cannot be "matched". A subtler example is contour features such as the finger and hand outlines considered in some of the prior art. With just a minor rotation of the finger or hand the actual points on boundary are now either occluded, on the part that turned away, or are completely inside the contour. Thus the features themselves, the points on the occluding contour, are not viewpoint invariant and hence not suitable for use in defining projective invariants. For polygons however, which are the subject of the prior art in invariants, the discontinuity of the surface does make the boundary points stable and hence useful for computing projective invariants. Such features may be useful for defining Euclidean or 2D ridged transform invariant features.

SUMMARY OF THE INVENTION

Pip T /" it i s n s ■- ^■ ' P ⁿ ES ^ι Q!P ^ι ' ^;; iϊ

^" A nfethoa ^" όf gengrating ^* t)tϋmetric signatures includes the steps of detecting a number of image features of a sub-region of a human. A quasi projective invariant is computed based on a subset of the image features. A number of quasi invariants may be computed based on another subset of the image features. A similarity metric for the quasi projective invariant may be determined. An image of a hand may be captured in the step of detecting image features. A number of projections of a curved feature may be computed, wherein the quasi projective invariant is obtained as a number of coefficients of a canonical functional representation of at least one the projections. A second group of image features may be captured from a second sub- region of a human. A second quasi invariant may be computed based on a subset of the second f features. The quasi projective invariant and the second quasi invariant may be combined to form a biometric signature.

In one embodiment, a biometric signature system includes a feature capturing system that captures a number of features. A process computing quasi invariants receives the features and computes a quasi projective invariant. A similarity computing process receives the quasi projective invariant and outputs a score. The system may include a biometric signature store coupled to the similarity computing process. The feature capturing system may be an imaging system. The process computing quasi invariants may compute a quasi invariant projective invariant. The process computing quasi invariants may compute a number of quasi invariants to form a biometric signature. The similarity computing process may determine a similarity between a stored quasi projective invariant and the quasi projective invariant. The feature capturing system may capture a first group of features from a first sub-region of a human and capture a second group of features from a second sub-region of the human.

In one embodiment, a biometric signature method includes the steps of capturing a number of features from a human. A projective invariant is calculated from a subset of the features. A similarity is determined between the projective invariant and a stored projective invariant. The human is enrolled to form the stored projective invariant. A number of quasi invariants may be determined from at least a portion of the features to form a biometric signature. A biometric signature may be compared to a stored biometric signature. An image of a hand may be captured as part of capturing a number of features. A number of projections of a curved feature may be computed, wherein the quasi projective invariant is obtained as a number of

coefficients of a canonical functional representation ot at least one the number of p Pro Cjec Ttio/ns U. S O S S H »9 ¹ ^t ≡ 3

i .,l,lustrated by way of ~ examp ,le, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

[0010] Figure 1 conceptually illustrates the major Geometry subclasses and the corresponding transformations that represent the changes that the subclass spans.

[0011] Figure 2 conceptually illustrates the major geometry subclasses and invariant properties preserved under each class.

[0012] Figure 3 conceptually shows the projection of four collinear points under an original and an alternative projection, and the corresponding line lengths. Clearly length 1 is not preserved under projection.

[0013] Figure 4 illustrates the projection of four coplanar lines that have one point in common. Again line lengths are not preserved, nor are angles.

[0014] Figure 5 illustrates projective invariants of curves using a canonical coordinate system. The 4 labeled points define the canonical coordinate system that is used to transform data from different projections. After two curves Ll and L2 closely match in the canonical system on the left, suggesting they are from the same feature, but they do not match using the transform on the right, indicating they are not from the same source.

[0015] Figure 6 illustrates the same hand seen from different viewpoints. The view on the left (a) is approximately frontal while the one on the right (b) is tilted with fingers toward the viewer and thumb slightly upward.

[0016] Figure 7 illustrates feature points on the fingers (drawn as a large X). The left

(a) shows four feature points suitable for ID cross rations. The right (b) shows 16 features suitable for use with 2D cross-ratios.

[0017] Figure 8 illustrates the layout of collinear features points for 1-D cross ratios

[0018] Figure 9 illustrates twelve five-point layouts for 2-D cross ratios. Out of the thousands possible these 12 were selected to reduce increase stability by reducing nearly degenerate situations.

[00ϊ9] ^{' ' '"} " ^"" Figure lβ^iltύstf&sHwo different qualified five-point sets superimposed on a drawing of the hand.

[0020] Figure 11 shows Response curves of four different Permutation and Projective

Invariant functions over a range of inputs.

[0021] Figure 12 shows the pair wise comparison of Cross-ratio based projective invariant-based biometric applied 10 pictures of 4 individuals.

[0022] Figure 13 shows the pair wise comparison of Cross-ratio based projective invariant-based biometric applied 10 pictures of different set of 4 individuals.

[0023] Figure 14 shows the effects of the positional noise on hand feature vectors composed of Il(τ), I2(τ), I3(τ), I4(τ), and cross ratios, measured by the distance between a genuine model and its (simulated) noisy counterparts.

[0024] Figure 15 shows a table of performance (Equal Error Rate as a %) of embodiments with different types of invariant and various lengths of hand feature vectors

[0025] Figure 16 illustrates feature points for a canonical coordinate system basis and three different curves in the hand plane that could be used for biometric signatures.

[0026] Figure 17 shows a flow diagram of the steps in computing projective invariant- based biometric signatures, including computing cross-ratio features, PPIs and projective invariant curves.

[0027] Figure 18 shows a flow diagram of the basic steps in a system using projective invariant-based biometrics for verification of identification.

[0028] Figure 19 shows a flow diagram of the steps in the enrollment process with projective-in variant biometric signatures.

[0029] ^' Figure 20 illustrates an imaging system view suitable for fusion of traditional face-recognition with perspective-invariant hand-geometry.

ψpψfM w^≥ ^■» };

[0030] Techniques, systems and methods for obtaining biometric signatures and identification are described. Broadly stated, embodiments of the present invention utilize specified geometric principles to provide means for accurate biometric identification using projective invariant or projective quasi-invariant features of a subregion of the human body.

[0031] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present invention. It will be apparent, however, to one skilled in the art that embodiments of the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form.

[0032] Certain elements of the embodiments of the present invention include various steps, which will be described below. The steps may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general- purpose or special-purpose processor programmed with the instructions to perform the steps. Alternatively, the steps may be performed by a combination of hardware, software, and/or firmware.

[0033] Certain elements of the embodiments of the present invention may be provided as a computer program product which may include a machine-readable medium having stored thereon instructions which may be used to program a computer (or other electronic devices) to perform a process. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, compact disc read-only memories (CD- ROMs); and magneto-optical disks, ROMs, random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or other type of media / machine-readable medium suitable for storing electronic instructions. Moreover, certain elements of the embodiments of the present invention may also be downloaded as a computer program product, wherein the program may be transferred from a remote computer to a requesting computer by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).

[0034] While, for convenience, embodiments of the present invention may be described with reference to physical, workstation, network, and domain access, single sign-on, application logon, data protection, remote access to resources, transaction security and Web security, the present invention is equally applicable to various other current and future

applications. Additionally, for sake of brevity, embodiments of the present invention are jj5 f ¹ - "j ' / y S D 5 / P Q «"[i ip "-! largely ^* described in the context of Hand geometry identification. However, it is contemplated that the techniques, systems and methods described herein will have broad applicability to other unique subregions of the human body, including but not limited to irises, eye sockets, teeth, skulls, retinas, ears, and feet.

I. INTRODUCTION

[0035] "Geometry" generally refers to as the mathematics of the properties, measurement, and relationships of points, lines, angles, surfaces, and solids of an object. In biometrics, geometry identification techniques, as the name implies, identifies persons by the geometric structure of a subregion or series of subregions of the human body. Hand geometry identification technique is based on the fact that nearly every person has reasonably different hand shape which does not change after certain age. Other subregions of the body share similar attributes of difference among individuals and consistency over time. However, for purposes of explanation, much of the discussion is focused on hand geometry, although one skilled in the art will recognize the wide applicability of the principles set forth.

[0036] The shape of hand is composed of certain measurements of the hand such as the length and the width of fingers, the thickness of fingers, the width of palm, the angle of some special points. In other words, the shape of hand can be described by looking at some geometric sizes of hand. However, the hand shape by itself might not be able to distinguish people from each other because the features extracted from the hand images might not be sufficient and accurate enough. Though, a variety of other features of a hand such as finger creases and palmprint make hands unique to different person, even for twins. This qualifies the hand a good biological identifier.

[0037] In nearly all hand biometric identification systems, feature vectors of a ^' 3-D hand object are obtained from its picture that is the 2-D representation of it. However, especially in peg-free, touch-free systems, the hand has gained freedom of movement in three dimensions. It can rotate, translate, tilt, or slant. If it moves towards or against the camera, the size of the hand in the picture will even be changed. In order to keep the equivalent substitution of 3-D with 2-D, the three dimension freedoms have to be brought down to two dimensions. Said explicitly, to get a correct match between two pictures of the same hand, the pictures should be taken from the exactly same distance from the hand and exactly same point of view with respect to the hand. However, it is difficult to restrict the moving of a hand without any aid. Thus any motion will change the 2-D appearance of the hand. For instance,

lant vertically or tilt horizontally will expose a portion of the hand by its side view. These

[0038] The distortion of 2-D hand images described above can be interpreted as the result of some projective transfoπnations. There exists a geometry hierarchy. Different combinations of motions such as rotation, translation, scaling, shear, tile, and slant, can construct different levels of geometric transformation, from the lowest level of Euclidean transformation to the highest level of Projective transformation. There are four levels of geometric transformations: Euclidean, Similarity, Affine, and Projective, and each transformation corresponds to one kind of geometry in a geometry hierarchy. Among different geometries, projective geometry has the largest collection of transformations and contains affine geometry that has the subset of similarity geometry. The smallest subset in the hierarchy is Euclidean geometry. The higher levels of subset comprise more transformations, as illustrated in Figure 1.

[0039] When any of the geometric transformations are applied to the object, there are always some features are preserved. These preserved features are so-called geometric invariants. There are different kinds of geometric transformations. Each geometric transformation preserves some features of the object. Figure 2 shows the invariant features preserved by the corresponding kind of geometric transformation. The number of the invariants decreases while the level of the geometric transformation moves up in the geometry hierarchy. When more geometric transformations are applied to an object, less features of the object will be preserved from changing. Euclidean transformation possesses the most number of invariants because it contains the least number of geometric transformations. On the other hand, projective transformation possesses the least number of invariants while contains most transformations. So when more degrees of freedom for moving are allowed for the object in front of the camera, more possible geometric transformations could be reflected in its 2-D picture. In the peg-free and touch-free biometric identification system, the hand or other subregion of the human body will likely have three degrees of freedom for moving. This implies that the 2-D picture will be experiencing the projective transformation. Consequently, the invariants the picture will possess are incidence and cross ratio.

[0040] The general principles of projective invariants describe how one can construct, from a collection of features, metrics that are invariant to projection. Thus, they retain the same value even if the features are subject to a projection such as that which occurs when imaging with a standard camera. Such measurements are not metrics, nor are they a distance measure in an image, rather they are quantities that can be computed and the computed value

will remain the same across different projective views. Projective invariants do not satisfy the

P C T ./' ^" IJi S I ^" « 5. ■ ^■■ ' P ^li;;; !f ^c ! P ^";;; i: mathematical definitions ^" bf metric ^" or distance measures, though since they themselves are numbers one can define the distance between measurements of the same projective invariant.

[0041] According to one embodiment, at least one of the principles of projective invariants described above is used in conjunction with features of the human body that that are invariant (or quasi-invariant) to the projection process inherent in the sensing of the "image," but still variable across individuals and hence contain biometric information. From such combination, accurate biometric signatures and identification procedures are produced. Projective invariants themselves do not satisfy the mathematical definitions of metric or distance measures, and are not directly employed as a biometric. However, since they themselves represented as numbers, if one or more such invariants are combined into a vector one can define a similarity measure between resulting vectors. For use as a projective invariant biometric, it is not sufficient that resulting properties are invariant. Instead, they need to also discriminate between different classes of interest, e.g. if a feature is defined as "the number of fingers on the hand," it is a projective invariant but not very discriminatory, and hence of little use as a projective invariant biometric.

[0042] According to different embodiments of the invention, image features may detected from the palm-side of a hand, the back-side of a hand, a fingerprint, a palmprint, a face, an ear, an eye-region, an iris, a retina, the teeth or jaw features, or any other subregion of the human body.

[0043] According to one embodiment, the feature points used to derive the invariant are restricted to approximately planar objects. The analysis is less complex if so restricted, as the mapping from scene points to image points is one to one. Most of the examples presented will be for planar subparts of objects. According to one embodiment, the feature points used to derive the invariant are not restricted to such planar objects. The full 3D case is of greater complexity, as no 3D invariant can be extracted from a single perspective image of a set of general 3D points, unless something further is known about the point configuration (for instance, that some of them are coplanar as is generally the case for points along the eye sockets even though the overall face has complex 3D structure. By using ordering constraints, points of inflections, or co-planarity, those skilled in the art of invariants can often find projective invariants (or quasi-invariants) for the specialized 3D curves or points that result. For general 3D curves, the more useful form of projective invariants use multiple images. Given the discussion in this document, those skilled in the art will be able to directly apply the results of multi-view 3D projective invariants to derive projective invariant biometrics.

'EαB'ilipdrSHøllibiϊwIen considering projective invariants for biometrics is the actual feature points used to derive the invariant. The features are to be invariant (or quasi- invariant) to the projection process inherent in the sensing of the image, but shall still vary across individuals and hence contain biometric information. The features themselves must be stable and be able to be re-identified if true projective invariance is to be computed. A simple example is cross-ratios of feature points - if they cannot be consistently labeled then the cross- ratio cannot be "matched". A subtler example is contour features such as the finger and hand outlines considered in the prior art. With just a minor rotation of the finger or hand, the actual points on boundary become either occluded, on the part that turned away, or are completely inside the contour. Thus the feature itself, the occluding points, is not viewpoint invariant (except for discontinuous edges as in polygons), and hence not suitable for use in defining projective invariants.

[0045] According to one embodiment, the invention supports multi-modal integration of both projective invariant features and traditional non-projective features which can often be computed from the same image, e.g. a outstretched hand an face contained in the same image. That a single sensor image can be useful for multi-biometric is an important enhancement that can reduce overall system costs

II. CROSS RATIOS

[0046] Projective geometry does not preserve Euclidean distances, angles or ratios of distances/angles. According to one embodiment of the invention, it is necessary to identify features that are projective invariant, yet discriminate individuals. The cross ratio, which is the ratio of ratios of some particular properties, is invariant to the projective distortion. There are a lot of other projective invariants, but the cross ratio is relatively straightforward, and is often considered to be the classic invariant.

[0047] According to one embodiment of the invention, a one-dimensional cross ratio is used. As is know to those skilled in the art, the one-dimensional cross ratio τ, is defined in terms of the distances between four collinear points. As shown in Figure 3, let four collinear points A, B, C, and D on line /. Points A ', B', C, and D ' are their projections on line / '. Then, the 1-D cross ratio can be defined as follows:

where ||D|| denotes the Euclidean distances between two points. This equation is very important because describes the intrinsic essential characteristics of any object under

projective transformations, so that these characteristics can be used as the invariant signature

P ^[ ) ^[" ": ^" ϊ ^" -''' ^' !JL B FX ff ^" ii ! ^!;;; ϊ / P ^ii;: l *3 if! ^{b '"■"} § of object. ThiiTactually models ^" p ^' art ^' Of the visual capability of human beings, which is used to distinguish or identify objects.

[0048] According to one embodiment, the features captured are the projections of at least 4 points from collinear subregion of a human body and the project invariant values are ID cross-ratios computed from the locations of subsets of at least 4 of these feature points. The general assumption for the use of the ID cross ratios as a (quasi) projective invariant is that the underlying feature is 4 points that are (approximately) co-linear. Examples would be joint spacings on a finger, teeth spacings on dental x-rays, spacing of zero-crossings of temporal signal. If more than 4 co-linear points exist, it is important that the features be such that the identification of the first point (say A above) is possible, e.g., the point closest to the tip of the finger or the back of the mouth.

[0049] According to one embodiment of the invention, a two-dimensional cross ratio is used. Under projective transformations neither area of planar object nor angle established by two intersecting lines is preserved. However, the cross ratios created by five coplanar points or four coplanar concurrent lines are invariant. Figure 4 shows a pencil of four concurrent lines, a, h, c, and d, in the same projective plane π. By projective transformation T, they are transformed into a\ b ', c ', and d', respectively, in projective plane II. On plane π, points A, B, C, and D are on the lines a, b, c, and d, respectively. Point O is the intersecting point of lines a, b, c, and d. No three of these five points, including O, should be collinear. On the other hand, on the transformed plane II, A ', B', C ', and D ' are points on the lines a ', b \ c \ and d\ respectively. Likewise, O' is the intersecting point of the lines. Since points set A ', B ', C, D ', and O' are the projections of points A, B, C, D, and O, the 2-D cross ratios of two five-point sets in Figure 4 can be computed by the following equation sm(ZAOC) sm(ZB0P) sm(ZA'O'C) sin(Z5'O'Z)') τ= sin(ZAOD) sin(ZBOC) ⁼ Sm(Z^ ¹ O ¹ D ¹ ) ^' Sm(Z^ ¹ O ¹ C") where Z- denotes the angle created by three points. This type feature assumes points where the reference point (ο) is distinguishable, and the points ABCD can be labeled. There are processes that may be used to symmetrize the cross-rations, but the problem is that when one symmetrizes the invariants discriminating power is reduced.

[0050] According to one embodiment, features captured are the projections of at least

6 points from co-planar subregion of a human body and the project invariant values are 2D cross-ratios computed from the 2D locations of subsets of at least 5 of these feature points. Generally, when comparing 2D cross ratios, it is very important to recall that they are highly

used. Those skilled in the art will recognize that the different 2D configurations can processed differently based on the underlying typology of the convex hull of the 5 points. This can improve the comparison and may be incorporated within the invention without great difficulty.

[0051] The next forms of projective invariants are those defined by a larger collection of points, or a set of distinguished points and the points on a curve. According to one embodiment, this form of projective invariant is used. One such an invariant is the projective curvature that characterizes all curves up to a projective transformation. However, the projective curvature depends on higher (at least seventh) order derivatives, so is very sensitive to noise. For simple curves, e.g. the principle curves of a palm or the shape of eyes, less complex approaches may be applied. In fact by some auxiliary points, the curves of interest may be transferred into a so called canonical system in which all projectively equivalent curves will be mapped to the same shape.

[0052] A canonical system for a curve may be defined as a coordinate system in which all the projective representations of a curve map to the unique shape. Essentially, a 2-D canonical system is a special projective plane. Other planes can be mapped onto this plane by any projective transformations. Those skilled in the art will recall that a projective transformation matrix can be determined by the correspondences of four coplanar points in different projective planes. So by mapping 4 reference points, the different shapes in other projective planes can be assumed to be the same shape in this canonical system. The correspondence created by this assumption can then be used to find the projective transformation matrix. As an example, assume that all quadrilaterals will be mapped to a unity square in the canonical system. Figure 5 gives an example. The eight linear equations created by this four-point correspondence can uniquely determine a projective transformation. Then other points that are on the same plane with the quadrilateral may be used as projective invariants of the object of consideration.

[0053] The resulting "curve matching" in the canonical reference space may use a robust similarity measure such as the Hausdorff distance, since its possible the ends of the curve will not have been equally well detected in both images. Some examples of biometric features where this type of projective invariant would be useful include features such as palm creases with distinguished features on the fingers, curves on the ear with distinguished high- curvature points within the ear, features on the eyes with junctions as feature points, and jawbone lines with individual teeth as reference points (in dental x-rays).

p c Ty IJ Si o 5./ E ^{Q i!;;;} ιt ;? ^ιι 3

[0054] A more general form of projective invariants of curve features does not require feature points but does require higher order representations of the curve so that it must be sufficiently smooth. According to one embodiment, this form of projective invariant is used. Weiss presents an algorithm that transforms a planar curve into a canonical system without knowing the correspondence of four points [Weiss-95], which is incorporated herein by reference. This algorithm can handle the invariants of more complicated curves, is resistant to partial occlusion, and does use a nodal cubic curve function along the curve. The mathematics are considerably more complex so we don't provide an example here, but those skilled in the art will be able to understand from the discussion herein how they can be applied for biometrics. This general form can be applied to many biometrically interesting curved features, if they are sufficiently smooth. All of the examples above for points and curves apply here, presuming there is sufficient resolution to fit the curve well. The tradeoff is that the implicit normalization required in developing the higher-order algebraic forms will reduce the discriminatory power of the resulting metric.

III. CAPTURE

[0055] According to one embodiment of the present invention, an imaging device is used to capture a number of features of at least part of one subregion of the human body. Such a device may be an analog or digital camera, an optical or other scanner, an imaging radar, a thermal imaging device, an ultrasound, a radar device, an electro-capacitive measurement device, or any other form of imaging device or feature detector used to capture or otherwise collect information, features, or images. The capture process may include a temporal component. Thus a measurement device that projects away time or a mixture of space and time to form a static array of measurements would also be included, allowing it to be potentially applied to dynamic signature, EEG, EKG, or voice measurements.

[0056] According to one embodiment, capture is "touch-free." The information, features, or images are thus captured without contact of any guide, peg, platform, or other device to provide image stabilization to the subregion at issue or any other part of the human body. There is no requirement for direct contact or constrained very close proximity to the sensor.

[0057] The term "image" is used throughout in a very general sense. According to one embodiment, it is used to mean the one or more dimensional vector of measurements that occur when the real-world process or subregion of the human body is subjected to a measurement apparatus that reduces the dimensionality in generating the signal.

pp D' T ^a / U S O S ./ iS: "3923

[ook >&-] The components ^" and subcomponents of (1) capture, (2) projection, (3) computation of the projective invariant features, and (4) matching against other biometrics may be done on different devices and in different locations.

IV. HAND IDENTIFICATION BY PROJECTIVE INVARIANTS

[0059] According to one embodiment, a digital camera is used as the imaging device to capture a hand for purposes of identification and the creation of a biometric signature . The user is directed to join their fingers as closely as possible when taking the hand pictures and keep their hand stretched flat. The result is only approximately planar and thus the cross rations are only quasi-invariant. But for reasonable imaging distance the variations from planarity with respect to viewing variations are very minor, much less than the error localization of the feature points. Users have the flexibility to pose their hands as they like, as long as major part of the hand plane is exposed to the camera. Therefore, the pictures of this hand plane taken from different viewpoints can be regarded as the projections of the original hand plane onto arbitrary projective planes. Figure 6 shows two examples of such pictures.

[0060] According to one embodiment, to acquire the 1-D cross ratios from a hand, at least four collinear points are extracted. There are a number of four-point or more sets existing on the hand plane. One can always find a straight line going through a finger. This line shall be identified as a middle line of a finger. Such a line can be located by connecting the middle points of finger widths. According to one embodiment, the intersecting points of this line and the crease lines on the finger give the collinear points. This is shown in Figure 7(a). In this embodiment, four points are extracted from each finger except the thumb. The four fingers give 16 feature points in total. This is shown in Figure 7(b). The four points from top to bottom on the pinky finger are denoted as 1-4, and those on the ring finger 5-8, those on the middle finger 9-12, and those on the index finger 13-16. Generating the four feature points on each finger qualifies them for computing the 1-D cross ratios. Thus there are four 1-D cross ratios, one from each finger, as shown in Figure 8 with the advantage that these features are invariant to individual finger movement. According to one embodiment, other feature points may be used. Care should be exercised in the choice to insure the feature is computable and invariant. For example, the finger tips, while easily found, are not even quasi-invariant with view because as the hand tilts down the point of occlusion moves quickly over the tip of the finger. If the resolution of the image is moderately reduced the two independent curves for middle joint, e.g. 2 and 3, will coalesce into a single feature providing 12 feature points on the hand.

[00611 According to one embodiment, the features captured are the projections of at

P C T/ U S O 5. / E! " ^» ! ^> 953 least 4 points from collinear subregion of palm-side or back-side of a hand and the project invariant values are ID cross-ratios computed from the locations of subsets of at least 4 of these feature points.

[0062] According to one embodiment, 2-D cross ratios are used as well since the object is a 2-D hand plane. Thus, the size of the feature vector will be increased, and the robustness of the technology will be improved. According to one embodiment, the 2-D cross ratio is computed based on two assumptions. The first is a coplanar five-point set, and the second is that no three points which include the reference point O should be collinear. The same 16 feature points noted above will be used again, and subsets will be identified satisfying the requirements. The first assumption is automatically satisfied because all the feature points are extracted from the hand plane. There are plenty of qualified five-point sets within the 4096 possible combinations of 16 points. Twelve example five-point layouts were selected, as shown in Figure 9. One 2-D cross ratio was obtained from each layout. In Figure 8, each five- point layout is corresponding to the five points O, A, B, C, and D. The points used in each layout are: (a) 6, 5, 9, 13, 14; (b) 12, 7, 5, 13, 15; (c) 9, 13, 14, 6, 5; (d) 15, 6, 5, 9, 13; (e) 10, 9, 13, 14, 15; (f) 6, 5, 13,14, 15; (g) 9, 13, 10, 6, 5; (h) 10, 7, 6, 5, 9; (i) 12, 11, 13, 14, 15; (j) 7, 5, 9, 10, 11; (k) 14, 12, 10, 9, 13; (1) 15, 7, 6, 5, 13. Figure 10 illustrates a qualified five- point set on the hand plane.

[0063] According to one embodiment, features captured are the projections of at least

6 points from co-planar subregion of a palm-side or back-side of a hand and the project invariant values are 2D cross-ratios computed from the 2D locations of subsets of at least 5 of these feature points.

[0064] However, as is know to those skilled in the art, cross ratios can be sensitive to positional errors, and it has been demonstrated that several pixels offset of the feature points can cause drastic changes in the cross ratio values. Positional errors are inevitable when the feature points are detected. So alternative projective invariant hand features that have stronger noise immunity may be employed. Ratios of symmetric polynomials of cross ratios derived from a five-point set are projective and permutation invariants (PPI), which means the value of a PPI feature is preserved under the projective transformations applied to the five-point set, and also independent of the permutation of the five points. However, different ordering of the five points results in different 2D cross ratio values. By fixing one of the five points as the intersection point, different cross ratios can be categorized into five groups. Each group

corresponds to one intersection point and has the permutation of the remaining 4 points. The 4! = 24 permutations in each group yield only six different cross ratios:

1 τ-1 T I ₁

⁷ I - ^T ' ^T 2 - _τ ' ^γ 3 ~ _T ' ^T 4 - _τ _ ! ' ⁷ S - ^ ₇ ' ⁷ O - ^{1 T} ' where τ is a 2D cross ratio produced by a particular five-point layout. The permutation invariant property of these six cross ratios can be expressed as:

' ^T 6 /{ _O ^ACD} where TT (•) denotes a permutation operation, and {O, A, B, C, D} are the five points determining τ,, / = 1, 2,..., 6, with point O being the intersection point. Six different cross ratios are resulted from the permutation of the remaining non intersection points A, B, C, D. A function that includes all six different 2D cross ratios τ, (/=1, 2,..., 6) with equal contribution has the advantage that that the function will have the same value under all possible permutations.

[0065] A symmetric polynomial of n variables xj, X ₂ , ..., X _n denoted as P(xi, X ₂ , ..., X _n ) remains the same value independent of any permutation of its variables and satisfies the following equation:

P(x _p x ₂ ,...,x _n ) = P{π(x _v x _2> ...,X _n ))

As described by Equation (5-3), a five-point set with one point fixed as the intersection point always generates six cross ratio values, while the permutation of the remaining four points will only change the order in which the cross ratios appear. So a symmetric polynomial using the six cross ratios X ₁ (/=1, 2,... ,6) as arguments is invariant to the permutation of the points. Thus, we have

A( ^T 1 > ^T 2 > ^τ 3 ' ^τ 4 > ^τ ₅ > ^τ 6 ){0,π(A,B,C,D)}) = ^P ( ^π { ^τ l » ^τ 2 > ^T 3 » ^τ 4 » ^τ ₅ » ^τ 6 ){O,A,B,C,D} ) = P^T ₁ , T ₂ , T ₃ , T ₄ , T ₅ , T ₆ ){o _ιλA c _, D} )

[0066] It is known that any function /having the symmetric polynomials as arguments is also invariant to the permutation {O, TT(A, B, C, D)). If we denote the original point set {O, A, B, C, D) as S, and the point set from the permutation {O, TT(A, B, C, D)) as S', then the permutation invariant property of function/ can be expressed as:

/( ^p ₁ ω/ ^» ₂ (τ _s .v.., ^p ,(τ _s .))=/( ^p _I (t _s i ^p ₂ (τ _s v-.,i!,ω)

^{is the} projective invariant ^τ s _> itself is ^a l ^{so a} projective permutation invariant (PPI). The elementary symmetric polynomials include: n

^• * 1 \ ^X l ; ^X 2'' ^•• ' ^■ ""« ) ~ / , ^X ι' ;=1

F ₂ (X ₁ , X ₂ , ■■■, X _n ) = / , X, _t X _U ,

I≤ι,≤ι ₂ ≤n

P ₁₁₁ (X ₁ , x ₂ ,...,x _n )= ∑ X, ^• x.

!<;,< • ≤i-≤n

[0067] The polynomial P _1n (X ₁ , X ₂ ,..., x _n ) is called the m-th symmetric polynomial ofx\,

...,x _n . Embodiments can use any of these as the basis for the PPI, and testing has been done using the first, the second, the third symmetric polynomials, and the second order power sum symmetric polynomial:

^J \W=∑*,= ^3,

/=i

-τ ⁶ +3τ ⁵ -5τ ³ +3τ-l

^ ₂ W= ∑ Wj = l≤i≤j≤β T^(T-I)

-2τ ⁶ +6τ ⁵ -5τ ⁴ -5τ ² + 6τ-2

P ₃ (τ)= ∑ T ₁ -T ₇ TA = i≤i≤j≤k≤6 τ ² (τ-l) ²

6 i 2τ ⁶ -6τ ⁵ +9τ ⁴ -8τ ³ +9τ ² -6τ + 2

P{τ)=∑τt ² =

M ' r ² (r-l) ²

Polynomials F '^(τ), Ps(τ), andP(r) are unbounded functions with nontrivial values. But the ratios of the polynomials are bounded. The ratio of the symmetric polynomials or the ratios of combinations of symmetric polynomials are also valid PPIs. The PPI ratios of the symmetric polynomials considered in this research include:

2τ ⁶ - 6τ ⁵ + 9τ ⁴ - 8τ ³ + 9r ² - 6τ + 2

/ _\ P ₃ (τ) _ -2τ ⁶ + 6τ ⁵ -5τ ⁴ - 5τ ² + 6τ -2 ^MT ' ^~ P ₂ (τ) ^~ - t ⁶ +3τ ⁵ -5t ³ +3τ -l ^'

I ₄ (τ) =

/ _J (T) is bounded between 2 and 2.8 as was shown by Meer et al [Meer-et-al-98], which is incorporated herein by reference. The PPIs / ₂ (τ), I ₃ (τ) and I AT) were developed by the inventors for use in projective invariant biometrics. / ₂ (τ) is bounded between 2 and 8.7, / ₃ (r) is bounded between 8 to 2744, and / ₄ (τ) is bounded between 7.4 and 5806. Figure 11 shows the response properties of these four PPIs.

[0068] Different embodiments can optimize tradeoffs between discriminatory power, computational cost and robustness to occlusion. From one 2-D cross ratio, potentially thousands of layouts can be obtained. Since they are easily distinguished, a multi-dimensional projective invariant biometric signature may be built which can handle missing features or even missing fingers. According to one embodiment, effective similarity measures incorporating Manhalanobis distance measures for each different topology are used.

[0077] Feature Vector Computation: According to one embodiment, the hand descriptor is represented by a feature vector which is composed of both 1-D and 2-D cross ratios which may also be combined into PPIs. In testing one embodiment, the feature vector V has a size of sixteen since sixteen cross ratios were computed.

F= {ψ;, ψ ₂ , ... ψ _N ], N= 16 where ψ, represents the 2-D cross ratios when z-l~12; ψ, represents the 1-D cross ratios when /=13~16. The distance, D, between two feature vectors of two hand images is calculated using the following equation:

^■ k where ψ/ and ψ/ are the /th cross ratios of the feature vectors of hand j and hand k, and

P C T . ^{■■•■'} U ψ Z/ +ψ/

[0069] Testing the cross-ratio based embodiment, a total of 31 hand pictures were collected from 14 persons. Each person was given an ID number, from 1 to 14, respectively. The pictures were also indexed. In the tables in Figures 12-15, PiJ represents the j ^th picture of Person # i. 465 different pairs were tested, one against another, with fairly promising results. Figures 12-15 show part of the experimental results. The entries indicate the distances, scaled by 1000, between the feature vectors of two hand images.

[0079] The tables in Figures 12-15 show that the pictures from the same person have small distance, and conversely, the pictures from different persons have large distances. Different tests were done based on various thresholds. When the threshold drops down to 0.007, FAR becomes zero; at the same time, FRR equals 2.8%. The Equal Error Rate (ERR) happens when the threshold is set around 0.012.

[0080] Testing for other embodiments used a variation of the feature vector normalization given by d :

d =

Hi ₂ ⁺ F ₂ Ii ₂ where ψl and ψ2 are two hand feature vectors, N is the dimension of the vectors, and ψ _x

¹ represents the ith component in the feature vector ψl. This similarity metric normalizes by the square of the vector and is may be more appropriate for some types of feature localization error. This was used to test the PPIs and compare their performance with the cross ratios, using hand data with 5000 random synthetic variations in feature location. Figure 16 shows the effects of the positional noise on hand feature vectors composed of Il(τ), I2(τ), I3(τ), I4(τ), and cross ratios, measured by the distance between a genuine model and its noisy counterpart

[0081] As already described, features for projective invariants do not need to be point or even point sets. Curves can also be used and figure 16 shows three different curves in the palm of a hand and the 4 points uses to define the canonical coordinate system for remapping

those curves. Of course any subset of 4 points could be used, though they would have

P C ^" ϊ ^" / ^' U S CP S S P '-if «ft ϊεt ^" B differing stability fn detection ^" and in remapping. While example shows "discrete curves", those skilled in the art will recognize they could use a functional form fit to the discrete curve, and also directly manipulate the resulting functional representation to allow for more efficient comparison.

[0082] Figure 17 shows a flow diagram of the steps in one embodiment for computing projective invariant-based biometric signatures, including computing cross-ratio features, PPIs and projective invariant curves. The first step in this embodiment is to match the computed point sets against patters for known qualified point sets. For the hand example used as the running example, this might mean locating the general hand orientation then choosing one of the patterns shown in figure 10. Given the qualified point sets the computation of cross ratios τ providing n features, which can also be used to compute nm different PPI features. Four of the points of the feature set can also be used as the canonical basis to define np different curve- based invariants. In total this provides for up to n + mn + np points. While there is some redundancy in the reuse of the same points multiple times, they are used in significantly different ways thereby providing improved noise resilience and allowing for a larger feature set needed for identification rather than just verification.

[0082] Having reviewed a collection of methods for computing the projective invariants and projective quasi-invariant we review the overall operation of a system using projective-invariant based biometrics. As show in Figure 18, the primary system operation is to capture the sensor data and compute the feature points from said data. Then a plurality of invariants or quasi-invariants, including at least one projective invariant is computed from subsets of the feature points. These quasi-invariant features are combined to form the biometric signature that can then be compared to stored signatures.

[0083] The enrollment process, depicted in figure 19, is similar except that a plurality of samples are computed for the subject and then tested for stability and discriminability. Enrollment would also address issues such as missing fingers or other features. For verification, it is straightforward for each individual to have a different set of features, as the only comparison is one-to-one. For. authentication/identification it is important the each subject can be compared with each entry in the database. Thus determination of which features to use could be pre-computed using a training gallery, allowing limited computation and storage, or it could a per individual weighting could be used allowing increase individualization at the cost of added storage and computation.

combines the planar projective-invariant features from a hand with the traditional biometric features from a face. A practical advantage of this multi-modal biometric is that both can be captured with a single image. The measurements from the system could be combined into a single biometric signature, or could provide two independent signatures that could be combined using any of the well-known approaches to biometric fusion, including but not limited to decision fusion, score level fusion, Borda count, Min-fusion, Max-fusion, etc. Figure 20 shows a sketch of what an acceptable data collection for such a fusion approach might produce.

[0085] REFERENCES [Keren-94]Daniel Keren, "Using Symbolic Computation to Find Algebraic Invariants",

IEEE Trans, on Pattern Analysis and Machine Intelligence, Vol. 16, No. 11, pp. 1143-

1149, 1994 [Weiss-93] Isaac Weiss, "Noise-Resistant Invariants of Curves", IEEE Trans, on Pattern

Analysis and Machine Intelligence, Vol. 15, No. 9, pp. 943-948, 1993. [Weiss-95] Isaac Weiss, "Local Projective and Affine Invariants", Annals of Mathematics and Artificial Intelligence, Vol. 13, pp. 203-226, 1995. [Meer-et-al-98] Peter Meer, Reiner Lenz, and Sudhir Ramakrishna, "Efficient Invariant

Representations", InternationalJournal of Computer Vision, vol. 26, no. 2, pp. 137-152,

1998.