Background of The Invention LCD panels are commonly used in electronic devices to display graphics and texts for conveying messages to the user. In an LCD panel, an LCD cell may be physically damaged thus displaying an undesirable intensity output. An LCD cell may also display an undesirable intensity due to faulty wiring or damaged memory chip.

For quality control, LCD panels are checked to ensure proper operations. Presently, inspection of LCD panel is commonly based on human operator. The main advantage of human inspection is its versatile capability towards various types of LCD defects.

However, with human operators, the consistency of inspection may change from time to time. There will be rare occasions when small defects are missed by a tiring eye. Therefore, there is a need to develop an inspection system to automatically detect for defects on an LCD panel.

The majority of automated LCD inspection methods have been relying upon electrical checking of LCD cells. An example of such a method is disclosed in U. S. Pat. No. 5,528,163 (Takahashi). This method requires the LCD to be in a state before assembly for electrical

probing. When the LCD panel is finally assembled and integrated with the main electronic device, another test will be required to check for proper integration. In this patent, we will focus on the visual inspection of LCD panel on the final integrated product.

One technique of LCD inspection is pattern comparison. Two images, one captured of an ideal LCD and the other of an LCD under inspection, are subtracted to obtain the difference. Any discrepancy between the two patterns indicates a defect in the LCD set, be it an LCD cell defect or an integration fault. One critical issue involved in this step is the miss-placement of the LCD set on the inspection platform. The miss-placement of LCD set will cause pattern miss-alignment between the two images under subtraction, thus leading to false alarms.

Many automated visual LCD inspection systems assume negligible placement error of the LCD panel and provide no compensation for it. One example of such a method is disclosed in U. S. Pat. No.

4,870,357 (Young et al.). It is costly and technically difficult to ensure placement repeatability to a high precision of LCD sets on an inspection platform. Therefore, it is desirable to align patterns for subtraction instead of using precision placement equipment.

The method disclosed in U. S. Pat. No. 4,942,539 (McGee et al.) is a general method of pose determination of 3-D objects. This method can be applied to the current problem of LCD miss-placement by determining the pose of the LCD set under inspection and

reconstructing its 2-D pattern from the image. However, this method requires a calibration phase for the camera which involves a high precision 3-D structure. This structure is costly to build and the whole system requires careful setting up.

The U. S. Pat. No. 5,696,550 (Aoki et al.) discloses a method of pose and perspective correction using linear interpolation.

Linear approximation of the non-linear perspective transformation is employed. This method is efficient but contains approximation error.

The patterns of two images involved in subtraction need to be aligned on a common reference frame before subtraction is carried out. The alignment process requires the consideration of the effects of object pose and image perspective. Next, the residue of subtraction needs to be analyzed to differentiate between regions of pattern difference from regions of no pattern difference, and between actual residues (due to pattern discrepancy) from false residues (due to image miss-alignment and noise).

Disclosure of The Invention <BR> To correct for the effects of pose and perspective, the LCD pattern in an image is mapped to an ideal rectangle. A camera model which relates a world point to its image point is used to derive the above mapping. The ideal rectangle, which is specified in the world coordinate, provides the common reference frame for image comparison. All images are transformed from the image coordinate to the common reference frame before subtraction.

An error score is evaluated at every point on the resultant image of image subtraction. To account for subtraction residue due to slight miss-alignment of images, the contrast information of the original image pattern is used to suppress the error scores around high contrast regions.

That is, The present invention is a method for detecting defects in the pattern displayed by a liquid crystal display (LCD), comprising the steps of: generating a pattern on said LCD; generating a camera image containing the whole panel of said ;LCD determining the locations of at least four non-collinear feature points in said camera image; determining the transformation parameters between the coordinate system of said camera image and a 2-D coordinate system on a plane in the 3-D space; transforming said camera image to said plane; subtracting the transformed image with a template pattern to produce an error image; and evaluating the residue in said error image for any pattern defect.

The present invention is also directed to a vision system for detecting defects in the pattern displayed by a liquid crystal display (LCD), said system comprising: an LCD panel driver for generating patterns on said LCD; a CCD camera for generating camera images containing the whole

panel of said LCD; means for determining the locations of at least four non- collinear feature points in said camera image; means for determining the transformation parameters between the coordinate system of said camera image and a 2-D coordinate system on a plane in the 3-D space; means for transforming said camera image to said plane; means for subtracting the transformed image with a template pattern to produce an error image; and means for evaluating the residue in said error image for any pattern defect.

Other embodiments of the present invention are as follows: i) In the method or system as described above, said feature points are the corner points of the frame of said LCD. ii) In the method or system as described above, said plane is a plane parallel to the XY-plane of the 3-D space. iii) In the method or system as described above, said XY-plane is a plane that coincides with said LCD panel. iv) In the method or system as described above, said template pattern is generated by said transformation or said means for transformation from a camera image of an ideal LCD pattern to said plane. v) In the method or system as described above, said step of subtraction or said means for subtraction takes the absolute

value of difference as the result of processing. vi) In the method or system as described above, said evaluation step or said means for error image evaluation includes the step of calculating the contrast information for the pixels in the template image.

Brief Description of The Drawings <BR> Figure 1 shows a schematic diagram of the system used in the current invention.

Figure 2 shows the flowchart of the current inspection algorithm.

Figure 3 shows a corner of the LCD panel in an image for visualizing the process of corner point detection.

Figure 4 shows the mapping between an LCD pattern in an image and a rectangle representing an ideal LCD panel.

Preferred Embodiments of The Invention Before inspection is put on-line, all template patterns produced by a good LCD panel are captured. These images will be compared against that produced by an LCD panel under inspection. The images captured of an LCD panel under inspection will be referred to as test images. The pose of the LCD set under inspection may not be secured on the inspection platform. Therefore, both template and test images under comparison must be transformed into a common reference frame for alignment before subtraction. After image subtraction, an image of the residue errors is produced. This

image is referred to as the error image. A scoring formula which measures the extent of residue error is evaluated at every pixel on the error image. The pixels with score values higher than a pre-set threshold level are determined to be defective.

Figure 1 shows the system set-up for the current invention. An LCD panel 10 is placed on an inspection platform 16, facing a color camera 13. A computer 15 instructs an LCD panel driver 12 to display a pattern on the LCD panel 10. The displayed pattern is captured by the camera 13 and sent to the computer 15. The computer 15 executes the inspection program to detect for any defect on the LCD panel 10. Template images for comparison are stored in a storage unit 14. The flowchart of the inspection algorithm is shown in Figure 2. The processes in the flowchart are described below.

Corner Detection (process 20) To derive the mapping between the image coordinate and the common reference frame, the four corner points of the LCD panel are located in the image and mapped to an ideal rectangle. For corner point detection, the LCD panel is lit with a uniform white pattern.

The case around the LCD will appear relatively dark. Thus, the margins of the LCD panel can be easily detected. The corner points are found by determining the two line equations of the two sides of the LCD panel that form a corner, followed by calculating the point of intersection between these two lines. In Figure 3, S1 and S2 form the two sides of a top-left corner. Each side of the LCD panel is determined by finding two points on it, such as points A and B for the side S1. Each point is detected by searching for

an intensity change along a line positioned to intersect the expected side of the LCD panel. These two points determine the equation of a line that passes through them. For better accuracy, the two points that form a side of the quadrangle are positioned close to the corner point. This is done to minimize the error caused by image distortion due to imperfect lens.

DLTParameters Calculation (process 21) The technique of image alignment is based on the direct linear transform (DLT) model. A reference to this calibration technique can be found in Y. I. Abdel-Aziz and H. M. Karara,"Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry", Proc.

Symposium on Close Range Photogrammetry, Urbana, Illinois, pp. 1-18, January 1971.

The DLT model tells how a point in 3-D space is mapped to a point on the image plane. By restricting the 3-D object to the XY-plane (i. e. taking Z=O), since the LCD panel is flat, the DLT mapping is expressed as: AlIX+AIZY+A14 A3lX + A32Y + 1<BR> <BR> <BR> <BR> <BR> <BR> <BR> AX+AY+1<BR> <BR> <BR> A31X+A32Y+1 where (u, v) refers to an image coordinate and (X, Y) refers to its

corresponding world coordinate on the XY-plane. Referring to Figure 4, the quadrangle 31 is the LCD pattern captured in a camera image. Each corner point detected on the image maps to the corresponding corner point of a rectangle 30 representing an ideal LCD panel. The coordinate of rectangle 30 is specified on the XY-plane. The XY-plane provides the common coordinate system for image subtraction. The dimensions of this rectangle is defined by the user. For ease of manipulation, the dimensions of this rectangle are set to the integer numbers of horizontal (W) and vertical (H) pixels that map to an image buffer. Ideally, the numbers of horizontal and vertical pixels in the rectangle should be equal to the resolution of the LCD panel.

There are eight parameters to be determined in equations (1) and (2); they are All, A12, A14, A21, A22, A24, A3land A32. The four mappings of the corner points between the camera image and the ideal rectangle on the XY-plane will produce eight equations fromkthe DLT equations. This allows the eight DLT parameters to be determined. In general, at least four non-collinear feature points are required to solve for the eight parameters.

Image Transformation (process 22) Given an image point (u, v), the terms X and Y can be rearranged to the left-hand-sides of equations (1) and (2) as: (u - A14)(A22 - vA32) - (A12 - uA320(v - A24)<BR> x = (3)<BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> (A11 - uA31)(A22 - vA32) - (A12 - uA32)(A21 - vA31)<BR> y-'24)(Au-uA,i)-(A,i-vA,i)(u-Ai,) (All-uA (A22-VA32)- (A12-UA32) (A21-VA31)

These two equations form the reverse transformation equations for transforming an image point to a point on the XY-plane. All images involved in subtraction are subject to this transformation.

Image Subtraction (process 23) The transformed images of the template image and the test image are subtracted pixel by pixel to obtain the error image. To avoid negative numbers, the absolute value of subtraction is taken as the residue.

Error Score Calculation(process24) Due to system noise and numerical error of computation, the two patterns under subtraction may not be perfectly aligned. This slight miss-alignment of patterns will produce small error residues on the error image at the pixel locations of high intensity contrast. Therefore, the judgement of error residues must take this into account and tolerate for the error residues which appear at high intensity contrast locations where the error residues may be contributed by pattern miss-alignment.

Every pixel on the error image is given an error score value indicating the extent of the difference at that pixel location between the two images involved in the subtraction process. Any error score higher than a pre-set threshold level indicates a pattern defect. The error score is defined as: Score (u, v) =max {hR (u, v)-acR (u, v), h (u, v)-ac (u, v), hH (u, v)- ac. (u, v)} (5)

where a = 2, a scaling factor. hR (u, v) = The magnitude of the error residue of the red band at coordinate (u, v). hG (u, v) = The magnitude of the error residue of the green band at coordinate (u, v). h, (u, v) = The magnitude of the error residue of the blue band at coordinate (u, v). cu,v)=max..{lR(u,v)-N(u,v)} cG(u,v)= maxi=1,...,8{IG(u,v) - NG,i(u,v) } CB(u,v) = maxi=1,...,i{IB(u,v) - Nb,i(u,v) }, where IR(u,v) = The red intensity level at coordinate (u,v) on the template image.

IG (u, v) = The green intensity level at coordinate (u, v) on the template image.

IB (u, v) = The blue intensity level at coordinate (u, v) on the template image.

NR, i (u, v) = The red intensity level of the i-th connected neighbour of (u, v) on the template image.

NG, i (u, v) = The green intensity level of the i-th connected neighbour of (u, v) on the template image.

NB, i (u, v) = The blue intensity level of the i-th connected neighbour of (u, v) on the template image.

The error score defined in (5) defines a quantitative measurement of pixel error, taking into consideration of the intensity discrepancy for the red, green and blue bands and the intensity contrast around a pixel. The terms CR (u, v), Cs (u, v) and CB (u, v) serve as measurements of intensity contrast around a pixel. These

terms reduce the magnitudes of the error scores at pixel locations of high intensity contrast. This reduces the possibility of false alarms due to pattern miss-alignment.

Possible Utilisation in Industry of The Invention This invention provides a general method and system for LCD inspection by pattern comparison. The module on image alignment allows the LCD under inspection to be placed loosely on the inspection platform. This module also corrects for perspective effect and produces a 2-D image of the pattern that appears on the LCD panel.