The invention relates to an image analysis unit for analyzing an input image.

The invention further relates to an image processing apparatus comprising: a receiving unit for receiving an input image; an image analysis unit for analyzing the input image, as mentioned above; and - an enhancement unit for computing an output image on basis of the input image, wherein the enhancement unit is controlled by the analysis output signal of the image analysis unit.

The invention iurther relates to a method of analyzing an input image.

The invention further relates to a computer program product to be loaded by a computer arrangement, comprising instructions to analyze an input image, the computer arrangement comprising processing means and a memory.

Often, an image enhancement device comprises a linear and a non-linear sharpness enhancement unit, for e.g. peaking and luminance transient improvement (LTI), respectively. Peaking is a technique that increases the sharpness impression, particularly of textures, by boosting the higher frequencies in the 2D-spatial video spectrum. LTI improves the sharpness impression, particularly of edges in the image, by shortening the transition time between start and end points of the edge. Typically both techniques are used to enhance all image parts and than the outputs are mixed together with fixed weights, independent of the image content. Unfortunately, this approach does not lead to optimal image quality for all images. However, distinguishing between edges and texture in order to control the actual usage of both techniques is not straightforward.

It is an object of the invention to provide an image analysis unit of the kind described in the opening paragraph, which provides an analysis output signal which is an indicator for the probability of presence of edges and/or texture in the input image.

This object of the invention is achieved in that the image analysis unit comprises: a linear filter for filtering the input image into a first filtered image; a nonlinear filter for filtering the input image into a second filtered image; and - an output unit for outputting an analysis output signal which is based on comparing the first filtered image with the second filtered image.

An observation of the applicant upon which the image analysis unit according to the invention is built, is that linear filters, like e.g. a Gaussian blur kernel, blur edges and textures alike, while nonlinear filters, like e.g. median filters on the other hand preserve edges although they blur textures. Applying both filter types to the same input signal and then comparing outputs of different filtering, provides a clue as to whether the input image locally corresponds to edges or texture areas.

In an embodiment of the image analysis unit according to the invention, the linear filter comprises: - a further linear filter for filtering the input image into a first intermediate image; and a first subtraction unit for subtracting the first intermediate image from the input image, resulting into the first filtered image.

It is advantageous to determine the effect of the linear filter independent of the effect of the nonlinear filter. Preferably, the first subtraction unit is arranged to compute the absolute differences between respective pixels of the first intermediate image and the input image. The size of the difference between respective pixel values is important for the analysis. The sign of the differences between respective pixel values is less important.

In an embodiment of the image analysis unit according to the invention, the nonlinear filter comprises: a further nonlinear filter for filtering the input image into a second intermediate image; and a second subtraction unit for subtracting the second intermediate image from the input image, resulting into the second filtered image. It is advantageous to determine the effect of the linear filter independent of the effect of the nonlinear filter. Preferably, the second subtraction unit is arranged to compute the absolute differences between respective pixels of the second intermediate image and the input image.

In an embodiment of the image analysis unit according to the invention, the analysis output signal is based on a difference between the first filtered image and the second filtered image. Preferably, the analysis output signal is proportional to the difference. The analysis output signal may be computed by multiplying the difference with a predetermined constant.

Typically, the actual operation of the linear filter and of the nonlinear filter are linked. Preferably, a first aperture of the further linear filter and a second aperture of the further nonlinear filter are substantially mutually equal. A typical aperture of the filters corresponds to 5*5 pixels for a standard definition video image. In an embodiment of the image analysis unit according to the invention, the further linear filter is arranged to compute a first one of the pixels of the first intermediate image by computing a weighted sum of pixel values of the input image. Preferably the further linear filter is an infinite or finite impulse response filter. A Gaussian filter has proven to be appropriate. In an embodiment of the image analysis unit according to the invention, the further nonlinear filter is an order statistical filter. Preferably, the order statistical filter is a median filter.

It is advantageous to apply the image analysis unit according to the invention in an image processing apparatus as described above. In an embodiment of the image processing apparatus according to the invention, the image enhancement unit is arranged to perform sharpening. Preferably, the image enhancement unit is arranged to perform a substantially linear enhancement of a first portion of the input image and to perform a substantially nonlinear enhancement of a second portion of the input image, the first portion of the input image corresponding to a first portion of the analysis output signal wherein a first output of the linear filter is lower than a first output of the nonlinear filter and the second portion of the input image corresponding to a second portion of the analysis output signal wherein a second output of the linear filter is higher than a second output of the nonlinear filter. In other words, the first portion is assumed to correspond to texture and hence a substantially linear enhancement is performed. The second portion is assumed to correspond to an edge and hence a substantially nonlinear enhancement is performed. Preferably the linear enhancement corresponds to peaking and the nonlinear enhancement corresponds to LTI. In this context "substantial" means that the output is based on a weighted combination of both linear and nonlinear enhancements

wherein a first one of these enhancements is weighted substantially stronger than the second one of these enhancements.

In another embodiment of the image processing apparatus according to the invention, the image enhancement unit is a noise reduction unit. Preferably, the noise reduction unit is arranged to perform a substantially linear noise reduction of a first portion of the input image and to perform a substantially nonlinear noise reduction of a second portion of the input image, the first portion of the input image corresponding to a first portion of the analysis output signal wherein a first output of the linear filter is higher than a first output of the nonlinear filter and the second portion of the input image corresponding to a second portion of the analysis output signal wherein a second output of the linear filter is lower than a second output of the nonlinear filter. In other words, the first portion is assumed to correspond to texture and hence a substantially non- linear noise reduction is performed. The second portion is assumed to correspond to an edge and hence a substantially linear noise reduction is performed. Preferably the linear noise reduction corresponds to blurring and the nonlinear enhancement corresponds to order statistical filtering. In this context "substantial" means that the output is based on a weighted combination of both linear and nonlinear noise reduction wherein a first one of these noise reductions is weighted substantially stronger than the second one of these noise reductions.

The image processing apparatus may comprise additional components, e.g. a display device for displaying the output images. The image processing unit might support one or more of the following types of image processing:

Video compression, i.e. encoding or decoding, e.g. according to the MPEG standard.

De-interlacing: Interlacing is the common video broadcast procedure for transmitting the odd or even numbered image lines alternately. De-interlacing attempts to restore the full vertical resolution, i.e. make odd and even lines available simultaneously for each image; and

Image rate conversion: From a series of original input images a larger series of output images is calculated. Output images are temporally located between two original input images.

The image processing apparatus might e.g. be a TV, a set top box, a VCR (Video Cassette Recorder) player, a satellite tuner, a DVD (Digital Versatile Disk) player or recorder.

It is a further object of the invention to provide a method of the kind described in the opening paragraph, which provides an analysis output signal which is an indicator for the probability of presence of edges and/or texture in the input image.

This object of the invention is achieved in that the method comprises: - filtering the input image into a first filtered image; filtering the input image into a second filtered image; and outputting an analysis output signal which is based on comparing the first filtered image with the second filtered image.

It is a further object of the invention to provide a computer program product of the kind described in the opening paragraph, which provides an analysis output signal which is an indicator for the probability of presence of edges and/or texture in the input image.

This object of the invention is achieved in that the computer program product, after being loaded, provides said processing means with the capability to carry out: filtering the input image into a first filtered image; - filtering the input image into a second filtered image; and outputting an analysis output signal which is based on comparing the first filtered image with the second filtered image.

Modifications of the image analysis unit and variations thereof may correspond to modifications and variations thereof of the image processing apparatus, the method and the computer program product, being described.

These and other aspects of the image analysis unit, of the image processing apparatus, of the method and of the computer program product, according to the invention will become apparent from and will be elucidated with respect to the implementations and embodiments described hereinafter and with reference to the accompanying drawings, wherein:

Fig. 1 schematically shows an input image comprising an edge area and a texture area; Fig. 2 schematically shows an embodiment of the image analysis unit, according to the invention;

Fig. 3 schematically shows a mapping of values of the analysis output signal to probabilities for particular types of image content;

Fig. 4 schematically shows an embodiment of the image processing apparatus according to the invention, comprising an enhancement unit which is arranged to perform sharpening;

Fig. 5 schematically shows an alternative embodiment of the image processing apparatus according to invention, comprising a noise reduction unit; and

Fig. 6 schematically shows two gain factors as function of the analysis output signal.

Same reference numerals are used to denote similar parts throughout the Figures.

Fig. 1 schematically shows an input image 100 comprising an edge area 102 and a texture area 104. The edge area 102 corresponds to a region in the image representing a relatively large transition of luminance values. Typically, that means that there is a first group of connected pixels having a relatively low luminance value and there is a second group of connected pixels having a relatively high luminance value. The first group of pixels and the second group of pixels are located adjacent to each other. Typically an edge corresponds to a border of an object in the image.

The texture area 104 typically corresponds to a pattern, e.g. of a surface. Typically, that means that there is a first group of pixels having a relatively low luminance value and there is a second group of pixels having a relatively high luminance value. The pixels of the first group of pixels and the pixels of the second group of pixels are interleaved.

Fig. 2 schematically shows an embodiment of the image analysis unit 200 according to the invention. The image analysis unit 200 is arranged to determine the texture probability, i.e. the probability that a portion of the input image corresponds to texture and to determine the edge probability, i.e. the probability that a portion of the input image corresponds to an edge. The image analysis unit 200 is provided with an input image at its input connector 208. The image analysis unit 200 is arranged to provide an analysis output signal which represents the texture probability and the edge probability. The analysis output signal is output at the output connector 210.

The image analysis unit 200 for analyzing an input image comprises: a linear filter 202 for filtering the input image into a first filtered image; a nonlinear filter 204 for filtering the input image into a second filtered image; and

an output unit 206 for outputting an analysis output signal which is based on comparing the first filtered image with the second filtered image.

The linear filter 202, the nonlinear filter 204 and the output unit 206 may be implemented using one processor. Normally, these functions are performed under control of a software program product. During execution, normally the software program product is loaded into a memory, like a RAM, and executed from there. The program may be loaded from a background memory, like a ROM, hard disk, or magnetical and/or optical storage, or may be loaded via a network like Internet. Optionally an application specific integrated circuit provides the disclosed functionality. The transfer function of the image analysis unit 200 is specified in Equation 1 :

That means that the value A(x,y) of the analysis output signal for a pixel with coordinates (x,y) is a function of the linear filtered input image I _Lιnear (x, y) , i.e. the first filtered image and the nonlinear filtered input image I _NonLιmar (x, y) , i.e. the second filtered image.

Preferably, the function is a specified Equation 2: A(x,y) = a + β * (I _NonLιnear (x,y) - I _Lιnear (x,y)) (2)

That means that the value A(x,y) of the analysis output signal is proportional to the difference between the linear filtered input image I _Lιnear (x, y) and the nonlinear filtered input image I _NonLιmar (x, y) ■ α and β are predetermined constants, e.g. α =128 and β =20 for luminance values of the input pixels being in the range of [0,255].

Preferably the linear filter 202 is a high-pass filter and the nonlinear filter 204 is another high-pass filter. A preferred implementation is based a combination of low-pass filtering and subtraction. Then the linear filter 202 comprises: - a further linear filter 212 for filtering the input image into a first intermediate image; and a first subtraction 214 unit for subtracting the first intermediate image from the input image, resulting into the linear filtered input image I _Lιnear (x, y) ■

The further linear filter 212 preferably corresponds to a Gaussian filter having a kernel, i.e. aperture of 5*5 pixels. Preferably, the first subtraction unit 214 is arranged to compute the absolute difference between pixels of the first intermediate image and respective pixels of the input image.

4 _^ (^) H ^/ (^)-4 _^ (*,}0l (3)

Similarly, the nonlinear filter 204 comprises: a further nonlinear filter 216 for filtering the input image into a second intermediate image; and - a second subtraction unit 218 for subtracting the second intermediate image from the input image, resulting into the nonlinear filtered input image I _NonLιnear {x, y) ■

The further nonlinear filter 216 preferably corresponds to a median filter having a kernel, i.e. aperture of 5*5 pixels. Preferably, the second subtraction unit 218 is arranged to compute the absolute difference between pixels of the second intermediate image and respective pixels of the input image. honUnear (*» ^j θ =1 ^J ( ^X , ^j θ ^" 4-1W (*> ^j θ I ( ⁴ )

Typically, the output of the image analysis unit 200 according to invention is a two-dimensional matrix of values, having a size corresponding to the size of the input image. A particular element of the two-dimensional matrix represents a value indicating the texture probability and/or edge probability for the corresponding pixel of the input image. If the value A of the analysis output signal A{x,y) of a particular pixel is relatively low then it is assumed that the particular pixel is located in an edge area, i.e. the edge probability is relatively high. This corresponds to I _Lιnear (x,y) > 1 ^' _NonLιnear (x,y) ■

However, if the value Λ of the analysis output signal A(x,y) of a particular pixel is relatively high then it is assumed that the particular pixel is located in a texture area, i.e. the texture probability is relatively high. This corresponds to I _Lιnear {x,y) < ■ Fig. 3 schematically shows a mapping of values A of the analysis output signal A(x,y) to probabilities for particular types of image content, i.e. edge probability and texture probability. Fig. 4 schematically shows an embodiment of the image processing apparatus

400 according to the invention, comprising an enhancement unit 404 which is arranged to perform sharpening. The image processing apparatus 400 comprises: a receiving unit 402 for receiving an input signal representing input images; an image analysis unit 200 for analyzing the input images, as described in connection with Fig. 2;

an image enhancement unit 404 for computing output images on basis of the - input images, wherein the enhancement unit for 404 is controlled by the analysis output signal of the image analysis unit 200; and a display device 406 for displaying the output images. The input signal may be a broadcast signal received via an antenna or cable but may also be a signal from a storage device like a VCR (Video Cassette Recorder) or Digital Versatile Disk (DVD). The signal is provided at the input connector 414. The image processing apparatus 400 might e.g. be a TV. Alternatively the image processing apparatus 406 does not comprise the optional display device but provides the output images to an apparatus that does comprise a display device 406. Then the image processing apparatus 400 might be e.g. a set top box, a satellite-tuner, a VCR player, a DVD player or recorder. Optionally the image processing apparatus 400 comprises storage means, like a hard-disk or means for storage on removable media, e.g. optical disks. The image processing apparatus 400 might also be a system being applied by a film-studio or broadcaster. The image enhancement unit 404 comprises: a linear enhancement device 408 for linear enhancement of the input images, e.g. a peaking device; a nonlinear enhancement device 410 for nonlinear enhancement of the input images, e.g. an LTI (luminance transient improvement) device; and - a combining unit 412 for combining the output of the linear enhancement device 408 and the output of the nonlinear enhancement device 410.

The transfer function of the image enhancement unit 404 is specified in Equation 5:

O(x,y) = f(E _Lιnear (x,y),E _NonLιnear (x,y)) (5) That means that the value O(x,y) of an output pixel with coordinates (x,y) is a function of the linear enhanced input image E _Lιnear {x, y) and the nonlinear enhanced input image E _NonLιnear {x,y) . Preferably the transfer function is as specified Equation 6:

O(x, y) = g _Lιnear (x, y)E _Lιnear (x, y) + g _NonLιnear (x, y)E _NonLιnear (x, y)) (6) ^{with a fu"st} S ^{ain factor and} S _N onUnear (X y) ^a second gain factor. The first gain factor g _Lιnear (x, y) and the second gain factor g _NonLιnear (x, y) are based on the analysis output signal A(x,y) . The first gain factor g _Lιnear (x, y) is proportional to the analysis output signal A(x,y) . That means that g _Lιnear (x, y) is relatively low if I _Lιnear is

relatively high compared with I _NonLιnear (x,y) and that g _Lιnear (x,y) is relatively high if I _Lιnear is relatively low compared with I _NonLιnear (x, y) ■ The first gain factor g _Lιnear (x, y) may be as depicted in Fig. 6 and specified in Equation 7: guneariW) = 1 + MW) ^" 128)/127 (7) The second gain factor g _NonLιnear {x,y) is inversely proportional to the analysis output signal A{x,y) . That means that g _NonLιnear {x, y) is relatively low if I _Lιnear is relatively low compared with I _NonLιnear {x,y) and thatg _NonLιnear (x,y) is relatively high if I _Lιnear is relatively high compared with I _NonLιnear (x,y) . The second gain factor g _NonLιnear (x,y) may be as depicted in Fig. 6 and specified in Equation 8: g _NonLιnear (x,y) = 1.5 + (128 - A(x,y))/S5 (8)

The image enhancement unit 404 is arranged to process the texture areas different from the edge areas. That means: less nonlinear enhancement (e.g. LTI) and more linear enhancement (e.g. peaking) for texture areas resulting in increased perceived sharpness; and - more nonlinear enhancement in edge areas, to increase the steepness of the edge and less linear enhancement in edge areas, to prevent relatively large over/under shoots appearing near edges.

Fig. 5 schematically shows an alternative embodiment of the image processing apparatus according to invention, comprising a noise reduction unit 502. The image processing apparatus 500 comprises: a receiving unit 402 for receiving an input signal representing input images; an image analysis unit 200 for analyzing the input images, as described in connection with Fig. 2; an image enhancement unit 502 for computing output images on basis of the input images, wherein the enhancement unit for 502 is controlled by the analysis output signal of the image analysis unit 200; and a display device 406 for displaying the output images. The input signal may be a broadcast signal received via an antenna or cable but may also be a signal from a storage device like a VCR (Video Cassette Recorder) or Digital Versatile Disk (DVD). The signal is provided at the input connector 414. The image processing apparatus 500 might e.g. be a TV. Alternatively the image processing apparatus 406 does not comprise the optional display device but provides the output images to an

apparatus that does comprise a display device 406. Then the image processing apparatus 500 might be e.g. a set top box, a satellite-tuner, a VCR player, a DVD player or recorder. Optionally the image processing apparatus 500 comprises storage means, like a hard-disk or means for storage on removable media, e.g. optical disks. The image processing apparatus 500 might also be a system being applied by a film-studio or broadcaster.

The image enhancement unit 502 comprises: a linear noise reduction device 508 for linear noise reduction of the input images, e.g. a blurring device; a nonlinear noise reduction device 510 for nonlinear noise reduction of the input images, e.g. an order statistical filter; and a combining unit 512 for combining the output of the linear noise reduction device 508 and the output of the nonlinear noise reduction device 510.

The transfer function of the image enhancement unit 502 is specified in Equation 9: O(x,y) = f{N _Lιnear {x,y),N _NonLιnear {x,y)) (9)

That means that the value O(x,y) of an output pixel with coordinates (x,y) is a function of the linear noise reduce input image N _Lιneαr (x,y) and the nonlinear noise reduced input image N _NonLιneαr (x,y) . Preferably the transfer function is as specified Equation 10:

O(x, y) = g _NonLιneαr {x,y)N _Lιneαr {x,y) + g _Lιmαr (x,y)N _NonLιmαr (x,y)) (10) with g _Lιneαr (x, y) the first gain factor and g _NonLιneαr (x, y) the second gain factor as depicted in

Fig. 6 and specified in Equation 7 and 8, respectively.

The image enhancement unit 502 is arranged to process the texture areas different from the edge areas. That means: less nonlinear noise reduction (e.g. median) and more linear noise reduction (e.g. blurring) for edge areas; and more nonlinear noise reduction and less linear noise reduction in texture areas, to prevent loss of detail.

Fig. 6 schematically shows two gain factors as function of the analysis output signal A(x,y) . On the x-axis the range of values A of the analysis output signal A(x,y) are indicated. On the y-axis the range of values of the gain are indicated. In Fig. 6 the first gain factor g _Lιneαr {A) as specified in equation 7 and the second gain factor g _NonLιneαr {A) as specified in equation 8 are depicted.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be constructed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements and by means of a suitable programmed computer. In the unit claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words are to be interpreted as names.