ID motion compensated dynamic frame insertion

FIELD OF THE INVENTION

The invention covers improvements on the Dynamic Frame Insertion technique that doubles the picture rate of a video stream, in order to reduce the motion blur due to the LCD's sample and hold principle.

BACKGROUND OF THE INVENTION

LCD displays typically suffer from a large motion blur. This is caused by the limited response time of the LC material and the sample-and-hold principle of the LCD. The contribution of the sample and hold effect can be reduced by using higher refresh rates in the display. Various techniques for generating the extra frames to achieve the higher refresh rate have been disclosed, such as black field insertion, grey field insertion and dynamic frame insertion. The latter one (DFI) has been disclosed by Kaneda Sadafumi, JP 2002351382A. See also Han-Feng Chen, et. al., Smooth Frame Insertion Method for Motion-Blur Reduction in LCDs, Samsung Electronics Co., Ltd., EuroDisplay 2005. DFI means doubling the frame rate of a video stream by generating a picture pair including a sharp picture and an unsharp picture from each source picture. The unsharp picture is a low passed filtered (blurry) version of the source picture. The sharp picture is such that average of the sharp and unsharp picture corresponds to the source picture again. If the sharp and unsharp are switched fast enough, then the result is still perceived with the correct intensity, without seeing any flicker. The details (in the sharp picture) are only "held" for half of the time, as they are not visible in the unsharp picture. This causes the intended motion blur reduction.

SUMMARY OF THE INVENTION It is an object of the invention to provide an improved video data processing.

The invention is defined by the independent claims. Advantageous embodiments are defined in the dependent claims.

In one embodiment, the motion compensated DFI as described here is an extension on the base principle of the prior art DFI, by using among others a motion

controlled technique to improve the overall picture quality performance. The motion compensation techniques as such are derived from known motion compensated temporal up- conversion methods.

Embodiments of the invention are based on the recognition that the basic DFI solution disclosed by Kaneda Sadafumi has the following problems:

1) In certain areas the difference between the unsharp picture and source picture cannot be compensated for in the sharp picture. This is because of the limited color range. Thus, the clipping in the sharp picture has to be compensated for in the unsharp picture again. This causes high frequencies in the unsharp picture, and thus details that are still being "held" at the same location, which was meant to be avoided in order to reduce the motion blur.

2) Other picture improvement processing in the video chain, amongst others the overdrive in the LCD panel to improve the response time, is typically not designed / tuned to deal with continuously oscillating pixel intensities as caused by DFI. This can lead to unwanted interference and side effects. In embodiments of the invention, the improvements to address these problems include using a ID filtering only, along the direction of the motion of the video stream content, and motion compensation (spatially moving the content) of the DFI-ed output, i.e. the image pair (LP", IN+HP") resulting from the DFI algorithm using the ID filtering. Applying motion compensation requires adjusting one of these two output images. Applying motion compensation on the unsharp output picture is preferred because it requires less intermediate storage (as IN+HP" needs to be computed before LP" and can be used as previous image during upconversion) and it balances the load more even (calculating IN+HP" requires more compute power than calculating LP").

Both problems are reduced by the following insight: only apply a low pass filtering in the direction of the motion. The rationale behind this is that the extra blur orthogonal to the motion only adds up to the "modulation depth" (i.e. the difference between the unsharp value and the source value). It thus one hand enlarges the chance of clipping in the sharp image and interference with overdrive and other picture improvement features, while on the other hand it does not contribute to the basic problem intended to be solved, i.e. the motion blur.

In an embodiment, problem 1 is solved by applying motion compensation on the unsharp image. This means that the pixels in the unsharp image are "moved" according the calculated motion of the objects in the picture.

Problem 2 is further reduced by modulating the DFI strength (amount of low pass filtering) by the amount of detected motion and/or the source pixel intensity. In order to compensate for errors and irregularities in the detected motion, some spatial and temporal low pass filtering can be applied to the motion vectors, before using them. These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 illustrates a first embodiment of the invention.

DESCRIPTION OF EMBODIMENTS

Fig. 1 illustrates a first embodiment of the invention. In operation, it performs the following functions:

1) On the input pictures of incoming video data I motion estimation ME is applied. This delivers motion vectors MV, at the granularity of pixel blocks, typically 8x8. The method for doing motion estimation falls outside the scope of this invention.

2) On the input picture I a low-pass filter LPF a one-dimensional gaussian blur is applied resulting in uncorrected unsharp pixel values LP. The gaussian blur is in the direction of the determined motion and having the blur strength (i.e. the standard deviation of the gaussian blur) modulated by the vector length, with a certain maximum being 4.5 in the implemented application. This upper limit depends on the response time of the target LCD panel. For a faster panel higher motion can be handled requiring a larger maximum gaussian blur. Also implementation cost determines the maximum allowed blur kernel.

3) A subtracter Sl determines an uncorrected modulation depth HP by subtracting the uncorrected unsharp pixel values LP from the source pixels of the input picture I. In order to get a blur in the direction of the motion, a horizontal and a vertical ID gaussian blur is calculated. These are mixed according to the horizontal and vertical vector components.

4) For the implemented application the target LCD panel shows interference with the overdrive limitations (resulting in visible artefacts) for especially darker source pixels. To resolve this, the modulation depth HP is reduced (clipped) by clipper Cl based on the source intensity. The clip factor is the source intensity, scaled by a certain ratio. This ratio is to be tuned to the final panel characteristics. After clipping we have a corrected modulation depth HP'.

5) An adder Al adds the corrected modulation depth HP' to the source pixels I to obtain the undipped sharp values I+HP'.

6) These undipped sharp values I+HP' are clipped again by clipper C2 to get clipped sharp values I+HP" in the valid pixel intensity range. The result is a sharp output picture SO.

7) A substracter S2 derives clip-corrected unsharp pixel values LP" by subtracting the clipped sharp values I+HP" from two times the source pixels of the input picture I. This guarantees the average intensity of the sharp and unsharp to be equal the source at all times. 8) Finally the unsharp values LP" are motion compensated by motion- compensation circuit MC. This is done by sub-pixel accurate backward mapping. For each motion compensated output a linear interpolated pixel is fetched from the non-compensated unsharp values, using the motion vector as addressing means. In the implemented application, the motion vectors have 2 fractional bits for sub-pixel accuracy, which is essential to reduce detail flicker in case of high frequencies in the unsharp output. The result is an unsharp output picture UO.

Note that ideally the embodiment is working in the correct color domain (linear light) when mixing the sharp and unsharp signals. Nevertheless, if not (gamma domain), the error that is made can be neglected.

The second embodiment is like the first one, with the simplification of doing horizontal processing only. This means that:

Only the horizontal components of the motion vectors are taken into account.

The gaussian blur is always horizontal only. - The motion compensation of the unsharp values is horizontal only.

The rationale behind this simplification is that it can save a significant amount of implementation cost, while still being effective for the majority of the use cases. Namely, statistically most video sequences that clearly benefit from the DFI concept are (near) horizontal pans. The third embodiment is like the first one, with the addition of a vector low- pass filter unit. This unit applies a small spatial and temporal low pass filtering on the motion vectors in order to avoid too fast transitions in the local amount of applied DFI that can cause some rest-flicker. In the implemented application the low pass is made by adding up the four

neighbor vectors by two times the current vector and two times the previous vector at the designated position.

The fourth embodiment is like the first one, with the addition of a vector interpolation unit. This unit will smooth the block based vector field by means of bi- linear interpolation per pixel.

The fifth embodiment is like the first one, where the source is first motion compensation and then blurred, instead of vice versa.

The improvement discussed in US application 60/821861 (Attorneys' docket PH006561), viz. a luminance-dependent mixing of input and the low-pass filtered (i.e. gaussian blurred) image, can advantageously be combined with the above described embodiments. The same holds for the improvement discussed in US application 60/821863 (Attorneys' docket PH006562), viz. to apply a motion-compensated interpolation to movie film originated images prior to the improved DFI described above. Advantageously, the same motion vector estimation / motion compensation hardware us used for the improved DFI and the motion-compensated interpolation to movie film originated images.

The invention can be applied in video processing pipelines for TV systems, e.g. backend-scaler ICs on the small-signal board of an LCD TV or TCON ICs on the LCD panel itself.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and/or by means of a suitably programmed processor. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.