The present invention relates generally to video processing, and to systems and methods for providing mitigating noise and/or blocking artefacts in video images. Video images are susceptible to a variety of undesirable noise and/or artefacts, which can result from characteristics of video data processing and displays used to display an image corresponding to the video data. For instance, video data is often compressed or otherwise processed for transmission, and then decompressed or otherwise processed for display. In addition, video data is often up-scaled or otherwise configured for use in generating an image on a particular type of display.

Noise reduction of image data has been realized using spatial and/or temporal filters that are based upon averaging likely-correlated picture elements. For the reduction of coding artefacts spatial or temporal filters have been used. These filters are generally designed specifically to reduce blocking artefacts or ringing, and are often based on detecting the edges of the blocks. However, previous approaches are susceptible to generating video exhibiting undesirable noise and/or artefacts, where image sharpness and quality is preserved.

These and other issues remain challenging to processing video for display, and in particular to presenting video images that are relatively noise and/or artefact-free. Various aspects of the present invention are directed to methods and systems for processing video in a manner that addresses challenges including those discussed above.

Example embodiments of the present invention are directed to pre-filtering video data by identifying, from within the video data, image regions such as pixel blocks that are centered upon one or more grids in the image, and by pre-filtering the identified regions to mitigate blocking artefacts that may be associated with the regions.

According to another example embodiment, video data representing images characterized by horizontal and vertical grids is processed as follows. For each image, a block grid detector is used to identify regions of the image that are centered on a horizontal grid or on a vertical grid. For each of the identified regions, video data representing central sub-regions therein is filtered to mitigate blocking artefacts.

According to another example embodiment, a system processes video data representing images characterized by horizontal and vertical grids. The system includes a block grid detector arrangement and a filter. The block grid detector arrangement identifies regions of each image that are centered on a horizontal grid or on a vertical grid. The filter arrangement filters video data representing central sub- regions within each identified region to mitigate blocking artefacts therein.

In a more particular example embodiment, a video processing system processes a compressed video bitstream for video data characterized by horizontal and vertical grids. A block grid detector arrangement identifies horizontal and vertical grids from the bitstream, decodes the bitstream, and outputs decoded video data together with information characterizing the identified grids. A spatial analyzer detects artefact conditions for each pixel block centered on one of the identified grids. A pre-filter arrangement uses the identified grids and detected artefact conditions to pre-filter the pixel blocks that centered on a grid in order to mitigate blocking artefacts therein. A secondary filter arrangement receives an output from the pre- filter arrangement and filters the decoded pre-filtered video data. That is, the secondary filter arrangement filters video data corresponding to the decoded video data, with certain data corresponding to the aforesaid centered pixel blocks being replaced with the filtered data.

The above summary is not intended to describe each embodiment or every implementation of the present disclosure. The figures and detailed description that follow more particularly exemplify various embodiments. The invention may be more completely understood in consideration of the following detailed description of various embodiments of the invention in connection with the accompanying drawings, in which:

FIG. 1 shows a system for filtering video data, according to an example embodiment of the present invention; FIG. 2 shows another system for filtering video data, according to another example embodiment of the present invention; and FIG. 3 shows a video processing and display system for multi-stage, grid- based filtering of incoming video data, according to another example embodiment of the present invention.

While the invention is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit the invention to the particular embodiments described. On the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the scope of the invention including aspects defined in the claims. The present invention relates to systems and methods for processing video in a manner that mitigates noise and/or blocking artefacts. While the present invention is not necessarily limited to such approaches, various aspects of the invention may be appreciated through a discussion of examples using these and other contexts.

In connection with various example embodiments, a video data processing approach involves mitigating white noise having Gaussian distribution and digital coding artefacts such as blocking, ringing and mosquito noise. This approach involves identifying regions of an image having a smaller centrally-located sub-region therein located at one or more grid lines of the image. In some applications, these regions include a plurality of pixels (e.g., a block of pixels), and the sub-regions include one (or a few) of the pixels located near the center of the plurality of pixels or pixel block. In some embodiments, this integrated approach is used for processing digitally encoded signals that have been converted from previously-transmitted signals of analog origin. This approach is also applicable to processing a variety of video types, including the processing of bitstream video data. According to another example embodiment, a block grid detector is used to identify information about the position of the horizontal and the vertical grid for an image, and about the size and the offset of the grid for vertical and horizontal directions of the image. This information is used to mark the pixels in the image, on which an additional low pass filtering is applied in the following manner. For a given pixel, which is marked as positioned on a vertical grid and its direct neighboring pixels on the same line, a horizontal low pass filter is applied. Similarly, a vertical low pass filter is applied to the pixels marked as positioned on a horizontal grid and to their direct neighboring pixels on the previous line and the next line of the image. The low pass filtered pixels are used to replace the appropriate pixels of the neighborhood of pixels (e.g., as used in a discriminating averaging filter). In some embodiments, this horizontal and vertical low-pass filtering approach is carried out to pre-filter a center pixel and only the direct neighboring pixels of a region in an image, when the centre pixel is situated on a horizontal or a vertical grid of the image. This approach facilitates a reduction of blocking artefacts, while preserving small details due to limiting the pre-filtering approach to the center (and surrounding) pixels. That is, pixels located near an edge of such a region are not processed via such low-pass filtering, maintaining sharpness and details corresponding therewith.

Various circuits, devices and/or software (and corresponding algorithm) approaches can be used to effect the grid detection and filtering as discussed above. For instance, a block grid detection approach that may be amenable to implementation with the present invention is described in R. Muijs, I. Kirenko, "A no-reference blocking artefact measure for adaptive video processing", submitted to EUSIPCO 2005, 13nd Eur. Signal Proc. Conf. (Antalya, Turkey) and fully incorporated herein by reference. Discriminating averaging filters amenable to use in connection with one or more example embodiments described herein may be implemented in accordance with one or both U.S. Patent No. 5,715,335 to de Haan et al, and to U.S. Patent No. 7,162,099 to Ojo, et al., both of which, and any priority documents cited therein, are fully incorporated herein by reference.

In some embodiments, the spatial filters described in U.S. Patent Nos. 5,715,335 and 7,162,099 are used to filter white Gaussian noise as well as digital coding artefacts as ringing and mosquito noise. The above grid-based horizontal and vertical low-pass filtering approach is applied with these noise and artefact filtering approaches, to facilitate desirable filtering of blocking artefacts and at the same time preserve sharpness and fine details. In some implementations, a low-pass filter is applied as a pre-filter, only on the pixels situated on the vertical and the horizontal grid and to the direct neighbors to reduce the blocking artefacts. Subsequently, a filter such as described in U.S. Patent No. 7,162,099 is applied to reduce ringing and mosquito noise.

Turning now to the figures, FIG. 1 shows a system 100 for filtering video data, according to another example embodiment of the present invention. The system 100 includes a block grid detector 110, an extended integrated filter 120 and a controller 130. An input signal 105 is analyzed at the block grid detector 110, which generates information about the size and the offset of horizontal and vertical block edges in the input signal and, in some instances, generates information characterizing the visibility of artefacts. For instance, where the input video signal is a compressed bitstream, the detector 110 can detect the position of block grid data, provide a decoded video signal to the filter 120, and provide visibility data to the controller 130 by, for example, extracting and analyzing corresponding coding parameters from an input compressed bitstream.

The block grid detector 110 generates both horizontal and vertical size and offset data, which are provided to the integrated filter 120 at 112. The integrated filter 120 pre-filters pixels located on both sides of a horizontal or vertical block edge, using the information provided at 112. The input signal with pre-filtered pixels is then provided at an output 140, which can be provided to additional filtering processes (e.g., to a discriminating averaging filter as described above). In applications where the block grid detector 110 generates and/or extracts information about the visibility of artefacts in images corresponding to the input signal 105, the detector provides the visibility information to a controller 130 for controlling the extended integrated filter 120. This visibility information may, for example, provide an indication of artefact visibility to the human eye for the video data as it would be displayed. The controller 130 uses this information to generate and provide control data such as a filtering threshold, aperture size and shape, and/or strength of blurring to the filter 120.

In some embodiments, the block grid detector 110 performs local spatial analysis around a center pixel for the input signal 105 to differentiate regions of the image. This analysis may involve one or more of detecting spatial activity, the presence of object edges, analog noise, film grain or other image characteristics. In a more particular approach, the block grid detector 110 performs local spatial analysis by differentiating three types of regions, respectively including high activity (texture) regions, flat regions and neighborhood of strong edges (e.g., as a potential source of ringing artefacts). The block grid detector 110 provides visibility data characterizing these types of regions (and corresponding artefact visibility) to the controller 130.

While shown as a single block, the block grid detector 110 may be implemented with separate components, such as to respectively perform block grid detection and local spatial analysis (e.g., the block grid detector 110 includes a detector circuit and a spatial analyzer circuit). In addition, the respective operations of the block grid detector 110 (as shown or separated) may be carried out on different domains. For instance, block grid detection can be carried out on a compressed bitstream, and local spatial analysis may be carried out upon raw pixel data. In this regard, the example block grid detector 110 shown in FIG. 1, as well as similar examples shown in the other figures (e.g., 210, 320), may be implemented with separate components or circuits.

For image regions near strong object edges or in flat areas, the controller 130 can set a filter threshold to a higher value, relative to the threshold as set for image regions exhibiting relatively high spatial activity. The threshold is correspondingly (and relatively) reduced in regions having strong object edges, where the visibility of a grid is generally lower due to a high local spatial activity of ringing or mosquito noise around the grid pixels. Other filtering parameters such as those described above may be similarly controlled to facilitate desirable video processing in each instance. This combination of using strong edge detection with a grid visibility metric (as provided by detector 110) is used to smooth image regions where ringing masks the block grid (i.e., reduces grid visibility).

In various example embodiments, the approaches described above and/or shown in FIG. 1 are applied to broadcast signals. For instance, where a digital television signal is broadcast over an analog channel, the above approach is implemented to mitigate noise corresponding to the analog transmission, while also mitigating digital artefacts. With this approach, undesirable image characteristics corresponding to both analog and digital are addressed, while maintaining desirable image quality.

FIG. 2 shows another system 200 for filtering video data, according to another example embodiment of the present invention. The system 200 can be operated in a manner similar to that described above in connection with FIG. 1 and otherwise.

The system 200 includes a block grid detector 210, an extended integrated filter 220 and a controller 230 (e.g., similar to that described above), and further includes a low-pass filter 215. The block grid detector 210 generates information characterizing the visibility of artefacts in images corresponding to the input video signal 205 (e.g., a bitstream as described above), and provides that information to the controller 230. In response to the visibility information, the controller 230 generates a set of filter control parameters, and provides the parameters to the low-pass filter 215 and integrated filter 220.

The block grid detector 210 generates both horizontal and vertical size and offset data (as above), which are provided at 212 to the low-pass filter 215. The low- pass filter 215 pre-filters pixels located on both sides of a horizontal or vertical block edge and passes these values to the integrated filter 220. The signal with filtered pixels is then provided at an output 240, which can be provided to additional filtering processes (e.g., to a discriminating averaging filter as described above). FIG. 3 shows a video processing and display system 300 for multi-stage, grid- based filtering of an incoming compressed video data bitstream from a video source 305, according to another example embodiment of the present invention. A filtering circuit 310 selectively pre-filters pixels of the bitstream that are located on or near grid lines to mitigate blocking (digital) artefacts, prior to filtering the incoming video data for noise.

The filtering circuit 310 includes a block grid detector 320 that identifies vertical and horizontal grid lines of an incoming video bitstream, decodes the compressed bitstream and provides the decoded bitstream and information regarding these grid lines to a filter 340. The block grid detector 320 also provides artefact visibility data 322 (e.g., as described above) to a low-pass filter controller 330, which uses the visibility data to generate filter control data 332 to control operational parameters of the filter 340 (e.g., to control parameters as relating to a threshold, aperture size and shape, strength of blurring or other characteristics).

The filter 340 passes pre-filtered video data 342 to a noise-based filter 350, which generally operates upon all of the pixels in the video data. The filtering circuit 310 provides the filtered output of the noise-based filter to a video display 370, which displays video using the filtered video data.

The system 300 can be implemented in a variety of manners. For example, in some applications, the filtering circuit 310 is integrated with the display 370. Certain applications are directed to using the display 370 to display broadcast video, such as digital video received over analog channels (i.e., where the video source 305 receives the data and converts it to digital data for processing).

In addition to the above, the various processing approaches described herein can be implemented using a variety of devices and methods including general purpose processors implementing specialized software, digital signal processors, programmable logic arrays, discrete logic components and fully-programmable and semi-programmable circuits such as PLAs (programmable logic arrays). For example, the above algorithms are executed on a microcomputer (a.k.a. microprocessor) in connection with certain embodiments, and as may be implemented as part of one or more of the devices shown in the figures. While the present invention has been described above, in the figures and in the claims that follow, various systems and approaches may be implemented in connection with and/or in addition to the example embodiments described above. For instance, embodiments described in reference to the figures may be implemented using different systems and approaches. Embodiments described without specific reference to the figures may be implemented with the figures. Other embodiments involve using different filtering approaches in combination with those described, with mitigation of blocking artefacts. Still other embodiments involve using different sub- regions or pixels in filtering grid locations to mitigate blocking artefacts, such as by filtering a central as well as additional pixels within a predefined range of the central pixel (i.e., in addition to immediately adjacent pixels). In this regard, those skilled in the art will recognize that many changes may be made thereto without departing from the spirit and scope of the present invention.