Description

Method and Device for Coding a Sequence of Digitized Images

This invention relates to a method and a device for coding a sequence of digitized images.

In video coding an optimization of distortion versus rate is of high importance. A weak optimization results either in a low image quality at a given rate or in an unnecessary high rate for a given image quality. This also means that if the optimization is very good a high image quality at a low bit rate can be achieved.

A target of the rate-distortion optimization is to minimize the distortion D for a given rate R _c of rate R, by appropriate selections of coding parameters, as shown by equation (D •

miniu] (1) by R < R _c

The requirement of equation (1) can be solved using a La- grange multiplier method or a dynamic programming [1] method. The Lagrange multiplier method represents equation (1) as

min{j]

(2) where J = D + λ • R

whereby J is the Lagrangian cost function and λ is the so- called Lagrange multiplier. Consequently, how to determine λ becomes a key problem in rate-distortion optimization. Supposing the rate R and the distortion D is differentiable eve ^¬ rywhere, the minimum of the Lagrangian cost function J is given by setting its derivative to zero, i.e.

^ = ^ ₊ X = O (3) dR dR

leading to

». = - * d£ A ) R

In fact equation (4) forms the basis of the Lagrange multiplier selection method since it indicates that λ corresponds to the negative slope of the rate-distortion curve. Intuitively, the overall performance of such a method will highly dependent on the accuracy of the rate and distortion models.

There are several publications [2, 3, 5] that go in for the rate-distortion optimization. Assuming a high rate for the rate R ₁ a typical high-rate (HR) approximation for model of R and D for entropy-constrained scalar quantization [4] is:

Q ² DHR = Y ⁽ 6 ⁾

whereby a and b are constants and Q the quantizer value describing half the distance of the reproduction levels. Taking R _HR and D _HR into account the Lagrange multiplier /L _HR can be determined as

λ _HR = -— = c • Q ² (7) dR

whereby c is a constant which is experimentally suggested to be 0.85 [3] . In summary, this method, referenced as High Rate HR-λ selection, is practical and efficient.

Actually, it has been adopted into the reference software of the state-of-art video coding standard - H.264/AVC (AVC - Advanced Video Coding) . On the other hand, this algorithm also has some drawbacks. First, HR-λ is only related to Q and no property of the input signal is considered. This means that

it can not adapt to different video sequences dynamically. More important, the "high rate" assumption is not realistic all the time, which will result in a poor performance for both rate and distortion models, especially for low bit-rate applications.

In a video coding algorithm the selection of coding modes, e.g. the mode selection of macroblocks, depends on the La- grange cost function J. In order to get good mode estimation the determination of the Lagrange multiplier λ gets a key problem. For example a known reference software for the video coding standard H.264/AVC calculates the Lagrange multiplier λ by

qp-12 λ = C ^■ 2 ⁶ (8)

whereby C is a constant and qp is the quantization parameter. As no property of the input video signal is considered in this method, it can not adapt to the input signals at all.

In order to achieve adaptivity, in document [5] a Lagrange multiplier λ estimation method is proposed based on the p- domain technique. Defining p as a percentage of zeros among quantized residual coeeficients within a coded image block of an image of the video, the rate R and distortion D are used as follows by taking [6] into account:

R _p = θ ^■ (1 - p) (9)

D _p = a ^{2 ■} _e - ^a(1 - ^p) (10)

whereby a and θ are coding constants, e is the exponential function and σ is the variance of the transformed residuals. Taking equations (9) and (10) into account the Lagrange multiplier λ _p can be calculated as:

λ _p = β • (ln{σ ² / D') + δ) • D' / R' (11)

Whereby β and δ are coding constants and R' is the rate used in the previously encoded frame and D' is the distortion of the previously encoded frame.

This method represents a first step towards adaptivity.

Thanks to the introduction of the variance of transformed residual coefficients, it is able to adapt videos dynamically. However it decouples quantization information Q from the La- grange multiplier λp, which results in a bad rate-distortion performance such as in the case of rate control.

The rate model is a kind of approximation [7] which may greatly degrade the performance of the algorithm because of the error propagation by the Lagrange multiplier' s direct derivation from rate R and distortion D.

In a patent application [8] a method for video coding a sequence of digitized images is known, wherein a prediction error is generated by the contents of each image by taking into account a rate distortion optimization criterion being dependent on a Lagrange multiplier, the prediction error resulting from predicting step is transformed into a plurality of transformed residual coefficients in each image, the transformed residual coefficients in each image are quan- tized, and the Lagrange multiplier is determined for each image based on a probability distribution of the transformed residual coefficients of the respective image, the probability distribution having its maximum probability at its mean zero and being formed such that the probabilities decrease to zero from the mean to large absolute values of the transformed residual coefficients, whereby the variance of the probability distribution is calculated from the plurality of transformed residual coefficients in the respective image.

Hence it is an object of this invention to provide a method and a device that achieves a better rate-distortion rate than known methods .

This object is accomplished by the independent claims. Advantageous developments can be achieved by the dependent claims.

The invention relates to a method for video coding a sequence of digitized image, comprising the steps of:

(a) predicting a prediction image for a content of the respective image taking into account a rate distortion optimization criterion being dependent on a Lagrange multiplier; (b) generating a prediction error signal based on a difference of the respective image and the prediction image (P) of step (a) ; (c) transforming the prediction error signal into transformed residual coefficients; (d) quantizing the transformed residual coefficients by quantization interval into quantized coefficients;

(e) determining the Lagrange multiplier for the respective image based on a rate and a distortion, both the rate and distortion are based on - a probability distribution of transformed residual coefficients of a previously coded image;

- a quantization interval and a rounding offset of the quantization interval of a respective image;

(f) selecting the probability distribution such that it is having its maximum probability at its mean zero and being formed such that the probabilities decrease to zero from the mean to large absolute values of the transformed residual coefficients;

(g) determining the rate based on an entropy for the quan- tized non-zero coefficients of the previously coded image, whereby the entropy is generated using the probability distribution, the quantization interval and the rounding offset.

This method shows the advantage that a variance of the transformed residual coefficients in the rate and distortion models are considered that results in adaptivity of the rate and distortion models according to images of the input video. The

accuracy is achieved due to the fact that the rate does not consider entropy of the quantized zero coefficients. The good result using this method is achievable from low to high bit rate input videos. The rate and distortion can be generated by both encoder and decoder without a need for additional side information.

The method can be enhanced by determining a basic entropy for the quantized coefficients of the previously coded image and by forming the entropy by multiplying the basic entropy by a probability of quantized non-zero coefficients. This enhancement shows a low requirement on calculation complexity.

Further on the distortion can be determined

and the rate by

R = S ^■ (l - P ₀ ) ^■ H

with

H = -Pn l og ₂ P ₀ - 2 J] P _n ^■ l og ₂ P _n n = l

(χ)dχ

'dx

whereby Po represents a probability of quantized zero coefficients, P _n a respective probability of a quantized non-zero coefficient and S a coding factor.

In particular the probability distribution can be defined by

1 —λ I I a Laplace distribution fι{x) = — λ^^e ^{" L} ' ^x ' with a standard

derivation σ defined as σ = V2. The variance thereof is de-

2 fined as σ . This kind of probability distribution gives a good estimation of the transformed residual coefficients in video coding. The λ _L is called probability distribution parameter.

By taking the Laplace distribution into account the distor- tion is determined as

e ^λLF (2λ _L Q + λ ² _L Q ² - 2λ ² _L QF) + 2 - 2e ^λi ° λ ² r/l - e ^λ ^ ^Q ,

and the rate as

R ₌ = ^{s ■ β"λL} '° ^"F) J- a - e-ido-r ⁾ , _{ln a} _ _β - ^χ .te-D

In 2

Hence the rate and the distortion models can be written in a very compact way, whereby the rate and the distortion only relies on the quantization interval Q, the rounding offset F of the image to be encoded and the λ _L probability distribution parameter. The coding factor S is only a constant factor that doesn't have to be recalculated for each image.

In one embodiment the coding factor S is determined such that a basic Lagrange multiplier, which is determined by a factor, most preferably assuming, times square of the quantization interval λ _ER = c*Q ² , and the Lagrange multiplier results in an identical value when the probability distribution parame- ter approaches zero. By this embodiment the coding factor S is adjusted in an easy way such that if the probability distribution reaches a uniform distribution the Lagrange multi-

plier for the method is identical to the basic Lagrange multiplier that represents the case of the known "high-rate" method (HR-λ) .

Setting the probability of quantized zero coefficients to

P _Q = f _L (θ) = — λ _L allows a simplification of the required calculation power to determine the Lagrange multiplier.

In another embodiment, if there is no previously coded image available, the Lagrange multiplier is determined by a factor, most preferably assuming 0.85, times square of the quantization interval λ = c*Q . This embodiment allows to initialize the coding of the sequence of the digitized images and to use the method for the images other than the first one.

In an embodiment of the method a ratio parameter is determined by a ratio of the rate of the previously coded image to a real rate after encoding of the previously coded image φ=R/Rr and the Lagrange multiplier is generated by multiply- ing the Lagrange multiplier by the ratio parameter for coding the respective image. This very low complex embodiment generates the Lagrange multiplier with a higher precision. This leads to a higher image quality of the respective coded image at a given rate or at a given image quality a reduced rate compared with a method other than this embodiment.

This embodiment can be enhanced by representing the ratio parameter by a percentage of macroblocks in a skip mode of the previously coded image up to the power of a coding parameter φ = γ ^Ps , by determining the percentage by an average of the percentage of macroblocks in a skip mode of a number of previously coded images and the coding parameter is generated by an average of the coding parameter of the number of previously coded images, most preferably assuming a number of 10, and by, if the coding parameter is exceeding a threshold value, most preferably assuming a threshold value of 2.0,

setting it to this threshold value. This enhancement leads to a stable determination of the Lagrange multiplier.

This enhancement and/or the recent embodiment can be further enhanced such that if the Lagrange multiplier exceeds an upper limit, most preferable 1.2 * Lagrange multiplier of the previously coded image, the Lagrange multiplier is set to this upper limit, or if the Lagrange multiplier falls below a lower limit, most preferable 0.8 * Lagrange multiplier of the previously coded image, the Lagrange multiplier is set to this lower limit. By this enhancement big variations of the Lagrange multiplier that may lead to ill solutions will be suppressed.

In an alternative embodiment to the recent embodiment a basic Lagrange multiplier is determined by a factor, most preferably assuming 0.85, times square of the quantization interval A HR = c*Q ² , if the Lagrange multiplier exceeds an upper limit, most preferable five times the basic Lagrange multi- plier, the Lagrange multiplier is set to this upper limit, or if the Lagrange multiplier falls below a lower limit, most preferable 0.2 times the basic Lagrange multiplier, the Lagrange multiplier is set to this lower limit. By this enhancement big variations of the Lagrange multiplier that may lead to ill solutions will be suppressed.

The method can be implemented in a video coding standard, e.g. H.264/AVC standard. By doing this the gains of this invention can also be used in standard video coders, e.g. H.264.

Besides the method a device for video coding a sequence of digitized image is also part of this invention, whereby this device covers at least one means that is capable of perform- ing the respective method steps as described above. With this device the method can be implemented and executed.

The invention and its advantageous developments are described by taking the following figures into account:

Figure 1 a schematic of a video encoder system;

Figure 2 simulation results of a first method for several test sequences;

Figure 3 simulation results of picture quality vs. rate of the first method and a high rate method;

Figure 4 comparison of a percentage of macroblocks in skip mode for a single image versus a ratio of a estimated rate to a real rate after encoding;

Figure 5 simulation results of a second method for several test sequences;

Figure 6 simulation results of picture quality vs. rate of the second method and a high rate method;

Elements with identical function and mode of action are marked with identical reference signs.

Fig. 1 shows an example of a video encoder based on the video coding standard H.264/AVC in a simplified form. In the encoder of Fig. 1, a sequence of digitized images I is compressed by encoding the so-called prediction error signal resulting from the prediction performed in a predicting step.

The prediction error signal PES is generated by supplying the input image I to an addition means A subtracting from the input image I a predicted image P determined in a corresponding prediction step. The prediction error signal PES is supplied to transformation means 1 performing a well-known transformation T on the digitized image resulting in transformed residual coefficients x. After a scaling step (not shown), the transformed residual coefficients x of each image I are sub-

jected to a quantization Q performed in a corresponding quantizer 2. The resulting transformed and quantized coefficients y are supplied to an entropy encoder 9 performing entropy coding EC. Furthermore, those quantized coefficients y are scaled and supplied to an inverse quantizer 3 performing an inverse quantization IQ on the coefficients. After this inverse quantization, the coefficients are subjected to an inverse transformation IT by inverse transformation means 4. The resulting decoded error signal is added to the predicted image P in an addition means A' . The image resulting therefrom is subjected to a deblocking step DB in a corresponding deblocker 5. As a result, a decoded image DI is obtained, which is supplied to a motion compensator 6 performing a motion compensation. The motion compensation is performed by the use of so-called motion vectors MV that are calculated in a motion estimation step ME in a motion estimator 7. The motion vectors describe the motion of picture blocks from one image to another. After performing the motion compensation MC, the predicted image P is obtained, which is supplied to the adder A in order to determine the prediction error.

The prediction based on motion compensation MC and motion estimation ME is a temporal prediction that is used in a so called INTER-frame coding. When rate distortion optimization according to the invention is used for this temporal prediction, different prediction methods for a respective image are applied. The prediction methods e.g. differ in the feature which images and how many images are used for determining the predicted image P. For each encoded image obtained by said different predicting methods, the cost function J according to equation (2) on the basis of the Lagrange multiplier is calculated, the Lagrange multiplier being determined by the method of the invention. The prediction method resulting in the lowest cost J for the respective image is used as the prediction method for this image.

Instead of the temporal (= INTER) prediction described before, the encoder in Fig. 1 may use a spatial (= INTRA) pre-

diction IP being performed by intra prediction means 8. In this INTRA prediction, the so-called macroblocks in each image are predicted from neighboring previously encoded macroblocks of the current image. To exploit spatial correlation among pixels in the image to be encoded, the coding standard H.264 uses three basic types of prediction, namely full- macroblock prediction for 16x16 luma (luma = luminance) or the corresponding chroma (chroma = chrominance) block size, 8x8 luma prediction or 4x4 luma prediction. For the full- macroblock prediction, the pixel values of an entire macrob- lock of luma or chroma data are predicted from the edge pixels of neighboring previously decoded macroblocks. FuIl- macroblock predictions can be preformed in one of four different ways that can be selected by the encoder for the pre- diction of each particular macroblock. Those different ways refer to vertical, horizontal, DC and planar prediction. For the vertical and horizontal prediction types, the pixel values of a macroblock are predicted from the pixels just above or to the left of the macroblock, respectively. In DC predic- tion, the luma values of the neighboring pixels are averaged and the resulting average value is used as a predictor. In planar prediction, a three-parameter curve-fitting equation is used to form a prediction block having a brightness slope in the horizontal direction and a brightness slope in the vertical direction that approximately matches the neighboring pixels .

In the above mentioned 4x4 luma prediction, the values of each 4x4 block of luma samples are predicted from the neighboring pixels above or left of a 4x4 block, and 9 different directional ways of performing the prediction can be selected by the encoder. The above mentioned 8x8 luma prediction basically uses the same concepts as the 4x4 prediction, but with a prediction block size that is 8x8 rather than 4x4 and with low-pass filtering of the predictor to improve prediction performance.

When rate distortion optimization is applied to INTRA or INTER prediction, different INTRA or INTER prediction modes are applied to the blocks, e.g. macroblocks, in each image, and the prediction mode leading to the lowest cost according to equation (2) is used for the respective blocks in the image, wherein the Lagrange multiplier λ is selected based on the method according to the invention.

In the following, a first embodiment of the invention will be described, where the Lagrange multiplier for rate distortion optimization is determined based on a Laplace distribution. The method of the first embodiment is also called first method and will result in a Lagrange multiplier Lapl-λ. Furthermore, results based on this embodiment will be presented when applying the first method in the reference software of H.264/AVC for intra prediction.

The Laplacian distribution applied for the INTRA predicted images has zero mean and is defined as follows:

λ _L = ^- (13) σ

In equation (12), x represents the transformed residual coefficients after applying the transformation step T in Fig.l λ _L is the distribution factor and σ ² is the variance of the transformed residual coefficients in an image I, whereby the variance indicates the property of the input video. Because of the one-to-one mapping between σ and λ _L , the latter will be used instead in the following expressions for simplicity.

In the H.264/AVC standard, a uniform reconstruction quantizer is used. Therefore, a basic entropy H of the quantized coef- ficients y in Fig.l can be written as follows:

H = -P ₀ ^■ l og ₂ P ₀ - 2 ∑ P _n ^■ l og ₂ P _n :i4)

17 = 1

P ₁₇ = £ ' f _L (x)dx (16)

Q indicates the quantization interval and F is the rounding offset used for quantization. P ₀ represents a probability of quantized zero coefficients and P _n of quantized non-zero coefficients .

The distortion D can be written as:

D= £ ^~F _ x ² f _L (x)dx + 2 X f _n ^f ^~F ( _* " nQ) ² f _L (x)dx (17)

17 = 1

In theory, the basic entropy expression equation (14) can be used to represent the rate directly. Nevertheless, the accuracy can be furthered improved. Assuming that pixels are coded independently, equation (14) actually describes the average entropy for each pixel. However, in video coding standards, such as H.264/AVC, a kind of run-length coding technique is applied to the quantized block level, which means successive zeros in zigzag order will be coded together to save bits. Consequently, if the successive zero coefficients after quantization take a big percentage, such as in low bit- rate inter-frame coding, equation (14) will be trapped into a low performance. The inventors discovered that normally the average entropy for zeros is much smaller than that for non- zeros. Therefore the rate R is formed on the basis of entropy, whereby the entropy is based on the basic entropy H taking only non-zero quantized coefficients into account, e.g.

R = S ^■ (l - P ₀ ) ^• H ( 1 8 )

Since the Laplacian distribution assumed for the residual coefficients is usually not an ideal Laplacian distribution, the rate R is compensated by a coding factor 5.

Putting equations (13) to (16) into equation (18) the rate R can be written as:

Similarly the distortion D can be written by taking equations (13) to (17) into account as:

Since λ _L is an inherent property of input videos, it is reasonable to be regarded as a constant during a short time, such as several images I. Thus equation (4) turns to

_{λ =} dD ₌ 8D/8 _O dR dR / dQ

By putting equations (19) and (20) into equation (21), the Laplacian distribution based on the Lagrange multiplier λ can be determined. The complete expression according to equation (21) is too long and will not be presented here. Practically, it can be easily computed by mathematical software or numerical methods. In one embodiment, a lookup table is used to determining λ in order to save computation time.

Defining p ₀ , such as

Po = ^L (O) = - ^λ i ' ( 22 )

the equations (19) and (20) become less complex. For example in the video coding standard H.264/AVC for INTER prediction and using as rounding offset F = Q /6, equation (21) becomes :

_{λ =} i|2 ^2_ _Q2 _ 8 í 3UIn 5 - In^ -inϋ-ln _Po , ^j ₊ ^

whereby O represents the big O notation and po is between 0 and 1 inclusively. Consequently, the high order items of p ₀ in (23), are mathematically neglectable. However, in practice, this can not be ignored when Q is big because Q always has a higher order than p ₀ in such items.

On the other hand, when λ _L , or p ₀ , approaches zero (or σ approaches infinity) , the Laplacian distribution will degrade to the uniform distribution. In the above example, the limitation of equation (23) will be

Hm X= Hm λ= ¹ ^ ^{■ Q2} _' < ²⁴ > λ _L →o PO → O

which mathematically matches equation (7) . Therefore, the HR- λ can be interpreted as a special case of the proposed first method Lapl-λ.

The proposed Lagrange multiplier selection first method Lapl- λ is a general method for the rate-distortion in hybrid video coding. In the following an example for implementation is given for an H.264/AVC environment. Hereby there are several practical issues related to the implementation with mentioning. Most of them are on the derivations of parameters for the Lapl-λ calculation.

First, Q in the equations (19) and (20) is determined by quantization parameter qp in H.264/AVC,

qp-12 Q = 2 6 (25)

And to be in line with the H.264/AVC reference software, the rounding offset F in the equations (19) and (20) is set to Q/3 and Q/ 6 for INTRA-image and INTER-image coding, respectively.

Second, the coding factor 5 in equation (19) can be calculated by comparing the degraded Lapl-λ to HR-λ, i.e. the comparison between equations (22) and (7) . Keeping c as 0.85 as in the H.264/AVC reference software, 5 for INTER-image coding in H.264/AVC will be

5 = ^^ = 0.317 (26)

18c

Similarly, the 5 for INTRA-image coding can be derived as 5=0.181. Of course, 5 can also be determined by experiments.

Third, λ _L in equations (19) and (20) is calculated according to its definitions for each image. As it is supposed to be constant over a short time, the λ _L of the previous image is used as the estimation to the current image. This means that for coding of the current image the probability distribution fL (x) of the previously coded image is used.

Finally, the λ for the Lapl-λ method can be calculated ac- cording to equations (19) to (21) . In practice, to avoid being trapped into ill solutions, the range of the final λ should be limited. In the current example implementation, it is set to between 20% and 500% of the corresponding value of HR-λ. It should be also noted that, for the very first image to be coded by an INTRA coding mode and the first image to be

coded by an INTER coding mode, as no λ _L information available, the HR-λ is used instead of the Lapl-λ.

The simulations for this example were made on JM 11.0, the most recent reference software of H.264/AVC. Since the first image I is always coded with HR-λ, only INTER-coded images (P-frames) are used for the result. Figure 2 shows the results of the simulation. The first column covers different test sequences Vl, ..., V7 that are:

Test Resolution Name

Sequence

Vl QCIF container

V2 QCIF news

V3 QCIF foreman

V4 CIF paris

V5 QCIF silent

V6 CIF mobile

V7 CIF tempte

The resolution CIF (CIF - Common Intermediate Format) represents 352x288 pixels and QCIF (QCIF - Quarter CIF) 176x144 pixels. The respective test sequences are well known at JVT (JVT-Joint Video Team of ISO/IEC MPEG and ITU-T VCEG) and are therefore not described in more detail. The second column covers the used quantization parameter qp. The third column HR-λ contains the results using the High Rate method for determining the Lagrange multiplier. In particular the first sub-column of the third column shows the average PSNR (PSNR - Peak Signal To Noise Ratio) , the second sub-column the Rate in Kbps (Kbps - kilo Bits per Second) and the third sub- column an averaged Laplace multiplier for HR-λ. The fourth column covers the test results using the Lapl-λ method. In the last column an averaged difference between the PSNR val- ues of the Lapl-λ and HR-λ is shown for each test sequence.

Fig. 3 shows a comparison of of the PSNR of the test sequences Vl and V5 using both methods Lapl-λ and HR-λ versus the rate R.

Since the rate-distortion optimization affects not only the compression on luma signal, but also that on chroma signal, the PSNR in the table is calculated according to

P = -(^P _L + P _c0 + P ₀₁ ) (27)

6

where P _Lf P _C o and P _CI are PSNR-values for luma, chromaO and chromal in a 4:2:0 video, respectively. The δPSNR indicates the average PSNR differences between RD-curves . Thus it can be used to as an objective evaluation on the whole perform- ance .

Thanks to the more accurate models for the rate R and distortion D, the proposed first method Lapl-λ shows a superior performance over HR-λ. The average gain in δPSNR for the seven test sequences is about 0.21 dB . The maximum gain in δPSNR is 0.66 dB for the test sequence Vl. As mentioned above, such gain obtained by a calculator for each sequence is a kind of average. The peak gain is actually even bigger. For example, Fig. 3 shows that the peak gain for the test se- quence Vl and the test sequence V5 is up to 1.1 dB and 0.5 dB, respectively.

However, for videos with big movements or rotations, such as V6 and Vl ₁ the gain by the LapE-1 - method is very little. It can be explained by the used rate function R. Normally the variance for the fast videos (test sequences) is much bigger than that for slow ones. That is the distribution in such case is close to the assumption of uniform distribution, which results that the HR-λ method can also work fine. There- fore in such situations the methods HR-λ and Lap-X share similar results. Contrariwise, for slow videos, the method of

HR-λ fails and the first method of Lapl-λ gives a better performance .

An enhancement to the presented first method can be achieved by a second method, called Lap2-λ. In hybrid video coding standards, such as H.264/AVC, a skip mode is a special INTER- coding method to encode one macroblock of the image I . The basic idea is to take the prediction as the reconstruction directly and code nothing but the skip mode flag. As in such a case only one bit will be coded for a single macroblock, skip mode is quite efficient for videos where high correlations exist among successive images, such as sequences with still background. If P ₃ is defined as a percentage of macrob- locks in skip mode for a single image, it can be regarded as a measure for the correlations among successive images to some extent. The inventors discovered that the rate R is highly related to P ₃ in a coded sequence.

Defining φ as the ratio of the estimated rate R by equation (19), indicated as R (Lapl-λ), to the real rate Rr collected after encoding, a relationship between φ and P _s on frame level can be investigated as seen in Fig. 4. The test sequence V3 with the parameters Main Profile of H.264 (JM12.1), five reference frames, QP=28 and 99 INTRA (P) frames were used.

From this figure, an approximately linear relationship between In (φ) and P ₃ can be observed, i.e.

ln(φ) * B ^■ P _s => φ * e ^B'Ps = y ^Ps (28)

where B and γ are coding parameters reflecting INTER-frame correlations of video sequences. Considering the definition of φ, the real rate Rr can be approximately established as

R(Lapl - λ) R(Lapl - λ)

R _r = (29)

Y

In fact, γ ^s can be regarded as a correction to the estimated rate R(Lapl-λ) after encoding. With such feedback, the rate R is dynamically refined at frame level so that a higher accuracy is achieved.

In the case of no shot transition, the inherent properties of test sequences, such as P _s and γ, can be reasonably regarded as constants during several images. While keeping the distortion D as defined in equation (20) and replacing the rate R as defined in equation (19) with the improvement as defined in equation (29) the improved Langrage multiplier can be written as

X ₁ = ^^^.y ^p s _{= λ} . _y ^P _{S (} 30)

¹ dR/dQ

That is, the proposed second method Lap2-λ is actually an extension to Lapl-λ by a dynamic rate factor γ ^s which reflects the correlations among successive frames in video sequences. It should be noted that if no skip mode is consid- ered, i.e. P _s equals zero, X _τ will reduce to λ of Lapl-λ- method.

Moreover, it is well known that uniform distribution is a special case of Laplace distribution when the Laplace parame- ter λ _L approaches zero. In such a case, the limit of X _τ is

Hm X ₁ = y ^{Ps ■} c ^■ Q ² , (3D λ _L →o

where c* is a coding constant. For a single frame where γ and P _s are both constant, equation (31) matches equation (31) mathematically .

The proposed Lagrange multiplier selection second method X ₁ can be used in rate distortion optimizations of any hybrid video coding standard. For example, it was embedded into the H.264/AVC environment. To determine λi for a frame before en-

coding, parameters needed in (30), such as γ and P ₃ , have to be estimated. Such estimations are based on the real values collected after the encoding of previously encoded images.

First, the real values for γ and P _s are updated at frame level. After the encoding of each image, P _s is obtained according to the statistic of the number of macroblocks in skip mode. Then the ratio φ is calculated with the estimated rate R (Lapl-λ) , and the real rate Rr for the coded image. After that the parameter γ is determined by equation (28) . The stability of Y is quite important since bigger γ may lead to bigger λi for the next frame and then even bigger γ. To inhibit such undesired positive feedback, the range of γ should be limited. Currently, it is set to a value of 2.0 at maxi- mum. Of course, a fixed value for γ may be used as well, through some adaptivity will be sacrificed.

Second, estimations on γ and P _s for λi calculation for the current image are made based on the averages of γ and P _s in previous ten images respectively. In fact, such setting can suppress the negative effect by sharp variations of γ and P _s in previous image while keep adaptivity.

Finally, to prevent being trapped into ill solutions the range of λi should be limited as well. In this example implementation, it is set between 90% and 500% of the corresponding value if X _HR . Moreover, to avoid big variations in the PSNR, the X ₁ of the current image is limited between 80% and 120% of that of the previous image.

The enhancement is verified by a reference software of H.264/AVC, called JM 12.1. The first 100 frames (one I (=INTRA) frame followed by 99 P (=INTER) frames) of each test sequence were coded with the following parameters using the H.264/AVC Main Profile:

- five reference frames,

- CABAC entropy coding,

- fixed quantization parameters,

- high complexity rate-distortion optimization,

- deblocking filter enabled.

It should be noted that the 8x8 transform was disabled in the simulation since the rate and distortion models aforementioned are both based on 4x4 transform.

Fig. 5 and Fig. 6 show the simulation results for P frames. I frames are not counted, because they were coded in the same way for all tests. Fig. 6 shows weighted averages of luminance and chrominance as PSNR.

Fig. 5 shows in column 1 the test sequences Vl, ..., V7 and the qp-value in the second column. In the third, fourth and fifth columns, the results of the three discussed methods HR- λ, Lapl-λ and Lap2-λ can be seen. The third column shows the Lagrange multiplier of the HR-λ method. The fourth column covers a first sub-column representing the Lagrange multi- plier of the Lapl-λ method and a δPSNR of the HR-λ and Lapl-λ methods. Similar, the third column represents the Lagrange multiplier of the Lap2-λ method and a δPSNR of the HR-λ and Lap2-λ methods.

The respective δPSNR indicates the average PSNR difference between two RD-curves (RD - rate distortion) . Therefore it is used as an objective evaluation on the whole performance. Thanks to the more accurate models, the proposed Lap2-λ shows a superior performance over both HR-λ and Lapl-λ methods. Compared with HR-λ, Lap2-λ obtains a δPSNR gain of 0.26 dB on average. Moreover, the peak δPSNR gain for single sequence can be much bigger: up to 1.3 dB, 0.6 dB and 0.5 dB gains can be observed from the figures for the test sequences Vl, V2 and V5 respectively. Compared with Lapl-λ, Lap2-λ achieves an improvement as well. For the last two test sequences mentioned above, the peak δPSNR gains are both above 0.15 dB . And for the test sequence Vl the peak gain is even more than 0.3 dB.

However, for videos with big movements, rotations or zooming, such as the test sequences V6 and Vl, the gains by Lap2-λ are quite limited. The reason is the same as that for Lapl-λ. That is, for fast sequences, the transformed residuals are much bigger than those for slow sequences due to the less precise prediction, which basically confirms the "high rate" assumption. Thus HR-λ, Lapl-λ and Lap2-λ show a similar performance. On the contrary, for slow sequences, the "high rate" assumption of HR-λ fails and the estimation of the rate R and distortion D by Lap2-λ method are much more accurate so that Lap2-λ method provides a better result.

As for the complexity of computation, Lap2-λ method is higher than HR-λ method while similar with Lapl-λ method. In fact, the computation mainly arises from the complex rate R and distortion D. As such computation only occurs once at frame level, it is neglectable compared with other modules of the encoder, such as motion estimation.

It should be noted that the teachings of all embodiments can be combined. The terms image and frame in all embodiments have the same meaning.

A device DD for video coding a sequence of digitized image I covers at least one means Ml that is capable of performing the respective method steps. A device DD with the means Ml and other means can be seen in Fig. 1. The means of the device DD can be implemented in hardware, software or in a com- bination of hard- and software. The device can be implemented in a video server that provides encoded videos e.g. for a video delivery service. The means Ml generates the Lagrange multiplier according to the first and/or second method. The Langrage multiplier can be used e.g. by the INTRA-prediction means 8.

Bibliography

[1] A. Ortega and K. Ramchandran, "Rate-distortion methods for image and video compression", IEEE Signal Processing Magazine, vol. 15, no. 6, pp. 23-50, 1998

[2] G.J. Sullivan and T. Wiegand, "Rate-distortion optimization for video compression, " IEEE Signal Processing Magazine, vol. 15, no. 6, pp. 74-90, 1998.

[3] T. Wiegand and B. Girod, "Lagrange multiplier selection in hybrid video coder control," in IEEE Int. Conf. on Image Processing, 2001, vol. 3, pp. 542-545

[4] H. Gish and J. Pierce, "Asymptotically efficient quantizing," IEEE Trans. Inf. Theory, vol. 14, no. 5, pp. 676-683, 1968

[5] L. Chen and I. Garbacea, "Adaptive lambda estimation in lagrangian rate-distortion optimization for video coding," in Visual Comm. and Image Processing, Jan 17-19 2006

[6] Z. H. He and S. K. Mitra, "Optimum bit allocation and ac- curate rate control for video coding via p-domain source modeling", IEEE Trans. Circuits System Video Technology, vol. 12, no. 10, pp. 840-849, 2002

[7] -, "A linear source model and a unified rate control al- gorithm for DCT video coding", IEEE Trans. Circuits

Syst. Video Technol., vol. 12, no. 11, pp. 970-982, 2002

[8] EP 06020948.3