The present invention relates to a method or system or system components for compensation and /or measurement of differential latency between displays located at endpoints of network as well as software for carrying out any of the methods and signal storage media for storing the software.

Technical Background

Different rendering machines and displays at the endpoints of IP networks can show different end-to-end latencies.

A typical chain consists of the following components:

(1) Source

(2) Encoder

(3) Network

(4) Decoder

(5) Renderer

(6) Display

Each hardware and/or software component in such a chain can in general have different processing time periods (processing latency] and these processing time periods can vary over time. Also the applied hardware and/or software solutions are evolving over time, and consequently the processing time periods can change. Also the network has in general a different delay to the different endpoints, and also this delay can vary over time (network latency).

Solutions for re-synchronizing data streams from the source to the destination are possible, for example as disclosed in RFC 3550, as long as the data can be controlled within a data processing framework and with high precision (e.g. <150 ms). Typically, the last synchronization point can be either just after the decoding (in the case of a software decoder) or just before the decoding (in the case of a hardware decoder). As such, the decoding, rendering and displaying add an unknown amount of latency that in general can be different between the different endpoints. This latency can in general be different for the various endpoints because the latency depends on the specific hardware and/or software implementations. Moreover this latency can vary over time for each endpoint.

Many applications require that the displayed images at the different endpoints are "data- synchronized" i.e. the displays must display the same frame at the same point in time. It is therefore important that the different end-to-end latencies can be measured in an easy way and compensated accordingly. Such a recalibration is necessary on a regular basis because the end-to-end latency for the various endpoints can change over time.

In US 4,540,982 the delay compensation is realized in hardware with flip-flops and AND- gates to implement multi-vibrator circuits. In essence a differential clock generator i.e. two clock trains, of the same frequency is implemented, that can be relative phase shifted. The two shifted clocks are used to compensate for all kinds of delays in electronic circuits.

In US 2007/0247470 Al a solution is described to reduce the latency of a display device. A controller controls the transfer of pixels from a frame buffer to the display module. The controller monitors the content of the frame memory. A minimum number of pixels must be available in the buffer memory before the transfer to the display module starts. This minimum number of pixels depends on the image format (e.g. number of lines and the refresh rate). In this solution the controller is not waiting till a full frame is available in the frame memory but starts to transfer the pixels from the frame memory earlier to reduce the latency caused by the display device.

Summary of the Invention

An aspect of the present invention is a method for measuring the end-to-end differential latency between displays and for putting this in vector form to perform differential compensation. The vectors are used to steer dequeue operations. The present invention relates in one aspect to a method of measurement of differential latency between displays located at endpoints of network , displays being driven by one or more rendering machines and having means for refreshing the images on different displays at the same point in time as well as means adapted so that the content of each displayed image can only change after the completion of previous displayed frame, for example the displays being frame locked and having double buffered swap locked memory operation, the method comprising:

Streaming test images over the network to the displays of the different endpoints, measuring end-to-end differential latency between the displays and for putting this in vector form.

The vectors are preferably used to perform differential compensation of the delays at the displays. For example, the vectors are sent to the displays e.g. to the frame buffers, to steer dequeue operations. The frame buffers can be FIFO's for example.

The method may include the steps of displaying the test images on the displays and capturing the displayed images at the different endpoints with an image recording device.

The image recording device can be a camera for example.

There is no requirement for the camera to be synchronized with the displays. In accordance with an embodiment of the present invention a measurement algorithm is carried out followed by delay compensation.

The present invention also provides a display unit for measurement of differential latency between displays located at endpoints of network , the display unit comprising:

displays being driven by one or more rendering machines and having means for refreshing the images on different displays at the same point in time,

means adapted so that the content of each displayed image can only change after the completion of previous displayed frame, means for receiving streaming test images from the network and supplying these images to the displays of the different endpoints, and

measuring end-to-end differential latency between the displays and for putting this in vector form.

The unit may be adapted to perform differential compensation of the delays at the displays using the vectors.

For example, the unit may be adapted to send vectors to the displays e.g. to the frame buffers. The displays may be adapted to steer dequeue operations using the vectors. The frame buffers can be FIFO's for example.

The unit may include means for capturing displayed images at the different endpoints with an image recording device.

The image recording device can be a camera for example.

There is no requirement for the camera to be synchronized with the displays. In accordance with an embodiment of the present invention, the unit is adapted to perform a measurement algorithm followed by optional delay compensation.

In one aspect the present invention provides compensation of frame latency differences between two or more networked displays. Embodiments of the present invention include means for measuring the differential latencies and for compensating by delaying the streams with lower latency. The compensation may be performed by delaying the streams with lower latency at the level of the frame buffer (FIFO) before or after decoding.

Accordingly an aspect of the present invention is that the unit is adapted to measure a differential latency between displays and for putting this in vector form to perform differential compensation. The vectors are used to steer dequeue operations. For example, the vectors are sent to the displays e.g. to the frame buffers, to steer dequeue operations. The frame buffers can be FIFO's for example. In a further aspect a system of measurement of differential latency between displays located at endpoints of network is provided, displays being driven by one or more rendering machines and having means for refreshing the images on different displays at the same point in time as well as means adapted so that the content of each displayed image can only change after the completion of previous displayed frame, the system comprising: means for streaming test images over the network to the displays of the different endpoints, and

means for measuring end-to-end differential latency between the displays and for putting this in vector form.

The system may be adapted to perform differential compensation of the delays at the displays using the vectors. For example, the system may be adapted to send vectors to the displays e.g. to the frame buffers. The displays may be adapted to steer dequeue operations using the vectors. The frame buffers can be FIFO's for example.

The system may include means for capturing displayed images at the different endpoints with an image recording device.

The image recording device can be a camera for example.

There is no requirement for the camera to be synchronized with the displays.

In accordance with an embodiment of the present invention, the system can be adapted to perform a measurement algorithm followed by optional delay compensation.

The present invention relates in a further aspect to a method of measurement of differential latency between displays located at endpoints of network , displays being driven by one or more rendering machines and having means for refreshing the images on different displays at the same point in time as well as means adapted so that the content of each displayed image can only change after the completion of previous displayed frame, for example the displays being frame locked and having double buffered swap locked memory operation., the method comprising:

Streaming test images over the network to the displays of the different endpoints.

displaying the test images on the displays and

capturing the displayed images at the different endpoints with an image recording device.

The image recording device can be a camera for example. There is no requirement for the camera to be synchronized with the displays.

In accordance with an embodiment of the present invention a measurement algorithm is carried out followed by optional delay compensation. An aspect of the present invention is a method for measuring the end-to-end differential latency between displays and for putting this in vector form to perform differential compensation. The vectors are used to steer dequeue operations. For example, the vectors are sent to the displays e.g. to the frame buffers, to steer dequeue operations. The frame buffers can be FIFO's for example.

The present invention also provides a display unit for measurement of differential latency between displays located at endpoints of network , the display unit comprising:

displays being driven by one or more rendering machines and having means for refreshing the images on different displays at the same point in time,

means adapted so that the content of each displayed image can only change after the completion of previous displayed frame,

means for receiving streaming test images from the network and supplying these images to the displays of the different endpoints, and

means for capturing displayed images at the different endpoints with an image recording device.

The image recording device can be a camera for example. There is no requirement for the camera to be synchronized with the displays.

In accordance with an embodiment of the present invention, the unit is adapted to perform a measurement algorithm followed by optional delay compensation.

In one aspect the present invention provides compensation of frame latency differences between two or more networked displays. Embodiments of the present invention include means for measuring the differential latencies and for compensating by delaying the streams with lower latency. The compensation may be performed by delaying the streams with lower latency at the level of the frame buffer (FIFO) before or after decoding.

Accordingly an aspect of the present invention is that the unit is adapted to measure a differential latency between displays and for putting this in vector form to perform differential compensation. The vectors are used to steer dequeue operations. For example, the vectors are sent to the displays e.g. to the frame buffers, to steer dequeue operations. The frame buffers can be FIFO's for example.

In a further aspect a system of measurement of differential latency between displays located at endpoints of network is provided, displays being driven by one or more rendering machines and having means for refreshing the images on different displays at the same point in time as well as means adapted so that the content of each displayed image can only change after the completion of previous displayed frame, the system comprising: means for streaming test images over the network to the displays of the different endpoints, and

means for capturing displayed images at the different endpoints with an image recording device.

Brief description of the drawings

Figure 1 shows an example of tiled display configuration of 4 x 3 displays that can be used with any of the embodiments of the present invention.

Figure 2 shows an embodiment of the present invention wherein images are streamed to a displays wall, images are captured and a measuring system is provided with feedback to the renderers for latency compensation. Figure 3 shows a more detailed embodiment of the present invention with a network and display and camera interfaces.

Figure 4 illustrates a latency difference example of 2 Td between display D _r and D _s

Figure 5 illustrates the images when there is a latency difference of 2 Td between display D _r and D _s .

Figure 6 shows vertical arrows are corresponding with the start of the camera's integration period as used in embodiments of the present invention.

Figure 7 shows camera image of the example of the tiled 4 by 3 displays.

Figure 8 shows a delay of n between displays D _r and D _s .

Figure 9 shows the situation for the case when T _c < T _d .

Figure 10 shows the case when T _c = T _d .

Figure 11 shows the case when T _c > T _d , i.e. a too low time resolution.

Figure 12 shows a frame buffer structure in accordance with an embodiment of the present invention with a FIFO with its controller, decoder, renderer and display

Figure 13 shows a frame buffer structure in accordance with an embodiment of the present invention with a dequeue-operation in the compressed domain.

Figure 14 shows a data path 1 for software decoding and data path 2 for hardware decoding in accordance with an embodiment of the present invention.

Figure 15 shows a measurement and delay compensation process and algorithm in accordance with an embodiment of the present invention.

Figure 16 shows examples of 6 data paths with g _x = 0, g ₂ = , g ₃ = l, g ₄ = 3, g ₅ = 2, 96 = 1

Detailed description of the illustrative

embodiments

Aspects of the present invention relate to a network with displays whereby different rendering machines and displays at the endpoints of networks can show different end-to- end latencies.

A typical chain consists of the following components:

[1) Data source (2) Encoder

(3] Network

(4) Decoder

(5) Renderer

(6) Display

The present invention in its various embodiments comprises methods and/or systems adapted to stream images such as specific test images over a network to displays at different endpoints. Preferably all the displays are driven by rendering machines. Preferably means for refreshing the images on different displays at the same point in time and means adapted so that the content of each displayed image can only change after the completion of previous displayed frame are provided. For example, the rendering machines are preferably frame locked and have double buffered swap locked memory operation. These features are important for data-synchronization. The images of the displays at the different endpoints are captured by an image recording device of which a camera is one possibility. The camera need not be/is not synchronized with the displays. A measurement algorithm is performed and a delay compensation can be performed.

Tiled display configurations and rendering machines

A plurality of displays {D , D ₂ , . . , D _l - see Figs. 1 and 2), for example projectors and screens such as rear projection displays, fixed format displays, CRT, LED, plasma, LCD, OLED displays or similar are arranged for example in a matrix structure of i by j displays to form a high resolution display wall (Figure 1, 10 in Figure 2]. The image display 10 is driven by one or more decoders and rendering machines 6-9. One rendering machine can drive one or more displays D; of the display wall 10. Data from a data source 3 such as a video source S is streamed e.g. via an encoder 4, in a compressed, encrypted or raw format, over a network 5 such as a data network, large area network, local area network, wireless network including RF or optical such as 1R network, or wired network, especially over an IP network to the rendering machines 6-9 and displayed on the high resolution display wall 10. A tiled display wall is an example. Any configuration of displays, tiled or not can be used with embodiments of the present invention. The insertion area of the video source will in general be spread over several displays D ₁ , D ₂ , - . , D _ll forming one logical video insertion. Due to different network latencies and/or processing latencies or other latencies the images on one or more of the displays D ₁ , D ₂ , - . , D _l can appear with different time shifts. In other words, image frames that were generated or captured at the same point in time can, due to the different network and processing latencies, appear on the displays of the display wall with different time delays.

In embodiments of the present invention means for refreshing the images on different displays at the same point in time and means adapted so that the content of each displayed image can only change after the completion of previous displayed frame is provided. For example, in an aspect of the present invention the decoders and rendering machines 6-9 are preferably frame locked, i.e. there is a means for frame locking. This means that synchronization signals such as vertical synchronization signals Vsync of all the relevant rendering machines and the displays of a group of displays e.g. of the display walllO have exactly the same frequency and phase. For example, one implementation system or method that can be used with any embodiment of the present invention is adapted to configure one rendering machine as master delivering the synchronization signals such as Vsync to each of the other rendering machines. Another system or method which can be used with any embodiment of the present invention is adapted to lock every rendering machine to an external synchronization signal such as Vsync supplied by an external synchronization signal generator. This latter implementation is called genlock. The rendering machines preferably have double buffered and swap locked memory operation. These features, e.g. vertical synchronization and double buffered memory operation with swap locking guarantees that the images on the different displays are refreshed at the same point in time and that the content of each displayed image can only change after the completion of previous displayed frame, i.e. can provide the means for refreshing the images on different displays at the same point in time and means adapted so that the content of each displayed image can only change after the completion of previous displayed frame. It is however still possible that frames, from the source 5, captured or generated at the same point in time are shown with different time delays on the screens, but the different time delays will be an integer multiple of the display's vertical time period. Thus although vertical synchronization and double buffered memory operation with swap locking are prerequisites for the data- synchronization they do not guarantee it alone. In accordance with embodiments of the present invention a method and/or a system and/or system components including software are provided as network interfaces 11 (Fig. 3), and to measure (e.g. camera 15 and measuring system 16) the different end-to-end time delays between streamed, processed and displayed image frames on the display wall 10. Once the end-to-end time delays are measured, embodiments of the present invention provide a method and/or a system and/or system components including software to compensate (latency compensation 17 - shown schematically) for the differences of the time delays. Consequently data-synchronization can be guaranteed. A more detailed version of figure 2 is shown in figure 3 for Ethernet 100 base IP networks.

Test pattern

In accordance with embodiments of the present invention a pattern such as a test pattern is streamed from the source S to the displays at the endpoints, e.g. a display wall. The test pattern should be designed to all the determination of delays. To determine delays the test pattern should ideally have a portion of the image which is the same for each display when it is rendered. The test pattern can be, for example a full white field with time duration of m vertical display periods. This full white field tests pattern is rendered on each of the displays. The display's frame rate (vertical synchronization frequency) is denoted f _d , the vertical display time period is therefore T _d = - and the time duration of the test pattern is

I d

therefore m T _d .

In Figure 4 and Figure 5, Figure 4 illustrates a latency difference example of 2 Td between display Dr and Ds. Figure 5 shows the related images. At ί = t _x both D _r and D _s are black. At t = t ₂ and t = t ₃ the streamed full white field is white on display D _r and still black on display D _s . At t = t ₄ both D _r and D _s show a white image.

Image recording device

In embodiments of the present invention an optical recoding means such as a camera is used as part of the measuring system. This device preferably captures the complete image presented by the plurality of individual displays, e.g. by the complete display wall, at a rate of f _c =— frames per second. The capture can be optically as done with a camera which images the complete wall 10 or the data may be read out from each display electronically and combined into a electronic version of a camera image, e.g. a canvas. The processing of the date whether it is captured electronically or by a camera which converts optical signals into digital signals is basically the same so that in the following only the camera will be described. It should be understood however that other means such as electronic capture can be used equally well. An electronic capture device can operate in the same way as a camera, e.g. with an integration time and a vertical time period. Optionally the camera can operate asynchronously with respect to the displays. Henceforth only the description of a camera will be made for simplicity. The integration time, also called electronic shutter time, of the camera is denoted T _int . The camera's vertical time period is denoted T _c .

The following conditions must be fulfilled:

(1) T _lnt « T _c

(2) T _c < T _d

The display system can communicate it's frame rate f _d to the measurement system, therefore the measurement system can always adjust its capturing rate f _c to fulfill the condition T _c < T _d . It is assumed that the differential latency is n T _d with n = 0, 1, 2, 3, ... p. The duration of the white field test pattern m T _d is chosen such that p T _d < m T _d .

In the time diagrams shown in Figs. 6 and 8 to 11, the vertical arrow corresponds with the start of the camera's integration period (see Figure 6).

The image of the display wall is imaged in the camera's imaging system, e.g. on the camera's sensor. Each tile of the display wall is therefore imaged on adjacent areas ....in of the camera's imaging system, e.g. on adjacent areas of the camera's sensor (Figure 7). Differential end-to-end latency Γ _Δί , measurement

It has already been explained that the differential end-to-end latencies, denoted T _AL , are an integer multiple of the display's vertical time period:

T _AL = n T _d with n = 0, 1, 2, ... p and p < m

Consider now a delay of n T _d between displays D _r and D _s (Figure 8). The display luminance in the center of the screen of displays D _r and D _s is respectively L(D _r ) and L(O _s ). The integration time (also called the electronic shutter time) of the camera is set at for example The camera is preferably adapted so that the camera response has only two stable states or in other words has a binary output. Such a camera produces a binary image. One way of doing this is as follows. The images of D _r and D _s are sampled by the camera. L(D _r ) and L(D _S ) are causing a an image related change in the camera, e.g. a charge build-up C(D _r ) and C(D _S ) in certain locations of the camera, for example in the CCD sensor or CMOS sensor of the camera C(D _r ) and C(D _S ). Consider now the response of the camera in the center of the images of D _r and D _s , this is respectively RC(D _r ) and RC(D _S ). The camera settings are adjusted in such a way that the camera responses reach the saturation level within the integration time T _int . If this is not possible then a Schmitt trigger circuit can be added. We have therefore a camera response that has only two stable states or in other words a binary output. These camera responses are available at the output of the camera at the beginning of the next camera frame. The camera responses show in this case a time difference Δ, the measured time difference is not necessarily equal to n T _d , but this is not a problem. For example the system can be adapted so that the end-to-end differential latency is adjusted until the time difference will be eliminated. It is guaranteed that if n T _d ≠ 0 then Δ≠ 0, and that if n 7^ = 0 then A= 0.

The time resolution of the sampling depends on the sampling frequency of the camera f _c =— and the width of the sampling pulse T _int . The critical situation happens at n = 1. In this situation it is necessary that at least one sampling pulse is situated in the time period T _d , therefore the condition T _c < T _d must be fulfilled. Figure 9 shows the situation for the case when T _c < T _d .

Figure 10 shows the case T _c = T _d and Figure 11 shows the case for T _c < T _d . In embodiments of the present invention means is provided for measurement of the relative time delays between the responses RC(Di) with t = 1, 2, . . , n that starts with the simultaneous sampling of the signals flC(Dj) by a sampling signal 5 ₂ (t) with the following properties: (1] S ₂ (t) has exactly the same frequency as the camera sampling signal

(2) 5 ₂ (t) is synchronous with the camera sampling signal

T

(3) S ₂ (t) is delayed over a time y with respect to the camera sampling signal

This sampling is initiated after the leading edge of the RC(D ) with the shortest latency and during a time m T _d . This results in vectors v ₁ , v ₂ ,v ₃ , ~' V _l with binary elements. The last sample value is omitted in each of these vectors. The number of elements of each vector is therefore equal to the integer part of the ratio minus one: int \ _ Qne or more data paths corresponding with the camera responses can have the shortest latency and one or more data paths corresponding with the camera response can have the longest end-to- end latency.

Algorithm and compensation

The vectors i7 _f of the camera responses are corresponding with data paths corresponding with display images Di.

The vector resulting from the sampling of the RCiD^) corresponding with the data path with the shortest end-to-end latency can be represented as:

The camera response from a display with a delay equal to T _d will result in a vector that can be represented as

'V

I I

1

LI

A camera response from display with a delay equal to 2 T _d will result in a vector represented as

In general a camera response from a display with a delay equal to n T _d will result in a vector with the first n elements equal to zero.

Means is provided to calculate the squared length of each of the vectors v _t with

*2

The squared length of a vector x— is calculated and is defined as: ||*|| ² = x + x +

We call the "reference vector" v _re f, the vector corresponding with the data path with the longest end-to-end latency. This reference vector has the shortest length: miniUVi W ² , i = 1, 2, 3, . . , /} = ||t; _re/ || ² > l

The condition ||v _re /-|| ² > 1 must be fulfilled. In case one or more vectors are the zero vector, then the end-to-end latencies are exceeding the time duration of the test pattern. In such cases the time duration of the test pattern must be increased.

For each data path the amount of required compensation is: T _d for ⁽ = 1, 2, 3, ... / After this compensation each of the data paths will show the same end-to-end latency. In case

τηίη{||ν _έ || ² , i = 1, 2, 3, .. , /} = 0

Then the time duration of the test pattern m T _d must be increased until: mmdlVi ll ² , i = 1, 2, 3, .. , /} > 1 and the measurement must be restarted.

The complete measurement and compensation process 100 is depicted schematically in Figure 15. The algorithm starts at step 102. In step 104 a variable m is set to a default value. In step 106 one or more test patterns are streamed and are received at the display unit comprising the displays, the measuring system and the image recording unit (see Fig. 3 for example). The measuring system measures the delay of the displayed images in step 108. In step 110 it is decided if one or more camera responses are zero. If YES it goes to step 112 and the value of m is increased and the algorithm is re-entered before step 106. If NO it is determined in step 114 whether the length of the vectors are all equal. If YES, the process terminates. If No the latency compensation is sent in step 116 to the frame buffer controllers after which the algorithm terminates.

The present invention includes in an aspect that compensation can be implemented for the delays. In one embodiment dequeue control of the frame buffer 20 between the decoder 18 and the renderer 24 for each of the data paths is used (see Fig. 12). The frame buffer 20 can be organized as a FIFO (first-in first-out) memory structure see Figure 12. The frame buffer 20 can store a limited number, for example 5, frames. The decoded frames are enqueued to the FIFO (frame buffer 20) and the dequeued frames are rendered (in 24) and displayed (in 26). The difference of the end-to-end latency between the slowest data path and any other data path i is:

(llVi ll ² - ^τ α f r . = 1, 2, 3, ....

And let's denote

9i = (ll^ll ² - Ike _/ Ι ) for i = 1, 2, 3, ...

The measurement system 16 of embodiments of the present invention is preferably adapted to send the amount of latency compensation to each of the controllers 22. This may be provided by a function 17 that may include hardware and/or software. For example each of the controllers 22 is preferably adapted to wait to dequeue its next frame to the Tenderer 24 during a time g _L T _d . The example below shows a the evolution of the frame buffer contents for ^ = 0, g ₂ = 0, g ₃ = l, g ₄ = 3, g ₅ = 2, g ₆ = 1.

It is also important to remark that in case there are also differences in the delays of the renderer and/or display part then this part of the differential delay will also be compensated because the measurement system is measuring the complete end-to-end latency.

Further embodiment

Another embodiment of this invention is to delay the dequeue-operations of the streams with latencies lower than the stream(s) corresponding with the reference vector in the frame buffer (FIFO) before the decoding, i.e. in the compressed domain. The compressed frames require less memory space compared with the decoded frames, for example a factor of 20. Therefore, larger frame buffers (FIFOs) occupying less memory space can be realized in the compressed domain. This is a lower cost solution especially in cases where large differential latencies must be compensated. Figure 13 shows schematically a compressed frame buffer 30 with a decoder 34 and its own controller 32 which feeds the decoder (decompressed) data to the frame buffer 20 with its own controller 22 and from there to be dequeued in the Tenderer 24 and displayed on the display 26. Another embodiment of the compensation

Another embodiment of the invention is a heterogeneous processing method or system wherein multiple streams are processed within one physical controller and wherein the data within the controller follows different data paths. This is the case when one controller is equipped with for example both a hardware decoder [hardware accelerated decoder based on for example System on Chip (SoC) or programmable hardware) and a software decoder. One stream is then decoded by the hardware decoder 52 (data path 1, Figure 14) while the other stream is decoded by the software decoder 42 (data path 2, Figure 14). In practice, the latency of the hardware decoder 52 is often lower than the latency in the software decoder 42.

In general the two streams will require different latency compensations. It is, in this example, possible to do the latency compensation in a similar way as in the previous embodiments at the locations A or D and B or C. Locations A and B are attractive because the size of the frame buffers (FIFOs) for the compressed frames are much smaller compared with the frame buffers (FIFOs) for the decoded frames. When two or more streams are processed by one physical unit then it is of course necessary to measure the frame latencies of each of these two or more streams processed by one physical unit relative to all the frames of the streams on the other displays of the display wall. These measurements can be done as in the case of a single stream per physical unit but with two or more consecutive measurements or with adapted test patterns where areas on the displays are allocated to each stream.

In this example one data stream is sent to the hardware decoder 52 and the other stream is sent to the software decoder 42. The hardware and software decoders 52, 42 have in general different latencies, but other elements in the paths of the streams towards the decoders could already have caused additional different latencies for the two streams. There are frame buffers (FIFOs) 50, 54; 40, 44 before and after the decoders 52, 42 respectively. For the hardware decoder 52 this is "FIFO compressed frames stream 1" and "FIFO decoded frames stream 1". For the software decoder 42 this is "FIFO compressed frames stream 2" and "FIFO decoded frames stream 2" (Figure 14].

It is possible to delay the dequeue-operations of the streams with latencies lower than the stream(s) corresponding with the reference vector at "A" and "D" for stream 1 and at "B" and "C" for stream 2. The decoded frames for both data path 1 and 2 are then sent over the bus 60 to a memory 62 such as a GPU memory from which it is displayed. The frames in the compressed domain require less memory space than the frames in the decoded domain. Therefore larger frame buffers (FIFOs) occupying less memory space can be realized in the compressed domain.

Example

I = 2

\\v ₂ \\ ² = 2

\\v ₃ \\ ² = 3

\\v ₄ \\ ² = 5

||v ₅ || ² =4

\\ve\\ ² =

minfllViH ² , £ = 1,2,3,4,5,6} = 2 for i = 1 and 2

Fori = 3: ||v ₃ || ² - data path 3 must be delayed with a factor (\\v ₃ \\ ² -\\v _l \\ ² )T _d = lT _d

Fori = 4: ||v ₄ || ² - \\v _t \\ ² = 5 - 2 = 3 = data path 4 must be delayed with a factor \v ₄ \\ ² -\\v ₁ \\ ² T _d = 3T _d For i = 5: ||v ₅ || ² - ||ΐ7χ || ² = 4 - 2 = 2 data path 5 must be delayed with a factor

(l|v ₅ || ² - ||v ₁ || ² ) r _d = 2 r _(i For t = 6: ||v ₆ || ² - ΙΙνχ ΙΙ ² = 3 - 2 = 1 => data path 6 must be delayed with a factor

Therefore:

9 _\ = 0- 5z = ^Q > 93 = 1- ^4 - , ^5 = 2, g ₆ - 1

The present invention includes within its scope the provision of computer programs to be used with the embodiments. The software may be in the form of a computer program product for execution in a processing engine such as a microprocessor, an FPGA and more generally in a computer based system. The computer program product may be stored on a suitable signal storage means such as an optical disk (for example DVD, CDROM), magnetic disk (for example diskettes, harddrive), solid state memory (for example flash memory, USB stick), magnetic tape or similar. The computer program product may include code segments that are adapted to receive streaming test images over a network when executed in a processing engine such as a microprocessor or FPGA. The computer program product may include code segments that are adapted to provide such images to displays and to provide displaying of the test images on the displays. The computer program product may include code segments that are adapted to capture the displayed images at the different endpoints with an image recording device.

The computer program product may include code segments that are adapted to operate the displays frame locked and having double buffered swap locked memory operation.

The computer program product may include code segments that are adapted to measure delay between the images displayed on the displays.

The computer program product may include code segments that are adapted to measure end-to-end differential latency between displayed images and for putting this in vector form.

The computer program product may include code segments that are adapted to use the vectors to perform differential compensation of the delays at the displays.

The computer program product may include code segments that are adapted to send the vectors are to the displays to steer dequeue operations. The computer program product may include code segments that are adapted to send the vectors to frame buffers.

The computer program product may include code segments that are adapted to provide delay compensation, e.g. to the rendering units or decoders.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.