Description

Method and device for data processing in a communication net- work

The invention relates to a method and to a device for data processing in a communication network.

Mobile terminals are known to be optionally equipped with a microphone and/or a camera and they may be able to convey content to a streaming server or to any other fixed or mobile device. The mobile terminal may utilize different encoding formats .

Mobile terminals are able to receive real time multimedia content from the streaming server or from another fixed or mobile device. Changing radio channel conditions due to user mobility, interference, fading, etc. result in a performance degradation of the delivery chain. Also, congestion on the packet-based radio access network inflicts problems regarding the performance of the delivery chain.

In a high speed streaming application a considerable amount of content is in transit, residing in various buffers along the delivery chain, in particular being distributed among various layers of the transport and radio protocol architecture .

In case the content needs to be adapted or converted, these buffers are detrimental to the user experience, as data stored within these buffers first has to be forwarded to the destination before the user may experience the benefits of the converted content. For example, if a content of low quality is to be upgraded to a content of higher quality (e.g., different encoding scheme) , the user does not receive such high quality content until all low quality content buffered has been conveyed. As a result, the end user application may receive low quality content even though the networking condi- tions are already suitable for transmitting good quality content to this end user application.

In addition, providing good quality real time services via radio access networks is problematic due to

- the sensitivity of the real-time applications regarding a delay;

- a delay jitter;

- scarce resources over the mobile backhaul; and - fluctuations of the radio channel.

Video encoders used for streaming purposes typically provide a parameter regarding a "committed bitrate", which is an average rate of a stream for a predetermined duration, e.g., a couple of seconds. Frames carrying multimedia content are delayed and/or dropped in case there is a point of congestion in the network where the committed bitrate exceeds the temporarily available bandwidth. An increased delay and a frame loss leads to quality degradation of the multimedia service. Such a transient congestion scenario may be mitigated by deploying adaptive streaming servers (also referred to as media servers), i.e. to reduce the rate of the video stream in case performance degradation is reported. On the other hand, it is desired to increase the media quality for a better user experience and thus the bitrate for the encoded data may have to be increased, because the perceived quality of the video stream depends (among other factors) on the bitrate. However, with the bitrate being increased an increased amount of network bandwidth is required.

Alternative coding schemes such as Scalable Video Coding (SVC) are developed by MPEG (Moving Picture Experts Group) and ITU-VCEG (Video Coding Experts Group) . SVC encodes data (e.g., audio and/or video) via several layers comprising, e.g., a basic layer and at least one enhancement layer. Such data may be encoded in portions, i.e. frames. The basic layer requires a low transmission bitrate only and it represents a minimum bandwidth requirement for streaming data (also refer- red to as streaming content, content or media stream) . The basic layer also provides a minimum quality experience to the user. At least one enhancement layer can be provided in addition to the basic layer. If enough bandwidth is available, the enhancement layer may be conveyed to the user, thereby improving the quality of the streaming data conveyed by the basic layer. There may be several enhancement layers, each providing an improvement of the underlying layer (s) . The basic layer and the enhancement layer (s) are subject to a pro- gressive coding mechanism, i.e., each subsequent (enhancement) layer improves the quality of the underlying layer (s) in a predefined manner, but the subsequent layer requires additional bandwidth. The order of enhancement layers is defined by the progressive coding mechanism. Advantageously, an enhancement layer can be removed with little effort from a multi-layered stream, thus providing a very effective tool for stream adaptation even in a resource-limited network node. The streaming server may provide the streaming data with a fixed number of layers (e.g., basic layer and at least on enhancement layer) , and the effectively transmitted number of layers can be changed (e.g., by cancelling the transmission of at least one enhancement layer) according to an adaptation decision .

Current adaptation systems are based on a quality feedback provided by the clients (e.g., receivers). Such quality feedback can be used to fine-tune the bitrate at the sender. Such an end-to-end adaptation mechanism can be used for a mobile environment that has an inherently high latency.

In case of a (nearly) real-time streaming scenario with a rather short output buffer, the latency (round trip time of the feedback from the client) is deemed too long for an effective adaptation. The efficient management of radio channel quality fluctuations and throughput degradation, e.g., due to mobility issues and/or handovers of clients, are difficult to handle in particular because of a high and fluctuating latency. Hence, the buffer may be under-run or the scarce resources of the mobile backhaul may be a constant bottleneck that further increases the latency and eventually limits the efficiency of the adaptation algorithm running on the streaming server.

Hence, the radio channel quality degradation, the user mobility and the capacity limited mobile backhaul are factors that may result in an increased latency of the quality feedback sent to the streaming server (or the producer) reducing the efficiency of the adaptation of the media rate and thus the quality of the user experience.

It is also an objective for mobile producers, i.e. content producers utilizing a mobile terminal, to optimize the quality of the transmitted multimedia stream, while adapting to the available bandwidth on a more or less regular basis. Such an adaptation of the data rate avoids a buffer overflow at the receiver.

The media stream encoder of the mobile terminal allows choo- sing from a wide range of bitrates, thus the mobile terminal may determine a radio link quality and estimate a future state to employ effective adaptation.

Furthermore, for mobile consumers of the streaming media distributed by a streaming server, the same problem applies. Since the radio link is the weakest and most volatile portion of the network path, disturbances over the air interface may result in a significant delay between the base station and the mobile terminal at the receiving side. This additional delay is introduced by the automatic retransmission attempts of the radio layers (e.g. via HARQ) . Thus, the quality feedback from the mobile terminal will arrive too late at the streaming data producer in order to be utilized in an efficient way.

The problem to be solved is to overcome the disadvantages stated above and in particular to provide an efficient solution for mobile streaming applications. This problem is solved according to the features of the independent claims. Further embodiments result from the depending claims .

In order to overcome this problem, a method for data processing in a communication network is provided

- wherein streaming data conveyed from a streaming server towards at least one client via a network is controlled by a media adaptation entity that is associated with or deployed at a network component of said network.

Hence, the media adaptation entity located in the sphere of the network allows to efficiently control streaming data, in particular determine quality assessments for a portion of the path from the streaming server to the at least one client. The media adaptation entity may efficiently control, in particular modify, the streaming data conveyed to the at least one client and/or inform the streaming server to change the type, coding scheme or format of the streaming data provided.

According to an embodiment, the at least one client is a device connected to the network, in particular via a radio in- terface (e.g., cellular radio link or proximity radio link) or a wired interface. The client may be a mobile terminal, e.g., any device capable of receiving or sending streaming data via a radio interface. The mobile terminal may in particular be a computer, a personal digital assistant, a video processing device, a set-top box, a mobile phone, a device with a mobile interface or the like.

In an embodiment, said network component is a base station serving the at least one client.

The base station can be any BTS, NodeB or eNodeB serving at least one mobile terminal. In another embodiment, said network component is a gateway connecting the streaming server and the network.

The gateway may be a device deployed between the streaming server and the network, wherein the gateway logically deployed at the border of the network can also be considered a component of the network.

In a further embodiment, the streaming data comprises at Ie- ast one of the following:

- Audio data;

- Video data;

- Interactive multimedia data (e.g., DIMS);

- User data; - Signaling data;

- Overhead data;

- Program data or program information.

In a next embodiment, the streaming data is coded in a sca- lable format (e.g., SVC) comprising in particular a base layer and at least one enhancement layer.

It is also an embodiment that the streaming data comprises several data streams of different format, in particular each of the several data streams requiring a different bitrate.

Hence, such several data streams may be conveyed in parallel from the streaming server to the network component. The media adaptation entity may decide how to process these parallel data streams, e.g., by only forwarding one of those data streams which meets the conditions of the network path towards the at least one client.

Pursuant to another embodiment, the media adaptation entity determines a quality of at least a portion of a path from the streaming server to the at least one client. According to an embodiment, based on the quality determined the media adaptation entity adjusts the data stream conveyed towards the at least one client.

According to another embodiment, based on the quality determined the media adaptation entity informs the streaming server to adjust the streaming data.

Such an adjustment may refer to a coding type, format, number of formats, number of enhancement layers in case of SVC or the like.

Adjusting the streaming data may comprise dropping portions of the streaming data, e.g., at least one enhancement layer in case of SVC. Also, the coding scheme may be changed or a request of changing the coding scheme can be sent to the streaming server.

In yet another embodiment, the media adaptation entity deter- mines a current and a predicted quality of at least a portion of a path from the streaming server to the at least one client .

Advantageously, a prediction of future behavior of the con- nections between the streaming server and the at least one client may be assessed or predicted by the media adaptation entity, e.g., based on statistical models considering past experiences regarding delay, congestion or quality of the connection .

According to a next embodiment, the quality determined comprises at least one of the following:

- A throughput over a radio link;

- A round trip time of a marked data packet; - A latency detected within the network;

- A congestion detected within the network;

- A buffer overflow and/or a buffer underrun. Pursuant to yet an embodiment, a stream producer provides streaming data towards the streaming server and/or towards a client .

As an option, the stream producer may be a mobile terminal and it may adjust the coding format and/or bitrate of the streaming data according to quality information of the link towards the streaming server and/or towards the client.

According to a further embodiment, an adaptation of the streaming data is provided by the stream producer in particular based on a quality of at least a portion of a path from the stream producer to the streaming server and/or to the client .

The problem stated above is also solved by a device comprising and/or being associated with a processor unit and/or a hard-wired circuit and/or a logic device that is arranged such that the method as described herein is executable there- on.

According to an embodiment, the device is a communication device, in particular or being associated with a network element, a base station, a streaming server, a stream producer or a gateway.

The problem stated supra is further solved by a communication system comprising the device as described herein.

Embodiments of the invention are shown and illustrated in the following figures:

Fig.l shows an overview of a mobile streaming scenario comprising a stream producer, e.g., a computer with a video camera, conveying content to a streaming server, wherein such streaming server supplies said content via a gateway, a radio access network and via a base station to a mobile terminal or any other mobile consumer;

Fig.2 shows an overview of a mobile streaming scenario com- prising a mobile stream producer (e.g. mobile phone with an embedded video camera) and a mobile terminal as destination;

Fig.3 shows a block diagram enabling a mobile streaming ad- aptation with a network buffer manipulation with a media adaptation entity being deployed or associated with a base station;

Fig.4 shows a block diagram enabling a different mobile streaming adaptation with a network buffer manipulation based on the diagram shown in Fig.3;

Fig.5 shows a block diagram enabling a mobile streaming adaptation, wherein a media adaptation entity is de- ployed within a gateway;

Fig.6 shows an overview of a mobile streaming scenario comprising a mobile terminal that acts as a mobile streamer (also referred to as "stream producer") and conveys streaming data via a base station and a radio access network to a streaming server, which forwards streaming data towards clients, e.g., a mobile terminal, a computer or any device that can be connected to the streaming server via a radio link or a wired connection;

Fig.7 shows a phone-to-phone streaming scenario comprising a mobile terminal acting as a mobile stream producer conveying streaming data via a proximity radio link to a client and/or via a cellular link to a base station, wherein the base station conveys the streaming data via a first radio access network, a core network and via a second radio access network to a next base station that conveys the streaming data further to a client;

Fig.8 shows a streaming scenario with a feedback loop at the side of the stream producer;

Fig.9 shows a message sequence chart visualizing the feedback loop functionality at the side of the stream producer between said stream producer (mobile termi- nal) , the base station and the streaming server;

Fig.10 shows a diagram visualizing a decision feedback loop functionality at the side of the stream producer based on the message sequence chart according to Fig.9;

Fig.11 shows a mobile streaming scenario comprising a receiver side feedback loop between a streaming server and a base station, said base station supplying a mo- bile terminal (client) being the destination of the streaming data;

Fig.12 shows a message sequence chart visualizing the feedback loop functionality at the side of the receiver between said client (mobile terminal) , the base station and the streaming server;

Fig.13 shows a schematic diagram to visualize a decision algorithm in a HSDPA environment.

Fig.l shows an overview of a mobile streaming scenario comprising a stream producer 101, e.g., a computer, a mobile phone, a video camera, conveying content to a streaming server 102. Said streaming server 102 supplies said content via a gateway 103 (e.g., a GGSN), a radio access network 104

(e.g., a cellular network or a WiMAX) and via a base station 105 (e.g., a BTS, a NodeB, an eNodeB or any access point) to a mobile terminal 106 or any other mobile consumer. The base station 105 and/or the gateway 103 can be utilized for streaming data manipulation purposes and/or for communicating with the streaming server 102 in order to change (inc- rease or reduce) the data rate supplied by the streaming server 102. Such manipulation or interaction can be done based on the condition of, e.g., the radio access network.

Hence, in the mobile streaming scenario as shown in Fig.l, the mobile terminal 106 receives the streaming data issued by the streaming server 102, which is attached to an operator's network or to the Internet. The streaming data is encoded in a compact form and transmitted to the mobile terminal 106 via a radio link or any (other) proximity access link.

It is noted that the mobile terminal may be any device capable of receiving or sending streaming data via a radio interface. The mobile terminal may in particular be a computer, a personal digital assistant, a video processing device, a set-top box, a device with a mobile interface or the like.

It is further noted that said streaming data may refer to audio and/or video data. Also, signaling data, user data or overhead data may be part of such information conveyed from one entity to another. Also, said streaming data may comprise program information or program code.

It is also noted that streaming data refers to any data conveyed from one network component (mobile of fixed) to another, in particular via at least one further network component. Such streaming data can also be referred to as, e.g., streaming content, content or media stream. The streaming data may in particular be data that is broadcasted from one network component.

Fig.2 shows an overview of a mobile streaming scenario comprising a mobile stream producer 201 and a mobile terminal 202 as addressee. The mobile stream producer 201 (which can be a mobile terminal of any kind) conveys the content via a base station 203 (e.g., BTS, NodeB, eNodeB) and a gateway 204 (e.g., a GGSN) to a streaming server 205. The streaming server 205 conveys the content via said gateway 204 and via a base station 206 (e.g., BTS, NodeB, eNodeB) to the mobile terminal 202. The base stations 203, 206 and the gateway 204 may be part of or may be associated with a radio access network 207.

The solutions provided herein in particular utilize the network between a streaming server and a media consumer (client) . The streaming server and/or the media consumer can be a mobile terminal. Hence, an adaptation point can be introduced in between the streaming server and the client, in particular at a or associated with a network element, e.g.,

- a base station (also referred to as BTS, NodeB, eNodeB) and/or

- a gateway (e.g., at an SGSN, a GGSN, an MME/UPE, an SGW, a HSGW, a PGW) .

These network elements are suitable to provide the suggested functionalities, as they are controlled by the mobile operators, in contrast to the mobile backhaul, which might be leased from a transport network service provider.

The base station can be utilized as adaptation point at the boundary of the mobile backhaul. The radio interface may buffer downstream data of a connection before a packet sched- uler selects it for transmission over the air interface. The amount of buffered data increases in case the channel quality deteriorates (due to the latency of the adaptation, a significant amount of data can be buffered at the base station before the data rate towards the base station decreases) . The approach provided allows the base station to manipulate media content stored in its buffers by selectively transmitting it according to a current and/or according to a predicted qual- ity of the radio link and/or according to a measured throughput of the air interface.

Also, the gateway can be utilized as adaptation point being located at the boundary of the core network and the mobile backhaul . The gateway may measure a RTT for each base station. This information can be used in order to detect a congestion on the transport network. Manipulating the media content and/or selecting the proper format and/or selectively discarding any enhancement frames can be provided in particular for the following reasons:

- to improve the level of service provided to the end user;

- to mitigate a congestion by reducing the load on the mobile backhaul links;

- to improve an efficiency; in case of a congestion, frames that are delayed for more than a predetermined time are dropped at the destination, which would result in a waste of bandwidth.

Media adaptation entity in base station

Fig.3 shows a block diagram enabling a mobile streaming adaptation with a network buffer manipulation comprising a media adaptation entity 301 deployed with a base station 302, wherein said media adaptation entity 301 receives a data stream 303 from a radio access network 304 and forwards a data stream 305 towards a mobile media consumer, e.g., a mobile terminal 306.

The base station 302 further comprises a buffer 307 and a packet scheduler 308, wherein the media adaptation entity 301 conveys the data stream 305 to the buffer 307 and the packet scheduler 308 utilizes said buffer 307 to convey data packets 309 to said mobile terminal 306.

The data stream 303 is encoded in a progressive encoding scheme, e.g., SVC, comprising a base layer 310 and at least one enhancement layer 311. The access network 304 (e.g., a radio access network, a cellular network or a broadband network) comprises a gateway 313 that is connected to a streaming server 314. Hence, the data stream 303 originates at said streaming server 314 and is conveyed via said gateway 313 and the access network 304 to the media adaptation entity 301.

The output of the buffer 307 can be utilized to measure a throughput of the converted data 305 and convey said measure- ment to the media adaptation entity 301 for further adaptation purposes. The media adaptation entity 301 may store a combination of base layer and enhancement layer information in the buffer 307. In addition, the media adaptation entity 301 may be provided with channel quality information 315 from an external source (e.g., from the mobile terminal 306 or from the packet scheduler 308) . All the information made available to the media adaptation entity 301 can be utilized at the media adaptation entity 301 to convert the data in a way that the data stream 305 corresponds to the actual chan- nel conditions and may utilize the available bandwidth in an efficient manner.

It is noted that a layered media encoding scheme (e.g., SVC) can be utilized. If streaming data is distributed via a basic layer and at least one enhancement layer, the media adaptation entity 301 may select the appropriate set of enhancement frames to be buffered. The incoming frames are checked, the basic and enhancement frames are identified and based on the reported radio channel quality and/or the measured throughput over the radio interface the media adaptation entity 301 may discard or store the enhancement frames of a layer (basic fames are always stored) . Thus, an appropriate set of layers can be selected for each single connection.

In order to avoid additional (and most likely unnecessary) load to be processed by the access network 304 (mobile back- haul) , the streaming server 314 is informed by the media adaptation entity 301 about layers that could not (or will not) be conveyed to the mobile terminal 306. Such information regarding streaming data not transmitted and/or dropped by the media adaptation entity 301 is conveyed via a feedback loop 316 to the streaming server 314 indicating, e.g., a maximum number of layers that can be utilized by the mobile terminal 306 (or by several mobile terminals) . This information can be used by the streaming server 314 to only transmit these layers (or information to an extent) which actually can be utilized. Hence, the streaming server 314 may cease transmitting some enhancement layers towards the base station 302.

Fig.4 shows a block diagram enabling a mobile streaming adaptation with a network buffer manipulation based on the diagram shown in Fig.3.

In contrast to Fig.3, a data stream 401 is conveyed from the streaming server 314 towards the media adaptation entity 301, wherein said data stream 401 comprises a data stream 402 of high quality, a data stream 403 of medium quality and a data stream 404 of low quality. The media adaptation entity 301 may decide upon information made available (e.g., throughput measured after the buffer 307 and/or channel quality information 315) which data stream 402 to 404 to convey towards the mobile terminal 306. The buffer 307 is used to store frames from the different data streams 402 to 404, hence, a quality change can be triggered by the media adaptation entity 301 by conveying a subsequent frame of a different data stream 402 to 404.

In the example of Fig.4 no layered media content encoding is required. The media adaptation entity 301 requests a particular content (requested by the mobile terminal 306) in several different formats (402 to 404) . As a result of the reported radio channel quality and/or as a result of the measured throughput over the radio interface, the media adaptation entity 301 selects which of the different formats 402 to 404 fits the considered single connection towards the mobile ter- minal 306 and it may selectively drop those frames that are not part of the selected format.

Additionally, if the set of n formats requested from the streaming server 314 turns out to be insufficient in a sense that the radio conditions are even better than the best format requested or in case the radio conditions are worse than the level of the worst format, the media adaptation may request (via the feedback loop 316) a different set of formats to be provided from the streaming server 314.

As stated above in connection with Fig.3, the feedback loop 316 may be used to change (reduce or increase) the number (and/or type of coding) of formats 402 to 404 to be provided by the streaming server 314.

In addition or as an alternative, a number of parallel formats can be selected as a tradeoff between the capacity of the mobile backhaul and the level of user experience. For ex- ample, in order to efficiently reduce the backhaul capacity required, only two formats may be requested to be provided in parallel by the streaming server 314. One format can be set to be a worst case scenario (i.e., a format providing the lowest bitrate) and a second format can be set to be applica- ble for a current situation determined by the media adaptation entity 301. Hence, a sudden drop of the radio network condition can be compensated by switching to this worst case format. Afterwards, the second format could be chosen providing a stream of relatively good quality. This second format can be adjusted by requesting a better format from the streaming server 314 after conditions of the channel improved for more than a given period of time (hysteresis) . Accordingly, a worsening condition of the channel could be met by requesting a lower bitrate to be provided as the second for- mat from the streaming server 314. However, in any case, the lowest bitrate format may be maintained as a fallback bitrate . Media adaptation entity in Cellular Gateway

The media adaptation functionality may be deployed and/or associated with a gateway (GW), e.g., a SGSN, a GGSN, an ISN, a MME/UPE, an SGW, a HSGW, a PGW.

The gateway may be a network element between the access network and the streaming server. The gateway can be considered to be also a part of the access network.

Fig.5 shows a block diagram enabling a mobile streaming adaptation, wherein a media adaptation entity 505 is deployed within a gateway 502.

A network 506 comprises a base station 501 that is connected to a mobile terminal 503. The network 506 is also associated with said gateway 502, which is connected to a streaming server 504.

The streaming server 504 conveys a data stream 507 comprising a base layer 508, a first enhancement layer 509 and a second enhancement layer 510 to the media adaptation entity 505. Based on a RTT measurement loop 511 with marked packets, the media adaptation entity 505 is aware of the status of the mo- bile backhaul, namely the quality of the transport link between itself and the base station 501. Hence the media adaptation entity 505 may process the data stream 507 received according to the measurement results obtained by the RTT measurement loop 511 and convey an adjusted data stream 513 (in this example comprising the base layer 508 and the first enhancement layer 509 only) towards the base station 501. The base station 501 forwards the data stream 513 (base layer 508 and enhancement layer 509) to the mobile terminal 503.

With the media adaptation entity 505 reducing the amount of traffic forwarded as data stream 513 (compared to the data stream 507), the media adaptation entity 505 may inform the streaming server 504 via a feedback channel 512. The stream- ing server may then adjust the traffic 507 accordingly to the information obtained via the feedback channel 512. The same applies in an analogue manner for increasing the data streams or for changing the coding of at least one data stream pro- vided by the streaming server.

Hence, as mentioned above, the media adaptation entity 505 may provide an adaptation functionality for adjusting data traffic towards the mobile terminal 503 and feedback may be provided to the streaming server. The media adaptation entity 505 sends marked packets to the base station which sends them back to the media adaptation entity 505. This way, the RTT over the mobile backhaul can be determined, a congestion can be detected and the media adaptation entity 505 can select the number of (enhancement) layers that are forwarded as data stream 507 towards the mobile terminal 503.

This embodiment may work without an adaptation entity at the base station 501; also, a feedback channel 512 is not re- quired.

As an option, the examples according to Fig.3 or Fig.4 and Fig.5 may be combined. Hence, a media adaptation entity can be provided with the gateway and with the base station. Then, the media adaptation entity of the gateway may perform a coarse shaping of the data stream, e.g., the number of enhancements layers transmitted from the streaming server and the media adaptation entity in the base station may provide a fine tuning functionality.

It is also an option with regard to the embodiment described above that the media adaptation entity 505 of the gateway 502 may provide a functionality similar to the case of the non- scalable content. Hence, the media adaptation entity 505 may collect information on the status of the load of the mobile backhaul by sending marked packets to the base station 501. The base station 501 receives the marked packets and forwards them back to the media adaptation entity 505. Thus, the RTT over the mobile backhaul can be determined and a congestion can be detected. The media adaptation entity 505 may thus select the amount of parallel data stream formats (e.g., several (independent) streams, each of a different quality level) that are forwarded via the base station 501 to the mobile terminal 503.

Further Examples and Advantages

An exemplary implementation may comprise a streaming server based on a RSTP/RTP streaming protocol stack, at least one base station (e.g., BTS, NodeB, eNodeB) serving a cellular network on the consumer side and at least one fixed and/or at least one mobile streaming client (s) with a RTP/RTCP-based content player. Any subset thereof may be applicable as well.

The media adaptation entity can be any hardware and/or software component running on (or being attached to) a base station or a gateway. Such component may in particular identify data flows that carry streaming media and data flows using layered video coding or using alternate streams with different formats. While streaming, the component may iteratively or continuously monitor radio link parameters (e.g., interference) or it may collect measured throughput. The component may also maintain a prediction model. Such prediction model could be realized as a hidden Markov chain-based estimate of the radio channel throughput.

Alternatively or in addition, the media adaptation entity running on or being attached to the gateway is enabled to identify streaming connections. The component may iteratively or continuously monitor the RTT of the mobile backhaul and detect an increasing delay. The measurement can be conducted by sending marked packets towards the base station. Upon re- ceiving these packets, the base station sends them back to the gateway. Hence, the component can calculate the current RTT of the mobile backhaul. Based on the measured radio channel quality or based on predicted radio channel quality or based on a congestion detected, the hardware and/or software component may decide upon a number of enhancement layers to be conveyed to the user. Alternatively, the component may select the appropriate format from several parallel streams of varying quality that fits into the measured end-to-end network throughput. Advantageously, the network buffers are not constantly filled with data that cannot be conveyed across the network to the mobile terminal. Hence, it can keep the streaming application interactive by controlling the delay caused by network buffers.

The frames corresponding to (enhancement) layers that shall not be transmitted to the mobile terminal are either removed from the buffer or they are discarded before being stored in the buffer. Hence, the media rate is adapted to the status of the radio link throughput or bandwidth over the mobile back- haul before any end-to-end feedback cycle is able to detect a change of such status.

In order to detect frames in the incoming flow, the hardware and/or software component may use a buffer to collect the packets of the frames. Since data of the frames may be transmitted in several packets, the component cannot decide which packet should be dropped by monitoring only the packets' data. Instead, the frames may be reconstructed before an enhancement layer is dropped.

In case of parallel formats of different quality levels (dif- ferent streams, each having a quality level of its own) , the server advertises the streaming data providing stream identifiers of the different streams. Thus, the component may check the stream identifiers provided with the RTP headers. In this example, no frame reconstruction is required as long as switching to an alternate stream is conducted at the frame boundaries . If the subscribers of the base station and/or the gateway receive a reduced set of enhancement frames, the hardware and/or software component may trigger the streaming server to stop transmitting the enhancement layers which will be dropped. This can be achieved with a secondary RTCP channel or by using any other signaling method. When any of the buffers is going to be empty and/or in case a (predicted) state of the radio link indicates that higher (enhancement) layers of the streams could be processed, the component may again request those layers from the streaming server. This feedback bears the advantage that the load of the mobile backhaul is effectively utilized, e.g., reduced in case of a radio throughput decrease.

If parallel streams are used instead of SVC (base layer and at least one enhancement layer) and none of the subscribers of the base station and/or the gateway receives the high quality stream, the component may indicate to the streaming server to stop sending this non-utilized stream.

Since, in general, in order to achieve the same quality the layered video coding requires more bandwidth than the single layer counterparts, other ways of media adaptations may be considered. For example, the base station may transcode the buffered media for each flow.

Stream Producer

Fig.6 shows an overview of a mobile streaming scenario com- prising a mobile terminal 601 that acts as a mobile streamer (also referred to as "stream producer") and conveys streaming data via a base station 602 (e.g., a base station) and a radio access network 603 (e.g., a cellular network or a WiMAX) to a streaming server 604. The streaming server 604 forwards streaming data towards clients 605, 606, e.g., a mobile terminal, a computer or any device that can be connected to the streaming server 604 via a radio link or a wired connection. On the client an application can be run that allows playing the streaming data.

The streaming server 604 can be attached to an operator's network or to the Internet. The mobile terminal 601 may generate and encode the streaming data to a compact format (e.g., to an MPEG4 stream) and transmit the streaming data towards the base station 602 via a cellular (GPRS, 3G, HSPA or LTE) or via a broadband (WiMAX) radio access link. The base station may be any access point, BTS, NodeB or eNodeB.

As an alternative, the mobile terminal 601 may utilize a Bluetooth or a WiFi interface and convey the streaming data over a proximity access link instead of the cellular link. The base station 602 (being a cellular base station or a proximity base station) forwards the streaming data to the streaming server 604, which distributes it to the clients 605, 606.

It is noted that streaming data is only one example of data to be conveyed from the mobile terminal 601 to the clients 605, 606. The data may not have to be real-time streaming data, it could also be recorded as a file and stored at the mobile terminal 601 and/or the streaming server 604. Further- more, said data may comprise audio, video and/or other user, overhead or signaling data. It is also possible that such data comprises program data that could be conveyed.

Fig.7 shows a phone-to-phone streaming scenario comprising a mobile terminal 801 acting as a mobile stream producer. The streaming data can be conveyed from the mobile terminal 801 via a proximity radio link to a client 802 and/or via a cellular link to a base station 803 (also, a proximity radio link could be used to reach said base station 803) . The base station 803 conveys the streaming data via a first radio access network 804, a core network 805 (e.g., the Internet) and via a second radio access network 806 to a base station 807 that conveys the streaming data further to a client 808. Hence, the streaming data, e.g., multi-media content or realtime video stream, can be exchanged between the mobile terminal 801 as content producer (or server) and the respective clients 802, 808.

The proposal provided herewith allows for two separate adaptations: One adaptation from the broadcasting entity (producer, streaming entity) to a streaming server and another adaptation from the streaming server to a client.

At least one of these adaptations (in particular both of them) may be applicable.

These two adaptations can be dealt with separately. For example, real-time modifications (e.g., stream-thinning, transcoding to another format) of primary streaming data provided by the stream producer can be performed by the streaming server (located, e.g., within the network) in order to create and provide a secondary content tailored to at least one client's needs. The stream producer may not have to adapt the stream parameters to each and every client, but only to the network conditions between itself and the streaming server .

An adaptation between the stream producer and a client affects the user experience of this client while an adaptation between the stream producer and the streaming server affects the user experience of all clients supplied by said streaming server.

An adaptation between the streaming server and its clients can be provided, e.g., on a per-client-cluster basis. A client cluster may comprise a set of clients that accommodate bit rates within a certain range. It can be ensured that streaming servers are not overloaded if the number of clusters is finite. The adaptation between the stream producer and the client is different, because a data loss within the primary content will affect the user experience for each such client.

Hence, the present approach also suggests a system comprising a streaming application, adaptation agent (s) associated or deployed with the stream producer, a streaming server, client-side adaptation agent (s) and streaming client (s).

The at least one agent of the stream producer may perform an adaptation for the radio link part of the network path directed from the stream producer towards the streaming server. Such an agent may be running on the stream producer as a standalone application; it may iteratively or continuously monitor a radio link quality and/or it may measure a throughput that has been reached. In addition, the agent could predict the link quality in the near future. Statistical models may be applicable for such prediction purposes.

Fig.8 shows a streaming scenario with a feedback loop at the stream producer. A mobile terminal 901 (acting as stream producer) conveys streaming data via a base station 902 and a radio access network 903 to a streaming server 904. The streaming server 904 supplies said streaming data to clients 905, 906 (clients with a radio interface, mobile clients or clients connected to the streaming server 904 via a wired interface) . The mobile terminal 901 comprises a hardware and/or software component (also referred to as agent) that checks the radio link quality, e.g., on a regular basis. This feed- back loop at the producer side is indicated by an arrow 907.

Additionally or as an alternative, the agent may work in a distributed fashion, i.e. a portion thereof may be deployed with the mobile terminal 901 and another portion of the agent may be deployed in the radio access network 903. This may provide a more accurate assessment of present and future radio conditions for the streaming application. If the predicted link quality is too low for the bandwidth requirements of the current rate of the streaming data, the bitrate of the encoder running at the mobile terminal 901 can be decreased. However, if the predicted link quality returns to the previous high state (e.g., after a handover has been processed) the adaptation may restore the previous (high) encoder bitrate and starts sending the streaming data at such a high bitrate.

A fast adaptation in the bitrate may lead (dependent on the frequency of changes and the level of chances) to an oscillating video quality at the receiver which may be deemed worse than a low but constant quality. For this reason, the adaptation may pursue a conservative approach when increasing the bitrate, e.g., the bitrate may be increased only when the radio network conditions are "consistent" over a predetermined period of time.

Fig.9 shows a message sequence chart visualizing the feedback loop functionality at the side of the stream producer between said stream producer (mobile terminal 901), the base station 902 and the streaming server 904. In a first step 1001, a streaming session between the mobile terminal 901 and the streaming server 904 is established, both units agree on a committed bitrate amounting to n kbps . In a message 1002, the base station determines a deterioration of the radio link and provides a "negative feedback" to the mobile terminal 901. The mobile terminal 901 watches a trend regarding the radio network conditions (e.g., waits for a predetermined period of time) and decides to change the encoding scheme to cope with a reduced bandwidth (see message 1003) . In a next step 1004, the mobile terminal 901 and the streaming server 904 continue the streaming session with modified encoding parameters, both units agree on a committed bit rate of n-x kbps (x being the reduction of the bitrate due to the deteriorated radio link condition) . Fig.10 shows a diagram visualizing a decision feedback loop functionality at the side of the stream producer based on the message sequence chart according to Fig.9.

The base station 902 sends the "negative feedback" message to the mobile terminal 901, in particular to an adaptation agent 1101 of the mobile terminal 901. The adaptation agent 1101 instructs Media Tools 1102 (providing recording services of audio/visual content) to reduce the committed encoder bi- trate, which reduces the encoder bitrate and provides recorded content to a transmission tool 1103, which forwards the streaming data to the base station 902.

For example, in case of an HSDPA link between a NodeB and a UE on which the stream producer is running, the "negative feedback" message may indicate "decrease transmission power on E-DPDCH"; as an alternative, the "negative feedback" message may indicate that the number of HARQ NACK messages increases or the "negative feedback" message may indicate any combination thereof. Additionally, the base station 902 may convey the "negative feedback" to the mobile terminal 901 via a Relative Grant Channel (E-RGCH) or via an Absolute Grant Channel (E-AGCH) . The mobile terminal 901 watches the resource availability by monitoring the received grants during a time interval. Additionally, the mobile terminal 901 might clear a "happy bit" on the Dedicated Physical Control Channel (E-DPCCH) several times consecutively, e.g., every 2 milliseconds, indicating to the base station 902 that it requires additional resources from the base station 902. In case the base station 902 can allocate additional grants to the mobile terminal 901, the encoding scheme is not changed. If, however, grant allocation cannot allocate resources to the mobile terminal 901, then the encoding scheme will be changed.

Considering the other part of the end-to-end system (see, e.g., Fig.11 as described below) between the streaming server and the clients, a similar concept to improve feedback accuracy may apply. If the client is a mobile device, this part of the communication comprises a radio link. In case of radio link quality degradation, retransmission may be required between the base station and the mobile terminal resulting in long end-to-end delays.

The base station can be aware of any radio link quality degradation and it may report the change of the radio link quality and/or the actual radio link condition to the streaming server. This way the last segment of the downstream network path, which suffers from high delays, can be omitted from the feedback loop. Thus there is no need to wait for the feedback from the downstream client, which would anyway be late due to the increased delay. In this scenario, also a conservative approach can be followed when the bitrate is to be increased and a proactive approach can be followed when the bitrate is to be decreased, because a low bitrate, i.e. a reduced quality of the streaming data, is preferable over an interruption based on a high bitrate that cannot be maintained.

The approach suggested herein also applies in case a distribution of the streaming data is not relayed by the streaming server. In such a scenario, the stream is directly transferred from the mobile stream producer to the (mobile) client. Hence, the stream producer may also monitor the radio link quality and it may adapt the bitrate accordingly to efficiently utilize the uplink bandwidth available. Furthermore, the base station to which the mobile client is attached may report radio link problems of the client directly to the mobile stream producer. Hence, the stream producer can adapt the rate of the streaming data sooner than it would be possible based on the feedback from the client consuming the streaming data.

Further Examples and Advantages

An exemplary implementation comprises a mobile terminal with a streaming producer application, a radio network (e.g., 2.5G, 3G, 3.5G, WiFi, Bluetooth, WiMAX) on the side of the stream producer (mobile terminal) , wherein an adaptation agent is provided with or implemented in the producer and/or the network. Also, a streaming server based on a RTSP/RTP streaming protocol stack, a radio network (cellular, WiFi, WiMAX) on the consumer (client) side and at least one fixed streaming client and/or at least one mobile streaming client can be provided. In case of a mobile client, the adaptation engine can be associated with and/or implemented in the mobile client and/or in the consumer side network. Such an ad- aptation engine may even be distributed among the clients and the consumer side radio network. Any subset of the above mentioned components may be applicable as well.

The agent can be implemented in software and/or hardware ac- tive on the stream producer, said agent being also referred to as adaptation agent. A producer side adaptation is performed via the producer side feedback loop 907 as shown in Fig.8. While streaming, said adaptation agent may iteratively or continuously monitor the radio link parameters (e.g., in- terference) or it may assess and/or maintain a prediction model. Such prediction model may be a hidden Markov chain- based estimate of the radio channel throughput.

Also, the adaptation agent may estimate an available through- put by using an amount of granted credits in case of HSUPA (NodeB assisted estimation option 1) or by measuring an amount of transferred data over a given sliding window according to an alternative embodiment. In the latter case, monitoring the amount of transferred data corresponds to monitoring the amount of received NACKs in HARQ (NodeB assisted estimation option 2) . When the number of NACK messages reaches a certain threshold, the adaptation engine may send an adaptation command towards the encoding process to decrease the committed bitrate of the encoder. In a WiMAX sce- nario, the mobile stream producer itself may monitor the amount of granted timeslots for RTPS traffic and compare it with an output packet buffer containing the queued packets of the stream. As the prediction model shows significant link degradation, the adaptation agent may direct an adaptation request towards the streaming application. Link degradation according to HSUPA may result in an insufficient amount of transmission power granted to the stream producer on the E-DPDCH channel and/or it may result in the number of NACK messages decreasing (see Fig.10) .

In a WiMAX scenario, a degradation of the radio link may be derived from the fact that the size of queued packets systematically exceeds the capacity of the timeslots allocated to the stream producer by the RTPS scheduler. Similar measures can be defined for basic 3G, GPRS and WiFi as well.

As an option, the bitrate parameter of the encoder can be decreased in order for the overall stream bitrate to remain below the predicted performance of the link. In this particular embodiment, in order to enable seamless media bitrate changes, the parameter may be delayed until a subsequent I- frame is being coded. Furthermore, the encoder may introduce a new I-frame at the adaptation point.

If the prediction model determines a significantly higher link quality, and the application allows adjustment to a higher bitrate, an adaptation request can be addressed to the streaming application. For example, as a result of such adaptation request the encoder bitrate may be increased according to the I-frame boundary.

The client side adaptation can be performed via a feedback loop deployed at the receiver side. Fig.11 shows a mobile streaming scenario comprising a receiver side feedback loop. A mobile terminal 1201 (stream producer) provides streaming data via a base station 1202 and a radio access network 1203 to a streaming server 1204. The streaming server 1204 conveys said streaming data via a radio access network 1205 and via a base station 1206 to a client 1207 (e.g., a mobile terminal). With the mobile terminal 1207 being the destination of the streaming data, the adaptation agent can be deployed at or associated with the base station 1206. This adaptation agent may evaluate the quality of the connection via the radio ac- cess network 1205 via the feedback loop 1208.

It is noted that the base station 1206 may be any BTS, NodeB, eNodeB for cellular applications or wireless access point in case of WiMAX. The base station 1206 may be aware of the channel quality and/or the throughput of each single user. This information can be used in order to detect and/or to predict channel quality degradation. Upon such detection and/or prediction of a degradation the adaptation agent may instruct the streaming server 1204 to decrease the bitrate of the streaming data. Thus, latency of the adaptation can be significantly reduced.

Fig.12 shows a message sequence chart visualizing the feedback loop functionality at the side of the receiver between said client (mobile terminal 1207), the base station 1206 and the streaming server 1204. In a first step 1301, a streaming session between the mobile terminal 1207 and the streaming server 1204 is established, both units agree on a committed bitrate amounting to n kbps . In a message 1302, the mobile terminal 1207 determines a deterioration of the radio link and provides a corresponding message to the base station 1206. The base station 1206 watches a trend regarding the radio network conditions (e.g., waits for a predetermined period of time) and decides to switch the bit rate. In such case, the base station 1206 sends a message 1304 to the streaming server 1204 indicating that the stream shall be adapted. In a next step 1305, the mobile terminal 1207 and the streaming server 1204 continue the streaming session with an adapted stream, both units agree on a committed bit rate of n-x kbps (x being the reduction of the bitrate due to the deteriorated radio link condition) . For example, in case of an HSDPA link between a NodeB and a UE on which the streaming client is running, the adaptation agent could work as follows. The UE may report a channel quality indicator (CQI) perceived to the NodeB using a re- porting mechanism that can range from, e.g., one report every 2 milliseconds to, e.g., an adaptive reporting scheme.

The NodeB may be aware of the streaming character of the data flow and the NodeB may comprise a software component extract- ing IP headers from the packets of the data flow. Hence, the NodeB is aware of an IP address of the streaming server and the NodeB may directly, on behalf of the streaming client, request the streaming server to change the bitrate, e.g., to switch to a lower bitrate.

An adaptation may comprise stream thinning and/or transcoding in particular for real time content and/or multiple tracks may be encoded in a 3GP file with different bit rates for non real time content.

The request for adaptation can be provided via an RTCP NADU or SR package or any custom RTCP package generically requesting to reduce the throughput towards this particular client. Alternatively or in addition, the request for adaptation can be provided using an RSTP SET_PARAMETER message in which an optional field "Air-Interface-State" can be adequately filled.

Fig.13 shows a schematic diagram to visualize a decision al- gorithm in a HSDPA environment. Several mobile terminals

1401, 1402 and 1403 each provide a CQI report to a base station 1404. The respective CQI values of each mobile terminal are depicted over time. The CQI provided by the mobile terminal 1403 (referred to as CQI _n ) reaches a first threshold thi thereby starting a CQI timer. At a time 1405, said CQI timer expires and the decision is made by the base station 1404 to inform the streaming server and to lower the threshold from thi to th ₂ . For example, a specific RTCP APP packet can be defined to request the lowering of throughput to this particular client as follows:

V=2 10101 0x204' (APP) I length | SSRC NodeB

'decrease amount'

This packet can be sent at the decision point.

It is also an option that a custom TCP/RTSP link is estab- lished between the base station (access point) and the streaming server via which stream adaptation commands can be transmitted towards the streaming server on behalf of the wireless clients served by the particular base station. For this purpose, RTSP methods may be extended by a custom method, e.g. "MOD_BW" (modify bandwidth) .

In case of a mode of operation without a streaming server, the adaptation functionality can be processed between the mobile producer and the mobile client (both may be mobile ter- minals) . For proximity links there is no need for two feedback loops, therefore the feedback loop can be active over the RTCP layer and no cross-layer functionality is required. In case of non-proximity links, the two separate feedback loops can be maintained.

List of Abbreviations:

3G Third generation of mobile standards APP Application-defined RTCP Packet ARQ Automatic Repeat-Request BTS Base Transceiver Station (a piece of equipment that facilitates wireless communication between UE and a network)

CQI Channel Quality Indicator

DPDCH Dedicated Physical Data Channel

E-DPDCH evolved DPDCH eNodeB evolved NodeB (NodeB in LTE system)

GGSN Gateway GPRS Support Node

GPRS General Packet Radio Service

GW Gateway

HARQ Hybrid ARQ

HSDPA High Speed Downlink Packet Access

HSGW High speed SGW (SGW in HRPD (CDMA evolution) network)

HSPA High Speed Packet Access

HSUPA High Speed Uplink Packet Access

IP Internet Protocol

ISN Intelligent Service Node

LAN Local Area Network

LTE Long Term Evolution

MME Mobility Management Entity

MPEG Motion Pictures Expert Group

NACK Negative Acknowledgment

NADU Next Application Data Unit

NodeB BTS in 3G/UMTS system

PGW Packet Data Node Gateway (GGSN in LTE terminology)

QoS Quality of Service

RSTP Rapid Spanning Tree Protocol

RTCP RTP Control Protocol

RTP Real-Time Protocol

RTPS Real-Time Polling Service (a QoS class in Wi- MAX) RTSP Real-time Streaming Protocol

RTT Round-Trip-Time

SGSN Serving GPRS Support Node

SGW Serving Gateway, (SGSN in LTE network) SR Sender Report

SVC Scalable Video Coding

UE User Equipment (mobile terminal)

UMTS Universal Mobile Telecommunications System

UPE User Plane Entity VCEG Video Coding Experts Group

WiFi Wireless LAN

WiMAX Worldwide Interoperability for Microwave Access