FIELD OF THE INVENTION The invention relates to a method and to an apparatus for processing a data stream. The method of the invention can be used within the context of, e. g., reproducing content information from the data stream.

BACKGROUND ART US patent 4,649,442 discloses an automatic program selector of a video tape recorder. In the video tape recorder, every time a recording start signal is detected, the corresponding content is automatically reproduced. The recording start signal is a control signal indicating the start of a recorded program. The automatic reproduction of the content upon detection of the signal allows a user to check whether or not the reproduced program is the desired one. Thus, when the signal is detected in the fast forward (FF) or rewind (REW) mode, the magnetic tape is stopped so as to reproduce a beginning portion of the detected program for a predetermined period of time. The FF or REW mode is automatically resumed if the user, during the reproduction of the detected program, makes no key operation.

SUMMARY OF THE INVENTION The known selector automatically renders the parts of a program without the intervention of the user, based on the recording start signal. The invention provides, among other things, a method for controllably reproducing pre-recorded content information that allows the user to decide him/herself which portions of the content or of the program are to be reproduced. The invention provides, e. g., a user-friendly method for retrieving a desired sequence or scene within the pre-recorded program. The invention further provides the user with the option of playing out content that starts as closely as possible to a location selected by the user within the content.

The invention provides a method for processing a data stream. While the data stream is being processed at a first speed, a command is supplied, e. g., by the user, to change the processing at the first speed to processing at a second speed. The processing at the first speed is stopped. The processing at the second speed is started at a location in the data stream

already passed while the stream was previously being processed at the first speed. The location for starting the processing at the second speed is re-adjustable to take into account circumstances examples of which are given below.

The expression"data stream"refers to, among other things, the concept of a linear succession of data. The data stream may be processed in either one of the forward and backward (or rewind) direction. The data stream may be processed at different speeds.

The invention allows switching from processing at the first speed to processing at the second speed, while taking into account the desired starting point for starting the processing at the second speed. Processing at the second speed is not necessarily started from the point in the stream where processing at the first speed was interrupted. The inventor has realized that there is a delay between the moment the processing at the first speed is actually stopped, and the moment at which the user actually wanted it to stop when he/she gave the command to switch to the second speed processing. The start point in the stream for the processing at the second speed has typically already been passed while changing from processing at the first to the second speed. The start point in the stream is located in the direction opposite to that of the processing at the first speed. The invention seeks to compensate for this delay.

This interval between the stop point and the start point may depend on, among other things, a profile of the user (or: user-history) regarding interaction with the processing apparatus, his/her habits, his/her reaction time, the magnitude of the first speed or on technical features of the apparatus processing the data stream. This interval between the stop point and the start point may be adaptive or even be dynamically, or real-time, adaptive.

The invention is used, for example, for the playing out of content information obtained from the processing of the data stream in a recording apparatus such as a video or audio tape recorder, or a HDD-based video recorder such as the one from TiVo or Replay.

For example, the user scans content information that is being played-out in a forward direction at a high speed (Fast Forward). The user spots a specific portion while in the Fast Forward mode and decides to resume playing out at normal speed beginning at that specific portion. The user sends a command to the apparatus to stop the fast forward processing. The user intends to stop the Fast Forward processing at the beginning of the specific portion.

However, this desired point is overshot due to latencies mentioned previously if no measures are taken to compensate for the latencies. In the invention, the start point is determined as mentioned above, upstream of the point where the Fast Forward processing actually stopped.

Information content may then be played in the forward direction at a normal speed starting at

a data preceding the fast forward stop point. As a result the information content is played-out beginning closely to where the user originally wished to switch from the fast forward mode to the normal playing mode.

In an embodiment of the invention, one or more locations in the data stream have been marked in advance by a respective bookmark pointer. Upon rewinding or Fast Forwarding and then instructing the system to start processing at normal play-out is started at a point indicated by a specific bookmark pointer. This specific bookmark pointer precedes the location where the processing at the first speed was stopped.

The term"bookmark pointer", as used herein, is any information that indicates a beginning, and possibly, an ending of a specific scene in the data stream ("bookmark pointer"is also abbreviated to"bookmark"). A bookmark may be a label embedded in the stream or a data item in the stream that can be recognized as such. A bookmark may also be provided separately from the data stream for, in combination with the stream. For example, the bookmarks can indicate beginnings of scenes or sequences in the content information, or the bookmarks indicate portions thereof, for which the user is expected to show or has shown interest.

The invention is also useful in a video-on-demand system or audio-on-demand system, wherein a remote server supplies to a client a video or audio stream over a data network. The network latencies between client-server add to the problems indicated above for which the invention provides a solution. The term"latency"refers to the noticeable propagation delay between cause and effect. In real-time interaction, network latency is a difficult problem to overcome. Today's Internet environment has considerable latency, typically in the order of seconds. High-speed analog modems introduce as much as 30 milliseconds of delay, which means 60 milliseconds end-to-end. Latency effects tend to get more serious with larger size of the network and with increased network load. Eliminating all network latency is impossible. Latency can be determined in various ways. One of the manners is to send a message that takes a round trip between the machines communicating over the network and then divide the round trip time by a factor of two. For example, on a Unix machine connected to the Internet, one can enter"ping-s <Internet name of a destination machine>"to send a number of round trip packages to the destination machine and obtain the round trip time. This pinging could be done once, e. g., at the initialization of the communication, or at regular time intervals to adapt for the dynamic changes in network congestion. The network latency thus determined is taken into account by the system to

determine the jump back in the stream when switching from processing at the first speed to processing at the second speed upon the user issuing the instruction.

BRIEF DESCRIPTION OF THE DRAWING The invention is explained in further detail, by way of example, and with reference to the accompanying drawing wherein: Fig. 1 is a diagram illustrating switching the processing of the data stream from fast forward to normal play; Fig. 2 is a diagram illustrating switching the processing of the data stream from fast backward (or rewind) to normal play; and Fig. 3 is a diagram of a data stream with bookmark pointers.

Throughout the drawing, same reference numerals indicate similar or corresponding features.

Fig. 4 is a functional block diagram of an end-to-end content delivery system.

Fig. 5 is a functional block diagram of an interactive end-to-end content delivery system.

DETAILED EMBODIMENTS Fig. 1 is a diagram that illustrates switching the processing of a data stream 100 from fast forward to normal play. Stream 100 is shown as a linear array, wherein the data are successively being processed from left to right in case of processing in the forward direction (fast forward and normal play-out). Stream 100 is initially processed at first speed 102 in the fast forward mode. When the user gives the command to stop the fast forward processing and begin the normal play out, latencies cause the fast forward processing to overshoot the data at which the user intended to resume normal play-out. Processing at first speed 102 actually stops at a stop point 108. If the normal play-out mode took over at stop point 108, the user would miss the data in overshoot 110. The processing of stream 100 at a second speed 104, here the speed corresponding to the normal play out mode, is started at a start point 106. In the invention, start point 106 is upstream of stop point 108. The distance between points 106 and 108 takes into account overshoot 110. As a result, a jump backward is needed in the switching of processing stream 100 from first speed 102 to second speed 104. Start point 106 should be the point where the user wants the processing to be started at the second speed 104.

The invention comprises determining start point 106. The jump backward is based on

overshoot 110 and takes into account, among other things, the reaction time of the user and system latencies.

Fig. 2 is a diagram that illustrates switching the processing of data stream 100 from rewind to normal play-out. Stream 100 is initially processed at first speed 102 in a fast backward or rewind mode. When the user initiates a request for switching the processing of stream 100 from rewind at speed 102 to normal play out at second speed 104, latencies cause the processing in the rewind mode to overshoot the intended switching point and stop at stop point 108. Processing of stream 100 at the second speed 104 should therefore start at start point 106 that lies upstream, in the rewind mode, of point 108. This requires a jump forward in the normal play-out mode. The jump is based on overshoot 110.

Fig. 3 is a diagram of data stream 100 provided with bookmark pointers 142, 144,146 and 148. Pointers 142,144,146 and 148 can be created in a variety of ways, examples of which are given below. The bookmark pointers have been added to stream 100 to provide system control signals. The control signals enable the system to find the proper starting point for starting the play-out when switching from fast forward or rewind to the normal play out mode.

A first example is based on detecting a start of a new scene (or of new data) in stream 100. In case data stream 100 comprises a video stream with successive encoded frames, bookmark pointers 412,144,146 and 148 can point to respective key-frames. The expression"key-frame", as used herein, indicates a frame in stream 100 that is a typical representative of part of the content. See for example, published International patent application WO 98/55942, herein incorporated by reference. A simple way of selecting a key-frame is to detect video scene changes in the content information. A key-frame representing a given scene is, for example, the first frame of the scene. A scene can be further analyzed to select a key-frame that is a meaningful representation of the scene. Each scene can then be further investigated to find a key-frame that is a better representation of the scene than the first frame.

A second example is based on detecting meta data accompanying stream 100.

Assume that stream 100 comprises, for example, a video stream with embedded closed caption subtitling meta data. Further assume that the user has expressed an interest in playing golf as indicated by his/her user profile or history. Whenever the word'golf is found in the closed caption subtitling data, a bookmark pointer is created pointing to the start of the corresponding scene.

A third example is based on detecting a certain object or movement in the content information of stream 100. For example, data stream 100 comprises an episode of a popular TV show. At a certain moment a guest star enters the scene. That particular moment in video stream 100 can get a bookmark.

A fourth example is based on detecting a certain sound or word. For example, stream 100 represents a movie with little dialogue. The start of a dialogue can get a bookmark by detecting voice in the movie's audio data stream.

A fifth example is based on bookmarks that indicate the content information of a portion of data stream 100, for which the user has shown or is expected to show interest.

Within this context, reference is made to U. S. Serial no. 09/585, 825 (Attorney docket US 000123) filed 6/1/00 for Eugene Shteyn for"Content with bookmarks obtained from an audience's appreciation", incorporated herein by reference. This document relates to a method for providing bookmarks to indicate elements or portions of content information that are likely to be of interest to a person. The creation of the bookmarks is based on, e. g., the appreciation registered while monitoring a test audience.

A sixth example is based on bookmarking the first frame of a GOP (Group Of Pictures). Data stream 100 comprises a video stream and comprises a sequence of GOPs, possibly MPEG2 compressed. The start of a GOP is typically I-frame encoded and is indicted by reference numeral 150. For example, data stream 100 is a compressed video stream, e. g., in MPEG2. In a compressed video stream the start of a new scene is typically encoded as an I-frame for encoding efficiency reasons. The 1-frame is then the first frame of a GOP.

The bookmarks can be created by, among others, the producer of the data stream ; the data stream provider or broadcaster; the user ; or an automated method. The user creates the bookmarks, for example, when he/she spots a desired element in the data stream.

The user sends a command, e. g., by means of a remote control, in order to create a bookmark for a location in the data stream corresponding to the desired element. Again, latency and reaction time are to be taken into account by the system when adding the bookmark to the stream.

Start point 106 can be a bookmark close to stop point 108. Data stream 100 is preferably stored using a random access memory (RAM) medium, e. g., an optical or magnet disk or a solid state memory. The RAM minimizes the time it takes to switch from first speed processing of the stream to second speed processing. Examples of data stream 100 and the storage medium are the following: an MPEG2-Stream of compressed movie on a DVD-

Video disk; a compressed TV-broadcast stored on an Hard Disk Drive (e. g. TiVo Personal TV); an MP3 compressed audio file in a Flash memory (e. g., the Philips Rush MP3 player).

The data stream may also be stored on a non-random-access memory medium, such as a magnetic tape, but, unless some random access buffer memory is available, the switch between first and second processing speed might not be smooth. A bookmark can be part of the data stream structure (e. g., embedded in private data in case of an MPEG stream) or can be kept in a separate data structure (e. g., a table-of-content file of a DVD-movie). In either case, a relevant bookmark is provided before the transition is made from processing at the first speed to processing at the second speed. The relevant bookmark is typically the latest one encountered in the data stream being processed at the first speed.

Fig. 4 is a functional block diagram of an end-to-end content delivery system 400. System 400 comprises a service/content provider 402, a STB 406, a display & loudspeaker 408, a link 422 between provider 402 and STB 406 over a network 404 and a link 424 between STB 406 and display & loudspeaker 408.

Provider 402 can supply content 412 to network 404 through its output to link 422. Provider 402 is, for example, a broadband cable operator, a Direct To Home (DTH) satellite operator or an ISP. Examples of content 412 are a pay-per-view movie, a television broadcast, or a real-networks streamed media clip. Examples of link 422 are a TV-cable, a POTS telephone/xDSL line, a wireless link between a DTH satellite and a parabolic antenna. Examples of network 404 are a cable head-end distribution system, a data network such as the Internet.

STB 406 comprises a microprocessor 416, a control interface 418 for the user, a local storage 420, an input for link 422 and an output to link 424. An example of STB 406 is a consumer digital cable receiver box with personal TV functionality. An example of microprocessor 416 is a Philips Trimedia processor. Such a processor is capable of performing tasks such as transport stream de-multiplexing and MPEG2 encoding or transcoding. Examples of local storage 420 are a HDD, a DVD+RW and a Flash memory.

User control interface 418 offers the user control over the STB. Storage 420 can contain data with properties as described above with reference to data stream 100. Microprocessor 416 is in control of the processing speed in case the data supplied via link 422 contains data stream 100.

Examples of display & loudspeaker 408 are a TV-set and an audio amplifier set.

STB 406 and display & loudspeaker 408 typically reside at the end-user's premises.

STB 406 and display & loudspeaker 408 can also physically be combined or its components can also be distributed for, e. g., cost or convenience reasons.

For example, provider 402 broadcasts a movie called'Never miss it again' over cable network 404. The user, referred to herein as'Never Home', has programmed STB 406, named'Philips super TV'to record"Never miss it again"because'Never Home' expects to be away from home during the broadcast. Microprocessor 416 knows when'Never miss it again'is being broadcast by using data from an EPG and a built-in timer.

Microprocessor 416 controls storage 420, a 100 GB HDD, and controls the recording of 'Never miss it again'during the live broadcast. Data stream 100, in this example'Never miss it again', resides physically in storage 420 after the broadcast. After'Never Home'returns to his home he can now search for his favorite scene of'Never miss it again'via control interface 418.

User control 418 offers trick-play control over previously stored data stream 100 on storage 420. Stream 100 could be, for example, a favorite TV-program. As an example, user control 418 is able to receive and process remote control (RC) commands. RC commands can include fast-forward (FF) play, fast-backward (FB) play and normal (1 x speed) play of the TV-program. Microprocessor 416 interprets the processed RC commands and processes stored stream 100, the TV program, from the storage accordingly. The result of this will be, e. g., a base-band AV-signal that will be output over link 424, such as an S-video connection, to for instance a TV-set.

The inventor also envisioned a user command such as'go to next bookmark' (or previous bookmark). This command allows the user to play a pre-recorded program from the next bookmarked location onwards. This command is very useful for a busy user when he wishes to quickly scan earlier recorded programs. As and example,'Never home'is only interested in the home-runs of last weekend's baseball play-off. Therefore'Never home'has instructed'Philips super TV', using voice controlled input, to bookmark all home-runs.

'Philips super TV'uses the closed caption data of the play-off to bookmark the home-runs.

'Never home'now uses'go to next bookmark'to quickly catch-up for the missed play-off.

'Never home'probably appreciates the above as an alternative to a user command such as 'Quick-skip' (jump ahead about one minute in a pre-recorded program) as implemented by Replay in their Personal TV STB offerings.

Features described above of system 400 can reside in a microprocessor executable program of STB 406. The program can be entered in STB 406 at the moment of production or can be entered at a later stage. In case the program is entered at a later stage, e. g. through downloading over a network such as the internet, STB 406 preferably has means for a program upgrade.

Fig. 5 is a functional block diagram of an interactive end-to-end content delivery system 500. System 500 comprises a service/content provider 502, a STB 506, a display & loudspeaker 408, a link 422 between provider 502 and STB 506 over network 404, an interactive link 520 between provider 502 and STB 506 over network 404 and a link 424 between STB 506 and display & loudspeaker 408. Physically links 422 and 520 can be the same one.

Provider 502 can supply content 512 to network 404 through its output to link 422. The examples of provider 402 given above can be the same as for provider 502.

Examples of content 412 are the same as for content 512. Microprocessor 510 enables provider 502 to play its content at different speeds. Interactive link 520 can be used to communicate to microprocessor 510 a requested speed for playing content 512. Data stream 100 is part of content 512.

STB 506 is similar to STB 416 but has some significant differences. STB 506 can control the play-out speed of data stream 100 although data stream 100 is not physically located in STB 506. Instead, microprocessor 516 remotely controls the play-out speed of data stream 100. Microprocessor 516 communicates this using link 520 which connects to microprocessor 510.

For example provider 502 can be an Internet service provider or a Video On Demand (VOD) provider which allows its content play-out speed to be controlled remotely.

When provider 502 receives a request for change of play-out speed network latencies in networks 520 and 422 are taken into account. This will be dealt with by microprocessor 510, 516 or 510 and 516 combined.

STB 506 could be a thin client STB with space, power and/or cost constraints.

A good example of this STB 506 would be that of a Web-pad device which even has display & loudspeaker 408 built in. The user can carry the device to wherever he wants and play-out content wherever he wants. The connection, links 422 and 520 to provider 502 can in this case be a wireless one, for example, based on Home-RF, IEEE802.11 or Bluetooth. Provider 502 could be a home-server at the premises of the user and could in turn be connected to an external content provider (not drawn in Fig. 5). Also provider 502 could be a remote server

using a wireless public network such as GSM or UTMS. When using these types of networks the network delays are to be known. STB 506 could for instance'ping'provider 502 in order to determine network delays. Another method could be, when the user requests changing the play-out speed of data stream 100, to time-stamp the command to request this.

Features described above of system 500 can reside in a microprocessor executable program of STB 506 and/or provider 502. The program can be entered in STB 506 at the moment of production or can be entered at a later stage. In case the program is entered at a later stage, e. g. through downloading over a network such as the internet, STB 506 preferably has means for a program upgrade.