Technical Field

[0001] This application relates to the technical field of data processing, more specifically to methods and apparatuses associated with service in support of browser for multi-media content.

Background

[0002] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

[0003] Advances in integrated circuit, computing, networking and related technologies have led to increased consumption of multi-media content on computing devices, including mobile computing devices. Often multi-media content are consumed using a browser, and increasingly multi-media content may be high definition (HD) video.

[0004] Recently, Hypertext Markup Language (HTML) 5 may be emerging as a widely accepted approach to deliver online multi-media content streaming service. It is expected that most platforms may need to enable hardware acceleration for video decoding and/or graphics processing, in order to smoothly play HD HTML5 videos. However, most prior art browsers have architectures that are generally difficult to modify to enable efficient exploitation of hardware acceleration for video decoding and/or graphics processing.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Embodiments of the present invention will be described by way of exemplary embodiments, but not limitations, illustrated in the accompanying drawings in which like references denote similar elements, and in which:

[0006] Figure 1 illustrates an example computing device configured to provide a multimedia service in support of a browser, in conjunction with a graphics processing unit and/or a media processor from outside of the browser, for consuming multi-media content;

[0007] Figure 2 illustrates example architectures of the browser and the multi-media service of Figure 1, in further detail; [0008] Figure 3 illustrates a method of operation for the browser and the multi-media service; and

[0009] Figure 4 illustrates an example non-transitory computer-readable storage medium having instructions configured to practice all or selected aspects of the method of Figure 3; all arranged in accordance with embodiments of the present disclosure.

DETAILED DESCRIPTION

[0010] Methods, apparatuses and storage medium associated with a multi-media service configured to support a browser for multi-media content are disclosed. In various

embodiments, a method may include receiving, by a multi-media service of a device, from a render engine of the browser operating on the device, through a browser engine of the browser, multi-media rendering commands associated with rendering the multi-media content on a display unit. The multi-media service may be operating outside the browser. Further, the method may include processing, by the multi-media service in conjunction with a graphics processing unit and/or media processor of the device, the multi-media rendering commands to render the multi-media content on the display unit, and emitting and returning signals to the render engine, through the browser engine.

[0011] Various aspects of the illustrative embodiments will be described using terms commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. However, it will be apparent to those skilled in the art that alternate embodiments may be practiced with only some of the described aspects. For purposes of explanation, specific numbers, materials, and configurations are set forth in order to provide a thorough understanding of the illustrative embodiments. However, it will be apparent to one skilled in the art that alternate embodiments may be practiced without the specific details. In other instances, well-known features are omitted or simplified in order not to obscure the illustrative embodiments.

[0012] Various operations will be described as multiple discrete operations, in turn, in a manner that is most helpful in understanding the illustrative embodiments; however, the order of description should not be construed as to imply that these operations are necessarily order dependent. In particular, these operations need not be performed in the order of presentation. Further, descriptions of operations as separate operations should not be construed as requiring that the operations be necessarily performed independently and/or by separate entities.

Descriptions of entities and/or modules as separate modules should likewise not be construed as requiring that the modules be separate and/or perform separate operations. In various embodiments, illustrated and/or described operations, entities, data, and/or modules may be merged, broken into further sub-parts, and/or omitted.

[0013] The phrase "in one embodiment" or "in an embodiment" is used repeatedly. The phrase generally does not refer to the same embodiment; however, it may. The terms "comprising," "having," and "including" are synonymous, unless the context dictates otherwise. The phrase "A/B" means "A or B". The phrase "A and/or B" means "(A), (B), or (A and B)". The phrase "at least one of A, B and C" means "(A), (B), (C), (A and B), (A and C), (B and C) or (A, B and C)".

[0014] Figure 1 illustrates an example computing device configured to provide multimedia service in support of a browser, in conjunction with a graphics processing unit and/or a media processor, from outside of the browser, in consuming multi-media content, in accordance with various embodiments of the present disclosure. As shown, for the illustrated embodiments, client computing device (or simply, client device) 102 may be coupled with, and receiving multi-media content streamed from multi-media server 132, through network(s) 134. Client device 102 may include processor and memory arrangement 104 configured to have operating system (OS) 122 and browser 120 operated therein, graphics processing unit 106, media processor 112, display unit 108, and networking interface 110. Further, OS 122 may include multi-media service 124. As will be described in more detail below, multimedia service 124 may be incorporated with teaching of the present disclosure to support browser 120, in conjunction with graphics processing unit 106 and/or media processor 112 and from outside of browser 120, in consuming multi-media content (e.g., streamed from multi-media server 132) on display unit 108. Accordingly, hardware acceleration of video decoding and/or graphics processing may be more easily provided for different platforms with different graphics processing unit and/or a media processor designs, through different implementations of the portions of multi-media service 124 associated with interfacing with the graphics processing unit and/or a media processor.

[0015] In various embodiments, as described earlier, processor and memory arrangement 104 may be configured to enable OS 122, including multi-media service 124, and browser 120 to be operated therein. Processor and memory arrangement 104 is intended to represent a broad range of processor and memory arrangement, including but are not limited to arrangements with single or multi-core processors of various execution speeds and power consumptions, and memory of various architectures with one or more levels of caches, and of various types, dynamic random access, FLASH, and so forth. [0016] In various embodiments, graphics processing unit 106 and/or media processor 112 may be configured to provide video decoding and/or graphics processing functions to browser 120, through multi-media service 124, while display unit 108 may be configured to enable multi-media content, e.g., HD video, to be rendered thereon. Examples of graphics processing functions multi-media service 124 may invoke graphics processing unit 106 and/or media processor 112 to assist may include, but are not limited to, graphics data transformation, lighting processing, triangle set up/clipping, polygon processing, and so forth. In alternate embodiments, graphics processing unit 106 and media processor 112 may be combined in part or in whole.

[0017] OS 122 and browser 120 (except for multi-media service 124, and the manner browser 120 and multi-media service 124 cooperate with each other) are intended to represent a broad range of these elements known. Examples of OS 122 may include, but are not limited to, Windows® operating systems, available from Microsoft Corporation of Redmond, WA, Linux, available from e.g., Red Hat of Raleigh, NC, Android™, available from Open Handset Alliance, or iOS, developed by Apple Computer of Cupertino. In various embodiments, browser 120 may be a web browser. Examples of browser 120 may include, but are not limited to, Internet Explorer, available from Microsoft Corporation of Redmond, WA, or Firefox, available from Mozilla of Mountain View, CA.

[0018] Similarly, multi-media server 132 and network(s) 134 are intended to represent a broad range of these elements known. Examples of multi-media server 132 may include, but are not limited to, a video server from Netflix, Inc. of Los Gatos, CA, or a video server from CNN of Atlanta, Georgia. Network(s) 134 may include wired or wireless, local or wide area, private or public networks, including the Internet.

[0019] In various embodiments, client device 102 may be a desktop computer, a laptop computer, a tablet computer, a smartphone, a personal digital assistant, a game console, or other devices of the like.

[0020] Referring now to Figure 2, which illustrates example architectures of browser 120 and multi-media service 124 of Figure 1, in accordance to various embodiments of the present disclosure. As shown, browser 120 may include browser engine 204 having user interface 202 and inter-process communication (IPC) bridge, and render engine 206 having media player 212 and various other components 208. Examples of various other components 208 may include, but are not limited to, an Extend Markup Language (XML) parser, a JavaScript interpreter, a display interface module, a networking interface module, and so forth. Browser components 202-208 (except for IPC bridge 210, and the manner browser engine 204 and render engine 206 cooperate with multi-media service 124) are intended to represent a broad range of these components known in the art, configured to perform their conventional functions.

[0021] In various embodiments, IPC bridge 210 may be configured to enable browser engine 204 to route multi-media content rendering related commands, on behalf of render engine 206, to multi-media service 124. Further, IPC bridge 210 may be configured to enable browser engine 204 to route emitted signals from multi-media service 124 to render engine 206.

[0022] In various embodiments, the multi-media content rendering related commands may include:

Start # launch/start media player of multi-media service;

Stop # stop/unload media player of multi-media service;

Play # start to play a multi-media content;

Pause # pause playing of a multi-media content;

SetUri # set uniform resource identifier (uri) of a a multi-media content to be played;

SetTarget # indicate whether playing is direct or indirect rendering;

SetPosition # seek to certain time;

SetPlaybackRate # faster or slower playback;

Set Volume # adjust volume;

Set Video Size # set video size;

GetPosition # get current video playback time;

Has Video # query whether the resource identified by the URI contains video;

HasAudio # query whether the resource identified by the URI contains audio;

GetPlayerState # query media player status, paused, initialized, or playing;

IsStreaming # query whether the media content file could be seeked;

SupportFullscreen # query whether the media player can be in fullscreen mode;

Get Video Size # query video size;

GetNaturalVideoSize # query natural video size property of the media content file;

GetBufferedBytes # query how many bytes have been buffered;

GetMediaSizeTime # query media duration;

GetMediaSizeBytes # query media content file size ;

GetBufferedTime # query how many seconds have been buffered.

[0023] In various embodiments, the emitted signals from multi-media service 124 may include:

E SIGNAL INITIALIZED # media pipeline is ready in multi-media service;

E SIGNAL EOF, # end of multi-media content file/stream;

E SIGNAL ERROR, # error happens with error id;

E SIGNAL BUFFERING, # media player is buffering to N%;

E SIGNAL BUFFERED, # media player finish buffering;

E SIGNAL SEEKED, # Actual seek finished, after Seek is called;

E SIGNAL STOPPED, # Actual stop finished, after Stop is issued; E SIGNAL PLAYSTATECHANGED, # Play state changed;

E SIGNAL EEDREPLY, # Please send a reply to indicate you are alive.

[0024] Still referring to Figure 2, for the illustrated embodiments, multi-media service 124 may include multi-media player manager 222 and multi-media player 224. In various embodiments, multi-media player manager 222 may be configured to manage the life-cycle of multi-media player 224, including the launching and unloading of multi-media player 224 (e.g., in response to the "start" and "stop" multi-media content rendering related commands). In various embodiments, multi-media player manager 222 may be configured to manage multiple multi-media players 224 at the same time, to concurrently play multiple multi-media contents. In various embodiments, multi-media player 224 may be configured to support playing of a multi-media content, in conjunction with graphics processing unit 106 and/or media processor 112. In various embodiments, multi-media player 224 may include IPC Host 232 and Player Control 234. IPC Host 232 may be configured to receive multi-media content rendering related commands from render engine 206, and transmit signals emitted by player control 234 to render engine 206, both through IPC Bridge 210 of browser engine 204. In various embodiments, player control 234 may be configured to process the multi-media content rendering related commands, and render a multi-media content, in conjunction with graphics processing unit 106 and/or media processor 112 (leveraging the processing power of the hardware in video decoding and/or graphics processing, such as video scaling, color conversion and so forth).

[0025] Figure 3 illustrates a method of operation for multi-media service 124 to support browser 120 in rendering a multi-media content, in conjunction with graphics processing unit 106 and/or media processor 112, in accordance with various embodiments of the present disclosure. As shown, method 300 may start at block 302. At block 302, media player 212 of render engine 206 may receive a play media request, e.g., from a user, through user interface 202. From block 302, method 300 may proceed to block 304. At block 304, media player 212 of render engine 206, may in response to the request, generate and issue various multi-media content rendering related service commands, e.g., "start" to start multi-media player 224 of multi-media service 124, "setURI" to set the URI of a video stream to be played, or "play" to start playing the video stream, and so forth. In various embodiments, method 300 may repeat block 304 to issue various commands over the course of playing a multi-media content.

[0026] On issuance of a multi-media content rendering related command (or hereinafter, simply, service commands), method 300 may also proceed from block 304 to block 306. At block 306, media player 212 of render engine 206 may forward the service command to IPC bridge 210 of browser engine 204. From block 306, method 300 may proceed to block 308. At block 308, IPC Bridge 210 may forward the service command to multi-media player manager 222 of multi-media service 124 if multi-media player 224 of multi-media service 124 have not been launched/started yet or the service command is a multi-media player 224 life-cycle management related command, or to multi-media player 224 of multi-media service 124, in particular IPC host 232, if multi-media player 224 of multi-media service 124 has been launched/ started. From block 308, for a multi-media player 224 life-cycle management command, method 300 may proceed to block 310. At block 310, multi-media player manager 222 of multi-media service 124 may process and respond to the service command accordingly, such as launching/ starting or stopping/unloading multi-media player 224. From block 308, for a rendering related service command, method 300 may proceed to block 312. At block 312, multi-media player 224 of multi-media service 124 may process and respond to the service command accordingly. For the illustrated embodiments, block 312 may include IPC host 232 processing and routing the commands to player control 234, and receive signals emitted by player control 234 for routing back to media player 212 of render engine 206, through IPC bridge 210 of browser engine 204. Block 312 may further include player control 234 issuing requests for video decoding and/or graphics processing service requests to graphics processing unit 106 and/or media processor 112 to accelerate the video playback.

[0027] For direct rendering (determined by render engine 206), from block 312, method 300 may proceed to block 318. At block 318, display hardware (such as, graphics processing unit 106 and/or media processor 112 with display unit 108), in response to requests made by Player Control 234, may render the multi-media content. For indirect rendering (determined by render engine 206) or for simply routing emitted signals back to media player 212 of render engine 206, from block 312, method 300 may proceed to block 314. At block 314, IPC bridge 210, on receipt of the indirect rendering related service commands or emitted signals, may route the indirect rendering related service commands or the emitted signals to media player 212 of render engine 206, through IPC Bridge 210. From block 314, method 300 may proceed to block 316. At block 316, media player 212 of render engine 206, on receipt, may process the indirect rendering related service commands or the emitted signal, and in turn, may issue requests to graphics processing unit 106 and/or media processor 112 to process and render the multi-media content, such as compositing with other HTML elements, rendering video as texture, do image processing to the video snapshots. [0028] Figure 4 illustrates an example non-transitory computer-readable storage medium having instructions configured to practice all or selected aspects of the method of Figure 3; in accordance with various embodiments of the present disclosure. As illustrated, non- transitory computer-readable storage medium 402 may include a number of programming instructions 404. Programming instructions 404 may be configured to enable a client computing device 102, in response to execution of the programming instructions, to perform all or some of the operations of method 300 earlier described with references to Figure 3. In alternate embodiments, programming instructions 404 may be disposed on multiple non- transitory computer-readable storage media 402 instead. As described earlier, in various embodiments, the programming instructions may be configured to implement multi-media service 124 or portions thereof.

[0029] Referring back to Figure 1, for one embodiment, at least one of the processor(s) of processor and memory arrangement 104 may be packaged together with computational logic configured to practice the method of Figure 3, in whole or in part. For one embodiment, at least one of the processor(s) of processor and memory arrangement 104 may be packaged together with computational logic configured to practice the method of Figure 3 to form a System in Package (SiP). For one embodiment, at least one of the processor(s) of processor and memory arrangement 104 may be integrated on the same die with computational logic configured to practice the method of Figure 3. For one embodiment, at least one of the processor(s) of processor and memory arrangement 104 may be integrated on the same die with computational logic configured to practice the method of Figure 3 to form a System on Chip (SoC). For at least one embodiment, the SoC may be utilized in a smart phone, cell phone, tablet, or other mobile device.

[0030] Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a wide variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described, without departing from the scope of the embodiments of the present disclosure. This application is intended to cover any adaptations or variations of the embodiments discussed herein. Therefore, it is manifestly intended that the embodiments of the present disclosure be limited only by the claims and the equivalents thereof.