DISPLAYING MULTIPLE VIDEO STREAMS

CROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application is a non-provisional filing of, and claims benefit under 35 U.S.C. §119(e) from, U.S. Provisional Patent Application Serial No. 62/279, 127, entitled "System and Method for Dynamically Displaying Multiple Video Streams", filed January 15, 2016, the entirety of which is incorporated herein by reference.

BACKGROUND

[0002] Due to the proliferation of video recording devices with networking capabilities, large numbers of video streams about events may be obtained. Streaming video content has increased in popularity. It is also becoming more common for users to want to watch multiple streams simultaneously on the same screen. The source of the video streams may be user generated content shared with peers (e.g., via Instagram, Facebook, Snapchat), subscription based content (e.g., cable channel subscriptions), publicly broadcast content (e.g., periscope.tv, Meerkat, Glide), or any combination of the above. Technology exists for viewing multiple streams simultaneously, and often times the user has to select one stream to focus on, or the configuration for displaying the multiple streams on the screen is preconfigured. Various layouts 150 that can be generated using available picture-in-picture / split screen processors are illustrated in FIG. 1. The screen video processor overlays selected input channels into various split-screen views 152, 154, 156, 158, 160 for presentation.

[0003] Another example of current split-screen technology is described in US Patent 9, 167,213 (by Raymond Bratton and Nathan Petersen, herein referred to Bratton et al.), which provides an approach for surveillance that shows multiple video feeds on one screen. Bratton et al. describes control of the incoming video stream, such as moving the video source (camera) to pan, tilt, re-center on a particular part of the image, and provides a way to share alerts and video when a specified event occurs (e.g., a door opening).

[0004] MyView: Customizable Automatic Visual Space Management for Multi-Stream Environment (Bin Yu, PhD Thesis, UIUC, 2006) addresses visual space management on a given screen for received streams. In particular, the system analyzes content of incoming video streams to determine how to display streams. For example, MyView may detect when a face is present in a video stream or a person present in a video stream is speaking, and uses such events as indications that the given stream may be more important. The system additionally asks users to manually provide scores for each "shot type" ranging over "must show (high priority)", "show if room available (low priority)", and "do not show". The shot types may be determined from content analysis of a video stream (determinations may be automatic based on a timer, while a determination of who is talking or silent or the presentation may also be manually adjusted).

SUMMARY

[0005] Described herein are methods and systems for some embodiments for determining an importance score of each of a plurality of streams, determining which streams to display to a user from among those that the user is currently viewing or has previously viewed, and allocating display space to each stream in accordance with the importance of the stream. In some embodiments, the importance score of the stream is based on the number of concurrent viewers of that stream.

[0006] In some embodiments, a method for dynamically displaying video streams on a display device includes the steps of displaying a plurality of video streams on a display device, wherein each video stream is displayed at a substantially similar size; responsive to receiving information regarding an increase in viewership for a video stream of the plurality of video streams, displaying the video stream at a relatively increased display size with respect to those of the remaining video streams of the plurality of video streams; and displaying the remaining video streams of the plurality of video streams at a smaller size with respect to the increased display size of the video stream in response to receiving information regarding a sustained increase in viewership for the video stream.

[0007] In some embodiments, the method includes displaying each video stream of the plurality of video streams at a substantially similar size in response to receiving information regarding a decrease in viewership of the video stream. In some embodiments, the display of the stream overlaps the others. In some embodiments, the remaining video streams of the plurality of video streams are displayed at a smaller size with respect to the increased display size of the video stream in a non-overlapping configuration.

[0008] In some embodiments, a method for dynamically displaying video streams on a display includes the steps of receiving two or more "relevant" video streams to be presented to a user, of which each video stream is currently presented to the user or was previously presented to the user, configuring the available display into one or more approximately equal sections, one per "presented" video stream; in case of two or more presented video streams, using one predefined stream to provide the audio; determining an importance score of each relevant video stream (whether being presented or not) over one SHORT running time window and one LONG running time window, wherein an importance score may be based on determining popularity of each video stream based on the number of viewers a stream has over a running time window, and/or determining the user's level of interest in each video stream over a running time window based on the user's emotional reactions, and/or determining the user's level of interest in each video stream over a running time window based on the user's eye gaze at a stream. The method further includes determining a configuration of the display such that a stream of higher importance over the SHORT time window is allocated a larger screen area (or begins being displayed if not the stream of higher importance is not currently being displayed) than a stream of lower importance, for example, by making the screen area allocated to a corresponding stream proportional to the importance score assigned to the corresponding stream, and switching the audio to the video stream that has the highest importance score over the LONG time window.

[0009] In some embodiments, a relevant stream includes one or more streams that the user has selected in the current session or a previous session or previous session within some time period (e.g., the last day). In some embodiments, a selected relevant stream may not be a stream currently being viewed.

[0010] In some embodiments, the method includes receiving levels of interest of other users

(viewing the same streams) and incorporating the levels of interest received from other users in determining the importance score assigned to a given video stream. In some embodiments, the streams are allocated areas that are anchored to particular points (e.g., corners) on the display's edge so that they remain somewhat stable and easy to find even as they change in size. In some embodiments, the areas allocated to the streams may overlap with each other (one stream may encroach on a stream's display area); a stream with a higher importance score in the SHORT time window may be placed in front of a stream with a lower importance score in the SHORT time window. In some embodiments, the areas allocated to a stream are resized only if another stream has higher importance scores in both the SHORT and LONG time windows. In some embodiments, the audio is switched only if the importance score over the LONG time window of a stream from which the audio is taken currently is sufficiently lower than the importance score of a stream with the highest importance score over the LONG time window.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] A more detailed understanding may be had from the following description, presented by way of example in conjunction with the accompanying drawings, wherein:

[0012] FIG. 1 is a schematic illustration depicting a prior art video viewing system.

[0013] FIG. 2 is a schematic block diagram depicting a client-server system for stream delivery from a server and rendering in accordance with some embodiments. [0014] FIG. 3 is a flowchart of a method for configuring and displaying a plurality of audio and video streams, in accordance with some embodiments.

[0015] FIG. 4 is a schematic illustration depicting a first frame sequence series, in accordance with some embodiments.

[0016] FIG. 5 is a schematic illustration depicting a second frame sequence series, in accordance with some embodiments.

[0017] FIG. 6A is a schematic illustration depicting an example communications system in which one or more disclosed embodiments may be implemented.

[0018] FIG. 6B is a schematic illustration depicting an example client device that may be used within the communications system of FIG. 6A.

DETAILED DESCRIPTION

[0019] A detailed description of illustrative embodiments will now be provided with reference to the various figures. Although this description provides detailed examples of implementations, the provided details are intended to be by way of example and in no way limit the scope of the application. Systems and methods relating to video compression may be used with wired and wireless communication systems described with respect to FIGS. 6A-6B.

Video Stream Resizing

[0020] Manually switching between streams and resizing the screens may be cumbersome for users. Manual switching may produce a lower quality of user experience given available technologies.

[0021] The pre-set approaches that rotate between fixed layouts and variations of the fixed layouts let the user set a layout manually, however both of these approaches presume that the user knows in advance what layout to use.

[0022] FIG. 2 depicts a block diagram of a client-server system 250 in accordance with some embodiments. As shown, FIG. 2 includes client device 200, stream server 220 and a plurality of users 225, each user downloading one or more streams from the stream server. Client device 200 may include components such as a display controller 205, a stream renderer 210, and an audio controller 215. Further, client device 220 may include a subset of these components and/or additional components. In some embodiments, the stream renderer module 210 is configured to receive selected incoming streams, along with any associated metadata (e.g., the number of concurrent viewers of each selected incoming stream). In some embodiments, the metadata is provided to the stream renderer 210 via the stream server 220. In some embodiments, stream renderer 210 may be configured to (i) guide the display controller 205 to control an external display that displays the selected streams for the user and to (ii) guide the audio controller 215 to select an audio signal associated with one of the selected streams to display to a user.

[0023] In some embodiments, stream server 220 is configured to provide the selected video streams to be processed by the stream renderer 210. In some embodiments, the stream server 220 is contacted by clients (e.g., other viewers). In some embodiments, the stream server 220 maintains statistics such as the concurrent downloads of each stream. A stream server 220 may incorporate such statistics as metadata along with each stream that a stream server 220 provides to the stream renderer 210.

[0024] FIG. 3 depicts a method 300, in accordance with some embodiments. As shown, method 300 for dynamically displaying video streams on a display may comprise for one embodiment of receiving relevant video streams 302, which for many embodiments comprises two or more "relevant" video streams to be presented to a user via an available display, each video stream currently presented to the user or was previously presented to the user, configuring the available display into one or more approximately equal sections, each presented video stream having a respective section, determining an importance score of each relevant video stream 304 (whether being presented or not) over a short running time window (which may also be referred to herein as a short time period) and a long running time window, determining a configuration of the display 306 such that a screen area allocated to a given stream is proportional to the importance score assigned to the stream (e.g., stream of higher importance over the short time window is allocated a larger screen area than a stream of lower importance), and switching, at step 308, the audio to the video stream that has the highest importance score over the long time window. For some embodiments, determining a display configuration 306 may be based on viewership information without calculating an importance score. For these embodiments, viewership information may be substituted for importance score in the descriptions shown below.

[0025] In some embodiments, a short time window corresponds to a duration of approximately 10 seconds while a long time window corresponds to a duration of approximately 20 seconds. These approximations are in no way limiting, and only offer one embodiment for relating short and long time windows.

[0026] In some embodiments, audio from a predetermined stream is selected during initialization. During runtime as stream popularity changes, the audio may change accordingly.

[0027] In some embodiments, determining popularity of each video stream (which may be used to determine a display configuration 306) is based on a number of viewers (or viewership) a video has over a running time window. In some embodiments, a stream that is not currently being presented may be brought forward for presentation if its short importance score exceeds the lowest importance score of a currently presented stream by a sufficiently large percentage. In some embodiments, a new stream to be displayed may exceed a currently displayed stream's importance score by at least 2%. For example, if the stream with the lowest importance score has a score of 1.255, that stream may be dislodged by a stream of score strictly greater than 1.28 (that is, with 19 or more viewers). A stream with an importance score equal to 4.00 may be dislodged by a stream of importance score strictly greater than 4.08 (that is, with 12,023 or more viewers). In some embodiments, for streams with exactly one viewer, a minimum of 0.5% may be used for the threshold percentage. In some embodiments, a relevant stream corresponds to a stream selected by a user in the current session or a previous session (e.g., a previous session within the last day). In some embodiments, determining the importance score assigned to a video stream includes receiving levels of interest (in the same streams) from the user. In some embodiments, determining the importance score assigned to a video stream includes receiving levels of interest (in the same streams) from other users.

[0028] In some embodiments, the method includes starting a new stream from a previous point in time with respect to a time of observation of its increased importance score (e.g., if a stream is determined to have a high importance score, a system may start the stream 20 seconds prior to the observation of its increased importance score - that is, to capture an event, a reaction to which by other users may be used to determine a stream's importance).

[0029] In some embodiments, the streams are allocated areas that may be anchored to particular points (e.g., corners) on the display's edge so that they remain somewhat stable and easy to find even as they change in size. In some embodiments, the areas allocated to the streams may overlap with each other. In such embodiments, a stream may encroach on another stream's display area; a stream with a higher importance score in the short time window may be placed in front of a stream with a lower importance score in the short time window. In some embodiments, the areas allocated to a stream are resized only if another stream has higher importance scores in both the short and long time windows.

[0030] In some embodiments, audio is switched only if the importance score over the long time window of the stream from which the audio is taken currently is sufficiently lower than the importance score of the stream with the highest importance score over the long time window.

[0031] FIG. 4 depicts a series of display layouts 400 as they change dynamically based on the importance scores for different video streams being recomputed on an ongoing basis. FIG. 4 shows only four video streams SI, S2, S3, and S4 for simplicity of illustration and more (or fewer) video streams may be used. The transition labels mention the most important of the displayed video stream.

[0032] Frame A 402 depicts the initial layout in which the four video streams are given equal area on the display. As stream SI gains importance (e.g., a larger number of users watching SI) over a short running time (denoted Short: SI), the allocated area of the screen for SI becomes larger, and in some embodiments is placed in front of and overlaps the other streams shown in Frame B 404). As SI gains importance over a long running time (denoted Long: SI), the screen allocation for S2, S3, and S4 recedes as the screen allocation for SI enlarges, and the audio may be switched to SI, as shown in Frame C 406. As S3 begins to gain importance during a short running time (Short: S3), the screen allocated to S3 becomes larger and covers up part of the screen allocated to SI as shown in Frame D 408. Further, the screen portions allocated to S2 and S4 remain unchanged as S3 is not large enough to overlap them. As S3 continues to gain importance over a long running time (Long: S3), the screen allocated to SI begins to recede while S3 grows, and the audio switches to S3, as shown in Frame E 410. Also shown in Frame E 410, the screen portions allocated to S2 and S4 grow as there is room for them, however this allocation may depend upon relative importance scores of the four streams.

[0033] In an alternative series starting at Frame B 404, S3 gains short term importance (Short: S3) right after SI, which causes S3 to be brought to the front rather than shrink, as shown in Frame F 412. In Frame G 414, all of the streams have an approximately equal importance score over short and long term running times (Short: Equal) and (Long: Equal), and the layout becomes equal for each stream (also shown in Frame A). In Frame H 416, S3 gains short term importance (Short: 3), and begins to grow and overlap the other streams (similar to Frame B 404 in regards to Short: SI).

[0034] Determining the importance score for a given stream may be done in a variety of ways. In some embodiments, the importance score may be determined based on a number of users currently watching the given stream. In some embodiments, the importance score is calculated using a logarithmic base 10 equation shown below in Eqn. 1 :

Importance Score = log _1Q (average number of viewers over time period)

[0035] Thus, if a given stream has 18 viewers over a given short time window, the short-term importance score is equal to 1.255. Similarly, if a stream has an average of 10,000 viewers over a long time window, the long-term importance score is 4.00. In some embodiments, the average number of viewers over short/long time periods corresponds to a straight average. Eqn. 1 above is only an embodiment of an equation used to determine the importance score; however, other methods of determining an importance score may be used.

[0036] In some embodiments, the importance score may be determined using physiological data. In some embodiments, the importance score may be determined by sounds made by users watching the given stream (e.g., gasps, cheers, screams, or other emotional reactions from the user and/or other users in the user's social circle who are currently viewing the same stream). In some embodiments, such emotions/reactions may be captured via a webcam and/or a microphone in a given user's environment, analyzed, and factored into importance score of a given stream. Techniques for detecting (from an audio or video stream) people laughing or otherwise crying out in a given stream may be adapted for determining the emotional reaction of the viewers of the given stream by placing cameras and microphones that record a viewer in his or her environment. Kim, et al. describe audio indexing and retrieval and mentions applications such as indexing and retrieving audio or video channels in Chapter 7 of their book. See Hyoung- Gook Kim, Nicolas Moreau, and Thomas Sikora, MPEG-7 Audio and Beyond Audio Content Indexing and Retrieval, Wiley 2005. Reuderink, et al. combine audio and visual channels to detect laughter. See Boris Reuderink, Mannes Poel, Khiet Truong, Ronald Poppe, and Maja Pantic, Decision-Level Fusion for Audio-Visual Laughter Detection, in A. Popescu-Belis and R. Stiefelhagen (eds.): Proceedings of the Workshop on Machine Learning for Multimodal Interaction 2008, Lecture Notes in Computer Science 5237, pp. 137-148, Springer- Verlag, Berlin Heidelberg, 2008.

[0037] In some embodiments, in response to receiving such indications about the various viewers of a stream, a system computes an "immediate ratio" of number of viewers reacting to the total number of viewers for whom the emotional data is available. For example, if there are

10,000 viewers and a system captures emotional reactions of 7,000 viewers at once over a short past interval of 5 seconds and over this interval of 5 seconds, 4,000 viewers reacted at least once, an immediate ratio may be calculated to be 4000/7000 = 0.57. In further embodiments, a system may compute the average of six short intervals (which may be the last six short intervals) to obtain the base rate of emotional reaction to this stream. For example, this base rate may be 0.33.

In some embodiments, an "immediacy factor" is a ratio of an immediate ratio and a base rate, in this example, 1.73. A system multiplies an importance score calculated above by an immediacy factor for all streams and uses the resulting values as an "effective importance score."

[0038] In some embodiments, the importance score may be determined based on crowd- sourced input from users watching a stream (e.g., "upvotes" or "likes"). In some embodiments, users may upvote a specific sports play or a witty remark made by a presidential candidate. In such embodiments, an immediacy factor may be determined for upvotes/likes in a similar way described above for viewer emotion/reactions, and used to update an importance score for a given stream.

[0039] In some embodiments, determining the importance score includes receiving an explicit specification of interest (e.g., a manual override) from a user about a user's interest in various streams. Such embodiments enable a user to watch what a user wishes to rather than the streaming being controlled by what other people are watching.

[0040] In some embodiments, displayed video streams may change dynamically but audio may switch in response to a request from a user for audio to switch to another stream (such as to the currently most prominent stream).

[0041] In some embodiments, streams may be crowd-sourced from individuals and streams may be selected based on social media relationships of the user with those individuals (e.g., whether a user is "friends" with a given individual or follows a given individual, or a user and a given individual mutually follow each other).

[0042] In some embodiments, an importance score for a new video stream from a given social media source may be based on an average or final important score computed for a previous video stream from the same social media source.

[0043] In some embodiments, a user may request to view one or more replay streams, meaning that a stream is not live at the time of watching. In such embodiments, one or more of the replay streams may be presented in which case the importance score may be calculated based on the popularity at the time each replay stream was originally viewed. In some embodiments, a cumulative popularity (or interest) is captured over each time segment of the replay stream was viewed.

[0044] In some embodiments, a stream that a user has previously viewed is included as a relevant stream. Thus, a stream that a user is no longer watching may be revived for presentation. In this manner, this disclosure goes beyond prior systems that involve a user's eye gaze (since those prior systems require the user to watch a video stream). Here the importance score may be determined based on potentially diverse sources.

[0045] Embodiments herein go beyond existing systems by providing a natural way for a user to benefit from displays capable of showing more than one video stream concurrently.

[0046] Embodiments described herein are user friendly and reduce explicit actions by the user. Some embodiments may be hands free so that the user would not have to look for a manual controller, such as a remote control. Further, the user may be concurrently engaged in some other relevant activity such as talking on the phone or with a friend nearby while embodiments herein allocate streams.

[0047] Embodiments described herein relate to automatically resizing displays of video streams because of changes in viewership. Moreover, embodiments herein include switching the audio in a way that is somewhat different from resizing the display.

[0048] Further, some embodiments described herein relate to resizing displays automatically based on viewership.

Exemplary Use Cases

[0049] FIG. 5 illustrates an embodiment depicting series of display layouts 500 for four college football games. FIG. 5 shows panes for video streams as being viewed by a user. A dashed border indicates the stream from which the audio is taken. Suppose user Mike is watching TV at home on a Saturday afternoon. Mike is deeply interested in college football and tries to watch as many games as he can. There are quite a few to choose from at noon on October 3, 2015. Texas is currently playing Texas Christian University (TCU), which is stream 3. Mike roots for Texas and tries to make sure he keeps tabs on how the Texas game is proceeding. For Texas, Mike tries to watch two channels (streams 3 and 4) concurrently if he can get them (e.g., WatchESPN and ABC streams of the same game). Mike's other channels are on the West Virginia-Oklahoma game (stream 1) (Oklahoma being long-time Texas rivals) and the Purdue- Michigan State game (stream 2) (Michigan State being rated No. 2). Initially, Mike is viewing all four streams in equally sized panes and is listening to the Texas-TCU ABC channel as shown in Frame A 502.

[0050] In the meantime, there is a long pass in the West Virginia-Oklahoma game followed by other exciting events (e.g., a touchdown, a fumble, or an interception). More viewers begin to watch the West Virginia-Oklahoma stream (where the relevant viewers may be those who are actively channel surfing and the popularity of a stream may correspond to how many viewers are viewing that stream). The West Virginia-Oklahoma pane gradually increases (shown in Fame B 504) and overlaps (shown in Frame C 506) the other games. As Mike continues to watch the West Virginia-Oklahoma pane, stream 1 becomes established. The other panes shrink so they no longer overlap with the West Virginia-Oklahoma pane, and the audio shifts to the West Virginia- Oklahoma stream as shown in Frame C 506.

[0051] Shortly thereafter, an exciting event occurs in the Texas-TCU WatchESPN pane

(stream 3) causing a gain in popularity. As the popularity (e.g., importance score) increases, the

Texas-TCU WatchESPN pane grows and overlaps the West Virginia-Oklahoma pane as shown in Frame D 508. Since Mike continues to watch the Texas-TCU WatchESPN pane, the other panes are resized (the large West Virginia-Oklahoma pane shrinks), and the Purdue-Michigan State and the Texas-TCU (ABC) panes both increase since there is space for them to increase, as shown in Frame E 510.

[0052] In an exemplary embodiment utilizing crowd-sourced live media streaming, a user Nancy is watching live media streams on her phone during Wimbledon. Nancy is interested in tennis and tries to watch as many streams as she can, especially in the semi-finals and finals. For the same game, there may be many streams available. Further, important matches may be likely to have relatively more streams than less important matches.

[0053] In some embodiments, there are a number of live media streams showing the men's finals. Without use of a previously described system, Nancy may have some difficulty continually identifying and paying attention to the right streams as a match progresses. However, in this use case Nancy selects a few live streams to watch. In some embodiments, the selected streams are produced/recorded by attendees sitting at various points in the stadium.

[0054] During the match, an interesting incident may occur on or off the court. For example, the players' families may make extravagant gestures, or one of their coaches may take issue with an umpire's line call. Alternatively, a celebrity attendee may draw attention. A stream focused on such activity may gain a high importance score because of its popularity (e.g., spike in popularity). In some embodiments, a particular stream may be streaming the game from a good angle or a videographer may have a special talent for producing a visually appealing stream, both of which may also gain popularity.

[0055] Thus, the relative importance of the various streams selected by Nancy changed live in a "real-time" manner (in sequences similar to those depicted in FIG. 5). As the popularity of streams increase or decrease, the most popular streams are given prominence on Nancy's screen by increasing the space occupied by those streams. A stream that remains prominent for a long running time window becomes established and becomes the source of audio as well.

[0056] An exemplary embodiment relating to crowd-sourced streams about emergency or terror attacks demonstrates a setting where a user may not select streams in advance. Such embodiments communicate with a server that communicates novel streams to the user. Such streams may be combined with other types of streams like sports, news, periscope, TV, etc.

[0057] In one embodiment, a user Peter is walking in the mall while using his smart phone. Peter is using his smart phone as a navigation aid for locating a store or restaurant in the mall. Suddenly, news of an emergency situation in progress appears. There are reports of two attackers apparently shooting people at an office building adjacent to the mall. The mall is placed on lock down, and people who are at the scene of the attack begin to post video streams of the attack from respective viewpoints.

[0058] Peter is subscribed to a service that provides video streams to its subscribers along with metadata about the popularity of those streams. In some cases, Peter may request to receive streams about this incident. Alternatively, he may have a standing request to be provided video streams of any police or natural emergency occurring within 10 miles of his location (e.g., if emergency personnel such as police, fire, or medical personnel are called to some emergency location that is within 10 miles of Peter's current location, the service will provide Peter with one or more streams from the emergency location).

[0059] Some of the streams posted on the site may be received by users near Peter (e.g., users who are in the mall, in the office complex that contains the attacked building, or in the general vicinity of the mall). In some embodiments, the service provides metadata including a popularity/importance score of the received streams to Peter's smart phone. Some embodiments comprise a process implemented on Peter's smart phone. Some embodiments may be configured to select a stream to be rendered with a greater display size and from which an audio stream is taken based on relative importance scores of each received stream.

[0060] In some embodiments, if a new, important event occurs, a stream that focuses on the new event may gain popularity and gain prominence on Peter's smart phone. For example, at some point during an incident, a SWAT team arrives and a user on the scene sees the SWAT team come up from a window and begins streaming a video. Then, the perpetrators escape in a black SUV, which is blocked by police half a block down from the incident. A second vehicle is also blocked in the other direction, at which point streams focused on the second vehicle spike in popularity, however when the second vehicle turns out to contain only innocent people, interest in those streams fades quickly. In some embodiments, smart glasses may be used to display popular streams, although the number of streams to be displayed on smart glasses may be less than a smart phone.

[0061] In some embodiments, a user may prefer to listen to audio from a particular user selected source, but likes to see other video streams. In an exemplary embodiment, a user Rhonda walks in the mall while using her smart phone. Just as in Peter's case above, Rhonda receives video streams whose prominence on her display changes dynamically. However, Rhonda finds that these video streams from disasters often produce a lot of indistinct noises and screaming, which she finds not informative and somewhat disturbing. Therefore, Rhonda may select an audio stream from a single video stream to listen to while the visual part of the plurality of video streams switches dynamically. Exemplary Hardware and Network Architecture.

[0062] The wired and wireless systems of FIGs. 6A and 6B are now described. FIG. 6A is a diagram of an example communications system 100 in which one or more disclosed embodiments may be implemented. The communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, and the like, to multiple wireless users. The communications system 100 may enable multiple wired and wireless users to access such content through the sharing of system resources, including wired and wireless bandwidth. For example, the communications systems 100 may employ one or more channel-access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), and the like. The communications systems 100 may also employ one or more wired communications standards (e.g. : Ethernet, DSL, radio frequency (RF) over coaxial cable, fiber optics, and the like.

[0063] As shown in FIG. 6 A, the communications system 100 may include client devices 102a, 102b, 102c, and/or 102d, Radio Access Networks (RAN) 103/104/105, a core network 106/107/109, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, and communication links 115/116/117, and 119, though the disclosed embodiments contemplate any number of client devices, base stations, networks, and/or network elements. Each of the client devices 102a, 102b, 102c, 102d may be any type of device configured to operate and/or communicate in a wired or wireless environment. By way of example, the client device 102a is depicted as a tablet computer, the client device 102b is depicted as a smart phone, the client device 102c is depicted as a computer, and the client device 102d is depicted as a television.

[0064] The communications systems 100 may also include a base station 114a and a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 110, and/or the networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, the base stations 114a, 114b may include any number of interconnected base stations and/or network elements.

[0065] The base station 114a may be part of the RAN 103/104/105, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, and the like. The base station 114a and/or the base station 114b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown). The cell may further be divided into sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, one for each sector of the cell. In another embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.

[0066] The base stations 114a, 114b may communicate with one or more of the client devices 102a, 102b, 102c, and 102d over an air interface 115/116/117, or communication link 119, which may be any suitable wired or wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, and the like). The air interface 115/116/117 may be established using any suitable radio access technology (RAT).

[0067] More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel-access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 103/104/105 and the client devices 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 115/116/117 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink Packet Access (HSDPA) and/or High-Speed Uplink Packet Access (HSUPA).

[0068] In another embodiment, the base station 114a and the client devices 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 115/116/117 using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A).

[0069] In other embodiments, the base station 114a and the client devices 102a, 102b, 102c may implement radio technologies such as IEEE 802.16 (Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 IX, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

[0070] The base station 114b in FIG. 6A may be a wired router, a wireless router, Home

Node B, Home eNode B, or access point, as examples, and may utilize any suitable wired transmission standard or RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like. In one embodiment, the base station 114b and the client devices 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, the base station 114b and the client devices 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the client devices 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, and the like) to establish a picocell or femtocell. In yet another embodiment, the base station 114b communicates with client devices 102a, 102b, 102c, and 102d through communication links 119. As shown in FIG. 6A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114b may not be required to access the Internet 110 via the core network 106/107/109.

[0071] The RAN 103/104/105 may be in communication with the core network 106/107/109, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the client devices 102a, 102b, 102c, 102d. As examples, the core network 106/107/109 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, and the like, and/or perform high-level security functions, such as user authentication. Although not shown in FIG. 6A, the RAN 103/104/105 and/or the core network 106/107/109 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 103/104/105 or a different RAT. For example, in addition to being connected to the RAN 103/104/105, which may be utilizing an E-UTRA radio technology, the core network 106/107/109 may also be in communication with another RAN (not shown) employing a GSM radio technology.

[0072] The core network 106/107/109 may also serve as a gateway for the client devices 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and/or other networks 112. The PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and IP in the TCP/IP Internet protocol suite. The networks 112 may include wired and/or wireless communications networks owned and/or operated by other service providers. For example, the networks 112 may include another core network connected to one or more RANs, which may employ the same RAT as the RAN 103/104/105 or a different RAT. [0073] Some or all of the client devices 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (the client devices 102a, 102b, 102c, 102d may include multiple transceivers) for communicating with different wired or wireless networks over different communication links. For example, the WTRU 102c shown in FIG. 6A may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0074] FIG. 6B depicts an example client device that may be used within the communications system of FIG. 6A. In particular, FIG. 6B is a system diagram of an example client device 102. As shown in FIG. 6B, the client device 102 may include a processor 118, a transceiver 120, a transmit/receive element 122, a speaker/microphone 124, a keypad 126, a display/touchpad 128, a non-removable memory 130, a removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and other peripherals 138. The client device 102 may represent any of the client devices 102a, 102b, 102c, and 102d, and include any subcombination of the foregoing elements while remaining consistent with an embodiment. Also, embodiments contemplate that the base stations 114a and 114b, and/or the nodes that base stations 114a and 114b may represent, such as but not limited to transceiver station (BTS), a Node-B, a site controller, an access point (AP), a home node-B, an evolved home node-B (eNodeB), a home evolved node-B (He B), a home evolved node-B gateway, and proxy nodes, among others, may include some or all of the elements depicted in FIG. 6B and described herein.

[0075] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the client device 102 to operate in a wired or wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While FIG. 6B depicts the processor 118 and the transceiver 120 as separate components, the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0076] The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114a) over the air interface 115/116/117 or communication link 119. For example, in one embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In another embodiment, the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive

IR, UV, or visible light signals, as examples. In yet another embodiment, the transmit/receive element 122 may be configured to transmit and receive both RF and light signals. In yet another embodiment, the transmit/receive element may be a wired communication port, such as an Ethernet port. The transmit/receive element 122 may be configured to transmit and/or receive any combination of wired or wireless signals.

[0077] In addition, although the transmit/receive element 122 is depicted in FIG. 6B as a single element, the client device 102 may include any number of transmit/receive elements 122. More specifically, the client device 102 may employ MTMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 115/116/117.

[0078] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the client device 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the client device 102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, as examples.

[0079] The processor 118 of the client device 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128. In addition, the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SFM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the client device 102, such as on a server or a home computer (not shown).

[0080] The processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the client device 102. The power source 134 may be any suitable device for powering the WTRU 102. As examples, the power source 134 may include one or more dry cell batteries (e.g., nickel- cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), and the like), solar cells, fuel cells, a wall outlet and the like.

[0081] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the client device 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 115/116/117 from a base station (e.g., base stations 114a, 114b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. The client device 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment. In accordance with an embodiment, the client device 102 does not comprise a GPS chipset and does not acquire location information.

[0082] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.

[0083] Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer- readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer- readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.