CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application is a non-provisional filing of, and claims benefit under 35 U.S.C. §119(e) from, U.S. Provisional Patent Application Serial No. 62/380, 184, entitled "Systems and Methods for Sourcing Live Streams," filed August 26, 2016, the entirety of which is incorporated herein by reference.

BACKGROUND

[0002] Major media, sports, and political events typically introduce numerous tweets, live Periscope streams, videos distributed in Vine and YouTube, and pictures posted in Instagram. The audience is typically active in sharing experiences online. The recently launched Twitter Lightning project searches and collects content from multiple sources and manually curates different streams. The service will packetize events together with full media coverage using crowd-sourced content from different social media sites.

[0003] News channels use social media for promoting their content. Links to news and announcement on upcoming events are distributed on Twitter, Instagram, Facebook, and other social media sites. In addition, news anchors provide live streaming video even during the live broadcast in the studio. Television channels, newspapers and media houses provide live coverage on breaking news, interviews, live reports and even real time reality TV. In addition, crowd source content is available in social media channels ranging from Facebook and Twitter to live video streaming, such as Periscope. TV viewers as well as web service and mobile application users are not able to combine different content sources. Unless the content creator is explicitly linking different streams to the production, relevant crowd-sourced streams, such as Periscope, remain isolated and difficult to find.

[0004] Social media content and shared live media are distributed via different services without contextual connection to each other. Collecting relevant content together and finding interesting events uses large amounts of manual trial-and-error. There is no automatic method, for instance, to continue following a debate in social media after live coverage on TV has ended.

SUMMARY

[0005] Systems and methods described in this specification enable users to discover, in various social media services, crowd-sourced media which is related to third-party reference content, such as a commercial news service. Contextual content identification, linking, and ranking is applied to create networks and clusters related to the given reference content. An exemplary real-time streaming service provides an interface for content metadata and contextual cues to deliver reference fingerprints. Continuous monitoring of available content in real-time streaming services enables users to find content clusters, track content evolution, and follow-up on the events after a third-party reference content stream has ended.

[0006] In exemplary embodiments, content streams related to the reference are clustered and made available for automatic content curation. The content linking and ranking provides control information for a rendering tool to create multi-view presentations.

[0007] Social media extensions of short news segments available via TV, web services, and newspaper mobile applications may offer wider coverage of real-time events and may allow continuing communication following an event or discussion when professional coverage or a news segment has ended. For example, a debate may continue seamlessly in social media after the live coverage on TV without limitations of TV channel capacity. Social media services link to relevant streams and create live compositions.

[0008] In an exemplary embodiment, networked service is provided for linking reference media streams to social media streams. The service receives location-indicating metadata associated with a source location of a reference media stream, and for each of a plurality of social media streams, the service receives location-indicating metadata associated with a source location of the respective social media stream. The service operates to compare the location-indicating metadata for the reference media stream with the location-indicating metadata for each of the plurality of social media streams to select at least one correlated social media stream that has a source location proximate to the source location of the reference media stream. The service then sends, to a user device displaying the reference media stream, information identifying at least one of the correlated social media streams.

[0009] The reference media stream and the social media streams may be live media streams. The reference media stream may be a broadcast audio-video stream. The social media streams may also be audio-video streams.

[0010] In some embodiments, the location-indicating metadata of the reference media stream includes information identifying at least one wireless access point in range of the source location of the reference media stream. The location-indicating metadata may further include information indicating a received signal strength of the at least one wireless access point.

[0011] In some embodiments, the service compares signal transients in determining which social media streams to identify to users. In one such embodiment, the service extracts signal transients from the reference media stream and from each of a plurality of the social media streams. The selection of at least one correlated social media stream then further includes selecting, from among social media streams having a source location proximate to the source location of the reference media stream, at least one social media stream having signal transients that substantially match the signal transients of the reference media stream. The extraction of signal transients may include performing filtering (e.g. high-pass or bandpass filtering) and thresholding. The transients may be, for example, transient audio signals or transient brightness changes in video signals. The comparison of transients may be performed in the sparse domain. In some embodiments, the comparison is made between the timing of signal transients.

[0012] In some embodiments, the service sends information identifying correlated social media streams in response to a user request. In such a method, the service receives a request for social media linking, wherein the request includes information identifying the reference media stream being displayed by the user device. The information identifying at least one of the correlated social media streams is sent in response to receiving the request for social media linking.

[0013] In such embodiments, the user device may perform a method as follows. The user device receives and displays the reference media stream. The device also receives the information identifying the at least one correlated social media stream. While displaying the reference media stream, the device presents to the viewer a user interface element enabling selection of the correlated social media stream. In response to detecting user selection of the user interface element, displaying the correlated social media stream to the viewer.

[0014] Further embodiments include systems with a processor and a non-transitory computer- readable storage medium storing instructions operative to perform the functions described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] A more detailed understanding may be had from the following description, presented by way of example in conjunction with the accompanying drawings.

[0016] FIG. 1 shows a social media user interface for posts related to a map layout of content available on a crowd-source application or server.

[0017] FIG. 2 is an example user interface that shows broadcaster video output overlaid with social media interfaces for a smart TV application.

[0018] FIG. 3 is an example user interface that shows broadcaster video output overlaid with social media interfaces for a mobile device application. [0019] FIG. 4 is a message sequencing diagram for selecting and presenting a social media stream.

[0020] FIG. 5 is an example user interface for selecting a content stream from a multi-view bundle.

[0021] FIG. 6 is a system diagram showing a system operative to perform extraction of contextual cues from primary and secondary content.

[0022] FIG. 7 is a timing diagram illustrating the temporal location of transients and contextual cues in primary and secondary content.

[0023] FIG. 8 is a block diagram of a system for extracting ongoing audio and video streams from selected services.

[0024] FIG. 9 is a block diagram of a system for linking broadcast and social media content streams.

[0025] FIG. 10 is a timing diagram illustrating a method of contextual similarity clustering.

[0026] FIG. 11 is a timing diagram illustrating a method of content similarity clustering.

[0027] FIG. 12 is a timing diagram illustrating linking of social media contextual cues to reference contextual cues using a sliding-window type comparison.

[0028] FIG. 13 is a schematic illustration of relationships between reference videos and crowd- sourced videos.

[0029] FIG. 14 is a system diagram showing a server-client type architecture for content sourcing relative to a reference.

[0030] FIG. 15A is an example user interface for a mobile device for stream selection.

[0031] FIG. 15B is an example user interface for a mobile device that combines stream selection with video playback.

[0032] FIG. 16 is an example user interface diagram for an application running on a mobile device.

[0033] FIG. 17A depicts an example wireless transmit/receive unit (WTRU) that may be used within a communications system.

[0034] FIG. 17B depicts an exemplary network entity that may be used within a communication system. [0035] The entities, connections, arrangements, and the like that are depicted in— and described in connection with— the various figures are presented by way of example and not by way of limitation. As such, any and all statements or other indications as to what a particular figure "depicts," what a particular element or entity in a particular figure "is" or "has," and any and all similar statements— that may in isolation and out of context be read as absolute and therefore limiting— may only properly be read as being constructively preceded by a clause such as "In at least one embodiment, ...." For brevity and clarity of presentation, this implied leading clause is not repeated throughout the detailed description.

DETAILED DESCRIPTION

[0036] This disclosure provides systems and methods to automatically detect and curate different media streams from different sources into a connected-event package. Crowd-sourced content streams are linked to reference material that is available in third-party applications. Media streams available in different content services are clustered around a reference based on contextual information embedded in the content itself and additional sensory information.

[0037] Embodiments disclosed herein may employ methods described in International Application No. PCT/US 16/47436, filed Aug. 17, 2016, entitled "Methods and Systems for Generating and Utilizing Contextual Watermarking," for discovering different streams originating from an event occurring in the same location and time instant.

[0038] Individual users (sensor nodes) providing content streams may be linked together in a content network and ranked based on content relevance. An automatic content curation provides contextual tools for clustering real-time content.

[0039] This specification describes methods to fetch additional content related to the third- party live reference stream from different social media services. A user is provided with a display of an additional multi-view composition of an on-going live stream for a TV or mobile application. A composition from a social media service may continue after a reference stream in a third-party application has ended. A user may choose to continue to follow an evolving event, breaking news, or a hosted debate in social media by selecting a "more" or "follow-up" button (such as the example shown in FIG. 15 A) after which an application fetches linked and ranked content from a source such as Periscope.

[0040] Systems and methods presented in this specification enable automatic content curation of source content (news, interview, live coverage) a user is viewing in a mobile application, web service, or TV show. Systems and methods also cluster connected (linked and ranked) media items available in social media, such as Periscope; create an automatic content package composition applying to linked and ranked content streams; and provide means for the user to select an interesting view of the evolving event.

[0041] FIG. 1 is an example user interface 100 of crowd-sourced media service application. Individual users attending events or participating in activities capture media content and share their experiences in various social media sites. People may share comments and observations about an event using social media services. They may also share live video content on crowd-sourced media services. FIG. 1 illustrates a user interface 100 identifying the locations 104, 106 of media stream from a crowd-sourced or social media service. The example shows locations 104, 106 of media content. The availability of media content is indicated using numbers 104, 106 on a map. Each number 104, 106 represents the number of video or other social media streams available at a particular location.

[0042] In embodiments disclosed herein, reference content streams may be provided by a commercial broadcaster (or reference content provider or reference media stream) or third-party application. Reference content in a third-party application may be a commercially-created news segment, interview, or reality TV show, for example. A third-party application pushes relevant contextual information of the stream to the content service API and requests all potentially related- content. Such a request message requests to provide matching content streams to a reference content stream. The request may contain the reference content stream and reference contextual information associated with the reference content stream.

[0043] Available content from one or more social media services, such as Periscope, is arranged, analyzed, linked to together, and ranked according to mutual relevance to the reference content. In this case, content clusters focus on the reference content. An automatic content composition picks a cluster and composes a media package around the reference. Clustering continues after a reference stream on a third-party application has ended and ranks social media content together. A user of a third-party application may continue following an evolving event in real-time from one or more social media streams that are relevant to a reference (while a reference is available).

[0044] FIG. 2 shows a picture of a broadcaster video output overlaid with social media interfaces 202, 204. FIG. 2 is an example display 200 for access to all available live stream related to breaking news. An application may be a button on video coverage of a smart TV or mobile application, as illustrated in FIG. 2. Access to relevant content streams in social media may occur with a push of a button 202 or other selection mechanism. For this embodiment, buttons 202, 204 appear if related content is available. At the bottom of the "Watch live on social media" box 202, there are icons 204 that appear for available content streams. [0045] In an exemplary embodiment, a broadcaster (e.g., CNN) transmits content stream details and relevant meta data to a Periscope API and receives back a list of content streams that match the breaking news segment. A user may follow a stream on a smart phone or TV application that has means to switch to another content stream based on the user selection. For one embodiment, if the broadcaster application is broadcasting recorded (non-real-time) content, the application will request corresponding recorded material available through social media (e.g., Periscope or Facebook Live).

[0046] FIG. 3 is an example of a screen display 300 for a third-party production broadcast application. This example is a video stream from a mobile device application, such as CNN. The top three-quarters of the screen shows the video stream currently playing 302 on the mobile device. The bottom quarter of the screen 304 shows a series of videos upcoming in the queue. The user may stop the current video from playing and skip to another video displayed along the bottom.

[0047] A user following a video news segment is presented a button indicating that there is crowd sourced live content available related to the event. For the example shown in FIG. 3, the button 306 appears in the lower right corner, where the button 306 says, "Watch on crowd-sourced media service." For this example, if the user taps the button 306, the UI 300 will open a multi- view content composition of a news outlet (such as CNN) and crowd-sourced live streams (such as Periscope or Facebook Live). If the news outlet (CNN) is recorded and shown later, the crowd sourced content may be both live real-time streams and recorded streams available via a crowd- sourced service, such as Periscope or Facebook Live.

[0048] FIG. 4 is a message sequencing diagram 400 for connecting a reference stream with a social media stream. FIG. 4 shows a timing diagram of the continuous process of creating content clusters around a reference content in a third-party application, composing a presentation, and rendering the presentation for a user. For one embodiment, a social media service may include a content access database 402, an analysis/clustering service 404, and an application programming interface (API) 406, where the API provides an interface to external components.

[0049] A third-party application 408 creates a reference stream 412 and presents a video stream to a user ("Presentation of Reference Stream") 414. A third-party streaming / broadcast service 408 obtains or generates contextual cues 416 and sends a request for related streams 418 to the API 406. The application, or a backend service hosting the stream 408, pushes contextual cues of the stream 420 over the social media service API (or "API") 406 to a clustering engine ("Analysis/Clustering Service") 404. A real-time media service (such as a Periscope server) 404 continually goes through a live stream selection or reference stream 422, clustering the streams to connect streams around potential events, and creating a media cluster using all content that matches the reference. This continuous process ("Link and Rank Streams vs. Reference Stream") 426 creates evolving networks when new relevant streams are found and connected to existing streams. Irrelevant streams are dropped or transferred to other clusters or networks. An analysis/clustering service 404 may retrieve content information 424 from a content access database 402. The analysis/clustering service 404 via the social media service API 406 returns details of relevant streams clustered around a reference ("Related Media Streams") 428. An indication of available streams 430 is communicated to a user mobile device / media player 410. Stream selection 432 is may be made via a user mobile device / media player 410. The user mobile device / media player 410 may follow the rendered composition of the evolving network or follow a single stream and the cluster network connected to that particular content stream. The user (or follower) may follow the event or a stream that goes from one event to another. A social media stream 434 is requested from a content access database 402 and presented 436 to the user device or player 410. For one embodiment, a social media stream may be retrieved from a user-sourced service, such as Periscope.

[0050] A user following a third-party content stream (e.g., on TV or the web) may want more information regarding the content. A third-party content creator streams contextual cues of the stream as a reference to a social media service API. A social media service API creates content clusters around third-party content and provides a composition to the user. An automatically clustered content bundle is rendered for the service user. Content clustering of linked streams may be a continuous process performed by the social media service.

[0051] FIG. 5 shows one embodiment of a user interface (UI) 500 for content discovery. FIG. 5 shows an example UI with a map and list of streams connected to each other and to references. In this example, a user searches for content in a map layout. The availability of each live stream within the selected region is indicated on the map (as black circles). Content from third-party services (such as commercial services and live news) may be indicated using a logo or symbol. Streams including third-party content that are linked together 508, 510, 512 with a content clustering tool may be shown with solid black lines connecting the source locations on the map. They are visually grouped together so that the user immediately notices that certain streams are related and most likely cover the same target. A stream close to the group that is not connected 506, 514 may be capturing a different, unrelated event. For the example shown in FIG. 5, there are three live streams at Wall Street that are covering the same target and are linked together. In this example, a broadcaster #1 (e.g., CNN) stream 502 is one of the live streams linked with two streams (such as Periscope). There is also a fourth live stream, but the fourth stream is related to a separate event. Further, in this example, there are two groups in Battery Park and a single unlinked stream. A broadcaster #2 (e.g., MSNBC) stream 504 is linked to one bundle in Battery Park. Battery Park is at the southern (or bottom) portion of the map of Manhattan.

[0052] In response to a user selecting 516 a group (or multi-view bundle 520), the UI may open a list of streams as shown on the right hand side of FIG. 5. The highest-ranked stream 522 regarding the information content or overall audio visual quality may be listed first. A user may select 518 one of the streams by selecting the list item with a contributor name (such as "Contributor Name 1A" 522, "Contributor Name IB" 524, or "Broadcaster #3 Live Stream" 526) or follow a group by selecting a collection line (such as "<Broadcaster #1 & #2 Streams>" 520). In the former case, the UI will render the selected stream. In the latter case, the UI will render a multi-view composition created automatically from the selection.

[0053] When a user selects a multi-view bundle (the first line of the selection in FIG. 5), the service adapts the rendering based on continuous cluster evolution. When a user selects an unlinked stream, the UI renders a given stream because there is no other related content available.

[0054] FIG. 5 shows an embodiment for a user interface that connects a social media client to broadcast provider content (or reference media stream). A broadcaster may use a server that may provide an API to query current live events that are being streamed, to query for matching content based on location and meta-data (or meta-text), and to provide matching available live streams. Crowd-sourced contextual information and reference contextual information may include location and meta-text.

[0055] In an example embodiment, an API may define a function QueryNewsService(). The requester makes an API function call and passes inputs of location, meta-data, and a time window. The API searches for a fingerprint that matches based on contextual cues such as location, time, audio content, and video content. Meta-data, optional text associated with a stream, may also be used as part of the search. The API returns content streams (or reference media streams) that are being broadcast or were recently broadcast and associated with the particular input location.

[0056] A social media client receives the list of available streams from the API and performs correlations to find matching content available from the social media service. The user receives, via a user interface, streams that match a live broadcast stream (or reference media stream). A network of linked live streams may be created around the reference content (for example, CNN or MSNBC). On the example user interface shown in FIG. 5, the user may tap on icon representing bundled content to open a list of available streams (automatic compositions and individual streams). A network of linked live streams is created around the reference content (e.g., CNN or MSNBC). Tapping a bundle on the map opens a list of available streams (automatic composition and individual streams).

[0057] For one embodiment, a social media server provides an interface for querying for related content. A social media API server may provide an API to query updated list of available streams and clusters, to query for matching content based on content fingerprint (location, audio cues, and meta-text input), and to provide matching available live streams requested.

[0058] An example embodiment uses an API function VideoStreamQuery(). Requesters make an API function call, passing a content stream, a fingerprint, and meta-data as inputs. A content stream is a video stream that will be correlated. A fingerprint is data that characterizes a stream based on contextual cues, such as location, time, audio content, and video content. Meta-data is optional, text associated with a stream. A client (e.g. a broadcaster) requests and receives a list of available streams from the API, and based on user selection, requests a live stream from the service.

Content collection in live streaming services

[0059] Live streaming of social media services shares captured live content with other users. A social media service also has an automatic content curation functionality to find emerging trends and events. A service has an API to access the clustering of live streaming content around predefined reference content.

[0060] To enable accurate contextual classification and finding related content, a secondary data stream may be attached to media content. A user device collects one or more sensor signals while capturing content (e.g., audio-visual content). Additional secondary sensor information may contain media content identification and classification values. For example, Facebook Bluetooth Beacons running iBeacon software may be applied as an information source for triangulating the location and orientation of an actual media stream. A reference stream may contain all available media content. If a reference stream lacks all available media content, a comparison may be done with only primary data and with cues extracted from the primary data. Examples of methods that compare audio signals with accelerometer signals are described later in this document.

[0061] In a first stage, secondary content information (e.g., contextual cues derived from secondary content information) is used to determine matching content and to enable reliable linking and ranking to the reference. Contextual information may be derived from crowd-sourced content streams and reference content streams. Derived contextual information may also include transients obtained by sparse filtering. In a second stage, linking is made available by comparing primary content (e.g., contextual cues derived from content streams). For some embodiments, comparisons of primary content may be a correlation of derived contextual information with derived contextual information from one or more live content streams selected from a group of crowd-sourced content streams. Secondary and primary content contextual cues are tied together with timing information. If a secondary content cue correlates, there is also a correlation with a primary content cue within a given time window (the time difference between primary and secondary context cues). The time difference check is a third stage in content clustering. Accurate timing enables matching and linking of content.

Metadata for clustering the content to the reference

[0062] Live media content capturing creates additional sensor data, such as prevailing radio conditions, radio signal strength, available Wi-Fi/BT devices, cellular network details, radio beacons, user motion, orientation and general environmental information, such as room reverberation. Collected material may be split into primary and secondary content. Primary content comprises the modalities in which users have an interest and intent to use for original capture. Typically, this is a mono- or stereo-audio and video stream or a still image. Additional sensor modalities are optional and depend on the capabilities of a recording device. Additional sensor modalities are classified as secondary content which may be stored as side information, metadata, or content descriptors for making an identification and creating a cluster of primary content with a reference. Contextual similarity and content clustering of different media streams to a reference content is conducted by collecting contextual cues of an environment with a predefined set of sensors and extracting contextual cues from an audio-visual content stream itself.

Event detection and classification

[0063] Multimodal content of primary and secondary sensor signals may be used by event detection algorithms and classified to create feature vectors and content identifiers. Secondary content not intended for media representation may be sampled with a lower sampling rate and lower dynamics compared to primary content. Secondary content may contain descriptive contextual cues about location, a captured event time window, user/content creator context, and environmental conditions at the point of content capture. A low sampling rate representation of secondary content may be sufficient. Sparse secondary content may be made up of a limited number and duration of bursts of data. Storage of content on predetermined intervals may be used for sparse compressed content or for significant transients or content changes when a sensor signal exceeds a predetermined threshold.

[0064] Contextual cues are extracted in a similar manner from primary content. Because primary content may be available in all recordings of an event, contextual cues may be determined based on transients and impulse type of features in the content. Because a plurality of devices recording content simultaneously lack synchronized timing, content-based timing of contextual cue extraction is more reliable. Both primary and secondary content may be used to synchronize time slots.

Capturing contextual cues

[0065] Physical sensor signals are seldom readily available in a sparse format when sampled with a given regular sampling frequency and converted to the digital domain. Although actual information content is typically far less than a given sampling frequency, a time domain signal may not be sampled with a lower rate without losing information.

[0066] A signal made up of sinusoidal harmonics, for example, may be efficiently represented in a time-frequency transform domain as a sparse impulse train. The following scenarios, among others, lend themselves well to sparse-signal representation: a sudden change in radio signal strength, number of Wi-Fi hot spots, cellular network details, or background noise level; a sound event, such as clapping of hands; a flash of light or acceleration of the structure in which the sensor nodes are attached.

[0067] For example, if a detected event is a time-domain transient, high-pass filtering may remove unnecessary data. Because an access method only determines whether devices detect an event simultaneously, more details about the signal are not captured. To capture contextual cues, recording of only transient incidents may be sufficient.

[0068] Alternatively, and in addition, content may be transformed, for example, with a discrete Fourier transform (DFT) or discrete wavelet transform (DWT) to enable a scalable representation. Harmonic signals are compressed in the DFT domain. And, for a wavelet transformation, a sparse representation may include the most significant coefficients in the beginning of the transform vector, or the highest magnitude coefficients.

[0069] A signal may be converted to a sparse domain by bandpass filtering and having, for example, a signal variance-based threshold function on signal level. When a filtered signal exceeds a predetermined level, the output is activated. For some embodiments, the level may be determined by an adaptive threshold function. The threshold may have, for example, a value two times the sensor signal variance in a given analysis window. The result is a sparse time-domain signal carrying information about only the event start and stop times. One embodiment may normalize the resulting sparse signal. Another embodiment may assign a value of 1 or -1 to a time-domain pulse in a sparse signal depending on the sign of the pulse. Another embodiment may normalize the signal level so that the inner product of the signal is unity. Classification cue coding

[0070] Contextual cues from primary and secondary content are dependent on each other. Therefore, contextual cues from primary and secondary content are coded and multiplexed as linked parameters. If criteria for a transient or an event is met (a transient or an event is found in the content), corresponding cues are extracted, compressed, and stored. The overall number of cues may be limited by a predetermined minimum trailing period after a set of cues is stored. In addition, the maximum number of cues of secondary content related to each primary content cue may be limited.

[0071] In some embodiments, secondary content is not to be stored anywhere in such a format, and only the corresponding contextual cues are stored with timing information. Some embodiments may store secondary content relative to primary content cues. Hence, a timestamp of secondary content cue may be determined as a temporal distance to a previous primary content cue. Timing information may be stored as relative information. Contextual cues for primary content may be stored after a predefined trailing period. Contextual cues of secondary content may be linked to primary content with relative timing information. In addition to relative timing, a sequence number may be used to link a cue to a correct primary content cue. The number of secondary content contextual cues may be limited to control data rates and data sizes. Some embodiments may use a predefined maximum number of cues that may be linked to each primary content cue.

[0072] FIG. 6 is a block diagram 600 of a system for extracting contextual cues from primary 602 and secondary 604 content. Primary audio-visual content 602 is compressed 606 with dedicated algorithms and forwarded to a live streaming service 616. To extract context 608, 610, a contextual cue estimation algorithm finds all the transients and events exceeding predefined thresholds. Extracted cues are compressed 612, 614 and coded with primary and secondary context dependent timing information as shown by the compression box 612, 614. Secondary content is processed in a similar manner to detect and to compress contextual cues. The compressed contextual cues with corresponding time stamps are stored as metadata 618 or payload header information for a live content stream. The metadata may be location-indicating metadata that contains information indicative of a source location (e.g. a video capture location) of a stream. For example, the location-indicating metadata may be express location information, such as GPS coordinates, other map coordinates, or address information. As another example, the location- indicating metadata may be data that is indirectly indicative of a source location, such as information identifying one or more wireless access points (e.g. by SSID) or other radio sources that are in range of the source location. The metadata may further include received signal strength information for the radio source or sources. Location-indicating metadata may be used to determine proximity of different source locations without necessarily requiring determination of an absolute position of the source locations. For example, two source locations may be determined to be in proximity when at least a threshold number of the same wireless access points (e.g. the same SSIDs) are detected at the two locations. Such locations may be determined to be in proximity even without access to any information regarding the absolute location of those sources or the detected access points.

Metadata format for clustering the content to the reference

[0073] FIG. 7 illustrates one embodiment 700 for dependent handling of contextual cues. Contextual cues 704 of primary content 702 are extracted followed by a predefined (e.g., one- second) trailing period 712. The contextual cues 706 of secondary content 708 are linked to primary content 702 with relative timing information 714, 716. In addition to relative timing 714, 716, a sequence number may be used to link a cue to a correct primary content cue. The number of secondary content contextual cues may be limited to control the data rate and data size. A predefined maximum number of cues 704 may be linked to each primary content cue. Contextual cues 704, 706 are extracted from both primary and one or more secondary content streams. The trailing period 712 limits the number of contextual cue parameters, limits the amount of storage space used, and reduces the complexity of stored data. Contextual cues 706 extracted from one or more secondary content streams 708 are coded relative to a previously recorded primary content cue.

[0074] Contextual cues from primary 702 and secondary 708 content are dependent on each other and are coded and multiplexed as linked parameters. If criteria for collecting a transient 710, 718 or an event is met (a transient or an event is determined in the content), the corresponding cues are extracted, compressed, and stored as metadata. The overall number of cues may be limited by a predetermined minimum trailing period 712 after a set of cues is stored. The trailing period 712 limits the number of contextual cues (or information) used. For example, a one-second trailing period 712 limits extensive storage and computation but maintains sufficient accuracy in comparing different content streams. In addition, the maximum number of cues of the secondary content 708 related to each primary content cue may be limited. For example, a loud sound effect may be classified as a contextual event within the primary content. The sound effect may cause the recording device to shake. The shake may be used as a contextual cue for secondary content. If so, the events are related and simultaneous, and therefore, the relative time difference between primary content cues and secondary content cues is zero. Hence, no additional time difference is used as a relative time cue in the clustering phase. Content clustering API

[0075] FIG. 8 shows a block diagram 800 for extracting ongoing audio and video streams from selected social media services, such as Periscope. FIG. 8 shows an exemplary content clustering process for an extended content service 802. Media streams available in different social media sites are extended with secondary data fields of contextual cues extracted from both primary and secondary content. The content may be clustered based on these contextual cues to third-party reference content. Third-party reference content and corresponding contextual cues are sent to an API service 810 for a clustering tool. Third-party content reference 808 content may comprise primary media content with available secondary sensor content. The secondary content contains, e.g., location coordinates, radio channel conditions, available Wi-Fi hot spots and corresponding signal strength. Secondary content may also contain information that is determined from primary content, such as reverberation and outdoor/indoor classification. Additional sensor signals, such as accelerometer, lighting, and location available in media content captured by individual users, may be applied to link social media content together when connection to a third-party reference content is established. These connections may be maintained using sensor content when third- party content has ended.

[0076] Methods may be applied to detect linked content and rank 804 relevance to each other as a content item network. As a result, content from the same location with similar contextual cues are linked together. On-going streams are clustered together 806 with a reference based on contextual cues determined from primary audio visual content and secondary sensor content.

[0077] The relevance of the on-going social media video stream compared to a reference stream is determined with ranking the content. The content stream having the most information in relation to other streams and to the reference stream is automatically ranked the highest and the most relevant. Other linked content streams may add a new perspective to the material and may add new information.

[0078] Content identification and clustering occurs iteratively. When connected content, e.g. a content cluster/network is detected, additional streams are continuously searched in the databases to find new related streams (e.g., as crowdsourced content providers may start and stop streaming, or the content location or environment may change). The process may be performed on a continuous basis, and new relevant streams may be detected. Old connections between streams may disappear when content similarity is lost. Additional details about how the content is analyzed, compared, ranked, and linked is provided below. Linked streams are listed in different clusters based on contextual characteristics and connections to a reference. These clusters are used to create a content pool for an automatic media content curation, composition, and rendering of media presentations.

Contextual content similarity

[0079] Contextual similarity and ranking of a particular content stream against a reference content in a service may be determined using only contextual cues of both primary and secondary content with timing information. The comparison of content (e.g., social media content streams) against a reference content stream is made using contextual cues and the timing between primary and secondary content cues. Clustering is split into three phases. Completion of all three phases enables linking of different content streams together.

[0080] FIG. 9 is a block diagram 900 for a system of the linking of broadcaster content 904 (or reference media streams) to social media content 902 (or streams). The three phases (or stages) 906, 908, 910 are shown in the center of the diagram. Contextual cues are derived (or extracted) 920, 922, 926, 928 from social media primary content 912, social media secondary content 914, broadcaster primary content 916, and broadcaster secondary content 918. Stage- 1 matches contextual cues for secondary content (sensors) 914, 918. Stage-2 matches contextual cues for primary content (audio/video streams) 912, 916. Stage-3 links 924, 930 the stage-1 and stage-2 clusters together. Each of these three stages is discussed in more detail below. Matching 906, 908, 910 is done using lightweight contextual cues derived from primary and secondary signals. Some embodiments may use a combination of the three matching stages as building blocks. A plurality of content streams that have linked audio-visual content (or a plurality of crowd-sourced content streams) associated with a reference content stream are sent to a user device to be displayed on a user interface. The user has the option to select from the group of content streams associated with the reference content stream. Social media primary content 912 may be social media feeds (or streams) from mobile devices. Social media secondary content 914 may include sensor data from a social media provider (or application) and may be generated to accompany a social media primary audio/visual feed. Broadcaster primary content 916 may be a broadcaster audio/visual feed (or reference media stream) from a professional production. Broadcaster secondary content 918 may include sensor data generated to accompany a broadcaster primary audio/visual feed (or reference media stream).

Stage 1: Contextual similarity clustering

[0081] FIG. 10 is a timing diagram 1000 for a contextual similarity clustering process. Content identification checks the secondary content modalities. One check compares similarity of location.

Available contextual clues are also compared. Reference secondary content 1002 is available in different sensor modalities. A reference may be available with secondary content (e.g., sensor data signals provided by a broadcaster, location coordinates, information about available Wi-Fi hot spots, background noise level, indoor/outdoor classification), or as a collection of already extracted contextual cues with timing information 1006, 1014, 1016 for a timeline 1008. If reference secondary content is available as sensor signals, the sensor signals are processed (e.g., by the social media service or clustering service) through a detection algorithm. Contextual cues 1004 related to transients 1012 are extracted with corresponding timing information.

[0082] Contextual cues 1004 are compared for similarity. Contextual cues 1004 from reference content 1002 and timing are correlated with corresponding data (e.g., corresponding contextual cues) from the analyzed media streams in social media. Contextual cues 1004 extracted from similar sensor modalities, such as location changes and indoor/outdoor classification, are compared. Comparison of contextual cues 1004, 1010 from different sensor modalities may also provide information. Contextual cues of crowd-sourced contextual information 1010 are matched with reference contextual information 1004 associated with the reference content stream.

[0083] For example, sound effects detected from the audio signal of a third-party reference content may be compared with contextual cues 1004 or events derived from an accelerometer signal associated with social media content. When a sufficient number of received secondary content cues match 1018, 1020 between social media content and reference content, the contextual similarity check is passed. When the contextual cues 1010 are sufficiently similar, content is checked for similarity.

Stage 2: Content similarity clustering

[0084] FIG. 11 shows a timing diagram 1100 for comparing contextual cues 1104 from reference material 1102 to corresponding contextual cues 1108 from social media content. FIG. 11 shows the comparison of primary content cues by extracting reference cues and corresponding timing information 1112 and timeline 1106. Contextual cues 1104 derived from primary content of reference material 1102 may be compared to contextual cues 1108 derived from primary content of social media streams to determine correlating or matching contextual cues. A similarity measurement extracts contextual cues 1104 on transients 1110 in the content 1102. A similarity measurement comprises correlating contextual cues 1104 of reference content (as well as timing information) with contextual cues and timing information extracted from social media content streams. Contextual cues may include sound effects or events, such as hand clapping, and sudden sound level changes. An identifying sound pattern, such as birds singing, may be a contextual cue. Visual content may have sudden changes in lighting, such as flashes of lights, turning lights on/off, and overall color or brightness changes. When compared signals have time domain transients or pulses with identical location and similar amplitude, there is a strong correlation and the recordings and measurements may be determined to be the same event.

[0085] FIG. 12 shows an alternative embodiment 1200 which may be separate from other embodiments. In this embodiment, a comparison entity may extract contextual cues from reference material 1202 and store the extracted contextual cues with corresponding timestamps as comparison metadata with a timeline 1204. A sliding window is used to compare contextual cues when reference cues are available. Cues extracted from social media content 1206 are compared as a sliding window over the reference cues. FIG. 12 shows the process when consecutive cues from the content under test (e.g., social media content) are compared step by step against the reference cues. Timing information of reference cues may be ignored. This alternative embodiment organizes the reference cues based on order of appearance. Contextual cues are compared against each other and when a correlating set is made, the process checks for another set of matches. When a sufficient number of consecutive cue sets are determined to correlate, the content may be linked to the reference. When a comparison entity detects a match to a reference cue, the corresponding timing information is attached to the tested cue.

Stage 3: Linking stage 1 and stage 2 clustering

[0086] Timing information is a factor in connecting primary and secondary content contextual cues. Primary content may have similar characteristics to both third-party reference content and social media content. In addition, sensory context may have similarities, and the cues may be correlated to one another. If the cues are not aligned together (the time information between the event in primary content and the event in secondary content is different), there is no match and content streams are not linked. Clustering may apply a binary decision process on timing information. If timing differs (e.g., by more than one tenth of a second), the content streams are not linked together in the same cluster. Timing information is a relevant factor because the relation between content similarity and context similarity ensures that social media content is relevant. Without tight timing thresholds, a video captured from a TV screen may be incorrectly linked to content.

Content clusters

[0087] FIG. 13 is a schematic illustration 1300 of relationships among content streams.

Different content clusters built around reference content may be understood as content item networks as presented in FIG. 13. Each live media stream is compared against a reference. Social media content is linked to a reference and to other social media content based on a similarity

(correlation) of primary and secondary content. The higher the correlation, the stronger the link.

The correlation may change over time and may become stronger or weaker in comparison with a reference. A link is evolving regarding the link itself and the strength of the link. Similarities may be determined based on a correlation of a reference content stream with a live content stream selected from a group of content streams that have linked audio-visual content.

[0088] When correlations of primary and secondary content cues are normalized (e.g. to the range 0 to 1), the overall linking strength may be the mean of the context (first stage) and content (second stage) correlations. A binary cue of timing information determines the final decision about the validity of context (first stage) and content (second stage) linking between the content streams.

[0089] An average correlation within a predefined time window is another measure for linking. Linking may be measured as a mean of the mean correlation in a given time window. For example, the strength of linking may be defined based on the average correlation within the last 10 seconds. Alternatively, the linking process may use a threshold, such as 50% of cue sets have a correlation above 0.5 within 10 seconds of a sliding-time window.

[0090] The network is created when relevant streams are found. The strength of the linking defines the content relevance. FIG. 13 shows a snapshot of the clustering at one particular time instant. The clusters are evolving in time. For example, during a broadcast news segment, new social media video streams (which are live) are linked to the network as long as the contextual analysis fits with the rest of the network. Similarly, depending on the content linking, different clusters may evolve by merging together or splitting into several new clusters.

[0091] Clusters of different media items are determined to be a network of content nodes. For the example shown in FIG. 13, the amount a node is filled in as black shows the ranking. The black circle corresponding to crowd-sourced video #lb (1310) (which has been created from available source matter has the strongest correlation, followed by reference video A (1302) and crowd- source (e.g., Periscope) video #la (1304). Reference video B (1308) is third, and crowd-sourced (e.g., Periscope) videos #2a, 2b, and 3b (1306, 1312, 1314) are tied for fourth. The line width represents link strength. The link between reference video A (1302) and crowd-sourced (e.g., Periscope) video #la (1304) is the strongest, while the link between reference video B (1308) and crowd-sourced (e.g., Periscope) video #lb (1310) is the second strongest. The rest of the links have the same amount of correlation.

[0092] The two different networks in FIG. 13 show the evolution of a content network. In the beginning, streams #lb (1310) and #2b (1312) are linked to reference B (1308) with different strengths. Later, new streams are connected and the network changes. The stream #lb (1310) becomes the dominant stream while streams #2b (1312) and #3b (1314) may add material that is only partly related. [0093] When an automatic tool determines a rendering scheme, the presentation may be covered mostly by switching between streams #1 and #2, while stream #3 may only contribute with snap shots. Since the stream #1 is the highest ranked, for example, the audio track may be taken from steam #1 and visual content may switch between streams #2 and #3.

Reference content based content sourcing architecture

[0094] FIG. 14 is a system diagram for a server-client type architecture 1400 for content sourcing relative to a reference. A content API 1406 creates a composition bit stream and/or multiple crowd-sourced content streams for streaming clients 1420 viewing content. Crowd- sourced content streams have associated crowd-sourced contextual information.

[0095] The server 1418, for example, may be a social media video live streaming service 1402 that is hosting crowd-sourced live streams and a clustering tool 1404 for selecting 1414 content. The client 1420 may comprise both a third-party news service backend (a server hosting local stream broadcast to streaming applications) and a streaming application in a smart phone or a smart TV, for example. The backend streams reference video to viewers and provides content cues 1416 to the server API 1406.

[0096] The clustering tool 1404 continuously fetches 1414 ongoing streams from extended content service 1402 databases, conducts linking and ranking, and clusters 1404 content against reference content 1416 available from a third -party application or service 1408. Clustering 1404 provides relevant live streams for the content API 1406 with the clustering information regarding the content relevance, as well as mutual linking and ranking. Contextual content cues 1416 are received from a third-party content service back end 1408 (the client of this architecture representing the reference stream).

[0097] The content API 1406 interfaces with third-party clients 1408. The third -party service back end 1408 is connected to the API 1406 for sharing the reference content 1416 and to receive related live streams. The API 1406 provides a streaming service with functionality to accept streaming requests and receive reference content cues 1416 of an on-going third-party stream in a client application. The API 1406 controls the clustering tool 1404 by providing reference content cues 1416. The API 1406 also returns a list of available live streams with different criteria to a client, bundles linked streams to a media composition, and interfaces with streaming (using a protocol, such as, HTTP live streaming (HLS)). The API's list of available live streams may include a selected geographical area or a global list that is based on popularity (number of viewers). The API's list may also include a list of followed users or a list generated based on relevance to reference material. Bundling linked streams into a media composition uses the highest ranked stream as a cover page (or landing page). The list of linked streams is communicated in a ranked order. The API 1406 interfaces with streaming for the selected content. The API 1406 creates an automatic composition of curated streams. Performing automatic composition of clustered streams in the content API 1406 is more efficient because only one bit-stream is transmitted.

[0098] A live streaming client has a user interface to select content within the streaming application 1410 running on a smart TV, a smart phone, or another device. A user may select automatically composed content comprising several different content streams or select one of the streams within the bundle. When a stream is selected, the client 1420 renders 1412 the content on the screen.

Variations

[0099] Third-party content may be recorded streams. A smart phone application, news service, or TV program may provide non-real-time content. The service may provide corresponding contextual cues into the live streaming service API and send a request for content clusters that are related to it. The corresponding contextual cues may be determined by a method for contextual watermarking.

[0100] The live streaming service may determine clusters of both live and recorded content that is linked to a reference. The functionality for content sourcing of recorded content may use the same UI as live content. Related live streams may be different and some of the social media streams that were previously live may no longer be available.

[0101] Different embodiments may use different weightings of contextual and content similarity correlations to determine a final linking of contents. One embodiment may use the mean of the contextual and content correlations. Another embodiment may use a weighting that favors actual content (or sensory context). Another embodiment may use a three-stage decision path as describe above but calculate the relevance of content based only on the correlation of content- based cues.

[0102] Some embodiments may group the processes of the three stages into alternate collections of processes. For one embodiment, stage 1 may analyze cues from secondary content. Stage 2 may analyze cues from primary content. Stage 3 may analyze inter-cue timing between cues from primary content and related/linked cues from secondary content.

[0103] Another embodiment may perform a linking process using only cues from secondary (stage 1) content. Another embodiment may perform a linking process using only cues from primary (stage 2) content. Another embodiment may perform a linking process using only inter- cue timing (stage 3). Another embodiment may perform Stage 1 and Stage 2 separately, and rank a social media stream based on the matches found in stages 1 and 2.

[0104] Another embodiment may perform Stage 1 to determine preliminary rankings for social media streams and use a threshold on those ranking to determine a highly ranked subset of social media streams. Stage 2 may be performed only on the highly ranked subset of social media streams to determine a final ranking for the social media streams of the subset.

[0105] Another embodiment may perform Stage 1 to determine an initial set of rankings/confidence values for social media streams. Stage 2 and/or stage 3 may be performed only if stage 1 failed to determine a sufficient number of matching social media streams with a high enough initial ranking or confidence level.

[0106] Another embodiment may perform an initial filtering of social media streams using location information (e.g., GPS data) relative to location information provided for reference content, and may perform one or more stages according to any of the above embodiments on only the social media streams which have a location within a threshold distance of the reference location.

[0107] Another embodiment may perform the stages, but the order each stage is performed may depend on what secondary sensor signals or secondary cues are available for the reference content and how many of these items correspond to similar or compatible secondary signals/cues available for the social media content streams.

[0108] One embodiment extends a mobile application. For example, a broadcaster back-end process may have functionality to represent a real-time content from a live interview happening on a street. Another embodiment performs this functionality in a broadcaster mobile application. For either a broadcaster back-end process or a broadcaster mobile application, a user browsing the content may choose to view a live stream. At the same time, the mobile application or service backend may transmit contextual information of the live video stream to a social media service content API. The social media video service clusters existing live streams in the service around the broadcaster reference stream and creates a multi-view composition out of all relevant live streams. The composition is streamed to the broadcaster application or backend service, and the user may follow additional content (via the social media composition) related to the original news segment. In some embodiments, the broadcaster backend operates to create a link to the social media composition only once per broadcast or stream. With this functionality, a news service is able to offer an improved user experience. The user may continue following the breaking news after a short news segment with crowd-sourced material created by social media users. [0109] FIG. 15 A is an example user interface 1500 for selecting a view from a content bundle. For example, a broadcaster mobile application or web page streams live content 1502 embedded in a news article. This embodiment is an alternative to the user interfaces of FIGs. 2 and 3 for a smart TV showing live coverage.

[0110] To receive wider coverage, a user presses 1506 the corresponding "more" button 1504 and/or a list of icons representing social media content. The UI displays video presentation mode. The application will fetch the linked content from a social media video API and render the multi- view composition 1508, 1510 of the content cluster.

[0111] For another embodiment, the application displays the multi-view composition 1508, 1510 in the main view of the screen. The lower portion of the screen displays a series of icons 1512 (middle mobile device of FIG. 15 A) or a map layout of icons 1514 (right mobile device of FIG. 15 A) for other linked streams. The other streams may be selected based on either the descriptive icon or the location on the map. The original broadcaster material may be highlighted on the list.

[0112] For some embodiments, the user may switch between views without any limits at any time. The composition 1508, 1510 will continue even after the original broadcaster coverage (or reference media stream) ended. The content ranking may be used to display first the content determined to be most informative.

[0113] FIG. 15B is an example user interface 1550 that combines stream selection 1556 with video playback. For one embodiment, a broadcaster application may have live stream 1552 selection available immediately in a map layout 1558. The map 1558 may simultaneously provide information about the location of the news event and list available streams. The map layout 1560 may continue to be displayed after broadcaster live coverage (or reference media stream) has ended 1554. The application continues following social media video streams (which may be live) that were clustered in the content network earlier. For one embodiment, the application may receive information that a broadcaster live coverage has ended 1554 and may use an ended broadcaster stream to continue to identify social media video streams (which may be live). Social media video streams may be linked together, such as using a ranking as illustrated in FIG. 13. In this case, crowd-sourced video streams may be clustered within a content network without a broadcaster live stream. The highest ranking social stream may be used as a reference to link other streams. A network may be maintained with linked streams that are within the same geographical range as the network that existed at the point the broadcaster live stream ended. Links to these available social streams may remain on a user interface. If the linked network of live social media streams is available, the user interface may maintain them as icons. The icons also may be available if broadcasted coverage is streamed again at later time.

[0114] For some embodiments, the broadcaster application may also show recorded content. The social media video API clusters social media content that was recorded at the same time as the broadcaster reference. In addition, the social media video API may search for live content that is related to the recorded broadcast video reference. Available streams are presented on a map view among a news article. Another embodiment displays the available streams as a list of icons, which may be used, for example, if the location information of the social media stream is confidential.

[0115] In response to a user pushing a content stream button (or otherwise selecting a content stream), the display will open either a list of available streams and or composition of all streams. The button may also have a list of icons displaying the content of the corresponding Periscope streams.

[0116] FIG. 16 is an alternative embodiment 1600 for a user interface. FIG. 16 is user interface diagram for an application running on a mobile device. As seen in FIG. 5, commercial content currently available is presented on a map with linked content 1606, 1608, 1610, 1612, 1614 from a live streaming service 1602, 1604, such as Periscope. A user may select a multi-view bundle 1616, which selects streams from broadcasters #1 and #2 (1620). A user may select 1618 a single stream 1622, 1624, 1626. Automatic multi-view presentation 1628 of streams displays on the main portion of the screen. Other available streams 1630, 1632, 1634 display along the bottom of the screen 1636.

[0117] A commercial news service may create a dedicated service for live video streaming. The service may use a social media API to determine content clusters that match each on-going live stream (or recorded streams for some embodiments). Live streaming availability may be presented via a map view. Linked streams are shown on the map with linking symbols.

[0118] In response to a user selecting a stream by tapping a linked content symbols (or otherwise making a selection), the application displays a list of streams in the bundle. For one embodiment, the first line corresponds to automatic composition of linked streams and other lines represent individual streams. Based on the selection, the application will render either the automatic composition or an individual stream. For some embodiments, the presentation view may also display the other available streams. This embodiment is similar to FIG. 15 A. Network Architecture

[0119] A wireless transmit/receive unit (WTRU) may be used as a system interface device in embodiments described herein.

[0120] FIG. 17A is a system diagram of an example WTRU 102. As shown in FIG. 17A, the WTRU 102 may include a processor 118, a transceiver 120, a transmit/receive element 122, a speaker/microphone 124, a keypad 126, a display/touchpad 128, a non-removable memory 130, a removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and other peripherals 138. The transceiver 120 may be implemented as a component of decoder logic 119. For example, the transceiver 120 and decoder logic 119 may be implemented on a single LTE or LTE-A chip. The decoder logic may include a processor operative to perform instructions stored in a non-transitory computer-readable medium. As an alternative, or in addition, the decoder logic may be implemented using custom and/or programmable digital logic circuitry.

[0121] It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment. Also, embodiments contemplate interfacing with base stations and/or the nodes that base stations may represent, such as but not limited to transceiver station (BTS), a Node-B, a site controller, an access point (AP), a home node-B, an evolved home node-B (eNodeB), a home evolved node-B (HeNB), a home evolved node-B gateway, and proxy nodes, among others, may include some or all of the elements depicted in FIG. 17A and described herein.

[0122] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 1 18 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While FIG. 17A depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0123] The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station over the air interface 115/116/117. For example, in one embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In another embodiment, the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, as examples. In yet another embodiment, the transmit/receive element 122 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.

[0124] In addition, although the transmit/receive element 122 is depicted in FIG. 17A as a single element, the WTRU 102 may include any number of transmit/receive elements 122. More specifically, the WTRU 102 may employ MFMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 115/116/117.

[0125] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, as examples.

[0126] The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128. In addition, the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SFM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0127] The processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. As examples, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel- zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li -ion), and the like), solar cells, fuel cells, and the like. [0128] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 115/116/117 from a base station and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.

[0129] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.

[0130] FIG. 17B depicts an example network entity 190 that may be used within a communication system. As depicted in FIG. 17B, network entity 190 includes a communication interface 192, a processor 194, and non-transitory data storage 196, all of which are communicatively linked by a bus, network, or other communication path 198.

[0131] Communication interface 192 may include one or more wired communication interfaces and/or one or more wireless-communication interfaces. With respect to wired communication, communication interface 192 may include one or more interfaces such as Ethernet interfaces, as an example. With respect to wireless communication, communication interface 192 may include components such as one or more antennae, one or more transceivers/chipsets designed and configured for one or more types of wireless (e.g., LTE) communication, and/or any other components deemed suitable by those of skill in the relevant art. And further with respect to wireless communication, communication interface 192 may be equipped at a scale and with a configuration appropriate for acting on the network side— as opposed to the client side— of wireless communications (e.g., LTE communications, Wi-Fi communications, and the like). Thus, communication interface 192 may include the appropriate equipment and circuitry (perhaps including multiple transceivers) for serving multiple mobile stations, UEs, or other access terminals in a coverage area. [0132] Processor 194 may include one or more processors of any type deemed suitable by those of skill in the relevant art, some examples including a general-purpose microprocessor and a dedicated DSP.

[0133] Data storage 196 may take the form of any non-transitory computer-readable medium or combination of such media, some examples including flash memory, read-only memory (ROM), and random-access memory (RAM) to name but a few, as any one or more types of non- transitory data storage deemed suitable by those of skill in the relevant art may be used. As depicted in FIG. 17B, data storage 196 contains program instructions 197 executable by processor 194 for carrying out various combinations of the various network-entity functions described herein.

[0134] In some embodiments, the network-entity functions described herein are carried out by a network entity having a structure similar to that of network entity 190 of FIG. 17B. In some embodiments, one or more of such functions are carried out by a set of multiple network entities in combination, where each network entity has a structure similar to that of network entity 190 of FIG. 17B. In various different embodiments, network entity 190 is— or at least includes— one or more of (one or more entities in) RAN 103, (one or more entities in) RAN 104, (one or more entities in) RAN 105, (one or more entities in) core network 106, (one or more entities in) core network 107, (one or more entities in) core network 109, base station 114a, base station 114b, Node-B 140a, Node-B 140b, Node-B 140c, RNC 142a, RNC 142b, MGW 144, MSC 146, SGSN 148, GGSN 150, eNode B 160a, eNode B 160b, eNode B 160c, MME 162, serving gateway 164, PDN gateway 166, base station 180a, base station 180b, base station 180c, ASN gateway 182, MIP-HA 184, AAA 186, and gateway 188. And certainly other network entities and/or combinations of network entities may be used in various embodiments for carrying out the network-entity functions described herein, as the foregoing list is provided by way of example and not by way of limitation.

[0135] Note that various hardware elements of one or more of the described embodiments are referred to as "modules" that carry out (i.e., perform, execute, and the like) various functions that are described herein in connection with the respective modules. As used herein, a module includes hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more memory devices) deemed suitable by those of skill in the relevant art for a given implementation. Each described module may also include instructions executable for carrying out the one or more functions described as being carried out by the respective module, and it is noted that those instructions may take the form of or include hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and/or the like, and may be stored in any suitable non-transitory computer-readable medium or media, such as commonly referred to as RAM, ROM, etc.

[0136] Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer- readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD- ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.