CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application No.

61/764,336 filed on Februaiy 13, 2013 the contents of which are hereby incorporated by reference herein.

BACKGROUND

[0002] Currently a large number of people use computing devices including, for example, mobile phones, tablets, as multimedia devices. The computing devices may be capable of playing audio visual content, e.g., music, video, and the like. Unlike conventional audiovisual devices, for example, TVs, home theater systems, and the like, mobile computing devices may provide a user with flexibility to adjust factors, for example, the orientation and viewing distance from the device. Based on these factors and other environmental factors, e.g., ambient noise the perceived audio quality may deteriorate. Rendering methods and systems, however, may provide inadequate mechanisms of alleviating effects of such factors. SUMMARY

[0083] Systems, methods, and instrumentalities are provided to implement user adaptive audio processing on audio-visual devices, e.g., mobile computing devices by using at least one of user's distance, user's age, user's gender, and/or ambient noise level in a single and/or multi user environment.

[0004J A device may receive information about a user from one or more sensors (e.g., infrared sensor, ultrasonic sensor, motion sensor, optical sensor). The device may calculate distance of the user (e.g., user's face) from the device. The device may determine the user's age and gender. The device may determine the noise level around the device. The device may adaptiveiy adjust at least one audio characteristic of the audio content, based on at least one of the distance of the user, user's age, user's gender, and/or the noise level.

[0005] A device may receive information about at least one user from at least one sensor.

The de vice may estimate at least one of age, gender or distance for each of the at least one user. The device may measure noise level. The device may adaptiveiy adjust an audio characteristic of the audio content for each of the at feast one user.

BRIEF DESCRIPTION OF THE DRAWINGS

[0086] FIG. 1A is a system diagram of an example communications system.

[0067] FIG. IB is a system diagram of an example wireless transmit/receive unit

(WTRU) that may be used within the communications system ilktstrated in FIG. I A.

[0008] FIG. 1C is a system diagram of an example radio access network and an example core network th t may be used within the communications system illustrated in FIG. 1 A.

[0009] FIG. ID is a system diagram of another example radio access network and another example core network that may be used within the communications system illustrated in FIG. 1 A.

[0010] FIG. IE is a system diagram of another example radio access network and another example core network that may be used within the communications system illustrated in FIG. 1A.

[0011] FIG. 2 illustrates an example of a threshold of quiet as a function of frequency for various ages.

[0012] FIG. 3 illustrates an example of a threshold of quiet for various ages and gender. [0013] FIG. 4 illustrates an example of sound localization using, for example, difference in inter-arrival time.

[0014] FIG. 5 illustrates an example of interaural level difference, interaural time difference, and/or interaural coherence.

[0015] FIG. 6 illustrates an exemplary spatial audio coding (SAC) system architecture.

[0016] FIG. 7 illustrates an example of variability of a user's distance from the mobile computing device, for example a device, e.g., mobile phone.

[0017] FIG. 8 illustrates an example of distance-volume relationship that may be adopted to control loudness of an audio signal.

[0018] FIG, 9 illustrates a block diagram of automatic volume adjustment using distance estimated by a user activity detection (UAD) module.

[0019] FIG. 10 illustrates an example of a block diagram of automatic volume adjustment using, e.g., user's distance from a device, age, and/or gender.

[002(5] FIG. 1 1 illustrates an example of a block diagram of automatic audio equalization system using, e.g., user's distance from a device, age, and/or gender,

[0021] FIG. 12 illustrates an example of a block diagram of automatic audio equalization system using, e.g., user's age, and/or gender.

[0022] FIG. 13 illustrates an exemplary adaptive streaming system.

[0023] FIG, 14 illustrates an example to adapt an audio bitsireani requested from the server based on the user's age and/or gender.

[0024] FIG. 15 illustrates an example of adaptation of volume to ambient noise level.

[0025] FIG. 16 illustrates an example of adaptation of volume to user's age, gender and distance, and noise level.

[0026] FIG. 17 illustrates an example of adjusting volume in a multiuser scenario.

[0027] FIG. 18 illustrates an example of using UAD to adjust for changed listener location.

[0028] FIG. 19A illustrates an exemplary audio system in mono sound.

[0029] FIG. 19B illustrates an exemplary audio system with a number of channels in stereo sound.

- ^ - [0030] FIG. 19C illustrates an exemplary audio system with a number of channels in

Dolby 5.1 surround sound.

[0031] FIG. 2.0 illustrates an example of speaker placement in a Dolby 5.1 sound system,

[0032] FIG. 21 illustrates an example of variation in the user's location from the ideal location.

[0033] FIG. 22 illustrates an example of adapting audio playback to a user's changed location.

[0034] FIG. 23 illustrates an example of adaptation of stereo balance using volume.

[0035] FIG. 24 illustrates an exemplary streaming system adapting to user's presence and location.

DETAILED DESCRIPTION

[0036] A detailed description of illustrative embodiments will now be described with reference to the various figures. Although this description provides a detailed example of possible implementations, it should be noted that the details are intended to be exemplary and in no way limit the scope of the application.

[0037] FIG. IA is a diagram of an example communications system 100. The communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast to multiple wireless users. The communications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), and the like.

[0038] The communications system 100 may include wireless transmit/receive units

(WTRUs) 102a, 102b, 102c, and/or 102d (which generally or collectively may be referred to as WTRU 102), a radio access network (RAN) 103/104/105, a core network 106/107/109, a public switched telephone network (PSTN) 108, the Internet 1 10, and other networks 1 12, though it will be appreciated that any number of WTRUs, base stations, networks, and/or network elements are contemplated. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and/or communicate in a wireless environment. By w ^r ay of example, the WTRUs 102a, 102b, 102c, 102d may be configured to transmit and/or receive wireless signals and may include wireless transmit/receive unit (WTRU), a mobile station, a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, consumer electronics, and the like.

[0Θ39] The communications systems 00 may also include a base station 1 14a and a base station 1 14b, Each of the base stations 1 14a, 1 14b may be any type of device configured to wirelessiy interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 1 10, and/or the networks 1 12. By way of example, the base stations 1 14a, 1 14b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 1 14a, 1 14b are each depicted as a single element, it will be appreciated that the base stations 1 14a, 1 14b may include any number of interconnected base stations and/or network elements.

[0040] The base station 1 14a may be part of the ^' RAM 103/104/105, which may also include other base stations and/or network elements (not depicted), such as a base station controller (BSC), a radio network controller (RNC), relay nodes. The base station 1 14a and/or the base station 1 14b may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not depicted). The cell may further be divided into cell sectors. For example, the cell associated with the base station 1 14a may be divided into three sectors. Thus, the base station 1 14a may include three transceivers, i.e., one for each sector of the cell. The base station 1 14a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.

[0041] The base stations 1 14a, 1 14b may communicat e with one or more of the WTRUs

102a, 102b, 1 02c, 1 02d over an air interface 1 1 5/1 16/ 1 17, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light). The air interface 1 15/ 1 16/ 1 17 may be established using any suitable radio access technology (RAT).

[0042] More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 1 14a in the RAM 103/104/ 105 and the WTRUs 102a, 1 02b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 1 15/1 16/1 17 using wideband CDMA (WCDMA).

WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) arsd/or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink Packet Access (HSDPA) and/or High-Speed Uplink Packet Access (H8UPA).

[0043] The base station 1 14a and the WTRUs 102a, 102b, 02c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 1 15/1 16/1 17 using Long Term Evolution (LTE) and/ or LTE- Advanced (LTE- A),

[0044] The base station 1 4a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.16 (i.e.. Worldwide Interoperability for Microwave Access (WiMAX)), CDMA200Q, CDMA2000 I X, CDMA2000 EV-DO, Interim Standard 2000 (IS- 2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

[0045] The base station 1 14b in FIG. 1 A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like. The base station 1 14b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802, 1 1 to establish a wireless local area network (WLAN). The base station 1 14b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). Or, the base station 1 14b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A) to establish a picocell or femtocell. The base station 1 14b may have a direct connection to the Internet 1 10. Thus, the base station 1 14b may not be required to access the Internet 1 10 via the core network 106/107/1 09.

[0046] The RAN 103/1 04/105 may be in communication with the core network

106/107/109, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) se dees to one or more of the WTRUs 102a, 102b, 102c, 102d. For example, the core network 106/ 107/109 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, and/or perform high-level security functions, such as user authentication. Although not depicted in FIG. 1 A, it will be appreciated that the RAN 103/104/105 and/or the core network 106/1 07/109 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 103/104/ 105 or a different RAT. For example, in addition to being connected to the RAN 103/104/105, which may be utilizing an E-UTRA radio technology, the core network 106/107/109 may also be in communication with another RAN (not depicted) employing a GSM radio technology.

[0047] The core network 106/ 107/ 109 may also serve as a gateway for the WTRUs 102 a,

102b, 102c, 102d to access the PSTN 108, the Internet 1 10, and/or other networks 1 12. The PSTN 108 may include circuit-switched telephone networks that provide plain old telephone sendee (POTS). The Internet 1 10 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and the internet protocol (IP) in the TCP/IP internet protocol suite. The networks 1 12 may include wired or wireless

communications networks owned and/or operated by other service providers. For example, the networks 1 12 may include another core network connected to one or more RANs, which may employ the same RAT as the RA 103/104/105 or a different RAT.

[0048] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system

100 may include multi-mode capabilities, i.e., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless Jinks. For example, the VVTRU 102c (FIG. 1 A) may be configured to communicate with the base station 1 14a, which may employ a cellular-based radio technology, and with the base station 1 14b, which may employ an IEEE 802 radio technology.

[0049] FIG. IB is a system diagram of an example VVTRU 102. The WTRU 102 may include a processor 1 18, a transceiver 120, a transmit receive element 122, a speaker/microphone 124, a keypad 126, a display/touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and other peripherals 138. It will be appreciated that the WTRU 102. may include any sub-combination of the foregoing elements while remaining consistent. Also, the base stations 1 14a and 1 14b, and/or the nodes that base stations 1 14a and 1 14b may represent, such as but not limited to transceiver station (BTS), a Node-B, a site controller, an access point (AP), a home node-B, an evolved home node-B (eNodeB), a home evolved node-B (He B), a home evolved node-B gateway, and proxy nodes, among others, may include some or all of the elements depicted in FIG. IB and described herein.

[0050] The processor 1 18 may be a general purpose processor, a special ur o e processor, a conventional processor, a digital signal processor (DSP), a plurality of

microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 1 18 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WT U 102 to operate in a wireless environment. The processor 1 18 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While FIG. IB depicts the processor 1 18 and the transceiver 120 as separate components, it will be appreciated that the processor 1 18 and the transceiver 120 may be integrated together in an electronic package or chip.

[0051] The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 1 14a) over the air interface

1 15/1 16/1 17. The transmit/receive element 12.2 may be an antenna configured to transmit and/or receive RF signals. The transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. The transmit/receive element 122 may be configured to transmit and receive both RF and light signals. t will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.

[0052] In addition, although the transmit/receive element 122 is depicted in FIG. IB as a single element, the WTRU 102 may include any number of transmit/receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. The WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 1 15/1 16/1 17.

[0053] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as UTRA and IEEE 802.1 1 , for example.

[0054] The processor 1 18 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 1 18 may also output user data to the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128. In addition, the processor 1 18 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132. The non-removable memory 130 may include random- access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. The processor 1 18 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not depicted).

[0055] The processor 1 18 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCad), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion)), solar cells, fuel cells, and the like.

[0056] The processor 1 18 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location mformation over the air interface 1 15/1 16/1 17 from a base station (e.g., base stations 1 14a, 1 14b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method.

[0057] The processor 1 18 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 138 may mclude an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.

[0058] FIG, 1C is a system diagram of the RAN 103 and the core network 106. As noted above, the RAN 103 may employ a UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 1 15. The RAN 103 may also be in communication with the core network 106. The RAN 103 may include Node-Bs 140a, 140b, 140c, which may each mclude one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 1 15. The Node-Bs 140a, 140b, 140c may each be associated with a particular cell (not depicted) within the RAN 103. The RAN 103 may also include RNCs 142a, 142b. It will be appreciated that the RAN 103 may include any number of Node-Bs and RNCs.

[0059] The Node-Bs 140a, 140b may be in communication with the RNC 142a.

Additionally, the Node-B 140c may be in communication with the RNC 142b. The Node-Bs 140a, 140b, 140c may communicate with the respective RNCs 142a, 142b via an lub interface. The RNCs 142a, 142b may be in communication with one another via an Tur interface. Each of the RNCs 142a, 142b may be configured to control the respective Node-Bs 140a, 140b, 140c to which it is connected. In addition, each of the RNCs 142a, 142b may be configured to carry out or support other functionality, such as outer loop power control, load control, admission control, packet scheduling, handover control, macro diversity, security functions, data encryption, and the like.

0Θ60] The core network 106 may include a media gateway (MGW) 144, a mobile switching center (MSC) 146, a serving GPRS support node (SGSN) 148, and/or a gateway GPRS support node (GGSN) 150. While each of the foregoing elements are depic ted as part of the core network 106, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0061 ] The RNC 142a in the RAN 103 may be connected to the MSC 146 in the core network 106 via an IuCS interface. The MSC 146 may be connected to the MGW 144. The MSC 146 and the MGW 144 may provide the WTRUs 102a, 102b, 102c with access to circuit- switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices.

[0062] The RNC 142a in the RAN 103 may also be connected to the SGSN 148 in the core network 106 via an luPS interface. The SGSN 148 may be connected to the GGSN 150. The SGSN 148 and the GGSN 150 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 1 10, to facilitate communications between and the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0063] As noted above, the core network 106 may also be connected to the networks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers.

[0064] FIG. ID is a sy stem diagram of the RAN 104 and the core network 107. As noted abo ve, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 1 16. The RAN 104 may also be in

communication with the core network 107.

[0065] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 1 16. The eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a. [0066] Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell

(not depicted) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink and/or downlink, and the like. The eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 interface,

[0067] The core network 107 may include a mobility management gateway (MME) 162, a serving gateway 164, and a packet data network (PD ) gateway 166, While each of the foregoing elements are depicted as part of the core network 107, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0Θ68] The MME 162 may be connected to each of the eNode-Bs 160a, 160b, 160c in the

RAN 104 via an S 1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer

activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may also provide a control plane function for switching between the RAN 104 and other RANs (not depicted) that employ other radio technologies, such as GSM or WCDMA.

[006.9] The serving gateway 164 may be connected to each of the eNode-Bs 160a, 160b,

160c in the RAN 104 via the S 1 interface. The serving gateway 164 may generally route and forward user data packets to/from the WTRUs 102a, 102b, 102c. The serving gateway 164 may also perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when downlink data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like,

[0070] The serving gateway 164 may also be connected to the PDN gateway 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 1 10, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices,

[0071] The core network 107 may facilitate communications with other networks. For example, the core network 107 may provide the WTRUs 102a, 102b, 102c with access to circuit- switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the core network 107 may include, or may communicate with, an TP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the core network 107 and the PSTN 108. In addition, the core network 107 may provide the WTRUs 102a, 102b, 102c with access to the networks 12, which may include other wired or wireless networks that are owned and/or operated by other service providers,

[0072] FIG. IE is a system diagram of the RAN 105 and the core network 109. The

RAN 105 may be an access service network (ASN) that employs IEEE 802.16 radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 1 17. As will be further discussed below, the communication links between the different functional entities of the WTRUs 102a, 102b, 102c, the RAN 105, and the core network 109 may be defined as reference points.

[0073] Referring to FIG. IE, the RAN 105 may include base stations 180a, 180b, 180c, and an ASN gateway 182, though it will be appreciated that the RAN 105 may include any number of base stations and ASN gateways. The base stations 180a, 180b, 180c may each be associated with a particular cell (not depicted) in the RAN 105 and may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 117. The base stations 180a, 180b, 180c may implement MIMO technology. Thus, the base station 180a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a. The base stations 180a, 1 80b, 180c may also provide mobility management functions, such as handof triggering, tunnel establishment, radio resource management, traffic classification, quality of service (QoS) policy enforcement, and the like. The ASN gateway 182. may serve as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to the core network 109, and the like.

[0874] The air interface 1 17 between the WTRUs 102a, 102b, 102c and the RAN 105 may be defined as an Rl reference point that implements the IEEE 802.16 specification. In addition, each of the WTRUs 102a, 102b, 102c may establish a logical interface (not depicted) with the core network 109. The logical interface between the WTRUs 102a, 102b, 102c and the core network 109 may be defined as an R2 reference point, which may be used for

authentication, authorization, IP host configuration management, and/or mobility management.

[0075] The communication link between each of the base stations 180a, 180b, 180c may be defined as an R8 reference point that includes protocols for facilitating WTRU handovers and the transfer of data between base stations. The communication link between the base stations 180a, 180b, 1 80c and the ASN gateway 1 82. may be defined as an R6 reference point. The R6 reference point may include protocols for facilitating mobility management based on mobility events associated with each of the WTRUs 102a, 102b, 102c.

[0076] Referring to FIG. IE, the RAN 105 may be connected to the core network 109.

The communication link between the RAN 105 and the core network 109 may defined as an R3 reference point that includes protocols for facilitating data transfer and mobility management capabilities, for example. The core network 109 may include a mobile IP home agent (MIP-HA) 184, an authentication, authorization, accounting (AAA) server 186, and a gateway 188. While each of the foregoing elements are depicted as part of the core network 109, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0077] The MIP-HA may be responsible for IP address management, and may enable the

WTRUs 102a, 102b, 102c to roam between different ASNs and/or different core networks. The MIP-HA 184 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The AAA server 186 may be responsible for user authentication and for supporting user services. The gateway 188 may facilitate interworking with other networks. For example, the gateway 188 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. In addition, the gateway 188 may provide the WTRUs 102a, 102b, 102c with access to the networks 112, which may include other wired or wireless networks that are owned and/or operated by other service providers.

[0078] Although not depicted in FIG, IE, it will be appreciated that the RAN 105 may be connected to other ASNs and the core network 109 may be connected to other core networks. The communication link between the RAN 105 the other ASNs may be defined as an R4 reference point, which may include protocols for coordinating the mobility of the WTRUs 102a, 102b, 102c between the RAN 105 and the other ASNs. The communication link between the core network 109 and the other core networks may be defined as an R5 reference, which may include protocols for facilitating interworking between home core networks and visited core networks.

[0079] A multimedia system may adjust audio playback characteristics, for example, volume and "stereo image" based on factors such as, e.g., the listener's age, gender, listener's location with respect to the sound source. The multimedia system may adjust the audio characteristics based on environmental characteristics, for example, ambient and/ or background noise to deliver an enhanced experience.

[0080] The methods, systems and instrumentalities described herein may be applied to the audio playback module in wireless iransmit/reeeive units (WTRUs), including for example consumer and mobile phones, other mobile multimedia devices (e.g., tablets, game consoles). [0081] Sound is a sequence of waves of pressure that propagates through compressible media such as air or water. For humans, hearing may be limited to frequencies within about 20Hz and 20 kHz. Noise may be an unwanted sound that may obscure a wanted sound signal.

[0Θ82] Hearing may be influenced by many factors, for example, age of the listener. A healthy young person may hear from sound waves in 20 Hz to 20 kHz range. Frequencies in the range of 500Hz - 2kHz may be most distinguishable by human auditory system. This range may correspond to a range of frequencies in human speech.

[0083] A middle aged adult may hear frequencies up to 12-14 kHz. In psyehoacoustics, threshold in quiet may represent the lowest sound level that may be heard at a given frequency in quiet conditions. FIG. 2 illustrates an exemplary plot of threshold of quiet as a function of frequency with age as a parameter, for example. As illustrated in FIG. 2, the frequency range may decrease with person's age.

[0084] Furthermore, hearing for men may worsen more quickly than for women. FIG. 3 illustrates an exemplary threshold of quiet for various ages and gender. As illustrated in FIG. 3, a 60 year old woman, for example, may have better hearing than a 60 year old man and even better than a 50 year old man. Volume level may be adjusted based on both user's age and gender,

[0085] Sound localization may refer to a listener's ability to identify, e.g., the location (or origin) of a detected sound in direction and distance. Localization may be described in terms of three-dimensional position, including, for example, the azimuth or horizontal angle, the elevation or vertical angle, and the distance {e.g., for static sounds) or the velocity {e.g., for moving sounds). The differences in timbre, intensity , and/or spectral qualities of the sound between the two ears may be termed as "interaurai cues".

[0086] As illustrated by example in FIG. 4, these interaurai cues may help the brain to orient, and to determine the orientation of sound. A sound emanating from a source 402 travels as sound waves 404. The sound waves 404 are heard by a listener 406. The sound waves 404 do not arrive at the ears of the listener 406 simultaneously. As illustrated, the distance the sound waves 404 must travel is farther for the left ear of the listener 406 than they are for the right ear, and the listener may subconsciously perceive this difference and locate the source 402.

[0087] Spatial auditory perception may be attributed to a plurality of parameters (e.g., cues) indicating how humans may localize sound in the horizontal plane. The parameters may include, for example, interaurai level difference (ILD), interaurai time difference (ITD), and/or interaurai coherence (IC). [0088] As illustrated in FIG. 5, direct (or first-arrival) sound wave fronts from a distant source (not depicted) may arrive at a human 506. A direct sound wave may impinge on the left ear of the human 506. The sound wave received by the right ear may be diffracted around the head of the human 506, with associated factors, for example, time delay and/or level attenuation. Effects from these factors may result in the interaural time difference and/or interaural level difference parameters. If sound is perceived by the human 506 in a reverberant environment, reflected sound may indirectly impinge on both ears, giving rise to the interaural coherence parameter.

[0089] MPEG spatial audio coding (SAC) or MPEG surround standard may use various techniques to capture a spatial image of a multi-channel audio signal by synthesizing the signal int o a compact set of parameters. As illustrated by example in FIG. 6, these parameters may be used to encode, transmit and reproduce the audio signals. An audio signal coded using the SAC may have better compression than, for example, encoding each of the audio channels separately. For example, SAC may compress a 5.1 channel audio signal as stereo with side information containing spatial parameters (cues). The side information may be used to reconstruct the 5.1 channel audio signal from the decoded stereo. The spatial parameters extracted from a multichannel audio (e.g., hierarchically) at an encoder may, for example, include channel level differences (CLDs), interchannel correlation (ICC), channel prediction coefficient (CPC), prediction error or residual. CLDs may represent the level differences between pairs of audio signals within a tree. ICC may represent coherence between the pairs of audio signals within a tree. CPC may represent the ability of the encoder to predict an audio signal from others. The prediction error or residual may be the error in the parametric modeling process. The prediction error may be relative to the original audio signal. The spatial parameters may capture aspects of the perceptual cues as described herein.

[0090] Mobile computing devices, e.g., smartphones, tablets may be equipped with stereo speakers. As illustrated by example in FIG. 7, a device 702 may provide audio information as sound 704. The location of the user 706 with respect to the device 702 may affect the perceived loudness and spatial image of the audio information. For example, as the user 706 increases the distance (d) between herself and the device 702, the audio level may get fainter.

[0091] FIG. 8 illustrates an exemplary distance-volume relationship. The distance- volume relationship may include a saturation region that may preserve natural decay of volume at very short distances (for example, between 12" and 18" as depicted).

[0092] FIG. 9 illustrates an exemplary automatic volume adjustment of a device 902

(e.g., a mobile device) playing sound 904 to a user 906, based on the distance from the device to the user. The distance of the user 906 from the device 902 may be estimated using input from one or more sensors in the device. The distance estimated may be mapped to a volume level for the sound 904. The volume may be used to control audio characteristics, e.g., speaker volume. The volume may be altered, e.g., made softer when the user 906 is relatively closer to the device 902 and louder when the user is farther away from the device.

[0093] in adaptive audio systems, the end users may control shape, and slope of the adaptation curve. The end users may manually control the volume. In FIG. 9, for example, the proposed volume adaptation logic may operate on a relative scale, and may be implemented in the volume control functions that solicit feedback from the user 906 to refine the volume knob.

[0Θ94] As noted with respect to FIGS. 2&3, human hearing varies with age and gender, particularly at certain frequencies. Computer vision techniques may be used to estimate a user's age based on the person's appearance, e.g., by analyzing photographs of the user. Various techniques may be used to classify the user's gender. A mobile computing device, for example a mobile phone, may be equipped with a sensor, for example a front facing camera. Using the information captured by the sensors on the device, age estimation and gender classification algorithms may be used to estimate the user's age and gender. Also, the user may enter age and/or gender information, or the device may have a user profile which specifies information about the user.

FIG. 10 illustrates an exemplary methods and/or systems to adjust volume on a computing device, for example, a mobile computing device, based on the user's age and/or gender. As illustrated in FIG.10, a sensor (e.g., light sensor, proximity sensor, camera) may send the information (e.g. sensor readings and/or image data) to a user activit (UAD) API. The UAD may be used to detect the user's face and distance. This information may be shared with the age estimation and gender classifier algorithm to estimate the age and the gender of the user. A distance-age-gender-volume level relationship may be used to map distance, age, and gender to a volume level. For example, the volume level v may be obtained from age , gender g, and distance d using the following expression (I):

v : + k ₂ f _! (d)/ ₂ (a, g), (I) where k, and 1¾ may be constants, f () may be a monotonically increasing function, and £>() may be a function reflecting gender, age and threshold of quiet at a central frequency (e.g., I kHz). The derived volume l evel may be provided as an input to the volume control of the speakers. [0096] FIG. 1 1 illustrates an exemplary system that may attenuate or intensify volume

(e.g., selectively) of a sound signal in different frequency bands. The selective attenuation or intensification of volume of the sound signal in different frequency bands may be used to reproduce ihe sound signal for a user of a particular age and, gender, ancl/'or the user's distance from the speaker characteristics.

[0097] The volume level, e.g., v _/ may be assigned to each frequency band . The parameter may be obtained from age a, gender g, and distance d, using the following expression (H): vi - ki + k ₂ f {d)f _Zii (a, g), (II) where k, and k ₂ , may be constants, f Q may be a monotonically increasing function, and ί¾;() may be a function reflecting relationship between gender, age and threshold of quiet at i-th frequency band.

[0098] FIG. 12 illustrates an exemplary system of an automatic audio equalization using the user's age and/or gender without addressing distance. The automatic audio equalization parameters «,· assigned to each frequency band i, may be obtained using the following expression (III): en = _Cl + k ₂ f (a, g), (III) where k, and k ₂ , may be constants, and () may be a function reflecting relationship between gender, age, and threshold of quiet at i-th frequency band.

[0099] FIG, 13 illustrates an exemplary adaptive streaming system. Adaptive streaming systems, such as http live streaming (HLS) and/or dynamic adaptive streaming over littp (DASH) may present several encoded streams to a streaming client device (UE), for example, a mobile computing device (e.g., a wireless transmit/receive unit (WTRU)) may be wirelessly connected to a long term evolution (LTE) network's base station (eNB).

[0100] Through the base station, and a gateway node, the device may send a request for an audio bitstream from a server. The device may select an encoded stream (e.g., a matching encoded stream), and play the matched stream to the user. The device may request the audio bitstream based on the user's age and/or the user's gender. The server may host a plurality of copies of an audio signal at different frequency ranges, for example, (20Hz, 20kHz), (20 Hz, 14 kHz), (20 Hz, 12 kHz), (20 Hz, 8kHz), and the like. The server may include encodings that may be tailored to shapes of thresholds in quiet for various gender and/or rate categories of users.

[0181] FIG.14 illustrates an exemplary WTRU 1402 wherein the user's age and/or gender may be estimated. The estimated age and/or gender may be used to determine the auditory frequency range (e.g., approximate auditory frequency range) of the user using an age- gender-frequency range relationship. Based on the selected frequency range, a suitable bitstream may be requested from the server. Audio signals with lower frequency range may require fewer bits to encode. Such an approach may adapt the frequency range to the user's age and gender.

[0182] FIG.15 illustrates an exemplary adaptation of volume to an ambient noise level by a WTRU 1502. Ambient noise may mask the audio signal played back on some computing device, e.g., the mobile phone's speakers. Such a noise may render the audio signal (or parts thereof) inaudible to the user. This phenomenon may be referred to as simultaneous masking. The masking signal (e.g., a noise signal) may have a higher energy level compared to the signal being masked (e.g., an audio signal). By amplifying the audio signal (or parts thereof), it may be rendered audible.

[0103] The microphone in WTRU 1502. may be used to measure (e.g., periodically) the ambient noise. The noise energy may be measured over a period of time. The volume level may be chosen such that the resulting audio signal energy may exceed the ambient noise level. To prevent sudden spikes in volume, volume adaptation may be performed. For example, the volume adaptation may be performed gradually. The volume adaptation may be performed when a noise signal may persist over a longer period of time.

[0184] As illustrated by example in FIG, 16, the volume level may be adapted based on the user's age, gender, distance, and noise level. These parameters may be mapped to audio characteristics, e.g., volume level using a five-dimensional relationship. For example, a rnonotonicaily increasing function may be used for this volume, using the following expression (IV):

v = ^ + k ₂ f (d)f ₂ (a, g)f ₃ (n), (IV) where k, and k ₂ , may be constants, f;() and f$0 may be rnonotonicaily increasing functions of distance and noise level, and where f ₂ 0 may be a function reflecting gender, age and threshold of quiet at a central frequency (e.g. lkHz),

[0105] As illustrated in FIG. 17, a group of users may be in proximity of a device, for example watching a video on a cell phone. In such a scenario, it may be desirable to adjust the volume level to accommodate the users. The users in front of the device may be detected using UAD. Their users' age, gender and/or distance may be estimated. The ambient noise level may be measured. The volume level corresponding to each user may be obtained by using a distance- age-gender-noise-volume relationship. The final volume level may be derived by using the maximum, median and/or the average of the volume le vels.

[0106] If an adaptive streaming system is used for streaming audio, the frequency range of a user may be estimated. An audio bitstream corresponding to the largest frequency range in the group may be requested. An audio bitstream corresponding to the median and/or the mean aural frequency range of the group may be requested,

[0107] Multi-channel audio content may be authored assuming that the listener is located at the "sweet spot" during playback. A listener located away from the sweet spot may suffer from lower perceptual quality. Audio content may be compressed using SAC. In such a case, the decoded audio may adapt to the listener's location, as illustrated by example in FIG. 18, Such an adaptation may include, for example, using UAD framework to determine the listener location, and/or using information about listener location in the decoder to adjust the spatial parameters. The parameters may include, for example, interaural level difference (ILD), interaural time difference ( f D), and/or interaural coherence (IC), as illustrated by example in FIG. 5. These parameters may correspond to the actual listener location, and may result in improved perceived audio quality.

[Θ188] A UAD framework may receive one or more inputs from available sensors in a system. For example, if a camera is available, the listener location may be determined by using computer vision techniques. A plurality of listeners and their locations may be identified using camera inputs. In mobile computing devices, a combination of sensors (e.g., proximity, accelerometer, gyroscope) may be used to determine location.

[0109] As illustrated in FIG. I9A, an exemplary audio system comprises a speaker 1903 emitting soundwaves 1904 that may be heard by a user 1906 in mono sound.

[0110] As illustrated in FIG. 19B, an exemplary audio system with a number of channels comprises a plurality of speakers 1903 emitting soundwaves 1904 that may be heard by a user 1906 in stereo sound. Stereophonic sound (e.g., "stereo") method of sound reproduction may create an illusion of directionality and audible perspective. This illusion may be achieved by using two or more independent audio channels through a configuration of, for example two or more loudspeakers in such a way as to create the impression of sound heard from various directions. The "stereo image" or "spatial image" of a multi-channel audio signal may concern spatial locations of the sound source(s). The location may be spatial physically and/or in depth. [0111] As illustrated in FIG, 19C, an exemplary audio system with a number of channels comprises a plurality of speakers 1903 and a relatively lower frequency speaker 1905 emitting soundwaves that may be heard by a user 1906, such as in Dolby 5.1 surround sound. Surround sound may be used for enriching the sound reproduction quality of an audio source with additional audio channels from speakers that may surround the listener {e.g., surround channels), providing sound from a 360° radius in the horizontal plane. Surround sound may be

characterized by a listener's location or "sweet spot" where the aitdio effect may enhance the perception of sound spa.tiaiiza.tion by exploiting sound localization, as illustrated by example in FIG. 20.

[ill 12] As illustrated by example in FIG. 21, an audio playback system, for example, a home theater, may have a video source 2.101 {e.g., a. television, projector, monitor and/or a tablet) and speakers 2103. The location of the speakers 2103 may be fixed, such that there is an ideal location for a listener 2106 according to how the playback system may have been designed. In practice, however, a listener 2106 may change positions or walk away during playback of the audio-visual content, causing the listener to be in a non-ideal location. This relative movement of the listener 2106 with respect to the ideal location may affect the spatial image of audio perceived by the listener,

[0113] As illustrated by example in FIG. 22, a plurality of sensors (not depicted) may be integrated in reproduction setup {i.e., at feast one of the video source 2101 and speakers 2103), including for example cameras, infrared sensors, ultrasound sensors, microphones, and/or other sensors to detect actual location of the listener 2.106. Based on the information from the sensors, the audio signal rendering may be adjusted to deliver the audio content to the listener's 2106 location, and may adjust to improve the audio content at the new location, or to deliver the same audio characteristics as at the listener's previous location. Location and'Or orientation beacons may be used to improve accuracy of detection and estimation of listener location. The beacons may be embedded in devices that may be a part of the reproduction setup {e.g., headphones, and'Or 3D glasses, and/or in active headsets). The beacons may be implemented as part of an application in mobile computing devices (such as smartphones and tablets). Mobile computing devices may be used to extend the range of detection and location estimation beyond the line of sight.

[0114] Adaptation of audio signals may be used to compensate for movements of the listener. As illustrated in the exemplary system of FIG. 23, at a first time, a left speaker 2303/ and a right speaker 2303r, each producing sound waves 2304, may have a user 2306 equidistantly disposed between them. At this time, the volume settings (VL and VR) for the left speaker 2303/ and the right speaker 2303r, respectively, may be equal.

[0115] At a second time, the user 2306 has moved to a new position, closer to the left speaker 2303/. If the user 2306 is to perceive equal volumes, the system must adaptively adjust the speaker volumes. The system may determine the new position of the user 2306 and calculate a volume adjustment factor (a) to be applied to the volume settings (VL and V _R ). The practical effect of this is that the system may adapt the stereo balance of the speakers by increasing the relative volume of the right speaker 2303r, and reducing the relative volume of the left speaker 2303/.

[0116] FIG. 24 illustrates an exemplary system for adapting to user's presence and location with respect to a UE. Adaptive streaming systems, such as HLS or DASH may present several encoded streams to the streaming client devices. In this case, the client may be responsible for matching and selecting an encoded stream and playing it to the user. Such system may be used for implementing adaptations to user location for improved quality of audio playback. In this case, the UAD framework may be used by the WTRU to request an audio representation that may be better suited for this situation. For example, if the system is using non-parametric stereo or multichannel coding technique (e.g. AC 3 and/or AAC codec), it may include several encoded renderings of audio to speakers. These renderings may be produced to enhance the user experience in a particular point in space relative to speakers. The client device may detect location of the user and request streaming of a representation corresponding to a target location (e.g., the closest target) to the user.

[0117] Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer- readable media include electronic signals (transmitted o ver wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a device, example, WTRU, WTRU, terminal, base station, RNC, or a host computer.