CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of EP Application No. 22305889.2, filed June 17, 2022 the contents of which are incorporated herein by reference.

BACKGROUND

[0002] There is a need for new and improved approaches as they relate to extended reality, such as augmented reality, virtual reality, mixed reality, and related realities.

SUMMARY

[0003] Extended reality may be created using one or more methods, devices, and/or systems. In an extended reality, there is a need to implement novel and improved features, such as features related to lighting and/or evolution of the lighting over time. In some cases, these features may be implemented by using lighting information. In some cases, the lighting information may adhere to a standardized format. In some cases, the lighting information may build off of a standardized format.

BRIEF DESCRIPTION OF THE DRAWINGS

[0004] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, wherein like reference numerals in the figures indicate like elements, and wherein:

[0005] FIG. 1A is a system diagram illustrating an example communications system in which one or more disclosed embodiments may be implemented;

[0006] FIG. 1 B is a system diagram illustrating an example wireless transmit/receive unit (WTRU) that may be used within the communications system illustrated in FIG. 1A according to an embodiment;

[0007] FIG. 1C is a system diagram illustrating an example radio access network (RAN) and an example core network (ON) that may be used within the communications system illustrated in FIG. 1A according to an embodiment;

[0008] FIG. 1D is a system diagram illustrating a further example RAN and a further example ON that may be used within the communications system illustrated in FIG. 1A according to an embodiment;

[0009] FIG. 1 E illustrates an example hardware configuration for operating an XR application as described herein;

[0010] FIG. 1 F illustrates an example hardware configuration for operating an XR application as described herein; [001 1] FIG. 2 illustrate an example process of an XR application;

[0012] FIG. 3 illustrates an example of KHRJights _punctual gltf extension;

[0013] FIG. 4 illustrates an example of EXTJightsJmageJjased gltf extension;

[0014] FIG. 5 illustrates an example of MPEG_media element for time-evolving lights properties;

[0015] FIG. 6 illustrates an example of a JSON file to control GLTF light sources;

[0016] FIG. 7 illustrates an example of MPEGJightingJiming extension at scene level;

[0017] FIG. 8 illustrates an example of MPEGJightingJiming extension at light component level;

[0018] FIG. 9 illustrates an example of video texture for EXTJightsJmage_based lights;

[0019] FIG. 10 illustrates an example of MPEGJightingJiming extension at light node level; and

[0020] FIG. 11 illustrates an example process of an XR application.

DETAILED DESCRIPTION

[0021] The present principles will be described more fully hereinafter with reference to the accompanying figures, in which examples of the present principles are shown. The present principles may, however, be embodied in many alternate forms and should not be construed as limited to the examples set forth herein. Accordingly, while the present principles are susceptible to various modifications and alternative forms, specific examples thereof are shown by way of examples in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the present principles to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present principles as defined by the claims.

[0022] The terminology used herein is for the purpose of describing particular examples only and is not intended to be limiting of the present principles. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising," "includes" and/or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Moreover, when an element is referred to as being "responsive" or "connected" to another element, it may be directly responsive or connected to the other element, or intervening elements may be present. In contrast, when an element is referred to as being "directly responsive" or "directly connected" to other element, there are no intervening elements present. As used herein the term "and/or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as"/".

[0023] It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the teachings of the present principles.

[0024] Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0025] Some examples are described with regard to block diagrams and operational flowcharts in which each block represents a circuit element, module, or portion of code which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in other implementations, the function(s) noted in the blocks may occur out of the order noted. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending on the functionality involved.

[0026] Reference herein to “in accordance with an example” or “in an example” means that a particular feature, structure, or characteristic described in connection with the example may be included in at least one implementation of the present principles. The appearances of the phrase in accordance with an example” or “in an example” in various places in the specification are not necessarily all referring to the same example, nor are separate or alternative examples necessarily mutually exclusive of other examples.

[0027] Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims. While not explicitly described, the present examples and variants may be employed in any combination or sub-combination.

[0028] FIG. 1A is a diagram illustrating an example communications system 100 in which one or more disclosed embodiments may be implemented. The communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), singlecarrier FDMA (SC-FDMA), zero-tail unique-word discrete Fourier transform Spread OFDM (ZT-UW-DFT-S- OFDM), unique word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC), and the like.

[0029] As shown in FIG. 1A, the communications system 100 may include wireless transmit/receive units (WTRUs) 102a, 102b, 102c, 102d, a radio access network (RAN) 104, a core network (ON) 106, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and/or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a station (STA), may be configured to transmit and/or receive wireless signals and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, television, projector, 3D display, 3D television, 3D projector, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and/or other wireless devices operating in an industrial and/or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and/or industrial wireless networks, and the like. Any of the WTRUs 102a, 102b, 102c and 102d may be interchangeably referred to as a UE.

[0030] The communications systems 100 may also include a base station 114a and/or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106, the Internet 110, and/or the other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a NodeB, an eNode B (eNB), a Home Node B, a Home eNode B, a next generation NodeB, such as a gNode B (gNB), a new radio (NR) NodeB, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and/or network elements.

[0031] The base station 114a may be part of the RAN 104, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, and the like. The base station 114a and/or the base station 114b may be configured to transmit and/or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage for a wireless service to a specific geographical area that may be relatively fixed or that may change over time. The cell may further be divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one for each sector of the cell. In an embodiment, the base station 114a may employ multiple-input multiple output (Ml MO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and/or receive signals in desired spatial directions.

[0032] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT). [0033] More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 104 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 116 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink (DL) Packet Access (HSDPA) and/or High-Speed Uplink (UL) Packet Access (HSUPA).

[0034] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A) and/or LTE-Advanced Pro (LTE-A Pro).

[0035] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR Radio Access , which may establish the air interface 116 using NR.

[0036] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for instance using dual connectivity (DC) principles. Thus, the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and/or transmissions sent to/from multiple types of base stations (e.g., an eNB and a gNB).

[0037] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like. [0038] The base station 114b in FIG. 1A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a roadway, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR etc.) to establish a picocell or femtocell. As shown in FIG. 1A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114b may not be required to access the Internet 110 via the CN 106. [0039] The RAN 104 may be in communication with the CN 106, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have varying quality of service (QoS) requirements, such as differing throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, and the like. The CN 106 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication. Although not shown in FIG. 1A, it will be appreciated that the RAN 104 and/or the CN 106 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 or a different RAT. For example, in addition to being connected to the RAN 104, which may be utilizing a NR radio technology, the CN 106 may also be in communication with another RAN (not shown) employing a GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0040] The CN 106 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and/or the other networks 112. The PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and/or the internet protocol (IP) in the TCP/IP internet protocol suite. The networks 112 may include wired and/or wireless communications networks owned and/or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 or a different RAT.

[0041] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). For example, the WTRU 102c shown in FIG. 1A may be configured to communicate with the base station 114a, which may employ a cellularbased radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0042] FIG. 1 B is a system diagram illustrating an example WTRU 102. As shown in FIG. 1 B, the WTRU 102 may include a processor 118, a transceiver 120, a transmit/receive element 122, a speaker/microphone 124, a keypad 126, a display/touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and/or other peripherals 138, among others. It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0043] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While FIG. 1 B depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0044] The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114a) over the air interface 116. For example, in one embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In an embodiment, the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit/receive element 122 may be configured to transmit and/or receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.

[0045] Although the transmit/receive element 122 is depicted in FIG. 1 B as a single element, the WTRU 102 may include any number of transmit/receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116. [0046] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11 , for example.

[0047] The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker/microphone 124, the keypad 126, and/or the display/touchpad 128. In addition, the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0048] The processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li- ion), etc.), solar cells, fuel cells, and the like.

[0049] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 116 from a base station (e.g., base stations 114a, 114b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.

[0050] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs and/or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a Virtual Reality and/or Augmented Reality (VR/AR) device, an activity tracker, and the like. The peripherals 138 may include one or more sensors. The sensors may be one or more of a gyroscope, an accelerometer, a hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, a humidity sensor and the like.

[0051] The WTRU 102 may include a full duplex radio for which transmission and reception of some or all of the signals (e.g., associated with particular subframes for both the UL (e.g., for transmission) and DL (e.g., for reception) may be concurrent and/or simultaneous. The full duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference via either hardware (e.g., a choke) or signal processing via a processor (e.g., a separate processor (not shown) or via processor 118). In an embodiment, the WTRU 102 may include a half-duplex radio for which transmission and reception of some or all of the signals (e.g., associated with particular subframes for either the UL (e.g., for transmission) or the DL (e.g., for reception)).

[0052] In a variation of the example of FIG. 1 B, there may be an architecture of an XR processing engine which may be configured to implement the methods described herein. WTRU 102 may be linked with other devices via their bus and/or via I/O interface. WTRU 102 may comprise one or more following elements that are linked together by a data and address bus: a microprocessor (or CPU), which is, for example, a DSP (or Digital Signal Processor); a ROM (or Read Only Memory); a RAM (or Random Access Memory); a storage interface; an I/O interface for reception of data to transmit, from an application; and a power supply (e.g., a battery). In one case, the power supply is external to the WTRU 102.

[0053] In each of mentioned memory, the word “register” used herein may correspond to area of small capacity (some bits) or to very large area (e.g., a whole program or large amount of received or decoded data). ROM comprises at least a program and parameters. The ROM may store algorithms and instructions to perform techniques in accordance with present principles. When switched on, the CPU uploads the program in the RAM and executes the corresponding instructions. The RAM comprises, in a register, the program executed by the CPU and uploaded after switch-on of the device, input data in a register, intermediate data in different states of the method in a register, and other variables used for the execution of the method in a register. WTRU 102 may be linked, for example via bus to a set of sensors and to a set of rendering devices. Sensors may be, for example, cameras, microphones, temperature sensors, Inertial Measurement Units, GPS, hygrometry sensors, I R or UV light sensors or wind sensors. Rendering devices may be, for example, displays, speakers, vibrators, heat, fan, and the like.

[0054] FIG. 1C is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0055] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and/or receive wireless signals from, the WTRU 102a.

[0056] Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and/or DL, and the like. As shown in FIG. 1C, the eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 interface.

[0057] The CN 106 shown in FIG. 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (PGW) 166. While the foregoing elements are depicted as part of the CN 106, it will be appreciated that any of these elements may be owned and/or operated by an entity other than the CN operator.

[0058] The MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and/or WCDMA.

[0059] The SGW 164 may be connected to each of the eNode Bs 160a, 160b, 160c in the RAN 104 via the S1 interface. The SGW 164 may generally route and forward user data packets to/from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like.

[0060] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0061] The ON 106 may facilitate communications with other networks. For example, the ON 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the ON 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the ON 106 and the PSTN 108. In addition, the ON 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 112, which may include other wired and/or wireless networks that are owned and/or operated by other service providers. [0062] Although the WTRU is described in FIGS. 1A-1 D as a wireless terminal, it is contemplated that in certain representative embodiments that such a terminal may use (e.g., temporarily or permanently) wired communication interfaces with the communication network.

[0063] In representative embodiments, the other network 112 may be a WLAN.

[0064] A WLAN in Infrastructure Basic Service Set (BSS) mode may have an Access Point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a Distribution System (DS) or another type of wired/wireless network that carries traffic in to and/or out of the BSS. Traffic to STAs that originates from outside the BSS may arrive through the AP and may be delivered to the STAs. Traffic originating from STAs to destinations outside the BSS may be sent to the AP to be delivered to respective destinations. Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP and the AP may deliver the traffic to the destination STA. The traffic between STAs within a BSS may be considered and/or referred to as peer-to-peer traffic. The peer-to- peer traffic may be sent between (e.g., directly between) the source and destination STAs with a direct link setup (DLS). In certain representative embodiments, the DLS may use an 802.11e DLS or an 802.11z tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and the STAs (e.g., all of the STAs) within or using the IBSS may communicate directly with each other. The IBSS mode of communication may sometimes be referred to herein as an “ad-hoc” mode of communication. [0065] When using the 802.11 ac infrastructure mode of operation or a similar mode of operations, the AP may transmit a beacon on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., 20 MHz wide bandwidth) or a dynamically set width. The primary channel may be the operating channel of the BSS and may be used by the STAs to establish a connection with the AP. In certain representative embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) may be implemented, for example in 802.11 systems. For CSMA/CA, the STAs (e.g., every STA), including the AP, may sense the primary channel. If the primary channel is sensed/detected and/or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

[0066] High Throughput (HT) STAs may use a 40 MHz wide channel for communication, for example, via a combination of the primary 20 MHz channel with an adjacent or nonadjacent 20 MHz channel to form a 40 MHz wide channel.

[0067] Very High Throughput (VHT) STAs may support 20MHz, 40 MHz, 80 MHz, and/or 160 MHz wide channels. The 40 MHz, and/or 80 MHz, channels may be formed by combining contiguous 20 MHz channels. A 160 MHz channel may be formed by combining 8 contiguous 20 MHz channels, or by combining two noncontiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, the data, after channel encoding, may be passed through a segment parser that may divide the data into two streams. Inverse Fast Fourier Transform (IFFT) processing, and time domain processing, may be done on each stream separately. The streams may be mapped on to the two 80 MHz channels, and the data may be transmitted by a transmitting STA. At the receiver of the receiving STA, the above described operation for the 80+80 configuration may be reversed, and the combined data may be sent to the Medium Access Control (MAC).

[0068] Sub 1 GHz modes of operation are supported by 802.11 af and 802.11 ah. The channel operating bandwidths, and carriers, are reduced in 802.11 af and 802.11 ah relative to those used in 802.11n, and 802.11 ac. 802.11 af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11 ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11 ah may support Meter Type Control/Machine- Type Communications (MTC), such as MTC devices in a macro coverage area. MTC devices may have certain capabilities, for example, limited capabilities including support for (e.g., only support for) certain and/or limited bandwidths. The MTC devices may include a battery with a battery life above a threshold (e.g., to maintain a very long battery life).

[0069] WLAN systems, which may support multiple channels, and channel bandwidths, such as 802.11 n, 802.11 ac, 802.11af, and 802.11 ah, include a channel which may be designated as the primary channel. The primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and/or limited by a STA, from among all STAs in operating in a BSS, which supports the smallest bandwidth operating mode. In the example of 802.11 ah, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support (e.g., only support) a 1 MHz mode, even if the AP, and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and/or other channel bandwidth operating modes. Carrier sensing and/or Network Allocation Vector (NAV) settings may depend on the status of the primary channel. If the primary channel is busy, for example, due to a STA (which supports only a 1 MHz operating mode) transmitting to the AP, all available frequency bands may be considered busy even though a majority of the available frequency bands remains idle.

[0070] In the United States, the available frequency bands, which may be used by 802.11 ah, are from 902 MHz to 928 MHz. In Korea, the available frequency bands are from 917.5 MHz to 923.5 MHz. In Japan, the available frequency bands are from 916.5 MHz to 927.5 MHz. The total bandwidth available for 802.11 ah is 6 MHz to 26 MHz depending on the country code.

[0071] FIG. 1 D is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment. As noted above, the RAN 104 may employ an NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0072] The RAN 104 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 104 may include any number of gN Bs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, gNBs 180a, 108b may utilize beamforming to transmit signals to and/or receive signals from the gNBs 180a, 180b, 180c. Thus, the gNB 180a, for example, may use multiple antennas to transmit wireless signals to, and/or receive wireless signals from, the WTRU 102a. In an embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum while the remaining component carriers may be on licensed spectrum. In an embodiment, the gNBs 180a, 180b, 180c may implement Coordinated Multi-Point (CoMP) technology. For example, WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and/or gNB 180c).

[0073] The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using transmissions associated with a scalable numerology. For example, the OFDM symbol spacing and/or OFDM subcarrier spacing may vary for different transmissions, different cells, and/or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using subframe or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing a varying number of OFDM symbols and/or lasting varying lengths of absolute time).

[0074] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and/or a non-standalone configuration. In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c without also accessing other RANs (e.g., such as eNode-Bs 160a, 160b, 160c). In the standalone configuration, WTRUs 102a, 102b, 102c may utilize one or more of gNBs 180a, 180b, 180c as a mobility anchor point. In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration WTRUs 102a, 102b, 102c may communicate with/connect to gNBs 180a, 180b, 180c while also communicating with/connecting to another RAN such as eNode-Bs 160a, 160b, 160c. For example, WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously. In the non- standalone configuration, eNode-Bs 160a, 160b, 160c may serve as a mobility anchor for WTRUs 102a, 102b, 102c and gNBs 180a, 180b, 180c may provide additional coverage and/or throughput for servicing WTRUs 102a, 102b, 102c.

[0075] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and/or DL, support of network slicing, DC, interworking between NR and E-UTRA, routing of user plane data towards User Plane Function (UPF) 184a, 184b, routing of control plane information towards Access and Mobility Management Function (AMF) 182a, 182b and the like. As shown in FIG. 1 D, the gNBs 180a, 180b, 180c may communicate with one another over an Xn interface.

[0076] The CN 106 shown in FIG. 1D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While the foregoing elements are depicted as part of the CN 106, it will be appreciated that any of these elements may be owned and/or operated by an entity other than the CN operator.

[0077] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 104 via an N2 interface and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, support for network slicing (e.g., handling of different protocol data unit (PDU) sessions with different requirements), selecting a particular SMF 183a, 183b, management of the registration area, termination of non-access stratum (NAS) signaling, mobility management, and the like. Network slicing may be used by the AMF 182a, 182b in order to customize CN support for WTRUs 102a, 102b, 102c based on the types of services being utilized WTRUs 102a, 102b, 102c. For example, different network slices may be established for different use cases such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for MTC access, and the like. The AMF 182a, 182b may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and/or non-3GPP access technologies such as WiFi.

[0078] The SMF 183a, 183b may be connected to an AMF 182a, 182b in the CN 106 via an N11 interface. The SMF 183a, 183b may also be connected to a UPF 184a, 184b in the CN 106 via an N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b and configure the routing of traffic through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions, such as managing and allocating UE IP address, managing PDU sessions, controlling policy enforcement and QoS, providing DL data notifications, and the like. A PDU session type may be IP-based, non-IP based, Ethernet-based, and the like.

[0079] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 104 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering DL packets, providing mobility anchoring, and the like.

[0080] The CN 106 may facilitate communications with other networks. For example, the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 112, which may include other wired and/or wireless networks that are owned and/or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to a local DN 185a, 185b through the UPF 184a, 184b via the N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b.

[0081] In view of FIGs. 1A-1 D, and the corresponding description of FIGs. 1A-1 D, one or more, or all, of the functions described herein with regard to one or more of: WTRU 102a-d, Base Station 114a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and/or any other device(s) described herein, may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more, or all, of the functions described herein. For example, the emulation devices may be used to test other devices and/or to simulate network and/or WTRU functions.

[0082] The emulation devices may be designed to implement one or more tests of other devices in a lab environment and/or in an operator network environment. For example, the one or more emulation devices may perform the one or more, or all, functions while being fully or partially implemented and/or deployed as part of a wired and/or wireless communication network in order to test other devices within the communication network. The one or more emulation devices may perform the one or more, or all, functions while being temporarily implemented/deployed as part of a wired and/or wireless communication network. The emulation device may be directly coupled to another device for purposes of testing and/or performing testing using over-the-air wireless communications.

[0083] The one or more emulation devices may perform the one or more, including all, functions while not being implemented/deployed as part of a wired and/or wireless communication network. For example, the emulation devices may be utilized in a testing scenario in a testing laboratory and/or a non-deployed (e.g., testing) wired and/or wireless communication network in order to implement testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and/or wireless communications via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and/or receive data.

[0084] Extended reality (XR), such as augmented reality (AR), virtual reality (VR), mixed reality (MR), and related realities may be interchangeable as discussed herein. In implementing XR technology, such as with an XR application running on a device (e.g., a WTRU that has or is connected to a head mounted display), scene description may be used as instructions of the application that may combine explicit and easy-to-parse description(s) of a scene structure and some binary representations of media content.

[0085] FIG. 1 E illustrates an example hardware configuration for operating an XR application as described herein. The WTRU 102 of FIG. 1 E is intended to be a variation and/or an augmentation of the WTRU 102 as presented in FIG. 1 B and/or 1 F. As shown, WTRU 102 may comprise (e.g., in addition to elements described herein with regard to other figures) an encoder 141, storage 142 (e.g., memory and/or data storage medium such as SSD, NVME, PCIe storage, etc.; e.g., in some cases storage 142 may be the same as other storage associated with WTRU 102 as described herein), and/or an input/output (I/O) 145a (e.g., this I/O illustrates that the encoder may input and out data externally; e.g., in some cases, the I/O is not needed and this data exchange may be internal to the device, like to and from storage 142 through the processing of an XR application, or in some cases the I/O is required and may be external from the device like received or accessed via the internet or a remote source, or in some cases the I/O is required in part operating partially one or more of the aforementioned cases). The I/O 145a may comprise any interface means described herein, such as a wireless transceiver, a wired transceiver, and/or the like. WTRU 102 may be cable of encoding a stream that may be subsequently decoded to be displayed.

[0086] FIG. 1 F illustrates example hardware configuration for operating an XR application as described herein. The WTRU 102 of FIG. 1 F is intended to be a variation and/or an augmentation of the WTRU 102 as presented in FIG. 1 B and/or 1 E. As shown, WTRU 102 may comprise (e.g., in addition to elements described herein with regard to other figures) an decoder 144, storage 143 (e.g., memory and/or data storage medium, which in some cases may be the same as other storage associated with WTRU 102 as described herein), input/output (I/O) 145b (e.g., this I/O illustrates that the decoder may input and out data externally; e.g., in some cases, the I/O is not needed and this data exchange may be internal to the device, like to and from storage 143 through the processing of an XR application, or in some cases the I/O is required and may be external from the device like received or accessed via the internet or a remote source, or in some cases the I/O is required in part operating partially one or more of the aforementioned cases), Head Mounted Display (HMD) 146a that is a part of the WTRU 102, an HMD 146b that is connected via a wire 147 to the WTRU 102, and/or a HMD that is connected to the WTRU 102 wirelessly (e.g., antenna/transceiver 148 to antenna/transceiver of 149). The I/O 145a may comprise any interface means described herein, such as a wireless transceiver, a wired transceiver, and/or the like. WTRU 102 may be cable of decoding a stream that was previously encoded in order to be displayed on an HMD 146 for a user. [0087] In one scenario a first WTRU may be responsible for encoding and may be separate from (e.g., remote over a wide area network or a local area network; e.g., local but physically separate using a direct communication means as described herein) a second WTRU that performs the decoding. In one scenario, one WTRU performs the encoding and decoding.

[0088] In one scenario, a WTRU 102 (e.g., of FIG. 1 B, 1 E, and/or 1 F), may use a GPU instead of, or in addition to, an encoder/decoder arrangement. The GPU may retrieve/receive scene information (e.g., locally or remotely) and process the scene information to generate display information. The display information may then be sent to the display (e.g., part of the WTRU, or external to the WTRU). For example, a WTRU may use a decoder to decode encoded elements (e.g., scene description such as JPG images, video, etc.), and then feed that decoded information into the GPU.

[0089] FIG. 2 illustrates an example process of an XR application. The XR application may be implemented by one or more WTRUs, as described herein. Initially, at 201 scene data may be determined. In one instance, scene data may be retrieved/received locally or remotely (e.g., from storage, or via the internet). At 202, the scene data is encoded to generate a bit stream. At 203, the bit stream is decoded into display information. At 204, the display information is displayed. The details of this process are further described herein throughout this description.

[0090] For example, a sequence of scenes (e.g., such as 3D scenes and/or XR scenes), may be provided to an encoder. The encoder may take one scene or a sequence of scenes as input and provide a bit stream representative of the input. The bit stream may be stored in a memory and/or on an electronic data medium and may be transmitted over a network or locally to a display. The bit stream representative of a sequence of scenes may be read from a memory and/or received from a network by a decoder. Decoder is inputted the bit stream and provides a sequence of scenes, for instance in a point cloud format.

[0091] The encoder may comprise several circuits implementing several steps. In a first step, the encoder may project each scene (e.g., from the scene data) onto at least one picture (e.g., 2D picture). Scene projection is any method of mapping at least three-dimensional points to a smaller dimension, such as a two-dimensional plane. For example, in methods for displaying graphical data based on planar (e.g., pixel information from several bit planes) two-dimensional media, the use of this type of projection may be used in computer graphics, engineering, and/or drafting. A projection circuit may provide at least one two-dimensional frame for a scene of sequence. A frame may comprise color information and depth information representative of the scene projected onto the frame. In a variation, some combination of one or more of the following may be encoded in separate frames: color information, light information, and/or depth information.

[0092] Metadata may be used and updated by the projection circuit. Metadata may comprise information about the projection operation (e.g., projection parameters) and/or about the way other information is organized within frames. [0093] In some instances, the metadata and the video data (e.g., bit stream from the encoding circuit that encodes a sequence of frames as a video, where pictures of a scene and/or a sequence of pictures of the scene may be encoded in a stream by a video encoder) may be encapsulated in a data stream by a data encapsulation circuit.

[0094] In one case, an encoder may be compliant with an industry standard encoder such as: JPEG, specification ISO/CEI 10918-1 UIT-T Recommendation T.81; AVC, also named MPEG-4 AVC or h264. Specified in both UIT-T H.264 and ISO/CEI MPEG-4 Part 10 (ISO/CEI 14496-10), HEVC (T recommendation, H series, h265); 3D-HEVC (an extension of HVEC, T recommendation, H series, h265); VP9; and/or AV1 (AOMedia Video 1).

[0095] The data stream may be stored in a storage (e.g., memory or other means as described herein) that is accessible, for example through a network, by a decoder.

[0096] The decoder may comprise different circuits implementing different steps of the decoding. The decoder may take a data/bit stream generated by an encoder as an input and provide a sequence of scenes (e.g., display information) to be rendered and displayed by a volumetric video display device, such as a WTRU that has or is connected to a Head-Mounted Device (HMD). The decoder may obtain the stream from a source. For example, the source may be one or more of the following: a local memory, such as a video memory or a RAM (or Random-Access Memory), a flash memory, a ROM (or Read Only Memory), a hard disk; a storage interface, such as an interface with a mass storage, a RAM, a flash memory, a ROM, an optical disc or a magnetic support; a communication interface, such as a wireline interface (for example a bus interface, a wide area network interface, a local area network interface) or a wireless interface (such as a IEEE 802.11 interface or a Bluetooth® interface); and a user interface such as a Graphical User Interface enabling a user to input data that is combined with other data to generate the display information.

[0097] The decoder may comprise a circuit for extracting data encoded in the data/bit stream. The circuit may take a data/bit stream as input and provide metadata corresponding to metadata encoded in the stream and a two-dimensional video. The video may be decoded by a video decoder which provides a sequence of frames. The decoded sequence of frames may comprise information (e.g., color and depth information, and/or any information disclosed herein). In a variant, the video decoder may provide two sequences of frames, one comprising color information, the other comprising depth information. In a variant, there may be as many sequences of frames as there are information. A circuit may use metadata to unproject information from decoded frames to provide a sequence of scenes. Sequence of scenes may have a possible loss of precision related to the encoding as a 2D video and to the video compression.

[0098] Hardware, such as an HMD connected to a WTRU, may display the output of the decoder such that a user can experience the extended reality.

[0099] Generally, XR is a technology enabling interactive experiences for a user(s) where the real-world environment and/or video content is enhanced by virtual content, which may be defined across multiple sensory modalities, including visual, auditory, haptic, etc. During runtime of the XR application, the virtual content (e.g., 3D content or audio/video file for example) may be rendered in real-time in a way that is consistent with the user context (e.g., environment, point of view, device, etc.). Scene graphs (e.g., such as the one proposed by Khronos / gITF and its extensions defined in MPEG Scene Description format or Apple / USDZ for instance) may be a possible way to represent the content to be rendered. They combine a declarative description of the scene structure linking real-environment objects and virtual objects on one hand, and binary representations of the virtual content on the other hand. Scene description frameworks ensure that the timed media and the corresponding relevant virtual content are available at any time during the rendering of the application. Scene descriptions may also carry data at scene level describing how a user may interact with the scene objects at runtime for immersive XR experiences.

[0100] Various XR applications may apply to different context and real or virtual environments. For example, in an industrial XR application, a virtual 3D content item (e.g., a piece A of an engine) is displayed when a reference object (piece B of an engine) is detected in the real environment by a camera rigged on a head mounted display device. The 3D content item is positioned in the real-world with a position and a scale defined relatively to the detected reference object.

[0101] For example, in an XR application for interior design, a 3D model of a furniture is displayed when a given image from the catalog is detected in the input camera view. The 3D content is positioned in the real- world with a position and scale defined relatively to the detected reference image. In another application, some audio file might start playing when the user enters an area close to a church (e.g., being real or virtually rendered in the extended real environment). In another example, an ad jingle file may be played when the user sees a can of a given soda in the real environment. In an outdoor gaming application, various virtual characters may appear, depending on the semantics of the scenery which is observed by the user. For example, bird characters are suitable for trees, so if the sensors of the XR device detect real objects described by a semantic label ‘tree’, birds may be added flying around the trees. In a companion application implemented by smart glasses (e.g., a type of HMD), a car noise may be launched in the user’s headset when a car is detected within the field of view of the user camera, in order to warn him of the potential danger. Furthermore, the sound may be spatialized in order to make it arrive from the direction where the car was detected.

[0102] An XR application may also augment a video content rather than a real environment. The video may be displayed on a rendering device and virtual objects described in a node tree may be overlaid when timed events are detected in the video. In such a context, the node tree may comprise only virtual objects descriptions.

[0103] In XR applications, scene description may be used to combine explicit and easy-to-parse description of a scene structure and some binary representations of media content. In one example, there may be syntax of a data stream encoding an extended reality scene description. The structure of an XR scene may comprise a container which organizes the stream in independent elements of syntax. The structure may comprise a header part, which is a set of data common to every syntax element of the stream. For example, the header part may comprise some metadata about syntax elements, describing the nature and the role of each of them. The structure may also comprise a payload comprising one or more elements of syntax. A syntax element may comprise data representative of the media content items described in the nodes of the scene graph related to virtual elements. Images, meshes, and/or other raw data may have been compressed according to a compression method. An element of syntax may be a part of the payload of the data stream and may comprise data encoding the scene description as described according to the present principles described herein.

[0104] Lighting may be an important element in an XR scene, and there is a need for the description format to describe different type(s) of light with specific properties. Lights may provide a scene with the ability to have shadows which are required for AR experience(s).

[0105] FIG. 3 illustrates an example of KHRJights punctual gltf extension. A Khronos gltf framework may provide such a scene description format that features an extension dedicated to light.

[0106] A KHRJights punctual extension may define three "punctual" light types: directional, point, and spot. Punctual lights may be defined as parameterized, infinitely small points that emit light in well-defined directions and intensities. These lights may be referenced by nodes and inherit the transform of that node.

[0107] The KHRJights punctual may support area light because this light type may be an efficient manner to integrate virtual objects in a real environment, may describe real lights efficiently, and may be incorporated into game engines like Unity or Unreal Engine. As discussed herein, the gITF format may include support for area light.

[0108] This area light may be a square quad, with the following parameters (e.g., additively to every light parameter already existing): 2D texture, width, and/or height of the area. It inherits of parent transform like other light type; for example, a light component that describes its properties may be referenced by a scene graph node that may feature transform elements (e.g., translation, rotation, scale) that may affect some light spatial properties (e.g., the size of the area light or the pose of a spotlight).

[0109] FIG. 4 illustrates an example of EXTJightsJmage_based gltf extension.

[0110] The EXTJightsJmageJjased extension may define an array of image-based lights at the root of the gITF and then each scene may reference one. Such an image-based light may provide an ambient light that is present all around the scene (e.g., not coming from a specific object). Each image-based light definition may comprise of a single cubemap that describes the specular radiance of the scene, the l=2 spherical harmonics coefficients for diffuse irradiance, and/or rotation and intensity values. The specularlmages array may contain references (e.g., indices) of images, one image being displayed on each inner face of a cube.

[011 1] The MPEG-I Scene Description framework may define a gITF extension (e.g., MPEG_animation_timing) that allows the control of the state of gITF animations over time. [0112] It may also define M PEG _accessorsjimed that implements accessors dedicated to timed data (e.g. , audio, video, or any data that may change over time).

[0113] The KHRJights _punctual and the EXTJights_image_based extensions may define static properties for each light component, and in a time-evolving scene a mechanism may be needed/implemented to describe pre-defined time-evolving light properties. For instance, a lighting animation for a set of spots in a virtual scene that change the colors and the intensity of the lights. Or a light that moves in the scene.

[0114] As described herein, there may be a device, method, and/or XR application that utilizes an extension that contains timed characteristics of a lighting source node. In some cases, this extension may be included in gITF 2.0 framework.

[0115] Lighting sources described by the scene description document may be controlled through a mechanism that describes time-evolving light properties. The activation timing of control events may be identified by the timing of a sample in a lighting media source. Once a lighting time event is activated modifications to be applied to the lighting may be determined from lighting data in the scene description data and/or information provided by the lighting sample.

[0116] This augmentation of the scene description may be defined at the scene level to describe a lighting animation that may involve several light nodes. Additionally/alternatively, it may be defined at the light node level, to animate a single light node.

[0117] In some cases, the subject matter descirbed herein may be read in the context of MPEG-I Scene Description framework using Khronos gITF extension mechanism to support additional scene description features.

[0118] FIG. 5 illustrates an example of a MPEG_media element for time-evolving lights properties.#

[0119] There may be a gITF extension MPEGJightingJiming which identifies time-evolving light properties, as disclosed further herein.

[0120] Timed samples that describe the evolution of the light(s) properties may be provided in a text JSON content file. This content may be described with the MPEG_media extension (e.g., such as defined in MPEG-

I SD framework), with a mime type equal to “application/json”.

[0121] FIG. 6 illustrates an example of a JSON file to control gITF light sources.

[0122] Each sample in the JSON file may contain information for each light element, such as

KHRJights -punctual or EXT_lights_image_based type. For the KHRJights_punctua\, there may be: an index in the gITF nodes array of the light node; a transform element that applies to the light node (e.g., a transformation matrix or a set of translation, rotation, and/or scale elements); and/or, all the nodes referencing the same light component are impacted by the new light properties. For both types, each sample may also comprise one or more light properties that override the initial static properties described in the light component from the gITF lights array (e.g., static light properties may be defined in the gITF file, and the properties contained in a JSON file may override them).

[0123] A timed accessor is preferably chosen for high evolution rate (e.g., at frame rate). Otherwise, a source index may be used to reference a set of timed samples.

[0124] The MPEGJightingJiming extension may be defined at a different level in the gITF file, such as: at the scene level to describe a lighting animation that may involve several light nodes or for EXTJightsJmage_based light that is unique to a scene; at a light component level for EXTJightsJmage_based lights, also for KHRJights _punctual lights if that same light animation is to be applied to several nodes referencing this same light component (e.g., no transform animation in this case); and/or, at the light node level (e.g., for KHR_lights_punctual lights), to animate a single light node with modification of the transform element.

[0125] FIG. 7 illustrates an example of a MPEGJightingJiming extension at scene level. Regarding scene level extension(s), a MPEGJightingJiming extension may specify the MPEGjnedia element containing the light samplesAt 701, the source file referencing in the MPEGJightingJiming extension may comprise timed samples for multiple light sources.

[0126] FIG. 8 illustrates an example of MPEGJightingJiming extension at light component level. Regarding light component level extension(s), one or several light items in the light components array may be replaced by a MPEGJightingJiming referencing an MPEG_accessorJimed, as shown at 801 and 802. In this case, all nodes referencing this light, may be animated. In some cases, the timed accessor may point to a buffer containing the light samples. In this case, the sample may only contain light component properties, and no index nor transform fields. The buffer pointed by the timed accessor may comprise time evolving parameters for the light item.

[0127] For image-based light, the light samples file may not be needed if all that is wanted is to animate the specular images: In the EXTJightsJmage_based elements, images of the cube map may be replaced by video textures as defined by the MPEGJexture_video (e.g., in the MPEG-I SD framework). For example, one image for a face of the cube (e.g., or any shape for that matter), may be replaced by a video texture for that face of the cube.

[0128] In one instance, the MPEGJightingJiming extension could also be applied to area lights if this type is included in the KHRJights punctual available types: the 2D parameter of the area light may be replaced by a video texture.

[0129] FIG. 9 illustrates an example of a video texture for EXTJightsJmage_based lights. The specularlmages parameters of the image_based light may reference video texture instead of static images, as shown at 901 .

[0130] Regarding light node level extension, to animate only one node, a MPEGJightingJiming extension may reference an MPEG_Media source. [0131] In one example, a WTRU may obtain scene data and related lighting information. This lighting information may augment/modify the lighting of the scene. The lighting information may be a video texture for one or more faces of the cube map (e.g., MPEGJexture_video). If the lighting information includes a video, then the video will contain both lighting image and timing information. The WTRU may render and display the scene based on this scene data and lighting information. At some point in time (e.g., in accordance with the video) there may be a change in the scene such that the lighting of the scene changes. In some instances, the scene may be 3D, may comprise one or more objects or areas, and may be interactive for a user of the WTRU. In some instances, color parameters may also be included in the lighting information.

[0132] FIG. 10 illustrates an example of MPEGJightingJiming extension at light node level. The source file, as shown at 1001 , referencing in the MPEGJightingJiming extension may comprise time evolving parameters for the node items and for the light it references (e.g., light parameters and/or transform parameters).

[0133] Table 1 is an example of parameters that may be included in MPEGJightingJiming extension.

Table 1

[0134] Each face may have an image texture, or a video texture. In one example, all faces of a cube map may be treated the same way. In one example, each face may be treated differently. In one case, there may be six videos/images for the cube map (e.g., one for each face of the cube). In one case, there may be one video/image for the cube map that is used by all faces. In one case, there may be fewer than six videos/images for the cube map (e.g., where at least one file is reused for two or more of the faces; a subset of faces may share the same video/image). In one case, there may be one video/image for the cube map that can be split into six sub-textures (e.g., one for each face of the cube). In one case, there may be one video/image that is used for all of the faces of the cube map.

[0135] FIG. 11 illustrates an example of a XR application. In one example, a WTRU may implement a method (e.g., an XR application may be executed by a processor of the WTRU and stored on a storage medium/memory of the WTRU). The WTRU, at 1101 , may obtain scene description data for a 3D scene. The WTRU, at 1102, may obtain lighting information. The lighting information may include one or more parameters for light sources for a cube map, such as a texture parameter, where the texture parameter may be a video. In one instance, the video may be applied to each face of a cube map. The WTRU, at 1103, may render the 3D scene based on the lighting information and the scene description data such that lighting of the 3D scene changes over time according to the video. The WTRU, at 1104, may display the 3D scene that was rendered (e.g., through an HMD that is a part of or external to the WTRU). In one instance, the one or more parameters may also include intensity or color. In one instance, the 3D scene may change overtime (e.g., as a result of an event, as part of the video). In one instance, the lighting information may be applicable to one or more light sources of the 3D scene. In one instance, the 3D scene changes may be part of an animation involving a light source of the 3D scene over time. In one instance, the scene is continuously being rendered since it is changing over time (e.g., as a result of the video file, which are a series of images linked to the progression of time). In one alternative case to the example shown, scene description data may include lighting data, and/or there may be one receiving step that retrieves all the necessary information for rendering the scene (e.g., one or more pieces of information/data/parameters disclosed herein).

[0136] In another one example, a WTRU may implement a method (e.g., an XR application may be executed by a processor of the WTRU and stored on a storage medium/memory of the WTRU). The WTRU, at 1101 , may obtain scene description data for a 3D scene. The WTRU, at 1102, may obtain lighting information and timing information. The lighting information may be associated with the timing information to augment lighting of the 3D scene. The lighting information may also include one or more parameters, such as a texture parameter, where the texture parameter may include video texture. The WTRU, at 1103, may display the 3D scene (e.g., through a HMD a part of or external to the WTRU). The WTRU, at 1104, may process the lighting information. The WTRU, at 1105, may determine an event has occurred relative to the timing information. The WTRU, at 1106, may augment the 3D scene based on the event and the lighting information such that lighting of the 3D scene changes over time according to the timing information and the lighting information. In one instance, the 3D scene may comprise an object or an area. In one instance, the one or more parameters may also include intensity or color.

[0137] In one example, a WTRU may comprise of one or more processors, wherein at least one of the one or more processors may be a graphics processor. The graphics processor may generate an output (e.g., extended reality) on an appropriate display (e.g., headset, glasses, immersive monitor, LCD, OLED, etc.). The output may be determined from one or more files that comprise data regarding a scene or a model (e.g., visual information, scene description data, etc.). The file may adhere to one or more standards, and may include one or more pieces of information and/or elements that relate to lighting (e.g., aforementioned extensions, flags, accessor, etc.). For example, the file may contain one or more lighting related elements, as disclosed herein, that when processed by the graphics processor modify the output such that lighting is presented differently. For example, if the output is of a scene or model (e.g., 3D or 2D), then the one or more lighting related elements may enable the output to display light in specific areas of the scene, wherein the light displayed may have one or more properties that further controls how exactly the light is displayed in the scene (e.g., intensity, power, color, lumens, etc.) and how the aspect of objects in the scene are impacted. This change may be based on the graphics processor calculating and/or processing one or more lighting related elements, as discussed herein. In one example, the one or more lighting related elements may include a timing aspect, as disclosed herein, including time-evolving light properties. There may be a lighting sample that provides the basis for applying and/or replicating lighting elements; this lighting sample may also include a timed element. Upon a predetermined event (e.g., as part of a timed event within the scene), a lighting event may be determined and activated, and the timeline (e.g., change of the light with respect to time) of the lighting may be determined by lighting data in the scene description data. Additionally, the information provided by the lighting sample may be applied. This changing/augmentation of the scene description may be defined at the scene level to describe a lighting animation that may involve one or more light nodes. Additionally/alternatively, it may be defined at the light node level, to animate a single light node.

[0138] In an example, there may be a method for timed lighting in extended reality experiences. A device may be configured to display a scene. The device may process lighting information, wherein the lighting information includes timing information related to augmenting lighting of the scene, or an object in the scene, or an area in the scene. The device may determine an event has occurred, such as a planned time event part of the scene. The device may augment the scene based on the event and the lighting information such that lighting of the scene, or an object in the scene, or an area in the scene, may change over time based on the timing information. In some cases, the lighting information may be applicable to several light nodes of the scene. In some cases, the change of the lighting of the scene is part of an animation describing the changes of several lights of the scene over time.

[0139] Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magnetooptical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.