1. Cross-Reference to Related Applications

[0001] This application claims the benefit of priority from U.S. Provisional Application No. 63/407,885 filed on 19 September 2022, and European Application No. 23151686.5 filed on 16 January 2023, each of which is incorporated by reference herein in its entirety.

2. Field of the Disclosure

[0002] Various example embodiments relate generally to video/image compression and, more specifically but not exclusively, to video/image encoding and decoding.

3. Background

[0003] Compression enables reduction of the amounts of memory needed to store videos/images and of bandwidth needed to transport the same. The Motion Picture Experts Group (MPEG) codec, based on the H.264 compression standard, is an example codec used for this purpose. Other codecs are also available in the marketplace. Many of the codecs are designed to be compatible with a variety of professional, consumer, and mobile-phone cameras.

BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS

[0004] Disclosed herein are various methods and apparatus for packing single-channel data into a multi-channel container, e.g., an MP4, TIFF, or JPEG container, to at least achieve good utilization of the container’s data capacity. In various embodiments, the packing is performed based on a mapping relationship represented by an n-dimensional curve or by a plurality of 2 ⁿ -way tree (e.g., octree for n=3) partitions of n-dimensional space. At least some embodiments are compatible with the existing hardware of a legacy video/image delivery pipeline, i.e., do not inherently rely on any hardware/infrastructure modifications thereof. Rather, such embodiments can advantageously be implemented in software and/or firmware, e.g., by interfacing the corresponding add-on data-processing modules with the existing codec, without altering the codec’s native container format(s).

[0005] According to an example embodiment, provided is a coding method, comprising: converting, with a processor, a plurality of scalar values of a received data stream into a corresponding plurality of n-dimensional values, the converting being performed using a mapper; assigning, with the processor, each of the n-dimensional values as a pixel value to a respective pixel of a virtual-image frame, where n is an integer greater than one; and compressing, with the processor, the virtual-image frame according to a type of a container for image data; and wherein the mapper is configured to map a scalar value to a corresponding n-dimensional value based on a relationship represented by an n-dimensional curve or by a plurality of 2 ⁿ -way tree partitions of n-dimensional space.

[0006] According to another example embodiment, provided is a non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations comprising the above method.

[0007] According to yet another example embodiment, provided is an apparatus for coding image data, the apparatus comprising: at least one processor; and at least one memory including program code; wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus at least to: convert, with an electronic mapper, a plurality of scalar values of a received data stream into a corresponding plurality of n-dimensional values; assign each of the n-dimensional values as a pixel value to a respective pixel of a virtual-image frame, where n is an integer greater than one; and compress the virtual-image frame according to a type of a container for the image data; and wherein the electronic mapper is configured to map a scalar value onto a corresponding n- dimensional value based on a relationship represented by an n-dimensional curve or by a plurality of 2 ⁿ -way tree partitions of n-dimensional space.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Other aspects, features, and benefits of various disclosed embodiments will become more fully apparent, by way of example, from the following detailed description and the accompanying drawings, in which:

[0009] FIG. 1 depicts an example process for a video/image delivery pipeline;

[0010] FIG. 2 is a flowchart of an encoding method that can be used in the video/image delivery pipeline of FIG. 1 according to an embodiment; [0011] FIG. 3 graphically illustrates the principle of operation of a lD-to-3D mapper that can be used in the encoding method of FIG. 2 according to an embodiment;

[0012] FIG. 4 graphically illustrates the principle of operation of a lD-to-3D mapper that can be used in the encoding method of FIG. 2 according to another embodiment;

[0013] FIGs. 5A-5B graphically illustrate the principle of operation of a lD-to-3D mapper that can be used in the encoding method of FIG. 2 according to yet another embodiment;

[0014] FIG. 6 is a flowchart of a decoding method that can be used in the video/image delivery pipeline of FIG. 1 according to an embodiment; and

[0015] FIG. 7 is a block diagram illustrating a computing device according to an embodiment.

DETAILED DESCRIPTION

[0016] This disclosure and aspects thereof can be embodied in various forms, including hardware, devices or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces, and application programming interfaces; as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and the like. The foregoing is intended solely to give a general idea of various aspects of the present disclosure, and does not limit the scope of the disclosure in any way.

[0017] FIG. 1 depicts an example process of a video/image delivery pipeline 100, showing various stages from video/image capture to video/image-content display according to an embodiment. A sequence of video/image frames 102 may be captured or generated using an image-generation block 105. The frames 102 may be digitally captured (e.g., by a digital camera) or generated by a computer (e.g., using computer animation) to provide video and/or image data 107. Alternatively, the frames 102 may be captured on film by a film camera. Then, the film may be scanned and converted into a digital format to provide the video/image data 107. [0018] In a production phase 110, the data 107 may be edited to provide a video/image production stream 112. The data of the video/image production stream 112 may be provided to a processor (e.g., one or more processors, such as a central processing unit, CPU, and the like) at a post-production block 115 for post-production editing. The post-production editing of the block 115 may include, e.g., adjusting or modifying colors or brightness in particular areas of an image to enhance the image quality or achieve a particular appearance for the image in accordance with the video creator’s creative intent. This part of post-production editing is sometimes referred to as “color timing” or “color grading.” Other editing (e.g., scene selection and sequencing, image cropping, addition of computer-generated visual special effects, removal of artifacts, etc.) may be performed at the block 115 to yield a “final” version 117 of the production for distribution. During the post-production editing 115, video and/or images may be optimized for viewing on a reference display 125.

[0019] Following the post-production 115, the data of the final version 117 may be delivered to a coding block 120 for being further delivered downstream to decoding and playback devices, such as television sets, set-top boxes, movie theaters, and the like. In some embodiments, the coding block 120 may include audio and video encoders, such as those defined by the ATSC, DVB, DVD, Blu-Ray, and other delivery formats, to generate a coded bitstream 122. In a receiver, the coded bitstream 122 is decoded by a decoding unit 130 to generate a corresponding decoded signal 132 representing a copy or a close approximation of the signal 117. The receiver may be attached to a target display 140 that may have somewhat or completely different characteristics than the reference display 125. In such cases, a display management (DM) block 135 may be used to map the decoded signal 132 to the characteristics of the target display 140 by generating a display-mapped signal 137. Depending on the embodiment, the decoding unit 130 and display management block 135 may include individual processors or may be based on a single integrated processing unit.

Various embodiments disclosed herein below can be used to implement the coding block 120 and/or the decoding unit 130.

[0020] A codec used in the coding block 120 and/or the decoding unit 130 enables video/image data processing and compression/decompression. The compression is used in the coding block 120 to make the corresponding file(s) smaller. The decoding process carried out by the decoding unit 130 typically includes decompressing the received video/image data file(s) into a form usable for playback and/or further editing. Example codecs that can be used in the coding block 120 and the decoding unit 130 include but are not limited to XviD/DivX codecs, MPEG codecs, and H.264 codecs.

[0021] A container is a digital file that contains video/audio data and any corresponding metadata organized into a single package. The metadata may include subtitles, resolution information, creation date, device type, language information, and the like. The container file interleaves the different data types in a manner that makes the components thereof readily accessible to the decoding unit 130. Different types of containers are typically identified by their respective file extensions, which include MP4, WAV, AIFF, AVI, MOV, WMV, MKV, TIFF, JPEG, HEVC, FLV, F4V, SWF, and more.

[0022] For example, the JPEG image file format is a popular choice for storing and transmitting photographic images, e.g., still images and individual video frames. Many operating systems have viewers that support visualization of JPEG image files, which are stored with the JPG or JPEG extension. Many web browsers also support visualization of JPEG image files. JPEG encoding typically includes the following operations:

(1) Transformation: color images are transformed from the RGB (Red, Green, Blue) color space into a luminance/chrominance space.

(2) Down Sampling: the down sampling is typically done for the chrominance components but not for the luminance component. For example, down sampling can be done for the image frame at a ratio 2:1 horizontally and 1 : 1 vertically (2h Iv).

(3) Organizing in Groups: the pixels of each color component are organized in groups of 8x8 pixels, often referred to as “data units.” If the number of rows is not an integer multiple of 8, then the bottom row is duplicated one or more times to achieve the needed multiplicity. Similar duplication may also be applied to the rightmost column.

(4) Discrete Cosine Transformation (DCT): the DCT is applied to each data unit to create an 8x8 map of transformed components. The DCT typically involves some loss of information due to the limited precision of the machine-based arithmetic.

(5) Quantization: each of the 64 transformed components in the data unit is divided by a separate number referred to as the Quantization Coefficient (QC) and then rounded to an integer. This operation typically causes additional loss of information. Large QC values tend to cause more loss. Many encoders rely on the QC tables recommended in the JPEG standard for quantization. (6) Encoding: the 64 quantized transformed coefficients (which are now integers) of each data unit are encoded using a combination of run-length encoding (RLE) and Huffman coding.

(7) Header: the last operation adds a header listing all relevant JPEG parameters therein. The corresponding JPEG decoder uses inverse operations to generate an image that closely approximates the originally encoded image.

[0023] As another example, TIFF image frames are made up of rectangular grids of pixels. The two axes of this geometry are termed horizontal (or X, or width) and vertical (or Y, or length). Horizontal and vertical resolution need not be equal. A baseline TIFF image divides the vertical range of the image into one or more strips, which are encoded and compressed individually (separately). The TIFF format is an alternative to tiled image formats, in which both the horizontal and the vertical ranges of the image are divided into smaller units. The data for one pixel include one or more samples. For example, an RGB image typically has one Red sample, one Green sample, and one Blue sample per pixel, whereas a greyscale image has only one sample per pixel. The TIFF format can be used with both additive (e.g. RGB) and subtractive (e.g. Cyan, Magenta, Yellow, Black, or CMYK) color models. In at least some examples, interpretation of the channel data is external to the TIFF container. The interpretation can be aided by metadata, such as, for example, the International Color Consortium (ICC) profile. The TIFF format does not constrain the number of samples per pixel, nor does it constrain how many bits are encoded for each sample. For example, three samples per pixel is at the low end of multispectral imaging supported by TIFF, whereas hyperspectral imaging (also supported by TIFF) may use one hundred or more samples per pixel. Support for the custom selection of the number of samples per pixel in the TIFF format is leveraged in at least some embodiments disclosed herein below. TIFF images may be uncompressed, compressed using a lossless compression scheme, or compressed using a lossy compression scheme. An example of a lossless compression scheme compatible with the TIFF format is the LZW (Lempel-Ziv- Welch) compression scheme.

[0024] Augmented reality (AR) applications, virtual reality (VR) applications, and various applications involving rendering of 3-dimensional (3D) scenes may typically generate additional image-data streams, each of which may take the form of a corresponding data sequence transmitted via a corresponding dedicated data channel. Examples of such data streams include but are not limited to depth data, vertex data, and index data. However, many of the above-mentioned types of containers (file formats) and the corresponding conventional hardware for the coding block 120 and the decoding unit 130 do not innately support single-channel transport, which disadvantageously creates difficulties for efficient encoding/decoding and compression/decompression of the above-mentioned additional data streams.

[0025] Various embodiments disclosed herein address at least some of the aboveindicated problems in the state of the art by providing various schemes for packing singlechannel data into multi-channel containers to achieve one or more of: (i) efficient utilization of the multi-channel container’s data capacity; (ii) preservation of the spatial and/or temporal coherency of the single-channel data stream in the packed form; and (iii) inhibition or avoidance of excessive compression artifacts in the decompressed single-channel data at the decoder unit 130. At least some embodiments are fully compatible with the existing hardware of the corresponding video/image delivery pipeline (such as 100, FIG. 1), i.e., do not inherently rely on any hardware/infrastructure modifications thereof. Instead, such embodiments can advantageously be implemented in software and/or firmware, e.g., by interfacing with the existing codec the corresponding relatively small data-processing modules, without altering the codec’s native container format(s).

[0026] FIG. 2 is a flowchart of an encoding method 200 that can be used in the coding block 120 according to an embodiment. The encoding method 200 enables encoding of a single-channel data stream (e.g., a data sequence) into an n-channel container, where n is an integer greater than one. The single-channel data stream can be, for example, the depth data stream corresponding to a 3D scene. The n-channel container can be, for example, one of the multi-channel containers mentioned above.

[0027] The encoding method 200 comprises receiving a next value of the single-channel data stream (in block 202). For the first instance of the block 202, the received next value is the first value of the stream. For any subsequent instance of the block 202, the received next value is the value of the stream that follows the previously received value.

[0028] The encoding method 200 also comprises selecting a next pixel of a virtual-image frame (in block 204). Herein, the term “virtual-image frame” refers to an image frame that is analogous to a conventional image frame. However, unlike the latter, a virtual-image frame does not represent a conventional image. Instead, different pixels of the virtual-image frame can be assigned any desired pixel values, e.g., values generated by a suitable mapper (e.g., see block 206 and FIGs. 3-5). The pixels in such virtual-image frame can be spatially arranged in the same manner as in a conventional image frame, e.g., in a rectangular array having the pixels thereof in rows and columns.

[0029] In some examples, in the first instance of the block 204, the top-left corner pixel of the frame is selected. In any subsequent instance of the block 204, the selection process may follow a raster pattern, or any other suitable predetermined pattern. The processing of the whole virtual-image frame in the encoding method 200 typically concludes after each of the frame’s pixels has been selected once. For example, with the raster pattern, the pixel selected in the last instance of the block 204, can be the frame’s bottom-right comer pixel.

[0030] In other examples, pixel processing orders other than the above-indicated sequential order are also used. In one example, random pixel selection is implemented. In various additional examples, the pixels of the virtual-image frame are processed in parallel within single instruction, multiple data (SIMD) vectorization; multiple instruction, multiple data (MIMD) multithreading; or a combination of thereof. Such alternatives can also be implemented in various examples of the decoding method 600 (see FIG. 6).

[0031] The encoding method 200 also comprises converting (in the block 206) the ID (1- dimensional, scalar) value received in the block 202 into a corresponding n-dimensional (nD) value using the selected ID-to-nD mapper. In some specific examples, an nD value can be represented by a vector in an nD space. In the Cartesian coordinate system, an origin-based vector v in the nD space is represented by a string of values (.n, xi, x„), where x, is the length of the projection of the vector v onto the coordinate axis corresponding to the / th dimension of the nD space. The number n is an algorithm parameter that depends on the container used in the encoding method 200. For example, for the JPEG and MP4 containers, the number n is n=3. For the TIFF container, the number n is the number of samples per pixel, which can be n>3 in at least some examples, as explained above. Several non-limiting examples of ID-to-nD mappers that can be used in the block 206 are described in more detail below in reference to FIGs. 3-5.

[0032] The encoding method 200 also comprises assigning (in block 208) the nD value generated in the block 206 to the pixel selected in the block 204. For example, for n=3 and containers supporting the RGB color scheme, the xi, xi, and xj components of the corresponding 3D vector are assigned to the selected pixel as the pixel’s R, G, and B values, respectively (in the block 208). As another example, for n=16 and the TIFF container, the i, 2, . . ., i6 components of the corresponding 16D vector are assigned to the pixel as the respective 16 samples corresponding to multispectral imaging (in the block 208). Other suitable assignment schemes may also be used in other examples of the block 208.

[0033] The encoding method 200 also comprises determining (in decision block 210) whether or not the end of the corresponding virtual-image frame has been reached. When it is determined that the end of the frame has not been reached (“No” at the decision block 210), operations of the encoding method 200 loop back to the block 202. Otherwise (“Yes” at the decision block 210), the (fully pixel-assigned) virtual-image frame is compressed in a conventional manner in accordance with the container format (in block 212). The container may then be directed from the coding block 120 to the decoding unit 130 as previously described (also see FIG. 1). Upon completing the operations of the block 212, the encoding method 200 is terminated.

[0034] FIGs. 3-5 illustrate several non-limiting examples of ID-to-nD mappers that can be used in the block 206 of the encoding method 200 according to various embodiments. For ease of presentation and without any implied limitations, the illustrated examples correspond to the number n=3. Based on the provided description, a person of ordinary skill in the pertinent art will be able to make and use various ID-to-nD mappers corresponding to other values of n, e.g., n=2 and n>3, without any undue experimentation. In at least some examples, the corresponding ID-to-nD mapper is implemented using one or more lookup tables (LUTs).

[0035] FIG. 3 graphically illustrates the principle of operation of a lD-to-3D mapper that can be used in the block 206 of the encoding method 200 according to an embodiment. More specifically, FIG. 3 shows a 3D logarithmic spiral 302 in a Cartesian coordinate system whose coordinate axes are labeled Xi, Xi, and X3, respectively. The spiral 302 can be used to map a range [0, D] of scalar values d onto 3D vectors (Y, Cb, Cr), where Y, Cb, and Cr denote the luminance, blue-difference chroma component, and red-difference chroma component, respectively. [0036] In some examples, a scalar value d (D > d > 0) is mapped onto the spiral 302 by finding the point on the spiral whose distance along the spiral, from the spiral’s origin O, is d. The cartesian coordinates (jf3, *2, xi) of the found point are then used to determine the corresponding values of Y, Cb, and Cr, respectively. In some specific instances, the following Eqs. (l)-(3) are used to program a processor of the coding block 120 to perform the corresponding computations on the fly in the block 206 of the encoding method 200: y = F(d) (3) where a and b are parameters that determine how the spiral 302 grows in radius; Cro and Cbo are (offset) constants; and F(d) is a function of d that determines the distance scale along the Y (luminance) axis. In some specific examples, the function F(d) is a quadratic function of d. In some other specific instances, Eqs. (l)-(3) are used to precompute a corresponding LUT, which is then accessed by the processor to perform the corresponding operations in the block 206.

[0037] When the mapping performed in the block 206 is implemented based on the spiral 302, such mapping is spatially and temporally coherent. In particular, such mapping is distance preserving. The mapping is also unique for any two points located on the spiral 302. Furthermore, for any three points located on the spiral 302, the distance relationship in the 3D space is the same as the distance relationship along the spiral 302 (i.e., in the ID space). For example, when point B lays between points A and C on the spiral 302, the mapping is such that the distance, in the 3D space, between points A and C is larger than the distance between points A and B and is also larger than the distance between points B and C. These properties resulting from the spatial and temporal coherency of the mapping are typically beneficial for achieving efficient compression, e.g., in JPEG, HEVC, MP4, and many other formats, and for inhibiting manifestations of compression artifacts in the decompressed single-channel data computed by the decoder unit 130.

[0038] FIG. 4 graphically illustrates the principle of operation of a lD-to-3D mapper that can be used in the block 206 of the encoding method 200 according to another embodiment. More specifically, FIG. 4 shows a 3D Peano curve 402 in the Cartesian coordinate system whose axes are labeled Xi, X2, and X3, respectively. The granularity of the Peano curve 402 is 2 bits per dimension. In additional embodiments, Peano curves of other granularities can similarly be used. In other additional embodiments, various nD Peano curves may be used to implement the corresponding ID-to-nD mappers with various granularities, where n=2 or n>3.

[0039] The Peano curve 402 is used to map a range [0, D] of scalar values d onto 64 different 3D vectors (Y, Cb, Cr) or (R, G, B). In some examples, a scalar value d (D > d > 0) is mapped onto the curve 402 by finding the point on the curve whose distance along the curve, from the curve’s origin O, is rounded to d. The cartesian coordinates (x , X2, n) of the found point are then used to determine the corresponding values of Y, Cb, and Cr, respectively, or of R, G, and B, respectively, for the pixel in question.

[0040] A Peano curve, such as the Peano curve 402, is an example of a space-filling curve (with endpoints) whose range covers the entire range of the corresponding n- dimensional hypercube. Other space-filling curve examples include but are not limited to Hilbert curves and Morton curves. Space-filling curves are special cases of fractal curves. Various ID-to-nD mappers suitable for implementing the block 206 of the encoding method 200 can be constructed using various suitable space-filling and/or fractal curves (with endpoints) in a manner similar to that described above in reference to the Peano curve 402 and FIG. 4.

[0041] FIGs. 5A-5B graphically illustrate the principle of operation of a lD-to-3D mapper that can be used in the block 206 of the encoding method 200 according to yet another embodiment. More specifically, FIG. 5A is a diagram illustrating an octreepartitioned 3D cube 500. FIG. 5B is a diagram illustrating an octree 510 used for the partitioning of the 3D cube 500.

[0042] In general, an octree, such as the octree 510 of FIG. 5B, is a tree data structure in which each internal node has exactly eight children. Octrees can be used to partition a bounded three-dimensional space (such as the 3D cube 500) by recursively subdividing the bounded space into eight octants. In a point-region (PR) octree, the node stores an explicit three-dimensional point, which is the “center” of the subdivision for that node. This center point defines one of the respective comers for each of the eight children. In a matrix-based (MX) octree, the subdivision point is implicitly the center of the space the node represents. The root node of a PR octree can represent infinite space. The root node of an MX octree represents a finite bounded space so that the implicit centers are well-defined. For example, the root node R of the octree 510 (see FIG. 5B) represents the 3D cube 500. The octree 510 is an example of a 2 ⁿ -way tree for which n=3. A person of ordinary skill in the pertinent art will readily understand how to construct various 2 ⁿ -way trees for other values of n without any undue experimentation. In additional examples, the value of n is 2, 4, 5, and so on.

[0043] The eight children of the root node R of the octree 510 are the nodes 0 through 7 (see FIG. 5B). The subcubes (octants) of the 3D cube 500 corresponding to the child nodes 0-7 are similarly labeled 0 through 7 in FIG. 5A. The subcube 7 is not directly visible in the view shown in FIG. 5A. For illustration purposes, only the children of the child node 1 are explicitly shown in FIG. 5B. These grandchild nodes are labeled 10 through 17 in FIG. 5B. The subcubes (octants) of the subcube 1 corresponding to the grandchild nodes 10-17 are similarly labeled 10 through 17 in FIG. 5A. The subcube 17 is not directly visible in the view shown in FIG. 5 A. For illustration purposes, only the children of the grandchild node 11 are explicitly shown in FIG. 5B. These great-grandchild nodes are labeled 110 through 117 in FIG. 5B. The subcubes (octants) of the subcube 11 corresponding to the great-grandchild nodes 110-117 are similarly labeled 110 through 117 in FIG. 5A. The subcube 117 is not directly visible in the view shown in FIG. 5A. A person of ordinary skill in the pertinent art will readily understand that each of the child nodes 0 and 2-7 similarly has grandchild nodes that similarly have great-grandchild nodes (not explicitly shown in FIG. 5A). A person of ordinary skill in the pertinent art will further understand that each of the grandchild nodes 10 and 12-17 similarly has great-grandchild nodes (not explicitly shown in FIG. 5 A). There is a total of 256 great-grandchild nodes in the octree 510. There is also a total of 256 corresponding great-grandchild subcubes in the 3D cube 500. The 256 great-grandchild subcubes partition the 3D cube 500 into 256 nonoverlapping portions, each having the shape of a cube.

[0044] In some specific examples, to implement a lD-to-3D mapper for the block 206 of the encoding method 200, the three dimensions Xi, X2, and X3 of the 3D cube 500 are assigned to represent the R, G, and B pixel values or the Y, Cb, and Cr pixel values, respectively. The range [0, D] of the scalar d values is divided into 256 intervals. Each of the intervals is assigned to a respective one of the 256 great-grandchild subcubes of the 3D cube 500. The cartesian coordinates (X3, z, xi) of the center of the corresponding greatgrandchild subcube are then used to determine the R, G, B (or Y, Cb, and Cr) pixel values for the scalar d value that is being mapped. In some specific instances, the above-explained relationship between the scalar d value and the coordinates (X , X2, i) of the sub-cube’s center is precomputed and tabulated in a corresponding LUT, which is then accessed by the encoder to perform the pertinent operations in the block 206.

[0045] In various examples, the octree encoding granularity, i.e., the number of subcubes in the 3D cube 500, is selected based on the desired accuracy and compression ratio intended for the encoder. The finer the cube’s subdivision, the more likely for the lossy compression performed in the block 212 of the encoding method 200 to result in an error during the corresponding decoding performed in the decoding unit 130 (also see FIG. 6). Thus, the selection of the octree encoding granularity may also need to be based on the error amount introduced by the lossy compression.

[0046] FIG. 6 is a flowchart of a decoding method 600 that can be used in the decoding unit 130 according to an embodiment. The decoding method 600 and the encoding method 200 are compatible with one another. As such, the decoding method 600 enables substantial recovery of the single-channel data stream encoded for transmission using the selected multichannel container format and the encoding method 200.

[0047] The decoding method 600 comprises decompressing a compressed virtual-image frame in accordance with the container format (in block 602). The decompression performed in the block 602 is an inverse operation to the compression performed in the block 212 of the encoding method 200.

[0048] The decoding method 600 also comprises selecting a next pixel of the decompressed virtual-image frame and reading the nD pixel value of the selected pixel (in block 604). In a representative example, the selecting operation in the block 604 of the decoding method 600 is implemented similar to the selecting operation of the block 204 of the above-described encoding method 200. As such, the reader is referred to the above description of the block 204 for pertinent details of the pixel selection process.

[0049] The decoding method 600 also comprises converting (in block 606) the nD pixel value read in the block 604 into a corresponding scalar value using a properly selected nD-to- 1D demapper. The selected nD-to-lD demapper is such that the demapping performed thereby is inverse to the mapping performed by the ID-to-nD mapper used in the block 206 of the encoding method 200. In at least some examples, the nD-to-lD demapper used in the block 606 of the decoding method 600 and the ID-to-nD mapper used in the block 206 of the encoding method 200 are implemented based on the same LUT. More specifically, the nD- to- ID demapper used in the block 606 is configured to look up a scalar value in that LUT based on the provided nD value, whereas the ID-to-nD mapper used in the block 206 of the encoding method 200 is configured to look up an nD value in the same LUT based on the provided scalar value. In various examples, the nD-to-lD demapper operates to account for errors introduced by lossy compression. For example, when the virtual-frame pixel RGB value (100, 200, 30) comes out after decoding as the RGB value of (94, 203, 31) due to errors introduced by lossy compression, the nD-to-lD demapper operates to correct the errors to return the original (100, 200, 30) RGB value. Such error correction is achieved, e.g., by selecting the granularity of the ID-to-nD mapping in such a way that the a typical scatter of the decoded points around the original constellation point is within the range closest (e.g., in the Euclidean distance sense) to the original constellation point as opposed to some other constellation point. This feature of the demapper is often referred to as the maximum likelihood detection.

[0050] The decoding method 600 also comprises outputting (in block 608) the scalar value determined in the block 606 as the next value of a corresponding data stream (data sequence). In the absence of losses and/or significant compression artifacts, the latter data stream is a copy or an approximate copy of the data stream received in the block 202 of the encoding method 200.

[0051] The decoding method 600 also comprises determining (in decision block 610) whether or not the end of the corresponding virtual-image frame has been reached. When it is determined that the end of the frame has not been reached (“No” at the decision block 610), operations of the decoding method 600 loop back to the block 604. Otherwise (“Yes” at the decision block 610), the decoding method 600 is terminated.

[0052] FIG. 7 is a block diagram illustrating a computing device 700 according to an embodiment. The device 700 can be used, e.g., in the coding block 120. A computing device similar to the device 700 can also be used in the decoding unit 130. Based on the following description of the device 700, a person of ordinary skill in the pertinent art will readily understand how to make, configure, and use a similar computing device for the decoding unit 130.

[0053] The device 700 comprises input/output (I/O) devices 710, a coding engine 720, and a memory 730. The I/O devices 710 may be used to enable the device 700 to receive at least a portion of the video/image stream 117 and to output at least a portion of the coded bitstream 122. The memory 730 may have buffers to receive image data to be encoded and compressed, e.g., by way of the video/image stream 117. The received image data may include, inter alia, the above described single-channel data streams. Once the data are buffered, the memory 730 may provide parts of the data to the coding engine 720 for processing therein. The coding engine 720 includes a processor 722 and a memory 724. The memory 724 may store therein program code, which when executed by the processor 722 enables the coding engine 720 to perform various coding operations, including hut not limited to the various coding operations described above in reference to some or all of FIGs. 2-5. The memory 724 may also store therein the above described LUTs that can be accessed by the processor 722 as needed.

[0054] Various aspects of the present invention may be further appreciated from the following enumerated example embodiments (EEEs)

[0055] EEE (1): A coding method, comprising: converting, with a processor, a plurality of scalar values of a received data stream into a corresponding plurality of n-dimensional values, the converting being performed using a mapper; assigning, with the processor, each of the n-dimensional values as a pixel value of a respective pixel of a virtual-image frame, where n is an integer greater than one; and compressing, with the processor, the virtual-image frame according to a type of a container for image data; and wherein the mapper is configured to map a scalar value to a corresponding n-dimensional value based on a relationship represented by an n-dimensional curve or by a plurality of 2 ⁿ -way tree partitions of n-dimensional space. Herein, a straight n-dimensional line is not an example of the recited “n-dimensional curve.” For example, a “curve” includes at least two portions that are not colinear with one another in the corresponding n-dimensional space.

[0056] EEE (2): The method of EEE (1), wherein n is greater than three.

[0057] EEE (3): The method of EEE (1) or EEE (2), wherein the corresponding n- dimensional value is a set including a red value, a green value, and a blue value, or a set including a cyan value, a magenta value, and a yellow value, or a set including a luminance value, a blue-difference chroma value, and a red-difference chroma value.

[0058] EEE (4): The method of any one of EEE (1) to EEE (3), wherein the plurality of scalar values are depth data, vertex data, or index data representing a 3-dimensional scene. [0059] EEE (5): The method of EEE (1), EEE (3), or EEE (4), wherein the n-dimensional curve is a spiral; and wherein n=2 or n=3.

[0060] EEE (6): The method of any one of EEE (1) to EEE (5), wherein the n- dimensional curve is a space-filling curve with two endpoints.

[0061] EEE (7): The method of EEE (6), wherein the space- filling curve is selected from the group consisting of a Peano curve, a Hilbert curve, a Morton curve, and a fractal curve.

[0062] EEE (8): The method of any one of EEE (1) to EEE (6), wherein the mapper is configured to determine the corresponding n-dimensional value by: finding a location on the n-dimensional curve having a distance from an endpoint thereof representing the scalar value, the distance being along the n-dimensional curve; and representing respective components of the corresponding n-dimensional value by a set of coordinates of the location in the n- dimensional space.

[0063] EEE (9): The method of any one of EEE (1) to EEE (5) and EEE (8), wherein the mapper is configured to determine the corresponding n-dimensional value by: identifying one partition of the plurality of 2 ⁿ -way tree partitions representing the scalar value; and representing respective components of the corresponding n-dimensional value by a set of coordinates of the one partition in the n-dimensional space.

[0064] EEE (10): The method of any one of EEE (1) to EEE (9), wherein the mapper is configured to use a lookup table precomputed based on the n-dimensional curve or the plurality of 2 ⁿ -way tree partitions of the n-dimensional space.

[0065] EEE (11): The method of any one of EEE (1) to EEE (10), further comprising: decompressing, with the processor or with another processor, a compressed image frame to generate a decompressed image frame, the decompressing being performed according to the type of the container, the compressed image frame having been generated by the compressing; and transforming, with the processor or with the another processor, a plurality of n-dimensional pixel values of the decompressed image frame into another plurality of scalar values, the transforming being performed using a demapper; and wherein the demapper is configured to perform a demapping operation that is inverse to a corresponding mapping operation of the mapper. [0066] EEE (12): The method of EEE (11), wherein both the mapper and the demapper are configured to use a same lookup table precomputed based on the n-dimensional curve or the plurality of 2 ⁿ -way tree partitions of the n-dimensional space.

[0067] EEE (13): A non- transitory computer-readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations comprising any one of EEE (1) to EEE (12).

[0068] EEE (14): An apparatus for coding image data, the apparatus comprising: at least one processor; and at least one memory including program code; wherein the at least one memory and the program code are configured to, with the at least one processor, cause the apparatus at least to: convert, with an electronic mapper, a plurality of scalar values of a received data stream into a corresponding plurality of n-dimensional values; assign each of the n-dimensional values as a pixel value of a respective pixel of a virtual-image frame, where n is an integer greater than one; and compress the virtual-image frame according to a type of a container for the image data; and wherein the electronic mapper is configured to map a scalar value onto a corresponding n-dimensional value based on a relationship represented by an n-dimensional curve or by a plurality of 2 ⁿ -way tree partitions of n- dimensional space.

[0069] EEE (15): The apparatus of EEE (14), wherein the electronic mapper is configured to: find a location on the n-dimensional curve having a distance from an endpoint thereof representing the scalar value, the distance being along the n-dimensional curve; and represent respective components of the corresponding n-dimensional value by a set of coordinates of the location in the n-dimensional space.

[0070] EEE (16): The apparatus of EEE (14), wherein the electronic mapper is configured to: identify one partition of the plurality of 2 ⁿ -way tree partitions representing the scalar value; and represent respective components of the corresponding n-dimensional value by a set of coordinates of the one partition in the n-dimensional space.

[0071] EEE (17): The apparatus of any one of EEE (14) to EEE (16), wherein electronic mapper is configured to use a lookup table precomputed based on the n-dimensional curve or the plurality of 2 ⁿ -way tree partitions of the n-dimensional space. [0072] EEE (18): The apparatus of any one of EEE (14) to EEE (17), wherein the at least one memory and the program code are configured to, with the at least one processor, further cause the apparatus to: generate a decompressed image frame by decompressing a compressed image frame according to the type of the container; and transform, with an electronic demapper, a plurality of n-dimensional pixel values of the decompressed image frame into another plurality of scalar values; and wherein the electronic demapper is configured to perform a demapping operation that is inverse to a corresponding mapping operation of the electronic mapper.

[0073] EEE (19): The apparatus of EEE (18), wherein both the electronic mapper and the electronic demapper are configured to use a common (e.g., a respective copy of the same) lookup table precomputed based on the n-dimensional curve or the plurality of 2 ⁿ -way tree partitions of the n-dimensional space.

[0074] EEE (20): The apparatus of any one of EEE (14) to EEE (19), wherein the plurality of scalar values are depth data, vertex data, or index data representing a 3- dimensional scene.

[0075] With regard to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the claims.

[0076] Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent upon reading the above description. The scope should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technologies discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the application is capable of modification and variation.

[0077] All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the technologies described herein unless an explicit indication to the contrary is made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

[0078] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in fewer than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.

[0079] While this disclosure includes references to illustrative embodiments, this specification is not intended to be construed in a limiting sense. Various modifications of the described embodiments, as well as other embodiments within the scope of the disclosure, which are apparent to persons skilled in the art to which the disclosure pertains are deemed to lie within the principle and scope of the disclosure, e.g., as expressed in the following claims.

[0080] Some embodiments may be implemented as circuit-based processes, including possible implementation on a single integrated circuit.

[0081] Some embodiments can be embodied in the form of methods and apparatuses for practicing those methods. Some embodiments can also be embodied in the form of program code recorded in tangible media, such as magnetic recording media, optical recording media, solid state memory, floppy diskettes, CD-ROMs, hard drives, or any other non-transitory machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing various embodiments described herein. Some embodiments can also be embodied in the form of program code, for example, stored in a non-transitory machine-readable storage medium including being loaded into and/or executed by a machine, wherein, when the program code is loaded into and executed by a machine, such as a computer or a processor, the machine becomes an apparatus for practicing various embodiments described herein. When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits.

[0082] Unless explicitly stated otherwise, each numerical value and range should be interpreted as being approximate as if the word “about” or “approximately” preceded the value or range.

[0083] The use of figure numbers and/or figure reference labels in the claims is intended to identify one or more possible embodiments of the claimed subject matter in order to facilitate the interpretation of the claims. Such use is not to be construed as necessarily limiting the scope of those claims to the embodiments shown in the corresponding figures.

[0084] Although the elements in the following method claims, if any, are recited in a particular sequence with corresponding labeling, unless the claim recitations otherwise imply a particular sequence for implementing some or all of those elements, those elements are not necessarily intended to be limited to being implemented in that particular sequence.

[0085] Reference herein to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the disclosure. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or additional embodiments necessarily mutually exclusive of other embodiments. The same applies to the term “implementation.”

[0086] Unless otherwise specified herein, the use of the ordinal adjectives “first,” “second,” “third,” etc., to refer to an object of a plurality of like objects merely indicates that different instances of such like objects are being referred to, and is not intended to imply that the like objects so referred-to have to be in a corresponding order or sequence, either temporally, spatially, in ranking, or in any other manner. [0087] Unless otherwise specified herein, in addition to its plain meaning, the conjunction “if" may also or alternatively be construed to mean “when" or “upon" or “in response to determining” or “in response to detecting,” which construal may depend on the corresponding specific context. For example, the phrase “if it is determined” or “if [a stated condition] is detected” may be construed to mean “upon determining” or “in response to determining" or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event].”

[0088] Also for purposes of this description, the terms “couple,” “coupling,” “coupled,” “connect,” “connecting," or “connected" refer to any manner known in the art or later developed in which energy is allowed to be transferred between two or more elements, and the interposition of one or more additional elements is contemplated, although not required. Conversely, the terms “directly coupled,” “directly connected,” etc., imply the absence of such additional elements.

[0089] As used herein in reference to an element and a standard, the terms “compatible” and “in accordance with” mean that the element communicates with other elements in a manner wholly or partially specified by the standard, and would be recognized by other elements as sufficiently capable of communicating with the other elements in the manner specified by the standard. The compatible element does not need to operate internally in a manner specified by the standard.

[0090] The functions of the various elements shown in the figures, including any functional blocks labeled as “processors” and/or “controllers,” may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor" or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and nonvolatile storage. Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

[0091] As used in this application, the terms “circuit,” “circuitry” may refer to one or more or all of the following: (a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry); (b) combinations of hardware circuits and software, such as (as applicable): (i) a combination of analog and/or digital hardware circuit(s) with software/firmware and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); and (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.” This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and/or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0092] It should be appreciated by those of ordinary skill in the art that any block diagrams and flowcharts herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in a computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0093] “BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS” in this specification is intended to introduce some example embodiments, with additional embodiments being described in “DETAILED DESCRIPTION” and/or in reference to one or more drawings. “BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS” is not intended to identify essential elements or features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.