Field of the invention

[0001] The invention relates to a method, device, and computer program for compressing and decompressing a colour image, and more specifically to the packetization of samples in a packed data set. The invention also relates to the packed data set produced by said method for compressing.

Description of prior art

[0002] Colour image sensors producing colour images are known in the industry. Video bandwidth requirements are growing fast, as video resolutions, frame rates and numbers of streams to manage are constantly increasing. The JPEG XS standard, a new standard for visually lossless low-latency lightweight image coding is currently being defined by the JPEG Committee (formally known as ISO/IEC JT1/SC29/WG1 ) and is described in:

• JPEG XS low-latency lightweight image coding system - Part 1 : Core Coding System (Hereafter Reference 1 )

(ISO /IEC JTC 1/SC 29 /WG 1 N91029 - 91th Meeting - online - April 2021 )

• JPEG XS low-latency lightweight image coding system - Part 2: Profiles and buffer models (Hereafter Reference 2)

(ISO/IEC JTC 1/SC 29/WG 1 N92027 92th Meeting - Online - July 2021 )

The JPEG XS standard defines a progression order, formally defined in Annex B, subclause B.7 of Reference 1. This progression order is dubbed “RLBC” for “Resolution-Line-Band-Component”. Note that a “band” in the JPEG XS standard is called “subband” in this document. This means that subbands are first serialized by component, then by subband, then by subband line, then by resolution.

Resolution level in the JPEG XS standard is similar to the decomposition level in this document. Lowest resolution level is numbered 0. Resolution level is increased by 1 after each bidirectional, i.e. vertical and horizontal, inverse DWT (Discrete Wavelet Transform) decomposition. Document “RICHTER THOMAS ET AL: “Bayer CFA Pattern Compression With JPEG XS”, IEEE TRANSACTIONS ON IMAGE PROCESSING, IEEE, USA, vol. 30, 16 July 2021 , pages 6557-6569” is a scientific paper describing in a less formal way what is defined in the JPEG XS standard as defined in Reference 1 and Reference 2 above.

[0003] Document WO2021084118 discloses an image processor for processing an image comprising pixels arranged in a Bayer pattern, and providing a highly decorrelated image.

[0004] Document US9332258 discloses a method and device for display stream compression, wherein coefficients are grouped, for each group, the greatest coded line index (GCLI) is determined and only the GCLI lowest weight bits of the coefficients are copied into the output stream together with the value of the GCLI. The GCLI is also known in the art as the bitplane count.

Summary of the invention

[0005] It is an object of the present invention to provide a more efficient method for compressing and decompressing of a colour image.

[0006] The invention is defined by the independent claims. The dependent claims define advantageous embodiments.

[0007] According to a first aspect of the invention, there is provided a method for compressing a colour image comprising colour image samples Sij arranged in a reference grid according to a sampling pattern, the reference grid comprising a number of lines and columns, the line index i starting at 0 and increasing with the lines, and the column index j starting at 0 and increasing with the columns, into a packed data set, comprising the steps of: a) performing a decorrelative transform, where for each position (i,j) of the reference grid, a component sample Ckij is determined, k being a component index depending on the sampling pattern, Ckij being a linear combination of the image samples Si’j’ at positions (i’,j’), where i’ is comprised between i-4 and i+4, and j’ is comprised between j-4 and j+4, producing component images CO, C1 , and C2, and, depending on the sampling pattern, an optional component image C3; b) repeating for each component image Ck, producing decomposed component images:

- Nlky being a number of vertical DWT decompositions to be applied to component image Ck, being an integer >= 0,

- Nlkx being a number of horizontal DWT decompositions to be applied to component image Ck, being an integer >= Nlky,

- considering the component image Ck as an auxiliary subband;

- if Nlky > 0, repeat, for n=0 to Nlky-1

- performing on said auxiliary subband a vertical DWT decomposition, by performing a lifting scheme with (5,3) filter-bank on the columns of said auxiliary subband, the resulting even numbered samples being a LLn subband, the resulting odd numbered samples being a LHn subband, keeping for each resulting sample a reference to the position (i,j) in the reference grid wherefrom they are determined;

- performing on said LLn subband a horizontal DWT decomposition, by performing a lifting scheme with (5,3) filter-bank on the lines of said subband, the resulting even numbered samples being a LLn subband, the resulting odd numbered samples being a HLn subband, keeping for each resulting sample a reference to the position (i,j) in the reference grid wherefrom they are determined;

- performing on said LHn subband a horizontal DWT decomposition, by performing a lifting scheme with (5,3) filter-bank on the lines of said subband, the resulting even numbered samples being a LHn subband, the resulting odd numbered samples being a HHn subband, keeping for each resulting sample a reference to the position (i,j) in the reference grid wherefrom they are determined;

- considering subband LLn as an auxiliary subband;

- if Nlkx > Nlky, repeat, for n=Nlky to Nlkx-1

- performing on said LLn-1 subband a horizontal DWT decomposition, by performing a lifting scheme with (5,3) filter-bank on the lines of said subband, the resulting even numbered samples being a LLn subband, the resulting odd numbered samples being a HLn subband, keeping for each resulting sample a reference to the position (i,j) in the reference grid wherefrom they are determined;

- considering subband LLn as an auxiliary subband.

According to the invention c) for each consecutive line i of the reference grid, for each component k,

(i) grouping all samples of said component k in said line i in one or more packets;

(ii) appending said one or more packets to the packed data set.

In processing step c) the method processes all the lines of the reference grid in the order of increasing line index i. For a given line, all the components are processed in any order, provided this order is the same when compressing and decompressing the packed data set. This order may be the order of increasing component index k.

In the context of the present invention:

- packetization is the formation of packets and appending of packets to the packed data set;

- a progression order results from the way component samples are grouped into packets and the way said packets are appended to the packed data set. The progression order of the invention results from step c) of the method for compressing of the invention. The progression order of the JPEG XS standard is defined by subclause B.7 of Reference 1 ;

- a packed data set is a data set obtainable by compressing a colour image with the method for compressing of the invention, the packed data set comprising a sequence of packets with the following characteristics that may be inferred from the method for compressing of the invention:

- each packet containing only data from a single decomposed component image;

- each packet containing only data from a single line of a decomposed component image;

- all packets containing data from a given line of a decomposed component image being located in the packed data set before any packet containing data from a subsequent line of any decomposed component image;

- determining whether an original data set is a packed data set according to the invention may be performed as follows:

- applying the method for decompressing of the invention on the original data set, producing an uncompressed data set;

- applying the method for compressing of claim 1 on the uncompressed data set, producing a packed data set;

- if the packed data set corresponds to the original data set, this original data set is a packed data set according to the invention;

- in an embodiment of the invention a packed data set may contain only the packets as formed and assembled in step c) of the method for compressing. In this embodiment, the device for compressing and the device for decompressing may have been set up for the same working parameters, such as image size, number of vertical and horizontal DWT decompositions, type of decorrelative transforms, sampling pattern and others. In another embodiment, the packed data set may comprise, in addition to the packets formed in step c) of the method for compressing, metadata specifying the working parameters used for compressing. Such metadata may be located in one header associated with the whole packed data set, or in several headers each one associated with a packet of the packed data set, or both;

- the grouping of all samples of a component k located in a line i of the reference grid into one or more packets may comprise considering all samples of the component k present in the line i, if any, and forming one or more new packets with these samples, said one or more new packets being not already present in the packed data set, each sample being assigned to one packet. If there is no sample of a component k present in a line i, then no new packet may be formed for this component k for this line i;

- step c) of the method for compressing of the invention is expressed in pseudo-code. What follows is an equivalent expression of the same step c) of the method in a more everyday language. A first repetitive loop is executed for each consecutive line i of the reference grid, i.e. for line index i going from 0 to line index of last line. For each execution of this first loop, a second repetitive loop is executed for each component k, i.e. for each value of the component index k. For each execution of this second loop, two steps are performed successively: (i) first, the grouping of all samples of the current component k located in the current line i of the reference grid in one or more packets that are not already present in the packed data set, each sample being assigned to one packet; (ii) then, the appending of said one or more packets formed in step (i) to the packed data set. The fact that packets formed in step (i) are not already present in the packed data set may be inferred from the method for compressing of the invention. If there is no sample of the current component k present in the current line i, then no new packet may be formed in the step (i) of the method and therefore no packet may be appended to the packed data set in the step (ii) of the method;

- subbands, component images, and decomposed component images are composed of even lines of even numbered samples and odd numbered samples, followed by odd lines of even numbered samples and odd numbered samples;

- when performing the decorrelative transforms and the lifting scheme operations for pixel positions near the borders of the image, when samples outside the borders of the image are needed for performing the computations, the values of the samples which are in mirror position of the missing samples with respect to the border of the image may be taken as replacement;

- when performing the lifting scheme operations, assigning low frequency samples to even numbered locations and high frequency samples to odd numbered locations is equivalent to assigning high frequency samples to even numbered locations and low frequency samples to odd numbered locations;

- when performing bidirectional, i.e. vertical and horizontal DWT decompositions, performing a vertical DWT decomposition followed by a horizontal DWT decomposition is equivalent to performing a horizontal DWT decomposition followed by a vertical DWT decomposition;

- when performing the decorrelative transform of step a), the linear combination may be taken from the immediately neighbouring pixels (i’ comprised between i-1 and i+1 , and j’ comprised between j-1 and j+1), or from the further neighbouring pixels (i’ comprised between i-2 and i+2, and j’ comprised between j-2 and j+2) , or from the still further neighbouring pixels (i’ comprised between i-3 and i+3, and j’ comprised between j-3 and j+3).

[0008] In a preferred embodiment, the number Nlky of vertical DWT decompositions is equal to 0 or 1 .

[0009] Between step b) and step c), an entropy coding of the data may be performed.

[0010] Said entropy coding may comprise an entropy coding of the bitplane count of groups of samples.

[0011] The step of grouping bitplane counts of groups of samples, and grouping data representing the bitplanes may be performed

[0012] In said sampling pattern, each pixel position may comprise samples arranged in a Bayer pattern, a Bayer pattern being, for each even i and j coordinates in the reference grid, a juxtaposition of a red pixel R at position (i,j), a first green pixel G1 at position (i,j+1 ), a second green pixel G2 at position (i+1 ,j), and a blue pixel B at position (i+1 , j+1 ), said decorrelative transform producing a component image CO for the positions of the red pixels, a component image C1 for the positions of the first green pixels, a component image C2 for the positions of the second green pixels, and a component image C3 for the positions of the blue pixels.

[0013] In said sampling pattern, each pixel position may comprise samples arranged in a 4:4:4 pattern, a 4:4:4 pattern being, for each i and j coordinates in the reference grid, a superposition at position (i,j) of a luminance pixel Y, a blue chrominance pixel Cb, and a red chrominance pixel Cr, or of a red pixel R, a green pixel G, and a blue pixel B, said decorrelative transform producing a component image CO for the positions of the luminance or red pixels, a component image C1 for the positions of the blue chrominance or green pixels, and a component image C2 for the positions of the red chrominance or blue pixels.

[0014] In said sampling pattern, each pixel position may comprise samples arranged in a 4:2:2 pattern, a 4:2:2 pattern being, for each i and each even j coordinates in the reference grid, an arrangement at position (i,j) of a luminance pixel Y, a blue chrominance pixel Cb, and a red chrominance pixel Cr, and a luminance pixel Y at position (i,j+1 ), said decorrelative transform producing a component image CO for the positions of the luminance pixels, a component image C1 for the positions of the blue chrominance pixels, and a component image C2 for the positions of the red chrominance pixels.

[0015] The decorrelative transform may be an identity mapping, where each sample of the colour image is mapped to a corresponding position on the component images.

[0016] The decorrelative transform may also be an in-line or a full Star- Tetrix transform. The in-line and full Star-Tetrix transforms are described in Annex F of Reference 1 .

[0017] According to a second aspect of the invention, there is provided a device for compressing a colour image comprising colour image samples Sij arranged in a reference grid according to a sampling pattern, the reference grid comprising a number of lines and columns, the line index i starting at 0 and increasing with the lines, and the column index j starting at 0 and increasing with the columns, into a packed data set, comprising: a) electronic components configured for performing step a) of the compression method; b) electronic components configured for performing step b) of the compression method; c) electronic components configured for performing step c) of the compression method.

[0018] According to a third aspect of the invention, there is provided a computer program comprising instructions which, when the program is executed by a computer, causes the computer to carry out the steps of the method for compressing of the invention. [0019] According to a fourth aspect of the invention, there is provided a packed data set obtainable by compressing a colour image with the method for compressing of the invention, the packed data set comprising a sequence of packets with the following characteristics:

- each packet containing only data from a single decomposed component image;

- each packet containing only data from a single line of a decomposed component image;

- all packets containing data from a given line of a decomposed component image being located in the packed data set before any packet containing data from a subsequent line of any decomposed component image.

[0020] According to a fifth aspect of the invention, there is provided a method for decompressing a packed data set obtainable by performing a method for compressing of the invention, comprising the steps of:

- unpacking the packets comprised in the packed data set;

- performing an inverse DWT decomposition;

- performing an inverse decorrelative transform.

[0021] According to a sixth aspect of the invention, there is provided a device for decompressing a packed data set obtainable by performing a method for compressing of the invention, comprising electronic components configured for:

- unpacking the packets comprised in the packed data set;

- performing an inverse DWT decomposition;

- performing an inverse decorrelative transform.

[0022] According to a seventh aspect of the invention, there is provided a computer program comprising instructions which, when the program is executed by a computer, causes the computer to carry out the steps of the method for decompressing of the invention.

Short description of the drawings [0023] These and further aspects of the invention will be explained in greater detail by way of example and with reference to the accompanying drawings in which:

Fig.1 represents schematically a portion of a colour image, the sampling pattern being a Bayer pattern;

Fig.2a represents schematically the component images resulting from a decorrelative transform of the colour image of Fig.1 , where each component sample Ckij is represented at the position (i,j) in the reference grid wherefrom it is determined;

Fig.2b represents schematically the component images resulting from a decorrelative transform of the colour image of Fig.1 , where component image samples are grouped in component images;

Fig.3 represents schematically the same portion of the component images as in Fig.2 after application of a single vertical DWT decomposition on component images CO, C1 and C3, and no decomposition for component image C2, where each component sample Ckij is represented at the position (i,j) in the reference grid wherefrom it is determined;

Fig.4 represents schematically the same portion of the component images as in Fig.3 after application of a single horizontal DWT decomposition on component images CO, C1 and C3, and no decomposition for component image C2, where each component sample Ckij is represented at the position (i,j) in the reference grid wherefrom it is determined;

Fig.5 represents schematically the same portion of the component images as in Fig.3 after application of a single horizontal DWT decomposition on component images CO, C1 and C3, and no decomposition for component image C2, where component image samples are grouped in component images and where samples are grouped by subband in each component image; only data for component image CO is represented;

Fig.6 represents schematically how a DWT decomposition performs a lifting scheme with (5,3) filter-bank on a line of samples; Fig.7 represents schematically subbands produced after no vertical DWT decomposition followed by 5 horizontal DWT decompositions of a component image;

Fig.8 represents schematically subbands in the case where component images CO, C1 and C3 are decomposed as in Fig.7 and component image C2 is not decomposed, and numbered arrows delimiting the samples for forming packets and serializing them in the packed data set according to the invention;

Fig.9 represents schematically subbands produced after one vertical DWT decomposition followed by 5 horizontal DWT decompositions of a component image;

Fig.10 represents schematically subbands in the case where component images CO, C1 and C3 are decomposed as in Fig.9 and component image C2 is not decomposed, and numbered arrows delimiting the samples for forming packets and serializing them in the packed data set according to the invention;

Fig.11 represents schematically subbands in the case where component images CO, C1 and C3 are decomposed as in Fig.9 and component image C2 is not decomposed, and numbered arrows delimiting the samples for forming packets and serializing them in the packed data set according to the JPEG XS standard;

Fig.12 represents schematically subbands produced in the case of a 3- component image with a 4:4:4 sampling pattern for 1 vertical and 5 horizontal DWT decompositions performed on each component image, representing how the packets are formed and serialized in the packed data set according to the JPEG XS standard (top) and the invention (bottom);

Fig.13 represents schematically subbands produced in the case of a 3- component image with a 4:2:2 sampling pattern for 1 vertical and 5 horizontal DWT decompositions performed on each component image, representing how the packets are formed and serialized in the packed data set according to the JPEG XS standard (top) and the invention (bottom); Fig.14 depicts a device for compressing a colour image;

Fig.15 depicts a device for decompressing a colour image.

Fig.16 panel a) represents schematically the grouping of samples from the first four lines of the image of Fig.4 in packets and panel b) represents schematically the order in which said packets are appended to the packed data set according to the progression order of the invention.

Fig.17 panel a) represents schematically the grouping of samples from the first four lines of the image of Fig.4 in packets and panel b) represents schematically the order in which said packets are appended to the packed data set according to the progression order of the JPEG XS standard.

Detailed description of embodiments of the invention

[0024] Fig.1 is a schematic example of a colour image obtained from an image sensor having a Bayer pattern. The reference grid represented by the squares comprises eight columns, for column index j going from 0 to 7 and six lines, for line index i going from 0 to 5, the top-left corner having coordinates (0,0). Each pixel position (i,j) may contain either a red, a first green, a second green, or a blue sample, according to the Bayer pattern. For example, reference sign 1 represents a blue sample at reference grid position (1 ,1 ) and reference sign 3 represents a red sample at reference grid position (2,4). In a real image, the number of columns and lines may be larger, such as 4096 x 2160.

[0025] Fig.2a is a representation of the data obtained after a decorrelative transform, where the red, the first green, the second green, and the blue image samples are replaced by samples of component images CO, C1 , C2, and C3, respectively. As depicted on Fig.2a, the decorrelative transform may produce a grid of component samples of same dimensions as the reference grid. The transform yielding a given component sample is centered on the corresponding colour image sample. Therefore, the spatial arrangement of input samples is preserved through the transform.

Here is a non-exhaustive list of such decorrelative transforms: - identity mapping, where each sample of the input grid is mapped to the corresponding position on the output grid, i.e. for any position (i,j) in the input/output grid, COij = Rij; C1 ij = G1 ij; C2ij = G2ij; C3ij = Bij;

- in-line Star-Tetrix transform;

- full Star-Tetrix transform.

The in-line Star-Tetrix transform and full Star-Tetrix transform are described in Annex F of Reference 1 . In Fig.2a, the samples of the component images are located in the reference grid at the positions of the corresponding colour image samples. In Fig.2b, four distinct component images CO, C1 , C2, and C3 are represented not in relation to the reference grid. However, with the knowledge of the sampling pattern and the decorrelative transform, one may determine the position (i,j) in the reference grid wherefrom a given component sample of the component images derives. For example, reference sign 2 of sample C2 on Fig.2b corresponds to sample of component image C2 at reference grid position (3,2) on Fig.2a.

[0026] Fig.3 is a representation of the data obtained after the application of a single vertical DWT decomposition on component images CO, C1 and C3, and no decomposition for component image C2. For component images CO, C1 , and C3, a subband LLO and a subband LHO are created. For component images CO and C1 , the subband LLO appears on grid lines 0 and 4, and the subband LHO appears on grid line 2. For component image C3, these subbands appear one line below. For component image C2, no DWT decomposition is performed, and therefore the original samples of the component image C2 are retained. Meaning of subscripts is as follows: LL = low frequency subband; LH = high frequency subband; E = even line; 0 = odd line.

[0027] Fig.4 is a representation of the data obtained after the application of one horizontal and one vertical DWT decomposition on component images CO, C1 and C3, and no decomposition for component image C2, i.e. after the application of a single horizontal DWT decomposition on component images CO, C1 and C3 on the data of Fig.3. LLO subband from each component image is decomposed into a new LLO and a HLO subband. LHO subband from each component image is decomposed into a new LHO and a HH0 subband. Each sample is located at the position (i,j) in the reference grid wherefrom it is determined.

[0028] Fig.5 is a representation of the data obtained after the application of one horizontal and one vertical DWT decomposition on component images CO, C1 and C3, and no decomposition for component image C2, i.e. after the application of a single horizontal DWT decomposition on component images CO, C1 and C3 on the data of Fig.3, where samples are first grouped by component, and then all samples from a given component are grouped by subband. Only data for component 0 is represented in this Figure. A LLO subband appears on the four positions in the upper left corner. A HLO subband appears on the four positions in the upper right corner. A LHO subband appears on the left-hand side of the bottom line, and a HHO subband appears on the right-hand side of the bottom line. A similar representation can be derived for components 1 and 3. For the non-decom posed component 2, the image is the C2 part of Fig.2b. Fig.5 presents a decomposed component image for component 0.

[0029] In performing the decorrelative transforms in step a) of the method, and the successive DWT decompositions in step b) of the method, it is required that the samples resulting from the transforms and decompositions keep a reference to the position (i,j) in the reference grid wherefrom they are determined. This may be achieved in different ways; three possible embodiments are described below:

- in a first embodiment, each sample resulting from a transform or decomposition may be stored together with the values (i,j) of the position in the reference grid wherefrom it is determined. This is done for each operation of the decorrelative transforms, as well as for each successive DWT decomposition.

- in a second embodiment, the resulting samples are memorized grouped by component and by subband, as represented in Fig.5.

- in a third embodiment, the samples resulting from a transform or decomposition are stored in an array at the position (i,j) in the reference grid wherefrom they are determined, i.e. at same line and column coordinates as the original data. Said array may be of same dimensions as the original colour image, or may even be the original colour image itself, the transforms and decompositions being then the so-called ‘in- place’ transforms and decompositions. Such an array of same dimensions as the original colour image is represented in Fig.2a, Fig.3, Fig.4, and Fig.16a.

According to said embodiments, step c) of the method is performed as follows:

- in the first embodiment, step c) of the method is performed by extracting from the storage the samples corresponding to each consecutive line of the reference grid and each component;

- in the second embodiment, step c) of the method is performed on the basis of address translation between the addresses in the storage, and positions (i,j) in the reference grid. For example, if the colour image is the image of Fig.1 : o it may be determined that pixel at position (1 , 1 ) of the colour image (reference sign 1 on Fig.1 ) corresponds to the 30 ^th sample after the beginning of the component images of Fig.2b (reference sign 2 on Fig.2b); o a similar correspondence may be determined for the storage of the subbands on Fig.5: pixel at position (2,4) of the colour image (reference sign 3 on Fig.1 ) corresponds to sample at position (2,4) of the image after DWT decomposition (reference sign 4 on Fig.4, sample of component image CO of subband LHO), and also corresponds to sample at position (2,1 ) of the subbands image for component image CO (reference sign 5 on Fig.5);

- in the third embodiment, step c) of the method is performed by scanning the storage line-by-line. For example, if the colour image is the image of Fig.1 , step c) may be performed by scanning the array in Fig.4 line-by- line.

[0030] Fig.16a provides a representation of the data obtained after the application of a series of image transforms on an example colour image comprising colour image samples arranged in a reference grid and obtained from an image sensor having a Bayer pattern, as illustrated in the first four lines of the image of Fig.1 . Said series of image transforms comprises: a) decorrelative transforms on said colour image, producing component images CO, C1 , C2 and C3, as illustrated in the first four lines of the image of Fig.2a and as performed in step a) of the method for compressing of the invention; b) one vertical and one horizontal DWT decomposition on said component images CO, C1 and C3, and no decomposition on said component image C2, producing decomposed component images, as illustrated in the first four lines of the image of Fig.4 and as performed in step b) of the method for compressing of the invention.

Each sample is located at the position (i,j) in the reference grid wherefrom it is determined.

Fig.16a and Fig.16b together provide a representation of the execution of step c) of the method for compressing of the invention, with Fig.16a illustrating step (i) and Fig.16b illustrating step (ii): c) for each consecutive line i of the reference grid, for each component k,

(i) grouping all samples of said component k in said line i in one or more packets;

(ii) appending said one or more packets to the packed data set.

The first line of the reference grid is first scanned. Four samples of component 0, C0_LL0, C0_HL0, C0_LL0, C0_HL0, are found and grouped in one or more packets. In the example, a single packet, packet 0, is formed and appended to the packed data set. Four samples of component 1 are subsequently found on said first line and processed similarly for forming packet 1 . No other component is found on the first line of the reference grid. Then, the second line of the reference grid is scanned, and packet 2 and packet 3 are formed from the samples of components 2 and 3 in a similar way. This is performed until all lines of the reference grid have been processed and a packed data set containing 8 packets is obtained.

The order in which said packets are formed and appended to a packed data set is according to the progression order of the invention: for each consecutive line i of the reference grid, for each component k, grouping all samples of said component k in said line i in one or more packets; appending said one or more packets to the packed data set.

For each line of the reference grid, the background fill of a cell of the reference grid indicates the packet into which the sample associated with the cell is grouped. The number associated with a packet indicates the serialization order of the packet in the packed data set, a packet associated with a lower number being appended first in the packed data set.

This example applies to a four-component image having four lines and eight columns, whereon a single vertical and a single horizontal DWT decomposition are performed on three component images, and no decomposition on one component image. Other examples, with images having a different number of components, lines, or columns, and whereon a different number of vertical or horizontal DWT decompositions are performed will show similar properties, included the differences with respect to the JPEG XS standard discussed below.

In this example, step (i) of the method produces a single packet for each component k present in each line i of the reference grid. In another example of the method, a plurality of packets may be produced in step (i) for each component k present in each line i of the reference grid. For example, packet 0 on Fig.16a may be replaced by two packets, comprising respectively samples C0_LL0, C0_LL0 in a first packet, and samples C0_HL0, C0_HL0 in a second packet.

Fig.16a provides an equivalent description of the progression order of the invention as Fig.10, with a notable difference that component images CO, C1 and C3 are decomposed with a single vertical and a single horizontal DWT decomposition in Fig.16a versus a single vertical and five horizontal DWT decompositions in Fig.10. Another difference is that in Fig.16a samples are represented in the reference grid, whereas in Fig.10 samples are grouped by component and by subband.

[0031] Fig.17a provides a representation of the data obtained after the application of the same image transforms on the same colour image as represented in Fig.16a. Each sample is located at the position (i,j) in the reference grid wherefrom it is determined. Fig.17 further provides a representation of the grouping of said data in packets (Fig.17a) and the order in which said packets are appended to a packed data set (Fig.17b) according to the progression order of the JPEG XS standard, as adapted from Table B.10 in Reference 1 . A difference between Table B.10 from Reference 1 and Fig.17 is the number of horizontal DWT decompositions for the decomposed component images: 5 horizontal DWT decompositions for Table B.10 versus 1 horizontal DWT decomposition for Fig.17.

The background fill of a cell of the reference grid indicates the packet into which the sample associated with the cell is grouped. The number associated with a packet indicates the serialization order of the packet in the packed data set, a packet associated with a lower number being appended first in the packed data set.

One can observe several differences between packets formed according to the progression order of the JPEG XS standard, as illustrated in Fig.17, and packets formed according to the progression order of the invention, as illustrated in Fig.16. A first difference is that a packet may contain samples from several components according to the progression order of the JPEG XS standard, whereas a packet only contains samples from a single component according to the progression order of the invention. A second difference is that a packet may contain samples from several lines of the reference grid according to the progression order of the JPEG XS standard, whereas a packet only contains samples from a single line of the reference grid according to the progression order of the invention. These differences cause the method of the invention to have advantages with respect to the method of the JPEG XS standard. These advantages are discussed below.

Fig.17a provides an equivalent description of the progression order of the JPEG XS standard as Fig.11 , with a notable difference that component images CO, C1 and C3 are decomposed with a single vertical and a single horizontal DWT decomposition in Fig.17a versus a single vertical and five horizontal DWT decompositions in Fig.11 . Another difference is that in Fig.17a samples are represented in the reference grid, whereas in Fig.11 samples are grouped by component and by subband. [0032] Fig.6 depicts how a horizontal DWT decomposition is performed on a line of samples containing a sequence of even (E) and odd (0) samples by decomposing said sequence in a high-pass (H) subband and a low-pass (L) subband. Odd samples are first replaced by performing a lifting scheme with (5,3) filter-bank on said line of samples, the resulting odd numbered samples being a H (high spatial frequency) subband, according to the equation

H _i = Oi -1/2 * (E _i .i + E _i+ i).

Even samples are then replaced by performing the operation

L _i = E _i +1/4 * (H _i .i + H _i+ i), producing a L (low spatial frequency) subband. As is well known, formulas where odd and even samples are interchanged in the above formulas are equivalent. When samples outside the image are needed, the corresponding samples at mirror position with respect to the border, inside the image, are taken. This results in the factor +1/2 instead of +1/4 at the left, below in the diagram (arrow going from H to L), and a factor -1 instead of -1/2 at the right, above in the diagram (arrow going from E to H).

[0033] In Fig.8, Fig.10, Fig.11 , Fig.12, and Fig.13, numbered arrows represent packets and associated numbers indicate serialization order in the packed data set, i.e. the order in which packets are appended to the packed data set; a packet associated with a lower number is serialized before a packet associated with a bigger number. A packet regroups all samples from all subbands covered by the corresponding arrow starting from the tail of the arrow to its head.

[0034] Fig.14 depicts a device 20 for compressing a colour image, comprising a module 21 for performing decorrelative transforms on said colour image producing component images, a module 22 for performing DWT decompositions on said component images producing decomposed component images, and a module 23 for packetizing said decomposed component images producing a packed data set. Each of these modules may comprise hardware such as gates and registers, FPGAs, ASICs and/or microcontrollers.

[0035] Fig.15 depicts a device 30 for decompressing a packed data set, comprising a module 31 for depacketizing said packed data set producing decomposed component images, a module 32 for performing inverse DWT decompositions on said decomposed component images producing component images, and a module 33 for performing inverse decorrelative transforms on said component images producing a colour image. Each of these modules may comprise hardware such as gates and registers, FPGAs, ASICs and/or microcontrollers.

Example 1

[0036] In a first example embodiment of the invention, the colour image is sampled according to a Bayer pattern. In step b) of the method, the number of vertical decompositions Nlky is equal to zero for all component images, and the number of horizontal decompositions Nlkx is equal to 5 for component images CO, C1 and C3, and is equal to zero for component image C2. Fig.7 is a representation of the subbands resulting from this decomposition, for component images CO, C1 and C3. Fig.8 represents the four decomposed component images together. According to the invention, successive packets are formed containing the samples corresponding to the four numbered arrows, marked “0”, “1”, “2”, and “3”, representing the result of the performance of step c) of the method. The first line of the reference grid is analysed and comprises components 0 and 1 . All samples of component 0 are represented as arrow “0”. These samples are packed in the packed data set as packet “0”. The samples of component 1 (arrow “1”) are then packed in the packed data set as packet “1”. The second line of the reference grid is analysed in the same way, comprising samples of components 2 and 3 corresponding to arrows “2” and “3”, which are packed in packets “2” and “3”, respectively. Subsequent lines of the reference grid are processed in the same way up to the last line of the reference grid.

Example 2

[0037] In a second example embodiment of the invention, the colour image is also sampled according to a Bayer pattern. The number of vertical decompositions Nlky is equal to one for component images CO, C1 and C3, and is equal to zero for component image C2, and the number of horizontal decompositions Nlkx is equal to 5 for component images CO, C1 and C3, and is equal to zero for component image C2. Fig.9 is a representation of the subbands resulting from this decomposition, for component images CO, C1 and C3. Fig.10 represents the four decomposed component images together.

Arrows numbered “0”, to “7” represent the result of the performance of step c) of the method.

The first line of the reference grid is analysed and comprises components 0 and 1 wherein samples of arrows “0” and “1” are treated, and packed in the packed data set as packets “0” and “1”, respectively.

The second line of the reference grid is analysed and comprises components 2 and 3 wherein samples of arrows “2” and “3” are treated, and packed in the packed data set as packets “2” and “3”, respectively.

The third line of the reference grid is analysed and comprises components 0 and 1 wherein samples of arrows “4” and “5” are treated, and packed in the packed data set as packets “4” and “5”, respectively.

The fourth line of the reference grid is analysed and comprises components 2 and 3 wherein samples of arrows “6” and “7” are treated, and packed in the packed data set as packets “6” and “7”, respectively.

Subsequent lines of the reference grid are processed in the same way up to the last line of the reference grid.

Fig.11 represents schematically subbands in a case similar to Fig.10, but where the numbered arrows delimiting the samples for forming and serializing packets are determined according to the JPEG XS standard.

One can observe that in the method of the invention, a given packet always contains samples from a single component, whereas in the JPEG XS standard, a packet may contain samples from different components.

Example 3

[0038] In a third example embodiment of the invention, the colour image is a 3-component image sampled according to a 4:4:4 sampling pattern where 1 vertical and 5 horizontal DWT decompositions are performed on each component image. Fig.12 represents schematically the subbands resulting from said DWT decompositions for all three component images; it depicts in the upper part the packets that are formed and serialized in the packed data set according to the JPEG XS standard, and in the lower part the packets that are formed and serialized in the packed data set according to the method of the invention. As in Fig.8, the numbered arrows represent the grouping of component samples in packets and the order in which said packets are appended to the packed data set.

Example 4

[0039] In a fourth example embodiment of the invention, the colour image is a 3-component image sampled according to a 4:2:2 sampling pattern where 1 vertical and 5 horizontal DWT decompositions are performed on each component image. Fig.13 represents schematically the subbands resulting from said DWT decompositions for all three component images; it depicts in the upper part the packets that are formed and serialized in the packed data set according to the JPEG XS standard, and in the lower part the packets that are formed and serialized in the packed data set according to the method of the invention. As in Fig.8, the numbered arrows represent the grouping of component samples in packets and the order in which said packets are appended to the packed data set.

[0040] For comparing how packets are formed and appended to the packed data set in the JPEG XS standard and in the present invention, Table 1 shows subbands numbering for an image with 3 components where each component image is decomposed by 1 vertical and 5 horizontal DWT decompositions. This applies for the images of Fig.12 and Fig.13. This is also applicable for a 4-component image in which 3 component images are decomposed by 1 vertical and 5 horizontal DWT decompositions, and the 4th component image is not decomposed, as is the case for Fig.10 and Fig.11 .

Table 1

Table 2 shows the JPEG XS progression order for the images of Fig.12 (top) and Fig.13 (top), as adapted from Table B.6 in Reference 1. The packet index corresponds to the numbered arrows of Fig.12 (top) and Fig.13 (top).

Table 2

Table 3 shows the progression order for the images of Fig.12 (bottom) and

Fig.13 (bottom), as resulting from the method of the invention. The packet index corresponds to the numbered arrows of Fig.12 (bottom) and Fig.13 (bottom).

Table 3

Table 4 shows the JPEG XS progression order for a 4-component image in which 3 component images are decomposed by 1 vertical and 5 horizontal DWT decompositions and the 4th component image is not decomposed. Adapted from Table B.10 in Reference 1. The undecomposed component image is treated as a single subband with 2 subband lines. Each subband line of the undecomposed component image is included in its own packet. See Table 1 for subbands numbering. The packet index corresponds to the numbered arrows of Fig.11 .

Table 4

Table 5 shows the progression order resulting from the invention in the same case as for Table 4, i.e. a 4-component image in which 3 component images are decomposed by 1 vertical and 5 horizontal DWT decompositions, and the 4th component image is not decomposed. The packet index corresponds to the numbered arrows of Fig.10.

Table 5

[0041] Table 6 provides a comparison of packets properties between the JPEG XS standard and the present invention in the cases addressed in the previous paragraph. Table 6

[0042] The compression method according to the invention differs and offers advantages with respect to the method of the JPEG XS standard in several aspects detailed hereafter. First, more packets may be generated according to the invention than in the prior art solution, while keeping the maximal packet size smaller than or equal to the minimal packet size in the prior art solution (see Table 6). Having smaller packets allows a reduction of the size of packet buffers. A packet buffer is a storage that accumulates a whole packet before making it available at its output, such buffer being needed for instance in order to smooth the data rate of the packed data set. However, smaller packets in a given packed data set means more packets, which increases the fixed cost needed to keep track of packet-related metadata in the packed data set, such as the position and size of each packet, which may take the form of packet headers, as in the JPEG XS standard. Said metadata does not contain any information related to the original colour image. Therefore, more metadata means less image quality for a given size of the packed data set. The invention allows a good trade-off between the size and the number of packets, allowing a reduction of the size of the packet buffers while keeping roughly the same amount of packet-related metadata compared to the prior art.

Second, according to the invention, a given packet contains only samples from a single line of a given decomposed component image, and therefore said packet contains only samples derived from a single line of the original colour image. In comparison, the JPEG XS standard prescribes that any packet containing samples from any DWT decomposed component image must contain samples from all DWT decomposed component images. Therefore, such packet may contain samples derived from several lines of the original colour image, for instance in the case of Bayer sampling pattern (see Fig.11 ). Having only samples derived from a single line of the original colour image in a given packet allows a reduction of the size of reorder buffers compared to the prior art. A reorder buffer is a storage that accumulates samples after forward decorrelative transforms and DWT decompositions at encoder, and outputs them in an order suitable for the inverse decorrelative transforms and DWT decompositions at decoder. Indeed, due to algorithmic dependence between DWT decompositions, samples resulting from a higher number of DWT decompositions are produced at encoder after samples resulting from a lower number of DWT decompositions. However, such samples resulting from a higher number of DWT decompositions are needed at decoder before samples resulting from a lower number of DWT decompositions. Therefore, reordering of samples is needed between encoder and decoder, which is performed by means of a reorder buffer. An example of reduction of the reorder buffer size in the case of Bayer sampling pattern is described hereafter. In the JPEG XS standard, a reorder buffer needs to accumulate at least two lines of data in order to output a given packet, because of the need to include samples from all DWT decomposed component images inside the packet, samples from one of these DWT decomposed component images being available only after two lines of data have been processed by the DWT decompositions. In addition, the fact that samples from the undecomposed component images are packed after samples from the decomposed component images means that they must be stored in the reorder buffer until all samples from decomposed component images have been processed. In comparison, according to the invention, a reorder buffer must only accumulate a single line of data because a given packet only contains data derived from a single line of the original colour image. In addition, samples from undecomposed component images are treated the same way as samples from the decomposed component images, therefore avoiding the need to store them until all samples from decomposed component images have been processed. These two features allow a reduction of the reorder buffer size compared to the prior art.

Finally, in a hardware implementation of the method in the form of a hardware device, a reduction of buffer sizes allows a reduction of the die area and thus a reduction of the fabrication cost of the hardware device, thereby providing a more efficient method for compressing and decompressing of a colour image. In a software implementation of the method in the form of a computer program, a reduction of buffer sizes allows a reduction of the memory usage and thus a reduction of the cost and power consumption of the computer executing the computer program, thereby also providing a more efficient method for compressing and decompressing of a colour image.

[0043] The drawings of the figures are neither drawn to scale nor proportioned. Generally, identical components are denoted by the same reference numerals in the figures.