The present invention relates to a computer-implemented system for image quality enhancement and color spaces conversion for manufactured surfaces printers, particularly for tiles decoration printers.

Background Art

It is known that color management is the process that controls how color is used and represented across different devices, such as monitors, scanners or printers. It often happens that the same color has a different outlook depending on the device is being visualized: color management tries to mitigate this effect.

The general principle, common to the various digital color management technologies, consists in assigning to each peripheral (monitor, scanner, printer) a so-called color profile that indicates its chromatic characteristics. Each image produced by a scanner or digital camera is associated with the color profile of the peripheral, which associates the relative chromatic coordinates to the peripheral coordinates. When the image is sent to a monitor or printer, the color management system calculates a color conversion between the input image profile and the output device profile so that the colorimetric coordinates of the input colors match the colorimetric coordinates of the output device. This ensures that the original colors and the colors of the printed or displayed image match.

It is also known that is mandatory to operate the correct management of colors for manufactured surfaces designing and decoration, because manufactured surfaces must have very specific properties, depending on the context and situation they will be used; for instance, colors should be warmer, brighter, colder or more vivid.

Concerning manufactured surfaces designing and decoration, the process starts from the scan of very large surfaces (from marble to wood to abstract surfaces) generating super high resolution RGB images, usually saved in tiff format. Then the obtained images are manipulated inside Photoshop: first they are converted in psb proprietary format using custom color profiles necessary to communicate with industrial printers, second these converted images are enhanced in a way that the output outlook after printing is as better as possible, depending on the context. Custom color profiles play a key role in this procedure because they affect the perceptual aspect of colors in different surfaces and are necessary to the printers they work with: instead, these specific printers work with particular multichannel output color profiles that map images in multichannel spaces, from the well-known CMYK to more complex five or six-channel spaces.

The conversion of images in these multichannel spaces is lossy, meaning that often it cannot reproduce the full range of color present in the initial high resolution RGB scan. The sensibility and experience of the designer are required to try to preserve and enhance color properties as they are asked from the customers. The process of conversion is automatized by Photoshop but the most difficult part, the enhancement, it is manually done through complex procedures like simulating the actual rendering of the surface once it will be printed in production.

Therefore, there was the need for a semi-automatic tool that would take care of the enhancement process, leaving the conversion to Photoshop.

Description of the Invention

The main aim of the present invention is to provide a computer-implemented system for image quality enhancement and color spaces conversion for manufactured surfaces printers that allow to automatically perform the enhancement and conversion process, effectively, quickly and reliably, in order to obtain images printed on the manufactured surface that are as close as possible to the initial scanned images considering the available color profile.

The above-mentioned objects are achieved by the present computer- implemented system for image quality enhancement and color spaces conversion for manufactured surfaces printers according to the features of claim 1. Brief Description of the Drawings

Other characteristics and advantages of the present invention will become better evident from the description of a preferred, but not exclusive embodiments of a computer-implemented system for image quality enhancement and color spaces conversion for manufactured surfaces printers, illustrated by way of an indicative but non-limiting example in the accompanying Figures, in which: Figures 1 shows a first possible embodiment of the computer-implemented system according to the invention;

Figures 2 shows a second possible embodiment of the computer-implemented system according to the invention;

Figures 3 shows a third possible embodiment of the computer-implemented system according to the invention.

Embodiments of the Invention

With particular reference to such illustrations, globally indicated with reference 1 is a computer-implemented system for image quality enhancement and color spaces conversion for manufactured surfaces printers.

Particularly, the computer-implemented system 1 can be used for tiles decoration printers.

However, it is not excluded the use of the method for the decoration of different types of manufactured surfaces.

Preferably, all the convolutional networks used in the computer-implemented system 1 are based over U-Net. However, the use of different network is not excluded.

As known, U-Net is a fully convolutional network (FCN) that uses only locally connected layers such as convolution, pooling and upsampling. Avoiding dense layers means both less parameters and the ability to work with variable image sizes, given that all connections are local.

The U-Net network consists of two paths:

- Contracting path: by decreasing the spatial dimension and increasing channels dimension the information on the global context is maintained via a compact feature map; - Expansive path: information on low-level features, which help in the precise localization of details, is reported thanks to the skip connections.

In order to properly train a deep neural network used by the computer- implemented system 1 according to the invention, there was the need to create a training dataset with as many images as possible.

The provided training dataset comprises images constituted by high resolution surfaces coming from marbles, woods, cements and others hand-drawn abstract surfaces.

Particularly, the computer-implemented system 1 comprises a training dataset with images divided in three groups:

- RGB images: initial images that have to be enhanced with a specific color profile.

- Converted images: starting from each of the RGB images, a converted image is given by applying the color profile inside Photoshop, with no other actions. According to the present invention, the converted images are obtained from the RGB images by means of dedicated libraries and without the use of Photoshop.

- Enhanced images for manufactured surfaces printers: for each of the RGB images an enhanced version of it was provided. These are images which were first converted then manually enhanced by a graphic designer in order to improve visual aspect and highlight the desired detail or color (for instance, highlighting the veins in a marble slab or accentuating shades of a certain color).

The concepts implemented by the computer-implemented system 1 can be mathematically formalized as follows.

A surjective non-invertible conversion function ƒ corresponds to the application of a color profile inside Photoshop. Given a [R, G, B] triple, the conversion function ƒ returns a n-tuple [C1;C2; ... ;Cn], where n is the number of output channels defined by the color profile.

Actually, the conversion carried out by Photoshop comprises a source space profile (in most cases Adobe RGB 1998) and a destination space profile involved using the CIELAB connection space to make the conversion. Inside each ICC profile Photoshop can find lookup tables that explain the mapping between the profile space and the CIELAB space and vice versa, so between the CIELAB space and the profile space. For instance, converting from multichannel space to Adobe RGB 1998 involves first the lookup table multichannel to CIELAB, then CIELAB to RGB.

An enhancement function g, which represents the enhancement work carried out by the graphic designer once the image has been converted in a multichannel space.

The process implemented by the computer implemented system 1 can be formalized as follows: where: x' is the original image in RGB space; x" is the converted image in multichannel space; is the enhanced image in multichannel space that can be used in manufactured surfaces printers; ƒ is the conversion function ƒO') = x"; g is the enhancement function

Particularly, according to the invention, the computer-implemented system 1 comprises at least a convolutional neural network configured for executing at least the following steps:

- receive in input an original image x' in RGB space;

- from the original image x' , elaborate an approximation in multichannel space or in RGB space of the enhanced image for manufactured surfaces printers;

- return to output the approximation of the enhanced image According to a first possible embodiment, schematically showed in Figure 1, the computer-implemented system 1 comprises a convolutional neural network 2 implementing an approximation of the conversion function ƒ and an approximation of the enhancement function g, wherein the conversion function ƒ is a function that converts an original image x' in RGB space in a converted image x" in multichannel space [C1;C2; ... ;Cn], where n is the number of output channels defined by a color profile, and wherein the enhancement function g represents the enhancement work carried out by a graphic designer on the converted image x" in the multichannel space for obtaining an enhanced image in multichannel space for the use with manufactured surfaces printers.

Furthermore, the convolutional neural network 2 is configured for executing the following steps:

- receive in input an original image x' in RGB space (step 21);

- elaborate an approximation x" of the converted image x" in multichannel space from the original image x' by means of the approximation of the conversion functionƒ, wherein (step 22);

- elaborate an approximation of the enhanced image in multichannel space, from the approximation of the converted image x" and by means of the approximation g of the enhancement function g, wherein (step 22);

- return to output the approximation of the enhanced image (step 23).

Furthermore, according to the first embodiment, the computer implemented system 1 is configured for executing a loss function used to train the network between ground truth enhanced image x" and predicted enhanced image (step 24).

Therefore, according to the first embodiment, both ƒ and g are approximated with a deep neural network 2 that takes as input an original image x' in RGB space and outputs an approximation of the enhanced image in multichannel space.

The model implemented by the convolutional neural network 2 according to the first embodiment can be summarized by the following expression: According to a second possible embodiment, the computer-implemented system 1 comprises a conversion unit 3 configured for execute the following steps:

- receive in input an original image x' in RGB space (step 31), and

- elaborate a converted image x" in multichannel space from the original image x' by means of the conversion function ƒ wherein ƒ(x') = x" (step 32).

Furthermore, according to the second embodiment, the computer-implemented system 1 comprises a convolutional neural network 4 implementing an approximation of the enhancement function g, wherein the enhancement function g represents the enhancement work carried out by a graphic designer on the converted image x" in the multichannel space for obtaining an enhanced image in multichannel space for the use with manufactured surfaces printers. Particularly, the convolutional neural network 4 is configured for executing the following steps:

- receive in input a converted image x" in multichannel (step 41);

- elaborate an approximation of the enhanced image in multichannel space, from the approximation of the converted image x" and by means of the approximation g of the enhancement function g, wherein

- return to output the approximation of the enhanced image (step 43).

Furthermore, according to the second embodiment, the computer-implemented system 1 is configured for executing a loss function used to train the network between ground truth enhanced image and predicted enhanced image (step 44).

Therefore, according to the second embodiment, the conversion function ƒis not approximated because it is applied before giving the image as input to the convolutional neural network.

Particularly, according to a preferred embodiment, the conversion unit 3 is configured for determining a correspondence between each [R;G;B] triplet and multichannel n-tuple. For example, the conversion unit 3 is configured for determining a correspondence between each [R;G;B] triplet and [C1;C2;C3;C4] quadruple referred to the 4-channel profile or [C1;C2;C3;C4;C5] quintuple referred to the 5-channel profile.

Therefore, the conversion unit 3 is able to create a perfect correspondence between a RGB value and its converted multichannel value. Given a [R, G, B] in position (h, w) where 0 ≤ h < 4096 and 0 ≤ w < 4096, the converted value in a color space is found in the same position (h, w) in the converted image with the desired profile.

The only function that is approximated by means of the convolutional neural network 4 is the enhancement function g that takes as input a converted image x" in multichannel and outputs an approximation of the enhanced image in multichannel space.

The model implemented by the convolutional neural network 4 according to the second embodiment can be summarized by the following expression:

With reference to the first and second embodiments, the computer-implemented system 1 further comprises a transformation unit for implementing a fast transform between a multichannel color profile and RGB color profile. Particularly, according to the first and second embodiments as disclosed above, the computer-implemented system 1 provides an output in multichannel space which was not usable because of the inability to save a multichannel image in .psb format and open it in Photoshop.

Therefore, the transformation unit is configured for implementing a fast transform of said an approximation of said enhanced image in multichannel space into an RGB color profile.

Particularly, the transformation unit is configured for executing at least the following steps:

- convert each value of a source multichannel color profile to its corresponding CIELAB value through the multichannel-to-CIELAB color lookup table inside the source color profile;

- then, convert each CIELAB value to its corresponding [R, G, B] value through the CIELAB-to-RGB color lookup table inside the destination RGB color profile.

According to a third possible embodiment, the computer-implemented system 1 comprises a first convolutional neural network 5 implementing an enhancement function over RGB images h, configured for executing the following steps:

- receive in input an original image x' in RGB space (step 51);

- elaborate an RGB enhanced image x ₊ ' from the original image x' in RGB space and by means of the enhancement function over RGB images h, wherein

Furthermore, according to the third embodiment, the computer-implemented system comprises a second convolutional neural network 6 implementing an approximation with a differentiable model of a conversion functionƒ, wherein the conversion function ƒis a function that converts an original image x' in RGB space in a converted image x" in multichannel space [C1;C2; ... ;Cn], where n is the number of output channels defined by a color profile, and wherein the second convolutional neural network (6) is configured for executing the following steps:

- receive in input the RGB enhanced image (step 61)

- elaborate an approximation in RGB space of the enhanced image from the RGB enhanced image and by means of the approximation of the conversion functionƒ wherein

- return to output the approximation x" in RGB space of the enhanced image (step 63).

Furthermore, according to the first embodiment, the computer-implemented system 1 system is configured for executing a loss function , used to train the network between ground truth enhanced image and predicted enhanced image (step 64).

Therefore, according to the first embodiment, the first convolutional neural network 5 implements a new function h which applies an enhancement over an RGB image input image x' and outputs an RGB enhanced image

It is important to note that it is assumed that: meaning that swapping the two operation conducts to the same result. This approximation is forced to loss level.

Successively, the second convolutional neural network 6 applies the conversion function

Particularly, the approximation implemented by the second convolutional network 6 approximate the application of the conversion function ƒ with a differentiable model and outputs an RGB image that can be saved in different formats, with the property that, once applied the color profile inside Photoshop, the converted image will look as close as possible to the ground truth enhanced image in multichannel space.

Particularly, this model tackles the need to make a LUT (Lookup Table) differentiable and the conversion function ƒ is nothing but a LUT that, given an [R;G;B] value, returns a [C1;C2; ... ;Cn] multichannel value, according to a color profile. Intrinsically, a LUT is a non- differentiable step function with derivative 0 everywhere.

According to the third embodiment, two possible modes are defined:

- a training mode, where e is the approximation ofƒ, necessary to allow backpropagation of gradient and imported as a pretrained model with freezed weights;

- a test mode, where is no longer needed because true ƒ is directly applied inside Photoshop.

The better approximation of e the better results in test mode.

The model in training mode implemented by the convolutional neural networks 5, 6 according to third embodiment can be summarized by the following expression: where: h: enhancement function in RGB space; approximation of the conversion function; loss used in train mode. Require to be differentiable.

The model in testing mode implemented by the convolutional neural networks 5, 6 according to third embodiment can be summarized by the following expression: where: h: enhancement function in RGB space; ƒ: true conversion function; loss used in test mode.

In the following section is described how first, second and third embodiment can be implemented.

Functions and h have been approximated with a convolutional network based over U-Net.

With reference to the first embodiment, the convolutional neural network 2 of the computer-implemented system 1 has been implemented using a U-Net where the input is a RGB image and the output is a multichannel enhanced image.

For example, the network can be trained with 17264 train images (80% of the total) and 4316 evaluation images (20% of the total), using as train loss a weighted sum of SSIM and L2 defined as where:

- SSIM (Structural Similarity Index Measure): measurement of image quality that considers three key features, namely luminance, contrast and structure over a window of pixels;

- L2 (also called L2-norm or Mean Squared Error): pixel-by-pixel measure defined as the average squared difference between the estimated values and the actual value. This forces precision at pixel level.

With reference to the second embodiment, the computer-implemented system 1 has been essentially implemented in the same way of the first embodiment; the only difference is that the input of the network is an already converted image in the destination profile space, while the output remains a multichannel enhanced image.

With reference to the third embodiment, for the implementation of the differentiable function ƒ by the second convolutional neural network 6 is fundamental learning the conversion that is done inside Photoshop when a color profile is assigned. Once the learning it is done, the model is imported pre trained with freezed weights. In doing so, the network is able to simulate the application of the specific color profile and can adjust the RGB image giving it particular properties in such a way that, once the real color profile will be applied during usage phase, the image will become the enhanced one, instead of a simple converted one.

The model according the third embodiment, like the others, has been implemented with a U-Net.

In order to properly learn the LUT (Look Up Table) of each profile, i.e. the conversion RGB to MULTICHANNEL, the original dataset, used for training the other models, can be extended with images belonging to COCO dataset (Common Object in COntext) by Microsoft.

Next step is to create the actual correspondence between [R G;B] triplet and multichannel n-tuple, for example [C1;C2;C3;C4] quadruple referred to the 4 channels profile and [C1;C2;C3;C4;C5] quintuple referred to the 5 channels profile.

By doing so, it is possible to create a perfect correspondence between a RGB value and its converted multichannel value. Given a [R, G, B] in position (h, w) where 0 ≤ h < 4096 and 0 ≤ w < 4096, the converted value in color space C is found in the same position (h;w) in the converted image with the desired profile.

To properly train the model, for each RGB image, the ground truth is the converted image according to the LUT defined above.

Therefore, according to the third embodiment, the first convolutional neural network 5 is implemented with a U-Net which takes as input a RGB image and outputs a RGB enhanced image. The first convolutional neural network 5 is then concatenated with the second convolutional neural network 6, the pretrained LUT model with freezed weights, also based on U-Net, which takes as input the RGB enhanced image and outputs a converted enhanced image which should be as close as possible to the real one.

In testing mode, the second branch of the network is not used and an RGB enhanced image is output and saved; then the user can take this image and apply, inside Photoshop, the desired color profile.

In practice it has been found that the invention achieves the intended aim and objects.

In particular, the computer-implemented system according to the invention automatically perform image quality enhancement and color spaces conversion for manufactured surfaces printers, effectively, quickly and reliably, in order to obtain images printed on the manufactured surface that are as close as possible to the initial scanned images considering the available color profile.