FIELD OF THE INVENTION

The invention relates to a method for determining physicochemical properties of a sample, particularly to a method for analyzing spectral data obtained from a sample in order to obtain a measure of one or more of sample type, of sample quality, and of a presence or an amount of one or more constituents of the sample.

The invention further relates to a device configured to perform a method for determining physicochemical properties of a sample from analysis of spectral data.

BACKGROUND OF THE INVENTION

Multivariate methods such as Principal Component Analysis (PCA), Partial Least Squares (PLS) and Artificial Neural Networks (ANN) have been widely used to build qualification models and/or quantification/regression models (calibrations) using infrared (IR) and near infrared (NIR) spectra or other types of spectra for the purpose of determining physicochemical properties of a sample.

However, the development and maintenance of these models require the input of a skilled specialist, such as a chemometrician or spectroscopist. Further, with these methods it is still very difficult to combine data from different instrument platforms, operating across different spectra ranges, or from different product types or to identify the instrument platform of origin for a given set of data.

Before applying the multivariate methods, the specialist uses his/her/their domain knowledge and visual skills to identify signatures in the spectra to pre-process the spectra, select relevant variables, apply down/up-weighting schemes etc. The specialist can also make crude approximations of the qualification of the sample (e.g. food type) and/or quantity of the constituents by visual inspection of spectra generated from the sample.

However, approaches involving such skilled specialists are very time consuming, costly, may be subjective and are vulnerable to human error. Also, with such approaches it is even more time consuming and complicated to combine data from different instrument platforms and different product types, particularly due to such data having different wavelength ranges, originating from different sample matrices or the like.

Therefore, there is a desire to provide a method for determining physicochemical properties of a sample which is largely or fully independent of the use of the skilled specialist such as chemometricians and spectroscopists to establish and maintain multivariate models.

There is also a desire to provide a method for determining physicochemical properties of a sample through spectroscopy which method is capable of in a simple and fast manner combining data from different instrument platforms and different sample types, even if such data has different wavelength ranges or originates from different sample matrices or the like.

SUMMARY OF THE INVENTION

It is an object of the present invention to mitigate at least one of the aforementioned problems or meet at least in part a one of the aforementioned desires.

According to a first aspect of the invention there is provided a method for determining physicochemical properties of a sample, the method comprising the steps of:

- providing a spectral data representation of the sample, where the spectral data representation is a representation of spectral data obtained from a sample as an image,

- providing a data processing device in cooperation with a learning data architecture,

- accessing the spectral data representation by the data processing device, and

- analyzing, by means of the learning data architecture and the data processing device, the spectral data representation accessed by the data processing device to obtain a measure of one or more physicochemical properties of the sample.

Thereby, and in particular in virtue of the provision and use of a learning data architecture, a method is provided which is largely or fully independent of the use of skilled specialists, which is thus both more time efficient and cheap in use, and which may be objectively and consistently applied.

Furthermore, the use of a spectral data representation in the form of an image representation of spectral data obtained from the sample (as opposed to the conventional vector representation of such spectral data) as an input to the data processing device and the provision and use of a learning data architecture for analyzing the input spectral data representation provides for a method which is capable of using any image type and format as an input and which is thus capable of in a simple and fast manner combining data from different instrument platforms and different product types, even if such data has different wavelength ranges, matrices or the like, to identify the product type and composition.

Combination of data from different instrument platforms and different product types allows for a more precise and efficient classification and quantification of the sample as well as for the development of a truly global and unified application model platform.

Still further, it has turned out that providing the spectral data representation in the form of a representation of spectral data obtained from the sample as an image and employing a learning data architecture for analyzing this spectral data representation provides for a method which is even capable of identifying the instrument platform from which the input spectral data representation originates. Identification of the instrument platform from which the input spectral data representation originates may be achieved by training the learning data architecture using a training set of spectral data representations in which each representation is labelled with data identifying the instrument platform originating the spectral data. This capability is of great advantage in that it allows for identification of influences from the instrument platform on the spectral data and/or the sample, and thus in turn for an even more accurate and consistent classification and quantification of the sample.

In some embodiments the one or more physicochemical properties is selected from the group of physicochemical properties consisting of: sample type; sample quality; and a qualitative and/or quantitative measure of one or more constituents of the sample.

In some embodiments the sample is a foodstuff.

In some embodiments the method may comprise the further step of generating the spectral data representation of the sample. Alternatively, the spectral data representation of the sample may be obtained in a different place and/or on a different platform.

In some embodiments the method further comprises the step of training the learning data architecture prior to the step of analyzing. The learning data architecture may be trained using one or more existing spectral data representations in the form of spectral images and results of earlier performed analyses.

Training the learning data architecture provides for even more accurate analysis results, even less dependency on the use of the skilled specialist and a general improvement of the above-mentioned advantages.

In some embodiments the learning data architecture is a deep network architecture comprising any one or more of ID, 2D or higher dimensional convolution filters, fully connected layers, dropout techniques, pooling layers, activation functions, early- stopping techniques and optimization and regularization techniques.

Such learning data architectures may be built up from scratch and trained with available data. This provides for the advantage of enabling customization of the learning data architecture to better match needs of the consumer and/or application.

In some embodiments the learning data architecture comprises any one or more of a Convolutional Neural Network (CNN), a Generative Adversarial Network (GAN), an Autoencoder and a Variational Autoencoder (VAE).

Such learning data architectures have the advantage of providing a strong data architectural platform on top of which the learning data architecture may subsequently be adapted and trained in accordance with the needs of the consumer and/or application. In other words, such learning data architectures are considerably faster to build up while still yielding accurate and consistent results of the analysis performed. In some embodiments the step of analyzing comprises performing by means of the data processing device and the learning architecture a regression on the spectral data representation to obtain a quantitative measure of one or more constituents of the sample.

In some embodiments the step of analyzing comprises performing by means of the data processing device and the learning architecture a classification of the spectral data representation to obtain an indication of sample type.

In some embodiments the step of analyzing comprises preforming by means of the data processing device and the learning architecture a classification of the spectral data representation followed by a regression of the spectral data representation in dependence of the classification. Thereby, the accuracy and consistency of the results of the analysis performed are enhanced.

In some embodiments the method further comprises the step of pre-processing the spectral data representation before the step of feeding the spectral data representation into the data processing device or before the step of analyzing.

Pre-processing has the advantage of allowing for cleaning up the input spectral data representation before the analysis to remove noise, sources of error, irrelevant spectral data and the like. This in turn provides for a further improvement of the analysis results and in general a further improvement of the above-mentioned advantages.

The step of pre-processing the spectral data representation may comprise any one or more of scaling, translation, rotation, flipping, addition of noise, variation of the lighting conditions and transformation of the perspective.

Thereby, a particularly efficient and well-functioning pre-processing is provided for.

In some embodiments the use of a discriminator as an element of the learning architecture may also be helpful in order to, during the analysis, detect deviations from such spectral data representations predetermined to be acceptable. In this manner for example adulterated samples or samples having constituents outside predetermined limits may be identified and classified.

In some embodiments the spectral data to be represented as an image comprises any one of transmission spectral data, absorbance spectral data, reflectance spectral data, pre-processed spectral data, a spectral correlation matrix, an n ^Lh order derivative of any one of transmission spectral data, absorbance spectral data, reflectance spectral data, Continuous Wavelet Transform (CWT) spectral data; outer product transformed spectral data and a spectral correlation matrix, where n is an integer being 1 or more, and combinations thereof.

These are all data types useful for providing input spectral data representations comprising a sufficient amount of data regarding the sample. The use of combinations of the above types of input spectral data is particularly advantageous since it makes it possible to avoid that the image constituting the spectral data representation comprises areas containing no information, so-called white spaces, or at least reduce the occurrence of such white spaces, while also providing more data useful in the analysis step. This in turn provides for a more efficient method and device using less data processing capacity and thus less power, as well as for a device which may learn faster.

In some embodiments the spectral data is obtained by performing any one or more of near infrared (NIR) spectroscopy, mid infrared (MIR) spectroscopy, visible (VIS) spectroscopy, ultraviolet (UV) spectroscopy, Raman spectroscopy, Nuclear Magnetic Resonance (NMR) spectroscopy, X-ray spectroscopy, fluorescence spectroscopy and Laser Induced Breakdown Spectroscopy (LIBS) on the sample.

These are all types of spectroscopy useful for providing the necessary spectral data for the spectral data representation in an efficient and easy manner comprising a sufficient amount of data regarding the sample.

In some embodiments the one or more physicochemical properties determined in the step of analyzing comprises determining any one or more of concentration and/or quantity of moisture, fat, protein, starch, salt, lactose, carbohydrates, solids, urea, ethanol, acids, oils and ash, and value of pH, density, freezing point depression and refractive index of the sample.

In some embodiments the one or more physicochemical properties determined in the step of analyzing comprises classification of the sample into sample type and/or adulteration or any other deviation from an expected sample type.

These are all parameters of particular interest in both the food industry and the agricultural industry when analyzing product samples.

The sample may be a foodstuff sample selected from of any one of dairy products, meat, wine, beverages, grain, feed and forage, fruits, vegetables, edible oils, and intermediaries of a final foodstuff product.

According to a second aspect of the invention, the above and other objects are achieved by means of a device configured to analyze one or more physicochemical properties of a sample, such as a sample of a foodstuff. The device comprises a data processing device and a learning data architecture, the data processing device being configured to access and/or generate a spectral data representation of the sample, where the spectral data representation is a representation as an image of spectral data obtained from the sample, and analyze, by means of the learning data architecture, the spectral data representation accessed by the data processing device to obtain a measure of one or more physicochemical properties of the sample, such as physicochemical properties selected from the group: the type of the sample, the quality of the sample, a quantitative and/or qualitative measure of one or more constituents of the sample. In some embodiments, the data processing device further is configured to pre- process the spectral data representation before analyzing the spectral image.

In some embodiments, the device further comprises a generator, which in some embodiments may be configured as an element of the data processor, specifically adapted to pre-process the spectral data representation before analyzing the spectral image.

In some embodiments, the learning data architecture is, prior to analyzing the spectral data representations, trained by means of one or more of existing spectral data representations in the form of spectral images and the results of prior performed analyses.

In some embodiments, the learning data architecture is a deep network architecture comprising any one or more of ID, 2D or higher dimensional convolution filters, fully connected layers, dropout techniques, pooling layers, activation functions, early- stopping techniques and optimization and regularization techniques.

In some embodiments, the learning data architecture comprises any one or more of a Convolutional Neural Network (CNN), a Generative Adversarial Network (GAN), an Autoencoder and a Variational Autoencoder (VAE).

In some embodiments, the device further comprises a discriminator configured to detect deviations from such spectral data representations predetermined to be acceptable. In this manner for example adulterated samples or samples having constituents outside predetermined limits may be identified and classified.

In further embodiments, the data processing device is further configured to perform a method according to any one of the above described embodiments.

According to a third aspect of the invention, the above and other objects are achieved by means of a computer program product in the form of a learning data architecture, which learning data architecture is configured to, when executed by a data processing device which has accessed or generated an image representation of spectral data obtained from the sample as a spectral data representation, analyze the spectral data representation to obtain a measure of one or more physicochemical properties of the sample, such as physicochemical properties selected from the group: the type of the sample, the quality of the sample, a quantitative and/or qualitative measure of one or more constituents of the sample.

In further embodiments, the learning data architecture is further configured to, when executed by a data processing device, perform a method according to any one of the above described embodiments.

It is noted that the features of the embodiments provided above are not necessarily mutually exclusive and that the invention relates to all possible combinations of features recited in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

This and other aspects of the present invention will now be described in more detail, with reference to the appended drawings showing embodiment(s) of the invention.

Fig. 1 shows a schematic diagram illustrating a first embodiment of a method and a device according to the invention.

Fig. 2 shows a schematic diagram illustrating a second embodiment of a method and a device according to the invention.

Fig. 3 shows a schematic diagram illustrating a third embodiment of a method and a device according to the invention.

Fig. 4 shows a schematic diagram illustrating further details of the third and a device embodiment of a method according to the invention.

Fig. 5 shows a schematic diagram illustrating a fourth embodiment of a method and a device according to the invention.

Fig. 6 shows a schematic diagram illustrating a fifth embodiment of a method and a device according to the invention.

Fig. 7 shows a schematic diagram illustrating a sixth embodiment of a method and a device according to the invention.

Fig. 8 shows a schematic diagram illustrating a seventh embodiment of a method and a device according to the invention.

Fig. 9 shows a schematic diagram illustrating an eighth embodiment of a method and a device according to the invention.

Fig. 10 shows an example of a spectrum to be used in a method according to the invention, the spectrum illustrating the absorbance of a sample plotted as a function of wavenumber.

Fig. 11 shows an example of a spectrum to be used in a method according to the invention, the spectrum illustrating the derivative of the spectrum according to Fig. 10.

Fig. 12 shows an example of a spectrum to be used in a method according to the invention, the spectrum illustrating both the absorbance of a sample plotted as a function of wavenumber according to Fig. 10 and its derivative according to Fig. 11.

Fig. 13 shows an example of a spectrum to be used in a method according to the invention, the spectrum being a contour spectrum.

Fig. 14 shows an example of an image of a spectrum to be used in a method according to the invention, the image illustrating the spectrum after applying the outer product of the spectrum itself and plotting the result.

Fig. 15 shows an example of an image of a spectrum to be used in a method according to the invention, the image illustrating the spectrum after applying a continuous wavelet transform (CWT) on the spectrum and plotting the result. Fig. 16 shows a schematic diagram illustrating a ninth embodiment of a method and a device according to the invention, and configured to use images such as those shown in Figs. 14 and 15.

Fig. 17 shows a schematic diagram illustrating a tenth embodiment of a method and a device according to the invention, and configured to use images such as those shown in Figs. 14 and 15.

As illustrated in the figures, the various embodiments of the invention are shown schematically to illustrate the general structures of embodiments of the present invention. Like reference numerals refer to like elements throughout.

DETAILED DESCRIPTION

The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, the scope of the invention is to be defined by the scope of the claims appended hereto.

With reference to Fig. 1, a device 100 according to a first embodiment of the invention is shown. Generally, and irrespective of the embodiment, the device 100 is configured to analyze one or more physicochemical properties of a sample, particularly one or more physicochemical properties selected from the group: sample type, sample quality, and a quantitative and/or qualitative measure of one or more constituents of the sample.

Generally, and irrespective of the embodiment, the device 100 comprises a data processing device 2 and a learning data architecture 3. Generally, and irrespective of the embodiment, the data processing device 2 is configured to access a spectral data representation 1 in the form of an image representation of spectral data obtained from a sample, and to analyze, by means of the learning data architecture 3, the spectral data representation 1 accessed by the data processing device 2 to obtain a measure of one or more physicochemical properties of a sample, particularly one or more physicochemical properties selected from the group: sample type, sample quality, and a quantitative and/or qualitative measure of one or more constituents of the sample. The data processing device 2 may be configured to execute the learning data architecture 3 in order to analyze the spectral data representation 1.

The learning data architecture 3 may be stored on a data storage device. Thus, the device 100 according to the invention may comprise a data storage device on or in which the learning data architecture 3 is stored.

In an aspect, the invention may thus concern a computer program product in the form of a learning data architecture 3. The computer program product may be stored on or in a data storage device or may be made remotely accessible to the data processing device 2. The computer program product is configured to, when executed by a data processing device 2 into which a spectral data representation of a sample in the form of a representation of spectral data obtained from the sample as an image is fed, analyze the spectral data representation to obtain a measure of one or more physicochemical properties of a sample, particularly one or more physicochemical properties selected from the group: sample type, sample quality, and a quantitative and/or qualitative measure of one or more constituents of the sample.

Fig. 1 also illustrates, by means of arrows, the various steps of a method according to the invention for analyzing a sample to obtain a measure of one or more physicochemical properties of a sample, particularly one or more physicochemical properties selected from the group: sample type, sample quality, and a quantitative and/or qualitative measure of one or more constituents of the sample.

Generally, and irrespective of the embodiment, the method according to the invention comprises the steps of providing a spectral data representation 1 in the form of an representation of spectral data obtained from the sample as an image, providing a data processing device 2 and a learning data architecture 3, feeding 200 the spectral data representation 1 into the data processing device 2, and analyzing 300, 500, 800, by means of the learning data architecture 3 and the data processing device 2, the spectral data representation 1 fed into the data processing device 2 to obtain a measure of one or more physicochemical properties of a sample, particularly one or more physicochemical properties selected from the group: sample type, sample quality, (one or both in the step classification 300) and a quantitative and/or qualitative measure of one or more constituents of the sample (quantification 800, cf. e.g. Fig. 5) ; or both classification 300 and quantification 500.

The learning data architecture 3 may be executed by means of the data processing device 2 in order to analyze the spectral data representation 1.

A common trend in computer vision and deep learning is to use the experience and knowledge of an existing neural/deep network, that has been trained on millions of unrelated images, and utilize the learnt weights of the model to solve another classification/regression problem. This method is known as transfer learning. Thus, one general example of a suitable learning data architecture 3 is a pre-trained learning data architecture.

An alternative is to design and build a deep learning architecture from scratch and train it with available data. Such an architecture will be specially tailored to analyze available spectral data. In the data architecture, 1-D, 2-D or higher dimensional convolution filters, fully connected layers, techniques like dropout, pooling layers, activation functions, early-stopping and optimization and regularization techniques can be used. Thus, another general example of a suitable learning data architecture 3 is a supervised learning data architecture.

Another alternative is to utilize the semi- supervised method of learning where very few labelled data points are used to learn. Generative Adversarial Networks (GAN) are one such semi- supervised data architecture that could be used for both classification and regression purposes. The advantage with the GANs is that it works with a limited set of labelled data and a large amount of unlabeled data. GANs are generally used for classification tasks but they can also be used for regression tasks.

Variational Autoencoders (VAE) is another group of semi- supervised data architectures or algorithms that can be used to solve the classification/regression problems. VAE consists of two subsystems - Encoder and Decoder. The Encoder compresses the input into a latent space hence reducing the number of features. These features can then be used in a deep network to build a classifier or regressor.

Thus, yet another general example of a suitable learning data architecture 3 is a semi-supervised learning data architecture.

In the following various examples of embodiments of a method and a device according to the invention will be described with reference to the figures and in association with the analysis of a foodstuff sample. It will be appreciated that other types of sample may be used without departing from the invention as claimed.

Example 1 - Classification of foodstuff sample

In the embodiments exemplified in Figs. 1-4, the target is to identify a food sample as belonging to a particular class or category. The input, or spectral data representation 1, to the device 100, 101, 102 and method can be any of those described below with reference to Figs. 10-15. The output of the device 100, 101, 102 and method will be the probability of a food sample being in each of the categories or types denoted A-D on Figs. 1 and 3 and A-E on Figs. 2 and 4. Examples of categories or types are given further below.

In a first embodiment illustrated on Fig. 1 the device 100 comprises a data processing device 2 and a learning data architecture 3 in the form of a pre-trained deep network in which the weights of the pre-trained deep network are reused. Pre-trained deep networks may be trained on millions of images to classify hundreds of classes. To customize and use an existing architecture fully connected layer(s) and a softmax/sigmoid layer near the latter layers of the existing architecture are added, and certain pre-trained layers might be removed. The weights of the existing pre-trained network are frozen and the net is trained to learn only the weights for the additional layers that are added. The new architecture can also be trained to fine tune all the weights or eventually train the network from scratch. This method is also known as transfer learning. In a second embodiment illustrated on Fig. 2 the device 101 comprises a learning data architecture 3 in the form of a deep network architecture that is trained from scratch to learn all the weights. An example could be a convolution based deep network. Such a network may use filters of different sizes and dimensions to perform the convolution operations. Pooling or un-pooling operations may be used to reduce/increase dimensions respectively after convolution.

The device 101 further comprises a generator 4 configured to perform, in a method step 400, a pre-processing of the spectral data representation 1 used as input. Such a generator 4 may in principle be provided in any of the embodiments of a device according to the invention and may be included as an element of the data processing device 2.

In this architecture the spectral data representation 1 used as input is randomly generated (herein also denoted artificial) spectra and true (herein also denoted real) spectra. It is noted that by real or true spectra it is intended to mean spectra originating from an actual physical sample. The spectral data representation 1 is fed into the generator 4 in a method step 400. The pre-processed output of the generator 4 is an image that is fed, method step 200, to the data processing device 2. The device 101 comprises in this embodiment a discriminator 9. The discriminator 9 may form part of the learning data architecture 3, or it may form a separate data architecture. The discriminator 9 is trained to predict if the spectrum is real or artificial. At the same time the discriminator 9 also classifies 300 the spectrum. Eventually, a discriminator 9 is trained that can classify spectra, learning from very few labelled spectra.

In a third embodiment illustrated on Figs. 3 and 4 the device 102 comprises a data processing device 2 and a learning data architecture 3 in the form of an auto-encoder. Generally, an auto-encoder is made up of an encoder 5a and a decoder 5b. The input to the encoder 5 a is the spectral data representation 1.

Referring particularly to Fig. 4, the encoder 5a maps (or encodes), in a method step 600, the input 6 to a latent space 8 in lower dimension. As shown on Fig. 4, the input here by way of non-limiting example has five dimensions. The decoder 5b then decodes, in a method step 700, the lower dimension layer to form an output 7, which has the same dimension, here five dimensions, as the input 6. An auto-encoder hence helps to represent an input in a lower dimensional latent space.

Referring to Fig. 16, an implementation of a device 108 according to the invention is shown implementing an auto-encoder to classify a foodstuff sample as either meat or cheese is illustrated. The device 108 comprises a data processing device 2 and a learning data architecture 3 in the form of a Convolutional Neural Network (CNN) configured as an auto-encoder.

In this example, the spectral data representation 1 used as input comprises the absorbance spectra for cheese, the absorbance spectra for meat and a label for each image for the class it belongs. Each of the absorbance spectra is represented as an image using either one or both of the methods of i) applying CWT (continuous wavelet transform) to the spectral data representing the spectrum and plotting it (cf. Fig. 15), and ii) taking the outer product of the spectral data with itself and plotting it (cf. Fig. 14).

Based hereon it is possible to classify a food sample as belonging to a specific type, here for example as being meat or cheese, using the above-mentioned spectral data representation 1 and using Convolutional Neural Network-based solutions. In this embodiment, the device 108 is therefore configured to perform a method as follows.

As the plot to be used should be an image, the scales along x-axis and y-axis are removed. All the images have a fixed dimension.

All the available spectra, for instance meat and cheese spectra, are merged into one dataset along with their respective class labels.

The available data is randomly split into three sets, in the ratio 80:10:10 - training, validation and test set respectively. Each image will be used as an input to the data processing device 2 and the learning data architecture 3, here a Convolutional Neural Network (CNN). The output is the label of the class to which the image belongs. The label could be generated using one-hot encoding.

The image is normalized by one of the following methods, dividing throughout by 255 or min-max normalization or mean centering and dividing by standard deviation.

A learning rate and optimizer (Adam, SGD etc.) is chosen.

A loss function is chosen to be binary cross-entropy or categorical cross entropy or a customized in-home loss function.

The network is trained, avoiding overfitting and observing validation loss as a key indicator of the generalization performance.

The accuracy, precision, recall and fl-score of the device 108 on the test set is checked to evaluate the performance.

Example 2 - Classification and Regression of foodstuff sample

In the embodiments exemplified in Figs. 5 and 6, a food sample is identified followed by prediction of the compositional or physical structure of the sample. The input, or spectral data representation 1, to the device 103, 104 can be any of those described below with reference to Figs. 10-15. The output of the device 103, 104 and method will be the probability of a food sample being of a specific class or category or type denoted A-D on Fig. 5 and A-E on Fig. 6, and further the constituents and quantitative composition of the food sample denoted A1-D3 on Fig. 5 and A1-D5 on Fig. 6. Examples of both types and constituents of foodstuffs are given further below.

In a fourth embodiment illustrated on Fig. 5 the device 103 comprises a data processing device 2 and a learning data architecture 3 in the form of a pre-trained deep network. To customize and use an existing architecture fully connected layer(s) is/are added and a softmax layer near the latter layers of the existing architecture may or may not be added, and certain pre-trained layers could optionally be removed. After the last layer of the classification network one or more fully-connected layers are added. These layers are input with features learnt from the hidden layers of the classification model. While training, the weights of the existing pre-trained network may be frozen and the network is trained to only learn weights for the additional layers that are added.

The network may also be trained to learn all the weights of the network from scratch. Thus, alternatively, the device 103 comprises a learning data architecture 3 in the form of a deep network architecture which is trained from scratch to learn all the weights.

This may be a sequence of models where the first model classifies and the second model regresses. The output from the hidden layers of this deep network architecture is passed to one or more of the layers prior to the output layers.

In a fifth embodiment illustrated on Fig. 6 the device 104 comprises a data processing device 2 and a learning data architecture 3 in the form of a deep network architecture that is trained from scratch to learn all the weights. The device 103 further comprises a generator 4 configured to perform, in a method step 400, a pre-processing of the spectral data representation 1 used as input.

In this architecture, the spectral data representation 1 used as input to the generator 4 is randomly generated (herein also denoted artificial) spectra and true spectra. The output of the generator is an image that is fed, method step 200, to the data processing device 2. The device 104 comprises in this embodiment a discriminator 9, which may form part of the learning data architecture 3, or which may be a separate data architecture. The discriminator 9 is configured to predict if the spectrum is real or artificial and then classifies, method step 300, the food sample. Next, using the features from the convolution layers of the discriminator 9 the learning data architecture 3 analyzes, method step 500, the composition of the food sample. Alternatively, the learning data architecture 3 may proceed directly to analyze, method step 800, the composition of the food sample.

Example 3 - Compositional Prediction of foodstuff sample

In the embodiments exemplified in Figs. 7 to 9, the compositional structure of the sample of foodstuff is predicted or analyzed. The input, or spectral data representation 1, to the device 105, 106, 107 can be any of those described below with reference to Figs. 10-15. The output of the device 105, 106, 107 and method (cf. step 800 on Figs. 7-9) will be the composition and the quantity of the constituents of the food sample in the same units as a chemical analysis, e.g. ppm, percentages, g/kg, denoted A1-A4 on Figs. 7 and 8 and A1-A5 on Fig. 9.

In a sixth embodiment illustrated on Fig. 7 the device 105 comprises a data processing device 2 and a learning data architecture 3 in the form of a pre-trained deep network. To customize and use an existing architecture fully connected layer(s) are added near the latter layers of the existing architecture, and certain pre-trained layers may be removed. The weights of the existing pre-trained network are frozen and the network is trained only to learn weights for the additional layers that are added.

The network may also be trained to learn all the weights of the network from scratch. Thus, in a seventh embodiment illustrated on Fig. 8 the device 106 comprises a learning data architecture 3 in the form of a deep network architecture, and the deep network architecture is trained from scratch to leam all the weights.

The device 106 further comprises a generator 4 configured to perform, in a method step 400, a pre-processing of the spectral data representation 1 used as input. In this architecture the input to the generator 4 is randomly generated spectra and true spectra. The device 106 comprises in this embodiment a discriminator 9, which may form part of the learning data architecture 3, or which may be a separate data architecture. The output of the generator 4 is an image that is fed to the discriminator 9. The discriminator 9 is configured to predict if the spectrum is real or artificial and to predict the composition of the food sample.

In an eighth embodiment illustrated on Fig. 9 the device 107 comprises a data processing device 2 and a learning data architecture 3 in the form of an auto-encoder. The auto-encoder is made up of an encoder 5a and a decoder 5b. The input to the encoder 5a is the spectral data representation 1. The auto-encoder maps the spectra to a new low dimension space and representation as described above in relation to Figs. 3 and 4. This low dimensional representation is then fed into the data processing device 2 and the learning data architecture 3 is used to analyze the contents of a sample of foodstuff.

Referring to Fig. 17, an implementation of a device 109 according to the invention is shown implementing an auto-encoder 5 to classify a foodstuff sample as either cheese or meat and to determine the protein content of the sample is illustrated. The device 109 comprises a data processing device 2 and a learning data architecture 3 in the form of a Convolutional Neural Network (CNN) configured as an auto-encoder.

In this example, the spectral data representation 1 used as input is constructed using the absorbance spectra for cheese, the absorbance spectra for meat, reference values for protein content in Cheese spectra (protein value for each cheese sample) and reference values for protein content in meat spectra (protein value for each meat sample). Each of the absorbance spectra is represented as an image using either one or both of the methods of i) applying CWT (continuous wavelet transform) to the spectral data representing each spectrum and plotting it (cf. Fig. 15), and ii) taking the outer product of the spectral data with itself and plotting it (cf. Fig. 14).

Based hereon it is possible to analyze and predict the protein content in the samples used to provide the input spectra, representing the spectra as images and using Convolutional neural network-based solutions. It is also possible to employ this technique to do food analysis by representing spectra as images. In this embodiment, the device 109 is therefore configured to perform a method as follows.

As the plot to be used should be an image, the scales along x-axis and y-axis are removed. All the images have a fixed dimension.

All the available spectra, here meat and cheese spectra, are merged into one dataset along with their respective reference values.

The spectra are randomly shuffled, keeping a constant seed.

The available data is divided into three sets, in the ratio 80:10:10 - training, validation and test set respectively.

Each image will be used as an input to the data processing device 2 and the learning data architecture 3, here a Convolutional Neural Network (CNN). The output is the reference value.

The image is normalized by one of the following methods, dividing throughout by 255 or min-max normalization or mean centering and dividing by standard deviation.

A learning rate and optimizer (Adam, SGD etc.) is chosen.

A loss function is chosen to be mean-absolute-error or mean-squared-error or a customized in-home loss function.

The network is trained, avoiding overfitting and observing validation loss as a key indicator of the generalization performance.

The Root Mean Square Error (RMSE) on the predictions of the device 109 on the test set is checked to evaluate the performance.

Figs. 10-15 show different examples of possible image representations of spectral data that has been obtained from a sample and which may be used as spectral data representations in a method and as input 1 for a device (100-109) according to the invention.

The spectral data representation may be any one of the following represented by e.g. different colors or line styles:

- spectral data, e.g. transmission, absorbance (cf. e.g. Fig. 10) or reflectance spectra,

- pre-processed spectral data, e.g. using Standard Normal Variate (SNV), mean centering, detrend, Multiplicative Signal Correction (MSC), baseline correction, etc.,

- spectral correlation matrix,

- an image generated by a Generative Adversarial Network (GAN) that can take as input i.a. any one of the above input formats,

- a contour plot (cf. e.g. Fig. 13),

- the n ^Lh order derivative of any of the above, where n is an integer being 1 or more (cf. e.g. Fig. 11), and - combinations of any of the above (e.g. Fig. 12 combining the spectra of Figs. 10 and 11).

The use of combinations of the above types of spectral data from which to construct the spectral data representations, such as e.g. the spectra shown in Fig. 12, may be desirable since it makes it possible to avoid that the input spectral data representations comprises areas containing no information, so-called white spaces, or at least reduce the occurrence of such white spaces. This in turn provides for a more efficient method and device suing less data processing capacity and thus less power, as well as for a device which may learn faster.

The input spectral data representation may also be pre-processed by e.g. data augmentation techniques like scaling, translation, rotation, flipping, adding noise to images, varying lightning conditions and transforming the perspective. The input spectral data representation may also be pre-processed by cleaning the spectral data. For instance, the spectral data representations shown in each of Figs. 10-12 are constructed using spectral data having an interruption or discontinuity at wavenumbers between 1500 and 1750, which corresponds to high absorption peaks for water. These peaks have been removed to remove noise and thus clean up the spectral data representation. Also, pre-processing may be used to remove or reduce white spaces in the input spectral data representation.

Pre-processing of spectral data may also be done by applying an outer product of the spectral data with itself (e.g. the spectral data representation in Fig. 14) or by applying a Continuous Wavelet Transform (CWT) to the spectral data (e.g. the spectral data representation in Fig. 15). CWT performs a convolution with data using the wavelet function. As an example the Ricker wavelet, also known as the ‘Mexican hat wavelet’, is applied to reflectance, transmission or absorbance spectral data and the resulting spectral data used in the construction of the spectral data representation shown in Fig. 15.

The spectral origin of the input spectral data representation 1 may by way of example be near infrared (NIR), mid infrared (MIR), visible (VIS), ultraviolet (UV), Raman, Nuclear Magnetic Resonance (NMR), X-ray, fluorescence, Laser Induced Breakdown Spectroscopy (LIBS) etc. plotted on any spectral axis (frequency, wavelength, wavenumber) or range.

The sample origin of the spectra of the input spectral data representation 1 , could be from any type of foodstuff, whether for humans or animals. Non-limiting examples of such foodstuffs are food products or agricultural products such as dairy products including milk, cheese and butter, meat, wine and other beverages, grain, feed and forage, fruits, vegetables, edible oils, and intermediaries of a final foodstuff product etc. Thus, the device and method according to the invention may be used in food, food industrial and agricultural applications. The physicochemical properties to be determined and modelled based on the input spectral data representation (image) may by way of example be compositional such as, but not limited to, the concentration or quantity of moisture, fat, protein, starch, salt, lactose, solids, urea, ethanol, acids, oils, ash etc., and/or physical properties, such as, but not limited to, pH, density, freezing point depression, refractive index, etc.

Alternatively, or additionally, the analysis may be based on qualification models such that the physicochemical properties to be determined may be a classification into any food sample type, such as, but not limited to, dairy products including milk, cheese and butter, meat, wine and other beverages, grain, feed and forage, fruits, vegetables, oils, and specific types of any one of the aforementioned etc., and/or adulteration or any other deviation (e.g. in a process) from the normal or expected spectra.

The person skilled in the art realizes that the present invention by no means is limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims. Additionally, variations to the disclosed embodiments can be understood and effected by the skilled person in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.