The present invention relates to machines for preparing and dispensing hot beverages by controlled infusion of a dose unit of a precursor substance containing one or more ingredients, for example coffee.

More specifically, the invention relates to a control system and method for a machine for preparing and dispensing hot beverages by controlled infusion of a precursor substance in granular or powder form, arranged in a dose unit, according to the preamble of claims 1 and 25, respectively.

Machines for preparing and dispensing hot beverages by controlled infusion of a dose unit are known which comprise means for automatically identifying the dose unit inserted into the machine, in order to classify said dose unit into one of a plurality of predetermined types or classes of dose units and consequently control at least one parameter or operating mode of infusion of the machine according to the type of dose unit recognized, or record and/or transmit machine usage data to a remote management system.

Automatic identification may be implemented with machine-learning techniques, for example through pattern recognition methods or artificial neural networks based on predefined classification schemes.

The utility model IT274555 of the same Applicant describes a machine for preparing beverages, in particular coffee, with the use of a pod or capsule selected from a plurality of types of pods or capsules having different features, in particular having the same dimensional features, but containing different substances or ingredients, the machine comprising a detector device adapted to detect at least one predetermined detectable feature or code identifying the type of pod or capsule inserted into the machine, and a processing and control unit to which said detector device is connected, which is arranged to implement an operating cycle of the machine which varies according to predetermined methods according to the type of pod inserted into the machine and recognized by the detector device, and/or to acquire data indicative of the methods of use of the machine by the user, such as the total quantity and/or the individual quantities of the various types of pods used in a given period of time, the possible use of “non-original” compatible pods, etc.

The detectable feature or code is for example a color, or a bar code, or a so-called RFID tag.

Automatic identification of the dose unit may also be implemented with machine-learning techniques, for example through pattern recognition methods or artificial neural networks based on predefined classification schemes, as taught by the international patent application WO 2019/154527.

WO 2019/154527 discloses a machine for preparing and dispensing a beverage, such as tea, coffee, hot chocolate, cold chocolate, milk, soup or baby food, comprising a module for recognizing a capsule inserted in said machine in a recognition position, comprising a camera for capturing an image of at least part of said capsule and a neural network computing device configured to determine the type of capsule among a plurality of predefined capsule types based on an image of at least part of said capsule captured by said camera.

Although a machine for preparing and dispensing a beverage of the type described above may guarantee the automatic recognition of a pod or capsule, it is not suitable for accurately recognizing and classifying objects that may change their features over time, such as coffee tablets or other precursor substances in granular or powder form, which are subject to deterioration due, for example, to the tablet storage conditions or exposure to non-optimal temperature or humidity environmental conditions, or their handling by a user.

Furthermore, even in the case of packaged dose units such as pods or capsules, the external surface of which is not perishable, the state of cleanliness of the image acquisition optics, which are located in proximity to the infusion chamber, or the wear of the illumination means of the dose units and the means for acquiring images during the life of the machine, especially if they are low-cost devices, may cause artifacts in the acquisition of the images of the dose units inserted in the machine, on which the automatic recognition process is conducted, leading to misclassification.

Further factors which influence the accuracy of the classification reside in the randomness of the spatial position of the dose unit inserted in the machine at the moment of acquisition of the image for recognition, in particular if this image is acquired in a transfer path of the dose unit to the infusion chamber, for example as a result of falling or by moving a housing compartment of the dose unit, in which the dose unit is not stationary.

The object of this invention is to provide a satisfactory solution to the problems described above, avoiding the drawbacks of the prior art.

More specifically, the object of the present invention is to provide a control system for a machine for preparing and dispensing a beverage based on an accurate and robust classification of the dose units of the beverage precursor substance fed to the machine, which classification is, as much as possible, independent of the conditions of the dose unit and the machine, during the entire operating life of the machine.

A further object of the invention is to make the classification of the dose units adaptable as the operating conditions of the machine vary or as the types of dose units which may be produced over time vary.

Another object of the invention is to provide a control system which may be made and integrated in a machine for preparing and dispensing a beverage in a cost-effective manner, for example for household machines of limited size and cost.

According to the present invention, these objects are achieved by a control system of a machine for preparing and dispensing hot beverages having the features set forth in claim 1.

Particular embodiments form the subject matter of the dependent claims, the content of which is to be understood as an integral part of the present description.

A further subject-matter of the invention is a method for controlling a machine for preparing and dispensing hot beverages having the features set forth in claim 25.

In summary, the present invention is based on the principle of carrying out automatic recognition of the dose units inserted into the machine through machine-learning classification techniques, preferably through a neural network or a similar determined classification model, where the classification is based on a set of training dose unit images representative of a plurality of predetermined types or classes of dose units in a plurality of conditions for acquiring images of said dose units. Specifically, the dose units comprise respective graphic recognition markings indicative of the dose unit classes, and the set of training images of dose unit comprises a plurality of primary (or original) images of dose unit samples pre-classified in a supervised manner and a plurality of synthetic images obtained by alteration of the plurality of primary images, according to a data augmentation technique, to take into account possible alterations in the actual samples of dose units or in the means and conditions of image acquisition of dose units during the operating life cycle of the machine, as well as to increase the amount of images for training.

Advantageously, the creation of a plurality of synthetic images makes it possible to simulate images of deteriorated dose units, artifact images of intact dose units, for example due to the deterioration of the acquisition means of said images or to non-optimal alignment of the dose unit with respect to the image acquisition means, or images of dose units different from the sample dose units, for example in the case of dose units for different beverages such as espresso or American coffee, which have different dimensions so that the acquired images may be susceptible to alteration due to optical distortion.

Advantageously, the graphic recognition markings indicative of the types of dose units comprise an inscription identifying the product line and additional denotation markings, allowing a first classification on the basis of the product line of the dose unit and a second classification on the basis of the mixture of the precursor material.

The term graphic markings refers to superficial (two-dimensional) graphic markings obtainable for example by printing or other application of an edible additional substance, hot marking or by applying a localized laser beam, which markings exhibit a different color from the color of the precursor substance of the dose unit, or a color contrasting with the color of the precursor substance, or in any case a luminous intensity or brightness different from the luminous intensity or brightness of the precursor substance in response to the illumination of the dose unit, or raised or notched (three-dimensional) graphic markings obtained by modifying the morphological properties of the surface of the precursor substance of the dose unit, for example by applying an edible additional substance or by mechanical and/or thermal impression on predetermined areas of the surface of the precursor substance adapted to form reliefs or engravings, which exhibit altered reflection properties and/or shadows when illuminated.

In one embodiment, the automatic recognition of the dose units may take place through the identification, in the acquired images, of the graphic recognition markings and the determination of a comparison index representative of the degree of similarity between the graphic recognition markings (or part of them) of the acquired image and the graphic recognition markings (or parts of them) of the training images of dose units or of a set of reference images of dose unit classes. The degree of similarity is defined by one or more comparison thresholds, which are preferably dynamically modifiable during the life cycle of the machine.

It is also possible to envisage that the automatic recognition of the dose units is implemented after a process of correcting the images acquired as a result of a deterioration of the classification accuracy or of the degree of similarity in comparisons between the graphic recognition markings of the image acquired and the graphic recognition markings of the training images of dose units or of a set of reference images of dose unit classes, or even on the basis of the elapsed operating time of the machine or of the number of dispensing actions performed. The image correction process may be implemented by applying predetermined correction models, or by updating the characteristic parameters, of the classification model according to instructions received from an external management system, which is temporarily connected to the machine control system through a local wired connection or a remote wireless connection.

Further features and advantages of the invention will be presented in greater detail in the following detailed description of an embodiment thereof, given by way of non-limiting example, with reference to the accompanying drawings, wherein:

Fig. 1 is a schematic representation, in sectional view, of a machine for preparing and dispensing hot beverages by infusion, according to the invention;

Figs. 2a-2c are exemplary representations of coffee dose units, in the form of tablets, bearing graphic recognition markings;

Fig. 3 is a block diagram of a control system for the machine of Fig. 1;

Fig. 4 is a block diagram of a neural network used by the system of Fig. 3;

Fig. 5 is an exemplary representation of the elements of the neural network of Fig. 4;

Fig. 6 is a representation of a plurality of synthetic images obtained by alteration of primary images of dose unit samples;

Fig. 7 is a representation of a plurality of additional synthetic images obtained by processing primary images of dose unit samples; and

Figs. 8a and 8b are flow charts of a training method and a control method of a machine for preparing and dispensing hot beverages by infusion according to the invention.

Referring to Fig. 1, a machine for preparing beverages, in particular coffee, from a compacted tablet containing one or more ingredients, in particular coffee powder, is denoted as a whole by 1.

Although the invention is described in reference to the use of a compacted tablet containing the ingredient, or ingredients, for the preparation of the beverage, it is not to be understood as limited to a dose unit in the form of a tablet, but it is also applicable in the case where, in place of a tablet, there is provided a dose of powder contained in a capsule, a pod or other similar package suitable for the preparation of beverages by infusion. For the purposes of the present invention, the term “precursor substance in granular or powder form” therefore contemplates both a product packaged in a rigid capsule or a flexible pod, and a compacted product in tablet form.

In the embodiment illustrated by way of example, the machine 1 comprises a machine body 2 with an operating region 4. The machine 1 has a lever 6 for pushing a dose unit 8 into the region 4 through an introduction opening 10, a closing device 20 operating on the dose unit 8 to enclose it in an infusion chamber delimited by two opposing cooperating elements 22, 24 (and possibly to pierce the coating film of the pod or the closure lid of the capsule), and a circulation system 26 to circulate a flow of hot water and/or pressurized steam through the dose unit in the infusion chamber.

The dose unit 8 contains a dose or portion of granular or powder material, of coffee or another substance intended for infusion to produce a hot beverage, in the form of a tablet having a self-supporting structure which does not require an outer casing, or packaged in a flexible water-permeable wrapper, typically a paper wrapper, usually referred to as a “pod”, or in a more or less rigid capsule.

The lever 6 is rotatable around a fulcrum 28 and is adapted to push the dose unit 8 towards the operating region 4 through the opening 10.

In the illustrated example, the closing device 20 includes an abutment element 22, a thrust element 24, movable with respect to the abutment element 22, and a clamping unit 30 operating on the element 24 to move it along a sliding direction.

The element 24 comprises a cup-shaped body defining the infusion chamber adapted to receive the dose unit 8, in a manner known per se.

The circulation system 26 is also of an essentially known type and comprises a water tank 40, a boiler 42 and a hydraulic circuit 44 including a circulation pump 46 between the boiler 42 and an inlet duct of the abutment element 22. A duct 48 connects an outlet manifold of the movable thrust element 24 with a beverage dispensing nozzle 50.

The machine 1 also comprises a pre-chamber 60 for the dose unit 8 in transit between the introduction opening 10 and the operating region 4.

The machine 1 is intended for use with dose units preferably provided with one or more predetermined graphic recognition markings identifying the class to which each dose unit belongs, in particular the type of ingredients or substances contained and the type of beverage which they allow to be made, or the product line. These recognition markings are of the optically detectable type, visible on a reference surface of the dose unit, for example they comprise one or more images bearing a product denomination and denotation markings.

Figs. 2a-2c illustrate by way of example some possible solutions according to which such a dose unit 8, in the form of a tablet, may be provided with graphic recognition markings including (at least) one inscription on its external surface and denotation markings in association with it.

In particular, Fig. 2a, 2b and 2c show dose units 8 bearing an inscription 70, for example a cartouche identifying the product line, and zero, one or two denotation markings 72 associated therewith, respectively, in the case in the example formed by simple symbols (for example: circles, squares) affixed at a corner of the cartouche, identifying the mixture of the precursor material. In the case of coffee powder, the mixture of the precursor material has a brown color, of variable intensity depending on the composition of the mixture or its storage conditions, while the inscription and denotation markings produced by printing or other application of an edible additional substance, heat marking or by applying a localized laser beam or embossing/engraving are of a contrasting color, for example dark brown or black. Naturally, a similar recognition effect may be obtained by making graphic markings of a contrasting color lighter than the background color of the mixture of the precursor material or in any case having a luminous intensity or brightness different from the luminous intensity or brightness of the precursor substance in response to the illumination of the dose unit. In some embodiments, the visibility of the graphic markings by contrast against the background of the precursor material may show at illumination wavelengths or at observation (image acquisition) wavelengths different from the spectrum of wavelengths visible to the human eye.

Dose units 4 of this type may be advantageously used in a machine of the type illustrated in Fig. 1, where a detector device (reader) 62 is positioned along the path of the dose units between the introduction opening 10 and the operating region 4, for example on one side of the pre-chamber 60. The machine 1 comprises an electronic processing and control unit 64 to which the code detector device 62 is connected and which is arranged to classify the dose unit and to implement an operating cycle of the machine for managing the dose unit, variable according to predetermined modes according to the class of the dose unit 8 introduced into the machine, whose image is acquired by means of the detector device 62, and/or to acquire data indicative of the modes of use of the machine 1 by the user, such as preferred types of beverages, etc.

The processing and control unit 64 may, for example, be arranged to vary, according to the type of dose unit and therefore of beverage, the duration of the infusion, and/or the quantity and/or the temperature and/or the pressure of water and/or steam passed through the dose unit.

The control system for the machine of Fig. 1 is illustrated in more detail by the block diagram of Fig. 3.

The dose unit 8 is shown (in the pre-chamber 60) between a pair of devices 80 and 82 adapted to detect its presence, for example by interrupting an infrared signal transmitted by an emitter device 80 to a receiver device 82. An image recognition and classification microcontroller unit 84, in communication with the devices 80 and 82, presides over the detection of the introduction of a dose unit into the pre-chamber and is coupled to one or more illumination modules 86 adapted to at least temporarily illuminate the pre-chamber to allow the acquisition of at least one image of at least one reference surface of the dose unit. Conveniently, the illumination modules 86 may include means for regulating the intensity and/or color (or, more generally, the wavelength) of the particular illumination light beam.

In general, the illumination light beam may be in the visible spectrum, or in another spectrum, for example infrared or ultraviolet, in one or more selected wavelength bands, with predetermined, and also variable, illumination angles.

Reference numeral 88 indicates an image acquisition module, for example including an RGB camera or a CCD sensor with LED ring illumination, used for the acquisition of the aforementioned images of the dose unit under the control of the microcontroller unit 84, which is arranged to trigger the shooting of one or more images of the dose unit inserted into the machine during its transfer towards the infusion chamber by means of the control of the illumination modules, as well as to carry out the recognition and classification of the images. The microcontroller unit 84 is coupled to the image acquisition module 88 via an adapter module 90 adapted to transfer the image data to the microcontroller unit 84. The microcontroller unit 84 is programmed to classify the detected images according to the procedure which will be described below, governed by a classifier model, such as a neural network, in the form of a computer program stored in an associated permanent memory module, which is optionally rewritable as a result of a connection of the microcontroller unit with an update module, for example a wired connection with a local update module or a wireless connection, for example via a local or wide area communication network, with a remote update module, in order to update the classifier model to new classes of dose units or at different times of the operating life of the machine.

The image acquisition module 88 may operate in the visible spectrum, or in another spectrum, for example infrared or ultraviolet, in one or more selected wavelength bands, at at least one predetermined acquisition angle, which is not necessarily frontal, for example also variable.

The microcontroller unit 84 is arranged to communicate with separate control means of the machine (not shown) which are arranged to control at least one parameter or operating management mode of the dose unit, such as for example an infusion parameter or operating mode of the machine or a parameter or operating mode relating to other steps, for example for controlling the compression of the dose unit during a pre-infusion step and/or during an infusion step, according to constant profiles or defined according to predetermined control curves, of the squeezing of the dose unit after infusion to remove residual water, of the automatic removal of the dose unit after infusion from the infusion chamber to a collection compartment for spent units, depending on the recognized dose unit class, according to the related data transmitted by the adapter module 90, or for communicating with a remote management system arranged for recording machine usage data, for example for counting the number of dispensing actions for statistical purposes and predicting possible wear of the image acquisition means. Advantageously, the results of the classification processes of the dose units gradually inserted into the machine during its operating life may be communicated over time to external management systems, temporarily associated via a local wired connection or a remote wireless connection, for example via a local or wide area communication network, from which the microcontroller unit may receive programming instructions, for example for updating the characteristic parameters of the classification model.

In a currently preferred embodiment, there are three images acquired for a dose unit inserted into the machine, and the microcontroller unit 84 selects the best image through a selection algorithm, for example a “voting” algorithm based on a voting strategy, such as majority voting Naive Bayes weighted majority voting.

In the pre-chamber, which communicates with the infusion chamber, there may be steam coming from the infusion chamber, represented in the image by the pictogram denoted as V, and the pre-chamber may therefore preferably be associated with fan means 92 for cleaning the detector device 62 integrated in the image acquisition and recognition module 88, for example for the suction of the steam or of the particles of precursor material in suspension produced by the previous infusion operations and the consequent cleaning of the optics of the image acquisition device, which fan means 92 are controlled by a respective electronic control module 94.

Finally, reference numeral 96 indicates a module for managing the electrical power supply to the system components.

Fig. 4 shows a currently preferred embodiment of the microcontroller unit 84, which operates as an electronic system for processing and automatic recognition of the dose units, of the machine-learning type, arranged (i.e., programmed) to receive as input at least one acquired image I of the dose unit and to classify said dose unit into one of a plurality of predetermined classes (or types) of dose units based on a set of training images of dose units, or classes of dose units taken in a learning step. In the currently preferred embodiment, the microcontroller unit 84 includes a convolutional neural network configured to receive as input a set of values of a predetermined matrix of pixels of the acquired image I of dose units, representative of the gray levels or intensity of the color channels, such as for example the RGB channels, and more generally indicative of the luminous intensity or brilliance, i.e. the amount of light that the illuminated surface of the dose unit reflects or emits (for example by phosphorescence or fluorescence of the edible additional substance with which the graphic markings are made) at the acquisition wavelengths, the neural network comprising an input layer, a plurality of cascaded hidden layers, and an output layer adapted to provide a classification of the dose unit into one of a plurality of manufacturer classes, and into one of a plurality of product classes. More specifically, a first classification is intended to recognize the product line of the dose unit inserted into the machine on the basis of the marking 70, for example by distinguishing between graphic markings relating to dispensable product lines, similar markings or makings relating to non-dispensable product lines (depending on the type of dispensing machine model), and a second classification is intended to recognize the mixture of the precursor material on the basis of the denotation markings 72.

In detail, the neural network comprises a first subset FR of hidden feature extraction layers and a second subset CL of hidden classification layers. The subset FE of hidden feature extraction layers, having the purpose of extracting the salient features from the images useful for the classification purpose, comprises a plurality of hidden layers, each of which includes a series of convolutional blocks in residual configuration, configured to extract a series of feature maps from an acquired image of the dose unit by repeated application of predetermined filters whose values are defined in a learning step of the neural network, which will be discussed later in the present discussion, together with a reduction in the size of the data representative of the image of the dose unit. The subset CL of hidden classification layers comprises two parallel processing legs CL1, CL2 each having a plurality of fully connected layers, which output corresponding vectors VI, V2 indicative of the respective classification outcomes, which are then concatenated into a single output vector VC.

More specifically, with reference to the example of Fig. 5, the subset FE of feature extraction layers includes five layers, each of which, as represented in the enlarged insert of Fig. 5, is configured to apply a pair of convolutional filters to an input matrix and ReLU activation functions according to a residual configuration, as well as to apply a pooling operation to halve the size of the input matrix at the output. As an alternative to the pooling operation, convolutional filters may be applied with a stride other than 1. Advantageously, this configuration of the feature extraction layers allows the problem of the vanishing gradient in the supervised training of the network to be mitigated by error back-propagation. Furthermore, the application of convolutional filters is performed according to the depthwise separable convolution approach, to speed up training convergence.

The sizes of the layers may vary according to the processing resources available on the machine.

The first layer is adapted to receive an input matrix representing the pixel values of a predefined area of the original image, in the exemplary case a central area of size 256 x 256 pixels, which is subjected to convolution with 16 filters of size 3x3 and stride 1 and application of ReLU activation functions in residual configuration, and subsequently to a max pooling operation with stride 2, resulting in the output of a plurality of 16 feature maps of size 128 x 128.

The second layer is adapted to submit the 16 feature maps calculated at the first layer, of size 128 x 128 pixels, to convolution with 32 filters of size 3x3 and stride 1 and application of ReLU activation functions in residual configuration, and subsequently to a max pooling operation with stride 2, resulting in the output of a plurality of 32 feature maps of size 64 x 64.

The third layer is adapted to submit the 32 feature maps calculated at the second layer, of size 64 x 64 pixels, to convolution with 32 filters of size 3x3 and stride 1 and application of ReLU activation functions in residual configuration, and subsequently to a max pooling operation with stride 2, resulting in the output of a plurality of 32 feature maps of size 32 x 32. The fourth layer is adapted to submit the 32 feature maps calculated at the third layer, of size 32 x 32 pixels, to convolution with 64 filters of size 3x3 and stride 1 and application of ReLU activation functions in residual configuration, and subsequently to a max pooling operation with stride 2, resulting in the output of a plurality of 64 feature maps of size 16 x 16.

Finally, the fifth layer is adapted to submit the 64 feature maps calculated at the fourth layer, of size 16 x 16 pixels, to convolution with 64 filters of size 3x3 and stride 1 and application of ReLU activation functions in residual configuration, and subsequently to a max pooling operation with stride 2, resulting in the output of a plurality of 64 feature maps of size 8 x 8.

Through a linearization operation (flattening), the pixel values of the 64 feature maps of size 8 x 8 are concatenated into a vector of size 4096, which is supplied as input to each leg CL1, CL2 of the subset CL of classification layers.

The subset of classification layers includes four layers for each processing leg, each of which is configured to perform a weighted linear combination of the input values and apply a relevant activation function, preferably a non-linear activation function, for example a ReLU function in the inner layers and a sigmoid or softmax function in the output layer.

The first layer is adapted to receive the input vector of size 4096, output by the subset of feature extraction layers, which is subjected to a non-linear weighted sum, resulting in an output vector of size 64.

The second layer is adapted to receive the vector calculated in the first layer of size 64, which is subjected to a non-linear weighted sum, resulting in an output vector of size 32.

The third layer is adapted to receive the vector calculated in the second layer of size 32, which is subjected to a non-linear weighted sum, resulting in an output vector of size 16.

Finally, the fourth layer is adapted to receive the vector calculated in the third layer of size 16, which is subjected to a non-linear weighted sum, resulting in an output vector of size 3, which represents the final result of the classification, which is preferably subjected to normalization using a softmax function to be expressed in terms of the probability that the dose unit belongs to each of the predefined classes.

The foregoing has been set out purely by way of example and it is to be understood that different convolutional network configurations, for example in the number of hidden layers, different filtering and pooling parameters, or different activation functions may be used. Further, that described above with reference to an input matrix representative of pixel values of a predefined area of the original image, applicable to a grayscale image in which each pixel has a grayscale level value, is extendable to color images by making appropriate reference to a plurality of input matrices representing the pixel values of a predefined area of the original image in the color channels, for example in the three RGB color channels.

The described neural network is trained in a supervised manner for the determination of the values of the convolutional filters and of the parameters of the activation functions of the subsets of hidden layers starting from a data collection comprising a plurality of images of dose units representative of all the different classifications required by the system, acquired by one or more training image acquisition modules, similar to the image acquisition module provided on board the machine, each of which has been preliminarily assigned at least one relative class, in the specific case the class indicative of the product line and the class indicative of the mixture of the precursor material. Preferably, the preliminarily classified dataset is divided into a first test dataset, comprising 20% of the dataset, a second training dataset, comprising 80% of the residual dataset, and a third validation dataset, comprising the remaining 20% of the residual dataset. The validation is for example carried out at each cycle by exchanging the validation data with the training data according to a cross-validation procedure with 5 partitions (k-fold cross validation, k = 5). Advantageously, the data of the test, training and validation datasets may be selected from the data collection manually or automatically, respecting as far as possible the percentage subdivision referred to above for each known class.

Conveniently, the data collection may include samples of different colors and different sizes (thicknesses), if any. The plurality of dose unit images representative of all the different classifications required of the system, constituting the data collection, comprises at least a plurality of primary images of dose units acquired by one or more training image acquisition modules starting from available samples of dose units, and a plurality of synthetic images obtained by altering said plurality of primary images according to a data augmentation technique. The plurality of synthetic images includes respective graphic recognition markings altered with respect to the predetermined graphic recognition markings or with respect to the reference surface of the dose unit bearing said predetermined graphic recognition markings, represented by way of example in Fig. 6. The alteration of the graphic recognition markings may be obtained by altering only said graphic markings with respect to the reference surface of the dose unit or by overall alteration of the images. In particular, the altered graphic recognition markings have altered morphological, dimensional, positional or optical properties with respect to the predetermined graphic recognition markings, and more specifically the altered graphic recognition markings include by way of example at least one of the predetermined graphic recognition markings translated, rotated, in a reduced scale or in an enlarged scale, distorted, incomplete, blurred, with altered light intensity or brightness, with altered color or with altered contrast.

The use of synthetic images with altered graphic recognition markings advantageously allows broader training of the neural network to be conducted and greater generalization to be provided to the neural network, even in the case of acquiring images with features altered with respect to the primary images acquired starting from available samples of dose units. In particular: the generation of synthetic images having graphic recognition markings translated vertically or horizontally, for example by 0.05% with respect to a primary reference image, allows the neural network to be strengthened, making the classification invariant with respect to the position of the dose unit in the pre-chamber 60; the generation of synthetic images having rotated graphic recognition markings, for example at a variable angle between -180° and +180° with respect to a primary reference image, allows the neural network to be strengthened, making the classification invariant with respect to the rotation of the dose unit in the pre-chamber 60; the generation of synthetic images having graphic recognition markings at a reduced or increased scale, for example at a variable scale between -0.10% and +0.10% with respect to a primary reference image, allows the neural network to be strengthened, making the classification invariant with respect to the axial position of the dose unit in the pre-chamber 60; the generation of synthetic images having distorted graphic recognition markings, for example at a variable pitch between -0. lOpx and +0. lOpx with respect to a primary reference image, allows the neural network to be strengthened, making the classification invariant with respect to acquisition speed of the image of the dose unit (or at the rate of fall of the dose unit) in the pre-chamber 60; the generation of synthetic images having incomplete graphic recognition markings, for example due to the absence of some portions with respect to a primary reference image within a predetermined incompleteness limit threshold, allows the neural network to be strengthened, making the classification invariant with respect to the completeness of the graphic marking and therefore more robust in the case of abrasions to the reference surface of the dose unit bearing the graphic marking, which cause the partial cancellation thereof; the generation of synthetic images having blurred graphic recognition markings with respect to a primary reference image allows the neural network to be strengthened, making the classification invariant with respect to the focusing of the dose unit in the pre-chamber 60; the generation of synthetic images having graphic recognition markings with an altered light intensity or brightness, respectively lighter or darker, for example in a brightness range between 0.35% and 1.25% with respect to a primary reference image, allows the neural network to be strengthened, making the classification invariant with respect to the color and reflectance of the dose unit in the pre-chamber 60, which may depend on the storage conditions of the tablet, on its state of deterioration (for example due to exposure to environmental conditions of non-optimal temperature or humidity) or even on the state of deterioration and cleanliness of the illumination means of the pre-chamber and of the image acquisition optics; the generation of synthetic images having graphic recognition markings with an altered color, for example in which the color of the graphic marking is lighter than the color of the background, with respect to a primary “dual” reference image in which the color of the graphic marking is darker than the color of the background, allows the neural network to be strengthened, making the classification invariant with respect to the color and the reflectance of the dose unit in the pre-chamber 60, which may depend on the manufacturing technique of the graphic recognition marking; the generation of synthetic images having graphic recognition markings with altered contrast, respectively lighter or darker than a predetermined reference contrast between the graphic marking and the reference surface of the dose unit in a primary reference image, allows the neural network to be strengthened, making the classification invariant with respect to the color and reflectance of the graphic marking of the dose unit, which may depend on the storage conditions of the tablet, its state of deterioration or the manufacturing technique of the reference graphic marking.

If the samples of dose units of the data collection do not have a sufficiently large population representative of a predetermined class, it is also possible to prepare a plurality of additional synthetic images obtained by manipulating the graphic recognition markings of the plurality of primary images or the primary images themselves with the aim of generating images that do not belong to the same class as the original image, for example by changing the number of denotation markings 72 associated with the inscription 70, as shown in Fig. 7. This may occur, by way of example, by applying pattern recognition techniques to an image of a dose unit in order to identify the inscription 70 and locate the associated denotation markings 72 (in the rectangular and circular survey areas shown in superimposition on the images of the left column), and image overwriting techniques in order to fully or partially remove the localized denotation markings 72 (central and right column).

Advantageously, it is possible to provide that the supervised training also includes acquisition of images of a dose unit without recognition markings (with features similar to the dose units to be recognized, i.e. with a background reference surface of said dose unit consistent in color and textures, or with different features, i.e. with a background reference surface of said dose unit that is not consistent) or images in the absence of dose units.

The supervised training of the neural network, on the basis of the data collection comprising primary images and synthetic images, is conducted, for example, by applying well-known error back-propagation algorithms, for example the Adam optimization algorithm for 300 cycles (epochs) of feeding the training dataset and updating the network parameters at each group of 32 images (batch size), with a learning rate of 0.01 with a decay factor of 0.5, patience of 10 epochs, minimum learning rate equal to 0.0000002 and label smooth equal to 0.2.

The supervised training of the neural network may adopt any known algorithm for minimization of the error function, such as a Categorical CrossEntropy error function.

Preferably, the subset of feature extraction layers may be provided with at least one embedding layer to optimize the classification by switching to a supervised mode in which, in addition to the above, a “Triplet Loss” technique is employed for supervision of the space of the features output from the feature extraction subset, in order to correct the distance in the reference space of the features between reference samples, samples to be recognized as “positive”, i.e. of the same class, and samples to be rejected as “negative”, i.e. of a different class, and to separate samples very similar to those to be recognized that may be confused with unsupervised training.

Fig. 8a shows a flow chart of the operations of an initialization process of a machine for preparing and dispensing hot beverages by infusion.

Specifically, at step 100 the acquisition of a primary image of a dose unit inserted in the machine, intended for a collection of training data, takes place, and at step 120 a preprocessing of the acquired image is performed, adapted to improve the contrast thereof, without distorting the pixel value. During the pre-processing, the acquired image is for example subjected to an analysis of the histogram and Otsu thresholding at step 122, which is particularly suitable for bimodal images, i.e. for images having histograms with a clear separation between two main peaks, which happens - with reference for example to the acquisition of images in the visible wavelength range - in the case of brown dose units, in which there are possibly graphic recognition markings that are in contrast, i.e. darker or lighter than the background of the mixture of the precursor material, for example due to the effect of black or white graphic traces on a brown background or due to the effect of concave or convex surface areas, or of engravings or reliefs, respectively dark or shiny with respect to the surrounding surface of the dose unit. Subsequently, an upper limitation of the pixel values of the image (clipping) is performed at step 124, then an equalization of the image histogram with background suppression is performed at step 126, and finally a zero flattening operation is performed at step 128, to minimize the effect of induced depth due to the image illumination and acquisition (except for the case in which the graphic markings include reliefs and/or engravings).

The above may be indifferently applied to a monochromatic image in gray tones, or to color images, by separating the application of normalization or equalization in the three channels.

At the next step 140, a transformation of the image is carried out in the format and dimensions acceptable by the neural network. In fact, to ensure high performance of the system in the classification of the dose units inserted in the machine, it is important to supply the neural network for feature extraction and classification with images that have the same size features as the images used in the neural network training step. For this, a scaling of the scanned image is preferably carried out. The acquired image is for example a 640x480 pixel image, and it is reduced to a 256x256 pixel image through a first cropping operation to select the central area of 480x480 pixel size of the original image, and a second bilinear interpolation operation to transform the cropped image into a useful image of 256x256 size, or in any case of a predetermined size based on the quantity of pixel values intended to be processed in the neural network, taking into account the balancing of opposing computational needs, i.e. memory saving and computational speed, and recognition efficiency needs.

At step 160, the data augmentation technique is applied to obtain synthetic images from the primary image, and at step 180, a normalization or standardization procedure of the image is applied to the primary image and to the synthetic images, comprising the subtraction from the image of its mean value and then the division by its standard deviation. In an alternative embodiment, the image pre-processing described at step 120 may also be performed on the synthetic images, subsequently to step 160.

Then, at step 200, the completion of the acquisition of the training data collection is verified and, if so, the previously described training of the neural network is performed, herein referred to collectively as step 220.

Following the training of the network, a quantization of the neural network model is performed at step 240, by approximating the floating point values to integer values, for example 8-bit integer values, based on a predetermined zero value and a predetermined scale factor, according to the formula: real value = (int value - zero point)* scale

The quantization operation allows a more compact representation of the neural network model to be obtained and neural network calculations to be performed with less memory occupation than would be required for calculations with floating point precision, reducing the space occupied by the neural network and the classification time.

Fig. 8b shows a flow chart of the operations of a control or management process of a machine for preparing and dispensing hot beverages by infusion.

Specifically, at step 500 the acquisition of an image of a dose unit inserted in the machine is performed, and at step 520 a pre-processing of the acquired image is performed, adapted to improve the contrast thereof, without distorting the pixel value. During the pre-processing, the acquired image is subjected to the processing applied to the training images of the neural network, in particular an analysis of the histogram and Otsu thresholding at step 522, an upper limitation of the pixel values of the image (clipping) at step 524, an equalization of the image histogram with background suppression at step 526, and finally a zero flattening operation at step 528.

At the next step 540, a transformation of the image is carried out in the format and dimensions acceptable by the neural network, for example as described above by resizing the image acquired with 640x480 pixels into an image of 256x256 pixels through a first cropping operation by selection of the central area of 480x480 pixel size and a second bilinear interpolation operation for transforming the cropped image into a useful image of 256x256 size. At step 580, a normalization or standardization procedure of the image is applied, comprising the subtraction from the image of its mean value and then the division by its standard deviation.

Then, at step 640, a quantization of the image is performed by approximating the floating point values obtained by normalizing or equalizing the image to integer values, e.g. 8-bit integer values, based on a predetermined zero value and a predetermined scale factor, according to the formula: real value = (int value - zero point)* scale

Then, at step 660, the classification of the image by the neural network takes place and an inverse operation to the quantization, by applying the formula recalled above, is carried out at the end of the classification to obtain the correct distribution of probabilities of the various classes.

Advantageously, the previously described neural network configuration and the processing of the input data thereof allow an image recognition and classification process to be executed with reduced memory resources (of the order of 1-2 Mbytes), such as are typically available on board a machine for preparing and dispensing hot beverages for household use.

Finally, at step 700, at least one of the following operations based on the knowledge of the class of the dose unit introduced in the machine is performed: control of at least one parameter or operating management mode of the dose unit such as for example a parameter or operating mode of infusion of the machine or a parameter or operating mode relating to other steps, for example for the control of the compression of the dose unit during a pre-infusion step and/or during an infusion step, according to constant profiles or defined according to predetermined control curves, of the squeezing of the dose unit after infusion to eliminate residual water, of the automatic removal of the dose unit after infusion, from the infusion chamber to a collection compartment for spent units; recording and/or transmission of machine usage data indicating the class of the recognized dose unit for statistical monitoring purposes, for example for the benefit of a remote telemetry system.

It should be noted that the proposed embodiment of this invention in the foregoing discussion is provided merely by way of example and not in limitation of the present invention. A person skilled in the art may easily implement the present invention in different embodiments which do not depart from the general principles outlined herein and are therefore included within the scope of protection of the invention defined by the appended claims. This applies in particular with regard to the possibility of acquiring moving images, i.e. a sequence of images (film) of the dose unit, for example in different spatial positions along a transfer path to the infusion chamber, which represents a dimensional extension of the case described above, whereby the image recognition unit includes a neural network adapted with respect to the previously described one to receive said sequence of images as input and carry out the classification of the dose unit on the basis of the sequence of images. In one embodiment, the neural network is arranged to act separately on the single images, applying a “voting” procedure to select the most probable classification. In an alternative embodiment, the neural network is arranged to act on a shooting video file of at least part of the transfer movement of the dose unit, for example comprising frames in which the dose unit is shot in its entirety by a wide-angle camera, whereby the neural network includes a matrix stack equal to the number of video frames in the case of an uncompressed file, or is adapted to classify a compressed file including one or more reference frames (fully described images) and predictive frames including respective partial information representative only of the variations with respect to the corresponding reference image.

Of course, without prejudice to the principle of the invention, the embodiments and the details of execution may vary widely with respect to that which has been described and illustrated purely by way of non-limiting example, without thereby departing from the scope of protection of the invention defined by the appended claims.