Inventors; Ze'ev Schmilovitoh, Asher Levi, Timea Ignat, Aharon Hoffman, Halm Egozi, Rail Regev, Yoseph Kashti and Farhad Geoola

Title: Grading Peanuts by Thermal Imaging

This patent application claims priority from, and the benefit of, U. S. Provisional Patent Application No. 62/060,620, filed October 7, 2014, which is incorporated In. its entirety as if fully set forth herein.

FIELD OF THE INVENTION

The present invention relates to a peanut grading and, more particularly, to a method and system for grading peanuts using thermal imaging.

BACKGROUND

In-shell peanuts make up a major portion of Israel's peanut export industry. It is not desirable to have pods with only one mature kernel among the peanuts that are marketed in-shell. Besides for the obvious quality issue, there are other considerations and reasons for separating single kernel / seed in-shell peanuts. For example, in-shell peanuts are usually roasted before marketing at the final destination. The heating in the oven causes burning of the unfilled end, thereby reducing the quality of the product as well increasing the number of rejections, resulting in the loss of a great deal of money to the exporters.

The packing houses attempt to separate the half full pods ("dead-end") using several methods such as air separators and "singles" drum separators. However, these methods are not sufficiently effective.

SUMMARY OF THE INVENTION

According to the present invention there is provided a method for grading peanut pods using thermal imaging, the method including the steps of: (a) heating the peanut pods for a predetermined heating time such that the peanut pods have a higher temperature than an ambient temperature; (b) allowing the peanut, pods to cool in the ambient temperature for a predetermined cooling time; (c) capturing a thermal image of the peanut pods after the predetermined cooling time has lapsed; and (d) determining occupancy level of each of the peanut pods, based on the thermal image.

According to further features in preferred embodiments of the invention described below the heating is thermal heating or microwave heating.

According to still further features in the described preferred embodiments the predetermined heating time for the thermal heating is in a range between 10 seconds and 40 seconds at a heating temperature in a range between 100º C and 160º C or between 15 seconds and 35 seconds at a heating temperature in a range between 110° C and 150° C. The predetermined cooling time for after the thermal heating is in a range between one and fifteen minutes or between five and ten minutes.

According to still further features the predetermined heating time for the microwave heating is in a range between 2 seconds and 20 seconds or between. 5 seconds and 15 seconds. The predetermined cooling time for the microwave heating is in a range between one and fifteen seconds or between five and ten seconds.

According to still further features the thermal image is captured by a thermal infra-red camera.

According to still further features the step of determining the occupancy level includes processing the thermal image and analyzing colors of the thermal image wherein relatively brighter colors indicate higher temperatures and relatively darker colors indicate lower temperatures, such that colors brighter than a predetermined threshold indicate occupancy and colors darker than the predetermined threshold indicate vacancy.

According to still further features the occupancy level is selected from the group of levels comprising: a full pod, a half-full pod and an empty pod. where occupancy of each of a first and a second zone of a single peanut pod indicates the full pod occupancy level; and wherein occupancy of only one of the first zone and the second zone indicates the half-full pod occupancy level; and wherein the vacancy of both of the first zone and the second zone indicates the empty occupancy level.

According to still further features the method further includes the step of: (e) sorting the peanut pods according to the occupancy level.

According to another embodiment there is provided a method for grading peanut pods using diermal imaging, the method including the steps of: (a) cooling the peanut pods for a predetermined time such that the peanut pods have a lower temperature than an ambient temperature; (b) allowing the peanut pods to warm in the ambient temperature for a predetermined warming time; (c) capturing a thermal image of the peanut pods after the predetermined warming time; and (d) determining an occupancy level of each of the peanut pods, based on the thermal image.

According to further features the thermal image is captured by a thermal infra- red camera.

According to still further features the step of determining the occupancy level includes processing the thermal image and analyzing colors of pixels of the thermal image wherein relatively brighter colors of the pixels indicate higher temperatures and relatively darker colors of the pixels indicate lower temperatures, such that colors brighter than a predetermined threshold indicate vacancy and colors darker than the predetermined threshold indicate occupancy.

According to still further features the occupancy level is selected from the group of levels comprising: a full pod, a half-full pod and an empty pod, where occupancy of each of a first and a second zone of a single peanut pod indicates the full pod occupancy level; and wherein occupancy of only one of the first zone and the second zone indicates the half-full pod occupancy level; and wherein the vacancy of both of the first zone and the second zone indicates the empty occupancy level.

According to still further features the method further includes the step of: (e) sorting the peanut pods according to the occupancy level.

According to another embodiment there is provided a system for grading peanut pods using thermal imaging, the system including: (a) a temperature controlling element capable of altering a temperature of the peanut pods; (b) a thermal imaging device configured to capture thermal images of the peanut pods, the thermal images being acquired after the temperature controlling element has altered the temperature of the peanut pods; and (c) a processing unit including an image processor, the image processor configured to receive the thermal images from the thermal imaging device and analyze a color gradient of the thermal images of the peanut pods in order to determine an occupancy level of each of the peanut pods.

According to still further features the thermal images are captured after the temperature controlling element has altered the temperature of the peanut pods and a predetermined time has lapsed so as to allow the peanut pods to partially return to an ambient temperature.

According to still further features the temperature controlling element is a heating element selected from the group of a thermal heating element and a microwave heating element, wherein the color gradient indicates occupancy if brighter than a predetermined gradient and vacancy if darker than the predetermined gradient- According to still further features the temperature controlling element is a cooling element wherein the color gradient indicates vacancy if brighter than a predetermined gradient and occupancy if darker than the predetermined gradient.

According to still further features the processing unit determines the occupancy level based of whether each of a first zone and a second zone of a single peanut pod is occupied or vacant.

According to still further features the system further includes: (d) a sorting device adapted to sort the peanut pods, the sorting device configured to sort the peanut pods according to the occupancy levels of the peanut pods.

The present invention discloses an innovative system and method which uses a thermal camera (e.g. IR imaging) which can detect thermal differences in the temperature of an object. The camera is able to acquire an image of a pod as well as the temperature of the various parts of the pod. By heating or cooling the pods for a short period of time, the image or images captured by the thermal camera show the different changes in temperature of the various pods as they return to ambient temperature. It can be determined from these images whether the pod is full, empty or half full.

DEFINITIONS

In order to clarify the language used herein, the following definitions relate to the listed terms.

Peanut pod, in-shell peanut - the peanut pod is the legume known as Arachis hypogaea which contains 1 to 4 seeds which are commonly referred to as peanuts. The major grading application of immediate invention is generally applied to pods appearing to contain 2 seeds (based on size and shape of the pod), although the same method and system can be used for pods appearing to contain three or four seeds.

First zone and second zone - refers to each of the "sections" of the 2-seed pod.

Peanut seed, kernel - the edible member of the legume. The terms 'seed ⁴ and 'kernel' are used interchangeably herein. The seed is commonly referred to as a peanut, even though the term peanut actually refers to the peanut pod. 2-seed pod - the general application of the immediate invention is for grading and/or sorting 2-seed pods which are peanut pods that appear (based on size and shape) to include 2 seeds.

Occupied, occupancy - including a seed. As used herein, the term occupied refers to the existence of a mature seed or kernel in the pod. A zone / pod including a partially formed seed, & deformed seed, undeveloped kernel, a decayed kernel, a shrunken kernel etc., are not included in the definition of occupied. I.e. a zone which has a seed which is not fully formed and/or matured is not considered occupied.

Vacant, vacancy - lacking a seed. Even if the zone includes a partially formed seed / kernel or a deformed seed etc., the zone is still considered, as defined herein, to be vacant.

Full peanut pod - a 2-seed pod including two seeds, one in each zone.

Half-full peanut pot - a 2-seed pod including only one seed. In such a pod, one zone is "occupied" with the seed and the other zone is "vacant''.

Empty peanut pod - a 2-seed pod where both zones are vacant, i.e. there is no seed in either zone.

The immediate invention is described below with reference to 2-seed pods which have a first zone and a second zone and having occupancy levels of a full pod (2 seeds), half-full pod (1 seed fully formed and one seed partially formed, deformed or unformed) and an empty pod (no seeds formed, i.e. both zones are empty or contain un-matured seeds / kernels). Nonetheless, the immediate system and method described above can be applied to 3-seed pods and 4-seed pods in a similar fashion. The 3-seed pods have three zones and occupancy levels of full, one-third full, two- thirds full and empty. The 4-seed pods have four zones and occupancy levels of full, one-quarter full, half-full (2 mature seeds and 2 unformed, partially formed or deformed seeds), three-quarters full and empty pods.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are herein described, by way of example only, with reference to the accompanying drawings, wherein:

FIO. la is a thermal image of a full peanut pod taken a few seconds after a short period of heating;

FIG. lb is a thermal image of a half-full peanut pod taken a few seconds after a short period of heating; FIG. 1c is a thermal image of an empty peanut pod taken a few seconds after a short period of heating;

FIG. 2a is a grayscale display of a thermal image of peanut pods;

FIG. 2b is a graph depicting the cool-down rate of the different types of pods over time;

FIG. 3 is a flow chart of the method of the immediate invention;

FIG. 4 is a diagram of one embodiment of the innovative system;

FIG. 5 is a diagram of a second embodiment of the innovative system;

FIG. 6 is a diagram of a third embodiment of the innovative system.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The principles and operation of a method and system for sorting in-shell peanuts using thermal imaging according to the present invention may be better understood with reference to the drawings and the accompanying description- Referring now to the drawings, Figure la is a thermal image of a peanut pod taken, a few seconds after a short period of heating. The thermal image (as well as those shown in FIGs 1b and lc) is rendered into a grayscale image where brighter pixels represent higher temperatures and darker pixels represent lower temperatures. The peanut pod is full (i.e. including two peanuts). As a result, the entire pod is brightly lit in a uniform manner.

Figure lb is a thermal image of a peanut pod which is half empty (i.e. including only one fully formed kernel). This type of peanut is referred to, in the industry, as a "dead-end". The full half of the pod, on the left hand side of the image, is brightly lit while the empty side is dimmer indicating a different thermal gradient / color due to the different temperature.

Figure lc is a thermal image of a peanut pod which, is empty. The thermal image of the pod is dimmer than the image of the full pod, indicating a lower temperature at the empty pod. The coloring is more or less uniform.

The thermal camera can be an Infrared (IR) CCD camera. The in-shell peanuts / peanut pods can be heated or cooled for a short period of time using any well-known method such as hot air stream or microwave heating or passing through a cooled chamber etc.

The novel sorting method suggested is based on changing the pod temperature (heating or cooling) and letting it balance in time with the ambient temperature. This process can be employed in short time. Images taken by the thermal camera during this process reveals differences in temperature between full or empty sections of the pods. The empty or vacant areas return to the ambient room temperature quicker than the full or occupied areas. The thermal image is capture during the window of time where the vacant section has returned (or nearly returned) to the ambient temperature while the occupied section has not. An algorithm of machine imaging is then applied to the images for sorting out the empty or half full pods from the full pods.

Experiment 1

Two experiments where performed to verify the system arid method. In the first experiment, the pods were heated with a stream of hot air at a temperature of 140 °C for 20 seconds.

Thermal Infra-red images where taken of the pods as they cooled down. A thermal camera ThermaCAM model SC6SS from FLIR Systems, Inc., Wilsonville, Oregon, USA was used to capture the images.

It took five minutes for a discernible difference between the full and empty segments to be observed. Images of peanuts were separated using an image segmentation processing algorithm. An analysis of the gray levels in each of the zones of the pod revealed differences in the temperatures of the zones in the pods after the aforementioned time interval. The empty part of the pod is cooler (darker) revealing the lack of seed.

Experiment 2

In the second experiment, the peanut pods were heated for 5 seconds in a 600W microwave oven. After five seconds, the thermal differential between the occupied parts of the pods and the empty parts of the pods was discernible.

Here too, the thermal camera ThermaCAM model SC655 from FLIR Systems, Inc., Wilsonville, Oregon, USA was used to capture the images after the

aforementioned time interval. An image processor converts the thermal infra-red image (TIR) into a grayscale image. The processing unit performs an analysis of the gray levels of the pixels in the image. If all the pixels are within a similar range then the pod is either full or empty. If the gray levels of the pixels are above a predefined threshold then the pod is full. If the levels are below the threshold then the pod is empty. If approximately half of the pixels are in a first gray level range and the other half arc in a second gray level range then one zone of the pod is occupied while the other is vacant. The empty part of the pod is cooler (darker) revealing the lack of seed. The occupied part of the pod is warmer (brighter) as it contains a mature seed. The empty part of the pod. cools down faster than the occupied side of the pod as the seed has a larger mass and retains the heat for a longer period. In an alternative method, the algorithm can divide the pod into two sections and compare the gray scale levels of each of the sections with similar results to those described above.

Figure 2a is a grayscale display of a thermal image of peanut pods which illustrates the different gray levels in the pods between the empty and full zones. The empty zones are darker, meaning that the empty section of the pods is cooler than the occupied section. In the Figure, the darker section of right-hand pod is highlighted with a circle for clarity. Based on the fact that the pod includes one darker zone and one lighter (brighter) zone, the processing unit of the system concludes that the pod includes a vacant zone (left zone) and an occupied zone (right zone), The TIR that is represented by the images in the Figure was taken after a short heating period similar to that discussed above for Experiment 2. The occupied zone contains a mature seed which retains heat for longer than the vacant zone which cools faster. Consequently, the occupied zone is more brightly lit and the vacant zone is darker. The left-hand pod includes 2 mature seeds and both zones are therefore brightly lit.

Figure 2b illustrates a graph depicting the cool-down rate of the different types of pods (full, half and empty) over time (in seconds), for the second experiment. Once the pods have been briefly heated the various pods begin to cool at different rates. As is visible from the graph, after five seconds, the temperatures of the different types of pods diverge sufficiently for the imaging processor to discern what type of pod it being observed.

The half-full pod including one seed reaches the highest temperature before beginning to cool as the single kernel retains heat for longer than the empty pod. The empty pod also heats quickly, but immediately begins to lose heat during the charted cooling period. After 5 seconds the pod has cooled significantly and after another 5 seconds evens out at ambient room temperature.

The full pod takes longest to heat having the largest mass and therefore reaches the lowest temperature of the three pods after the short heating period. On the other hand, the full pod also takes the longest to cool down as the larger mass retains the heat for longer. After 5 seconds, the different temperatures of the full and half-full pods become evident. Between 5 and 15 seconds the difference is clearly notable, with the difference between the pods continuing, albeit to a lesser degree, for at least another 30 seconds, as evident from the graph.

The image processing is extremely rapid and can even be accomplished while the pod is falling from the conveyer belt to the sorting machine. The sorting machine receives instructions from the processing unit that is responsible for image processing. The instructions instruct the machine to sort the pod according to the pods type (full, half or empty) as calculated by the processing unit. Figure 3 is a flow chart of the method of the immediate invention, in a preferred embodiment. In step 10 the conveyor (e.g. a conveyer belt or a vibrating conveyor feeding or some other conveying mechanism) feeds the pods through the system. In step 12 the pods are exposed to a temperature altering element for a predetermined amount of time. In one embodiment, the temperature altering element is a heating element such as a thermal heater or a microwave heater. In an alternative embodiment the temperature altering element is a cooling element such as an air conditioning device, a refrigerating mechanism or a cooling chamber, etc.

in the former embodiment, the peanut pods are heated, in step 12, for a predetermined heating rime at a predetermined temperature until the peanut pods have a higher temperature than the ambient room temperature.

In step 14 the pods are allowed to cool in the ambient room temperature for predetermined cooling time. Exemplarily, the conveyer belt is manufactured at a particular length and advanced at a calculated speed so that the predetermined cooling time is afforded between the heating step and the next step.

Exemplarily, the predetermined heating time, using a thermal heater, is in a range between 10 seconds and 40 seconds at a heating temperature in a range between 100° C and 160° C. Further exemplarily, the predetermined heating time for the thermal heating is in a range between 15 seconds and 35 seconds at a heating temperature in a range between 110° C and 150° C. For thermal heating, the predetermined cooling time is preferably between one and fifteen minutes. In other embodiments, the predetermined cooling time is in a range between five and ten minutes.

Alternatively, in preferred embodiments for microwave heating, the predetennincd heating time is in a range between 2 seconds and 20 seconds. In other embodiments, the predetermined heating time is in a range between 5 seconds and 15 seconds, !n preferred embodiments, the predetermined cooling time is in a range between one and fifteen seconds. In other embodiments, the predetermined cooling time is in a range between five and ten seconds.

In step 16 at least one thermal image of the pods is captured by a thermal imaging device such as the T1R camera discussed elsewhere. The thermal image is captured after the predetermined cooling time has lapsed.

In step 18 a processing unit (including at least an image processor) processes the thermal image in a manner that allows the processing unit to make a determination as to the occupancy level of each of the peanut pods captured in the image. The determination is based on identifying the warmer and cooler regions of the pods where warmer regions are depicted as being brighter and cooler regions are depicted as being darker. The brighter zone indicates that the zone is occupied by a mature seed / kernel. The darker zone indicates that the zone is vacant of a mature seed (even though the zone may contain a partially formed, or deformed seed, is discussed above).

If a 2-seed pod has two bright regions then the pod is determined to have a full occupancy level. If the pod only has one bright region and the other region is darker, then the pod is determined to be a "dead-end" as it has only a half-full occupancy level. If both regions (or zones, as they regions are referred to herein) of the 2-seed pod are darker than a predetermined threshold (e.g. a given grayscale color gradient), then the pod is determined to be empty.

In the latter embodiment, where the temperature altering element is a cooling element, in step 12 the pods are exposed to a cooling element for a predetermined amount of time which is sufficient for the pods to have lower temperature than the ambient room temperature.

In step 14, the pods are allowed to warm up in the ambient room temperature for a predetermined warming time calculated to warm the vacant zones of the pods while the occupied zones remain distinguishably cooler.

In step 16 the pods are imaged with the thermal imaging device after the predetermined warming time and before the occupied zones warm to the ambient temperature.

In step 18 the images are processed to determine the occupancy levels of the pods. The process is the same as described for the heating embodiment, although the results are opposite. The vacant zones retain the cold for a shorter period of time, as they have less mass. The occupied zones remain cold for longer is the larger mass takes longer to warm. Therefore the colors brighter than a predetermined threshold indicate vacancy and colors darker than that predetermined threshold indicate occupancy.

In some embodiments, including both the former and latter embodiments, the process continues with a sorting of the graded pods. In step 20, a sorting command is sent to a sorting device. In some embodiments, if no sorting action is needed then the sorting command is not sent. In other embodiments, even if a sorting operation is not needed, a sorting command is still sent. Here, the sorting command instructs the sorter not to activate. Where relevant, in step 22, the sorting mechanism performs a sorting operation. As both steps 20 and 22 are optional, they are depicted in the Figure with a broken-line border. In step 24, either a full pod or a half-full pod exits the system via Exit 1. In step 26 other pod (either half-full or full) exits the system via Exit 2.

One example of a sorting device is an air pulse nozzle which dispenses a pulse or burst of pressurized air on command. In an. exemplary embodiment, discussed in. greater detail below, a pod falls from the conveyor between steps 14 and 1.6. During the fall the pod is imaged by the camera, the image processed and a determination made regarding the occupancy level of the pod, and before the pod lands on a lower conveyer belt, the pod is either subjected to a pulse of air from the sorter mechanism, or simply allowed to land on a second conveyer belt, un-harassed. The pulse of air changes the trajectory of the pod, so that it lands on a different conveyer belt. Other sorting methods are known in the art and considered within the scope of the invention.

A diagram of one embodiment of a system of the immediate invention is depicted in the diagram of Figure 4. In Figure 4, the system configuration includes the following components: a conveyer belt 102, a temperature chamber / heating element / temperature controlling element 104, thermal camera 106, sorting device 108 and processing unit 110, including an image processor (not shown).

The conveyer belt 102 conveys peanut pods 101. Temperature controlling element 104 is depicted as microwave device. This is merely exemplary and it made clear that the heating device may be a thermal heating device such as a heating coil etc. or a cooling device such as an air conditioning or refrigerating unit. The microwave device heats the pods for a predetermined amount of time, such as S seconds.

The conveyer belt continues to convey the peanuts to a drop area 114 where the pods fall from the conveyer belt. In the first stage of the fall, the thermal camera 106 captures at least one TIR image of each pod. More than one pod can be imaged at the same time. If multiple pods are captured in a single image, an image segmentation processing algorithm is used to separate the pods. The camera relays the image to the processor 110 (including the image processor 112) which determines whether the pod is full or half mil and the sorting command is then relayed to the sorting device 108. Exemplarily, the sorting device can be a pulse air nozzle which issues a pulse of air to correct the fall trajectory of the pod according to the sorting command.

For example, full pods arc allowed to fall without being subjected to a pulse of air from the air nozzle. A divider 116 ensures that the pod falls onto a right-moving conveyer belt 118R which conveys to the pod to the collection area for full pods. A half-full pod is recognized by the processor which sends a command to the air nozzle to issue a precisely timed pulse which alters the trajectory of the pod's fall so that the pod lands on a left-moving conveyer belt 118L which conveys the defective pod to a gathering area for the "dead end" pods. It is made clear that other possible sorting mechanisms and/or methods can be used to sort the pods based on the sorting command resulting from the analysis of the thermal image.

Another possible configuration of the system in shown in Figure S. Figure 5 is a diagram of a second embodiment of the innovative system which depicts a similar system to that depicted in FIG. 4 and the same reference symbols have been used for the same elements. The heating element is excmplarily depicted in the Figure is a thermal heating element 104' including a heating coil. Of course, any type of effective heating element may be used in substitution. The conveying mechanism in the Figure is a vibrating feeding conveyor 102' which is vibrated by vibrators 113. The conveyor conveys and orientates the pods in a sequential, longitudinal orientation for improved efficiency and uniform presentation of the pods to the imaging device. In one embodiment the pods fall one at a time. The remaining elements in the Figure are substantially equivalent to the corresponding elements in FIG. 4.

Yet another configuration is shown in Figure 6. FIG. 6 is a diagram of a third embodiment of the innovative system which illustrates an isometric view of a system 200 that exemplarily includes four conveyor channels and four sorting devices. A four-channeled vibrating feeding conveyor 202 conveys longitudinally orientated pods 201 through a temperature controlling element, such as a cooling chamber 204. As the pods fall from the conveyor, a TIR camera (or other thermal imaging device) 206 captures a TIR image of the pod. The image is relayed to processor 210 which includes an imaging processor (not shown).

As discussed above, the processor determines which pods are full and which are half full based on the thermal images (the current and preceding exemplarily systems relate to systems configured to sort between full and half-full pods, where empty pods have been removed from the pool of pods by other methods such as air separation, known in the art). Exemplarily, full pods are allow to fall without being disturbed by the sorting device and land on a right moving conveyer belt 218R. Pods determined by the processor to be half full are subjected to a blast of air from the sir pulse nozzle sorter device 208 which 'pushes' the pods to land on a left-moving conveyer belt 218L.

In the immediate configurations, the four pods can even be sorted simultaneously. One pod is propelled down each channel of conveyor 202 and the pods drop simultaneously (or in a non-synchronous manner) in front of camera 206. A single thermal image can capture all four pods, and using the aforementioned image segmentation processing algorithm (or some similar method), the processor separates each pod in the image. The images of the pods are analyzed to determine whether they are full or half-full and a respective sorting command (e.g. either "issue a pulse" or "do not issue a pulse"; alternatively no command is issued if no pulse is needed) is issued to the corresponding sorting device positioned along the fall path of the respective pod. A pulse is either issued or not, for each of the four pods - even simultaneously. The receiving conveyer belts (right and left moving belts) are sufficiently wide to receive all of the pods which arc processed in parallel. If the processing demands are two high for a single processing unit, the two or more parallel processors can be employed. While the invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications and other applications of the invention may be made. Therefore, the claimed invention as recited in the claims that follow is not limited to the embodiments described herein.