FIELD OF INVENTION

The present invention provides a process to observe, annotate, classify and segregate unstripped bunches (USB). More particularly a palm oil milling process to observe, annotate, classify and segregate USB from empty fruit bunches (EFB) continuously in real-time mode.

BACKGROUND

Oil palm (Elaeis guineensis) is cultivated for the production of fresh fruit bunches (FFBs) due to its stability, high yield and low cost. The FFBs are then converted into a variety of products including foods, cosmetics, detergents and biofuels. To date, approximately 85% of global crude palm oil (CPO) is produced in Indonesia and Malaysia. CPO is extracted from FFBs in processing facilities known as palm oil mills. A typical milling process consists of several operational units. FFBs undergo sterilisation, threshing, digestion and pressing to produce pressed liquid and cake. The pressed liquid is clarified and purified to produce CPO, while the pressed cake undergoes nut separation, nut cracking, kernel separation and drying to produce palm kernel (PK). Most palm oil mills in Malaysia will send the PK to a kernel crushing plant for production of crude palm kernel oil (CPKO) before refinery processes where CPO and CPKO are further refined into higher quality edible oils and fats. Throughout the milling process, biomass such as palm kernel shell (PKS), empty fruit bunches (EFB), unstripped bunches (USB), pressed empty fruit bunches (PEFB) and palm pressed fibres (PPF) are generated as by-products. Meanwhile, large amounts of strong wastewater, which is known as palm oil mill effluent (POME) are produced during sterilisation and clarification operations. [Source: Hybrid Approach for Optimisation and Analysis of Palm Oil Millf

Oil losses are inevitable during mechanical processing due to various factors, one among them being inefficient machinery. Normally oil losses occur in EFB, press cake fibres, steriliser condensate, separator sludge, decanter cake, USB and spillages. Generally, the milling oil losses due to mechanical extraction are poor production control and poor FFB quality. The approximate oil loss from USB in palm oil mill is 0.05%. The results from the analysis showed that the oil loss caused by USB in six mills exceeded the approximate value. It is not possible to measure the actual value of oil lose in USB accurately as USB have various amounts of loose fruits attached to it.

[Sources: Improving Mill Oil Extraction Rate under the Malaysian National Key Economic Area \ Hassan, N.H. Muhammad, Z.A. Rahman, Improving Mill Oil Extraction Rate under the Malaysian National Key Economic Area, Palm Oil Eng. Bull. (2012) 33-47]

USB signifies that the sterilization and threshing processes are not efficient and hence need to be worked on further to reduce losses from USB. [Source: O. Walat, N.S. Bock, Palm oil mill OER and total oil losses, Palm Oil Eng. Bull. (2013) 11-16]]

USB is currently measured visually by the number of USB produced. A trained operator is placed at the EFB scrapper conveyor to manually observe and evaluate USB from EFB. The evaluation is done every 2 hours with 100 bunches being evaluated at any one time. Additionally, the EFB must be turned over by the operator using a hook so that the whole bunch can be observed visually. This manual process is inefficient and unreliable as it is heavily dependent on the human eye. It is also not safe and dangerous for the operator to use a hook at the conveyor.

Publication entitled “The effect of storage time of chopped oil palm fruit bunches on the palm oil quality, Agriculture and Agricultural Science Procedia 2: 165-172, 2014’ describes that chopped FFB improved the heat penetration into the inner layers of the fruits and help in reducing USB since the fruitlets can be detached easily. The findings also showed that chopped FFB can only be stored less than 30 minutes prior to sterilization process to obtained good quality CPO. This prior art does not disclose a process which replaces the manual means of observing, evaluating and segregating USB from. EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data . sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm FFB.

Publication entitled “Performance evaluation of automating the sterilisation process in a palm oil mill, International Journal of Computer Applications in Technology 5(l):53-66, 1992’ explored the development for the control of the sterilization process based on the USB. Evaluations were made in one of the local palm oil mills in Malaysia, based on the number of USB using the conventional method via naked eye observation. The developed system improved the capability of monitoring the process. However, the paper was published in 1992 and no image processing algorithm of any kind was used in this system. This prior art does not disclose a process which replaces the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm FFB.

Publication entitled “Image-based processing for ripeness classification of oil palm fruit, Proceeding International Conference on Science in Information Technology 8987575:23- 26, 2019’ focused on FFB ripeness classification using image-based processing. In this paper, the classification method of palm fruit is aimed to distinguish three classes of fruit ripeness, namely raw, under-ripe, and ripe. The focus of this work starts from the segmentation process by applying the thresholding using the Otsu method. Following this, the colour extraction features were employed by calculating two kind features, including the mean and standard deviation based on four-color components: red, green, blue, and grey, hence there are eight features produced. Lastly, classification is applied using the support vector machines method. This method was tested usingl60 images with the successful rate indicated by an accuracy value of 92.5%. This prior art does not disclose a process which replaces the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm FFB.

The article entitled “Automation of Palm Oil Mills” describes that the function of monitoring and regulating process operations non-manually or remotely to check or maintain the required processing conditions and product quality, the use of instrumental techniques of analysis in the laboratory and the use of computers for data processing and other miscellaneous applications. Applications where automation is justifiable and the impact of changing trends in automation technology, especially the recent influx of microprocessor-based systems, are discussed briefly. Applications in the palm oil mill where automation is fairly well- developed and is being developed, as well as ideas for future applications are reviewed. An example of a practical approach to automation in palm oil mills is given using automatic control of crude oil dilution. This prior art does not disclose a process which replaces the manual means of observing, evaluating and segregating USB from EFB and coun ting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm FFB.

The article entitled “Plantation-Wide Automation of Palm Oil Mills” describes about a laboratory-scale and pilot-scale studies on a new process for continuous sterilisation demonstrated its technical and economic viability and a commercial-scale system was subsequently built in the MPOB Palm Oil Mill Technology Centre in Labu. This prior art does not disclose a process which replaces the manual means of observing , evaluating and segregating USB from EFB and coun ting of USB percentages by visual observation of the human eye (by an operator), by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm, fruitlets from the oil palm FFB.

The continuous sterilization process provides the impetus for new paradigms in the design and operation of palm oil mills. The use of technology that is simple and uncomplicated to ensure that the system is competitively priced. It eliminates the use of sterilizer cages, rail tracks, overhead cranes, tippers, transfer carriages and tractors and thereby facilitates the design and construction of palm oil mills having significantly smaller footprints than conventional mills. This system provides a real-time process monitoring, automatic control, centralised motor control, visual surveillance / CCTV system, supervisory control and data acquisition (SCAD A) system and human-machine interface / supervisory control and data acquisition [Source: https://pro-infosys.com/palm-oil- milling-automation/ ]. This prior art does not disclose a process which replaces the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm, oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm. FFB.

Pro-Infosys Biotec can offer fundamental support by developing a logical industrial automation integration solution for palm oil mill applications such as FFB input monitoring and control system, fruitlet separation and cooking system, sterilizer and back pressure receiver control system, central monitoring and control room, sterilized fruitlet separation and control system, clarification station control system, production oil measurement system, POME measurement system, tank gauging system, boiler combustion control system, despatched oil security and tracking system, remote access/monitoring system, sand filter and plant control system, loading ramp control system, steam management control system, optimization control system for screw press and digester, optimization control system for sludge separator and decanter, oil thickness control system and moving floor system [Source: Pro-Infosys Biotec Sdn. Bhd.]. This prior art does not disclose a process which replaces the manual means of observing, evaluating and segregating USB from. EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm. FFB (efficient separation of oil. palm fruitlets from the oil palm FFB.

The article entitled “Overview of Automation in A Palm Oil Mill-the Sterilisation Process” describes about sterilisation of oil palm fruits is the first step in the sequence of processes to extract the oil. It is typically a batch process, using steam for heating or 'cooking' the fruits. Sterilisation is carried out in a way which requires the proper sequencing of valve movements for deaeration, steam inputs, steam blow-offs and condensate removal. Since a palm oil mill normally has multiple sterilisers, the task is to ensure the proper sequencing of all the associated valves. In this review, the sterilisation process is taken as a case in the study of automation in a palm oil mill. This article discusses the sterilisation process in the palm oil mill and elaborates on the control requirements. It also examines the features of some installed systems. This prior art does not disclose a process which replaces the manual means of observing, evaluating and. segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm, oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm FFB,

Intelligent Process Automation & Data Acquisition System (iPADAS) is the foundation for business excellence and innovation and provides the modern palm oil mill with a comprehensive industrial monitoring and resource planning solution much needed to gain the competitive edge, increase productivity and improved business decision making. iPADAS is at the core of increasing mill efficiency and productivity by centrally monitoring and provide real time data for mill operation and critical decision making related to the ecosystem, mill operation, production planning and quality control [Source: http://www.dolphineng.com/ipadas.html]. This prior art does not disclose a process which replaces the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm FFB.

While palm oil milling industry is trailing behind refinery and oleochemical industry in Industry 4.0, it’s indeed encouraging to see few initiatives have been originated. Sime Darby Plantation has built a pilot mill with a number of sensors, controllers, and near- infrared spectrophotometer. This plant incorporates also supervisory control and data acquisition technology — which is presently not widely used in palm oil mill — and will bring the industry up to Industry 3.0, and potentially Industry 4.0, level. There is also another palm pilot mill built by Novozymes Malaysia, a global biotechnology company headquartered in Bagsvserd outside of Copenhagen, to test their biological solutions and study mass balance flows. They also use digitalisation tools and data analysing skill to research the palm oil milling process in this pilot mill. [Source: https://www.malaymail.com/news/what-you-think/2020/ 02/ 19 / embracing-industry - 4.0-in-palm-oil-milling-hong-wai-onn/ 1838892, Embracing Industry 4.0 in palm oil milling — Hong Wai Onn] SUMMARY OF THE INVENTION

The present invention provides a palm oil milling process to observe, annotate, classify and segregate unstripped bunches (USB) continuously in real-time mode, the process including the steps of sterilizing the oil palm fresh fruit bunches (FFB) on a sterilizer for a time period of between 1 to 120 minutes at a temperature range of between 95 °C to 145 °C and a pressure range of between 1.7 x 10 ^{^} 5 pa to 3.1 x 10 ^{^} 5 pa to produce sterilized fruit bunches (SFB), threshing the SFB to strip the oil palm fruitlets from the SFB to produce USB and/or empty fruit bunches (EFB), whereby the USB and/or EFB are dropped onto an EFB conveyor to be carried to the end of the EFB conveyor, capturing images and/or video feed of the USB and/or EFB moving on the EFB conveyor using at least one imaging device, providing the images and/or video feed of the moving USB and/or EFB continuously to a deep learning algorithm to detect, annotate and classify the images and/or video feed into an USB or an EFB, wherein percentages of the USB are calculated and recorded as a moving average of 100 bunches, after the images and/or video feed has been annotated by any type of image annotation means and the USB will be automatically segregated from the EFB.

BRIEF DESCRIPTION OF THE DRAWINGS

Above recited features of the present invention may have been referred by embodiments, some of which are illustrated in the appended drawings. The appended drawings illustrate only typical embodiments of this invention and are therefore not considered limiting of its scope as the invention may perform effectively to other equally effective embodiments.

These and other features, benefits and advantages of the present invention will become apparent by reference to the following figures: -

Figure 1 illustrates the flowchart of a conventional oil palm milling process.

Figure 2 illustrates the conventional oil palm milling process.

Figure 3 illustrates the present invention.

Figure 4 illustrates the results of the deep learning algorithm of the present invention. Figure 5 illustrates the classification of images with 1 (for USB) and 0 (for EFB) using the deep learning algorithm.

Figure 6 illustrates the successful classification of the images into USB and EFB.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE PRESENT INVENTION

All prior arts as listed and referred to above do not specifically describe a process which replaces the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator). Hence, there remains a need in the art to provide a process to address the above problems or to at least provide an alternative with regards to an automated means and in real-time mode of observing, evaluating and segregating USB from EFB, specifically by using data sciences via image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (i.e. efficient separation of oil palm fruitlets from the oil palm FFB) as per present invention. Automation of USB identification and segregation via deep learning algorithm in the palm oil mill is not a common knowledge and cannot be found from the prior arts, hence, the inventors strongly believe that this invention is novel and inventive.

Background

Freshly cut fruit bunches and detached oil palm loose fruitlets are transported to the palm oil mill where they are sterilised to deactivate the lipolytic enzymes (as the quality of the oil will deteriorate due to free fatty acids, which increases through bruising and damaging of the fruitlets on FFB through harvesting and transportation to the palm oil mills), loosen the fruitlets on the bunch, soften the fruitlets, condition the kernels and cause protein to coagulate. The sterilisation process uses live steam at about 3 bar for a certain period of time. In the conventional milling process, FFB and oil palm loose fruitlets are loaded into cages and pushed into sterilizers and the FFB are cooked in batches. The cooking of the fruitlets happens using steam, which is let in by the opening of inlet valves which could be easily controlled by an automated programme. Once the fruitlets have been cooked in the cages, the steam is exhausted from the steriliser and the fruit cages are pulled out of the steriliser. The steriliser is usually a pressure vessel with the bottom part lined with liners of mild steel or stainless steel for easy replacement for wear and tear purposes. The body of the steriliser is insulated so that the heat loss is minimised.

Stripping the fruitlets from the sterilised bunches (threshing) is carried out in a thresher which is a rotating cage with bars that allows the fruitlets to pass through, but retains the empty bunches. The loosened fruitlets are collected by a conveyor below the cage and the empty bunches emerge at the end of the cage. The separated fruitlets are then fed to a digester, which is a cylindrical, steam -jacketed vessel kept at 90°C to 100°C by the injection of live steam. It is fitted with beater arms that break up the fruits and liberate the oil. The digested contents are then fed continuously to a screw press that produces a liquid stream consisting of oil, aqueous phase, and press cake containing the fruit fibre residue and the palm kernels. The liquid stream is passed to a settling tank via a vibrating screen that returns what it retains to the digester. The oil recuperated from the settling tank is first passed through a purifier to further remove the impurities from the oil and then dried using a vacuum dryer. The sludge collected in the settling tank is passed to a decanter that separates this sludge into a heavy effluent phase and a light, oily phase that is returned to the clarifier tank. This conventional process is summarised in Figure 1.

Problem Statement

Unstripped bunches (USB) are defined as empty fruit bunches (EFB) having more than 20 oil palm fruitlets which are still attached to the EFB after sterilization and threshing processes and this signifies that the sterilization and threshing processes are not efficient in the miding process. USB are currently measured visually by the human eye, whereby a trained operator is placed at the EFB conveyor to manually observe, evaluate and segregate USB from EFB. The EFB must be turned over by the operator using a hook so that the whole bunch can be observed visually. However, most operators do not flip the EFB and they just observe the EFB from the top angle only. The observation, evaluation and segregation is usually done for 100 bunches per session at an hourly or two-hourly interval. This manual process is inefficient and also unreliable as it is heavily dependent on the operator. It is also not safe and dangerous for the operator to use a hook as the conveyor moves at a fast speed.

Solution Statement

With the advancement of data analytics and instrumentation technologies, the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by an operator, visually by the human eye can be replaced by using image processing and data sciences, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) in order to provide an efficient means of processing the oil palm FFB (i.e. efficient separation of oil palm fruitlets from the oil palm FFB), to control the USB level to be below control levels as set by individual oil palm mills, preferably below 10%, in order to reduce losses from oil being lost from USB being discarded without being processed further. Conventional means vs. Present Invention

The table below illustrates the differences between conventional means (Figure 2) vs. present invention (Figure 3):

Problems from conventional means are explained as follows: - a. Observing, evaluating, segregating USB from EFB and counting percentages of USB are conventionally done by trained operators who are stationed at the EFB conveyor, hence, the consistency and accuracy will most definitely vary from one person to another. Efficiency of the operators are also affected after long hours of work, hence, observation, evaluation, segregation of USB from EFB and counting percentages of the USB will not be done at full capacity. b. USB observing, evaluating, segregating USB from EFB and counting percentages of USB are done in hourly or 2-hourly intervals for every 100 bunches per session. Hence, many bunches will travel via the EFB conveyor uninspected and percentages of USB undetermined, which does not allow for any corrective measures to be implemented and if implemented, would not be fast enough. c. USB calculations and percentages are written down manually in logbooks which requires space to store the logbooks. Past data are also not easily searchable via this means as there are so many logbooks in place. Apart from that, the operator could easily record data wrongly or miss recording some data, hence, incomplete data on USB is recorded on a daily basis. d. A dedicated worker is required to be stationed at the EFB conveyor for observing, evaluating, segregating USB from EFB and counting percentages of USB instead of being able to use these dedicated personnel for other more pressing and important sections in the palm oil milling process. e. There is a safety risk hazard when the operator uses the hook to check the bunches on the EFB conveyor and also when the operator needs to remove / segregate the USB from the conveyor to another trailer or conveyor, whereby accidents could happen as the conveyor moves at a fast speed.

Benefits of the Present Invention

The benefits of the present invention are as follows: continuous, consistent and 100% observation, evaluation and calculation of percentages of USB can be conducted using the present invention as opposed to manual means; real-time image capturing, monitoring and measuring of the percentages of the USB in the palm oil mill can be done; processing of data in real-time mode allows an automated response so that adjustment to the parameters of the processes can be done automatically and immediately at the sterilizer and/or thresher so that to increase efficiency of detachment of the oil palm fruitlets from the oil palm FFB; segregating USB from EFB via automated means instead of manual way of being removed by the operator into a separate trailer or conveyor; images and data saved into databases instead of being manually recorded in logbooks and stored in specific areas in the mill; saves manpower (cost) and time; reduces human error; efficient; and safe.

According to Figure 3 , the present invention provides a palm oil milling process to observe, annotate, classify and segregate USB continuously in real-time mode, the process including the steps of sterilizing the oil palm fresh fruit bunches (FFB) on a sterilizer (1) for a time period of between 1 to 120 minutes at a temperature range of between 95 °C to 145 °C and apressure range ofbetween 1.7 x 10 ^{^} 5 pa to 3.1 x 10 ^{^} 5 pa to produce sterilized fruit bunches (SFB), threshing the SFB to strip the oil palm fruitlets from the SFB to produce USB and/or empty fruit bunches (EFB), whereby the USB and/or EFB are dropped onto an EFB conveyor (5) to be carried to the end of the EFB conveyor (5), capturing images and/or video feed of the USB and/or EFB moving on the EFB conveyor (5) using at least one imaging device (8), providing the images and/or video feed of the moving USB and/or EFB continuously to a deep learning algorithm (9) to detect, annotate and classify the images and/or video feed into an USB or an EFB, wherein percentages of the US B are calculated and recorded as a moving average of 100 bunches, after the images and/or video feed has been annotated by any type of image annotation means and the USB will be automatically segregated from the EFB.

The Objectives of the Present Invention

The objective of the present invention is to use data sciences using image processing, machine learning (deep-learning) algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (i.e. efficient separation of oil palm fruitlets from the oil palm FFB), to produce USB below control range, preferably below 10% and to reduce losses from oil being lost from USB being discarded without further processing.

The objective of the present invention also provides continuous, consistent and 100% observation, evaluation and calculation of percentages of USB as conventional means does not allow for all EFB to be inspected 100% as the operator stops evaluation after 100 bunches in order to manually record and calculate the USB percentages and takes a break before starting the process again with another 100 bunches.

Also, the present invention provides a means to constantly monitor, evaluate and determine the effectiveness of the sterilization and threshing processes at all times of operations and corrective actions can be implemented immediately to have the oil palm fruitlets separated efficiently from the FFB. Corrective measures are sometimes not undertaken via the conventional means, and if any corrective measures are taken, it does not happen in an immediate manner (specifically in real-time mode) as per present invention.

The present invention also provides real-time image capturing, recording and storing of the USB images in determined databases. This provides automated analysis of output data in real-time mode and calculation of percentages of USB to determine corrective measures to be undertaken immediately. Generating data in real-time allows for actions to be taken immediately in order for parameters of the processes at the sterilizer and thresher to be adjusted to be more effective and efficient.

The present invention further provides an automated means to segregate USB from EFB for the USB to go through the sterilization and threshing processes all over again to enable all oil palm fruitlets to be effectively detached from the bunches to prevent further oil loss from the USB being discarded without being processed further.

Also, the present invention provides a means of monitoring via electronic devices such as a mobile device, laptop / desktop or any compatible devices whether at the mill or away from the mill in order to monitor or be aware of the situation at the mill in real-time mode. The parameters or any corrective measures to take can also be adjusted using any electronic devices or any compatible devices whenever necessary, whether at the mill or away from the palm oil mill.

The present invention further provides data to be recorded automatically in databases unlike conventional means whereby the data is recorded manually in logbooks and it is usually not complete (nor reliable) as sometimes data recording could be missed by the operators and observations are also not 100% done. Recording data manually takes a lot of space and cannot be searched easily and/or historical data traced back whenever needed. Digital data as generated doesn ^’ t take up any space and can be viewed / analysed / searched at any point in time without any problems, and from any location as long there is access to the server.

Further, the present invention also provides a means to reduce reliance on human as there is no need for a dedicated operator to be placed at the EFB conveyor, also a means to overcome labour shortage currently faced in the palm oil industry.

Further Information

The present invention provides a palm oil milling process to observe, annotate, classify and segregate USB continuously in real-time mode which includes the following steps:

Sterilizing the oil palm FFB on a sterilizer (1) for a time period of between 1 to 120 minutes, preferably between 80 to 120 minutes at a temperature range of between 95 °C to 145 °C and a pressure range of between 1.7 x 10 ^{^} 5 pa to 3.1 x 10 ^{^} 5 pa (25 to 45 psig) to produce sterilized fruit bunches (SFB). Sterilization happens as the oil palm FFB moves on the sterilizer (1). The speed range of the sterilizer (1) is usually in a range of between 10 to 60 rpm but it can differ from mill to mill depending on the size, gear box and revolutions per minute (rpm) of the sterilizer (1) and also mill throughput.

The SFB is then carried on the SFB conveyor (2) to the thresher (3). The SFB goes through the threshing process in order to strip the oil palm fruitlets from the SFB to produce USB and/or EFB, whereby the USB and/or EFB are then dropped onto an EFB conveyor (5) to be carried to the end of the EFB conveyor (5). Speed of the thresher (3) is generally in a range of between 18 to 28 rpm but can vary from mill to mill depending on the size of the thresher, mill throughput and others.

The images and/or video feed of the USB and/or EFB moving on the EFB conveyor (5) are captured using at least one imaging device (8). “Imaging device(s)” for this present invention means any and all types of instruments (mechanical, di gital or electronic) which are capable of recording, storing, viewing or transmitting visual images such as still camera, motion picture camera, video camera, camcorder or any other instruments or format which are capable of recording, storing, viewing or transmitting the visual images. Any camera can be used for the present invention, preferably a high -re solution video or CCTV (closed circuit television) camera to provide sharp and accurate data for interpretation by the specific algorithm designed to be used. The images and/or video feed of the moving USB and/or EFB are continuously provided to a deep learning algorithm (9) to detect, annotate and classify the images and/or video feed into an USB or an EFB. At least one imaging device (8) such as a video camera is placed on top of the EFB conveyor (5) or multiple video cameras placed at various angles from the EFB conveyor (5) can be used to capture images and/or video feed of the USB and/or EFB while travelling on the EFB conveyor (5).

Percentages of the USB are calculated and recorded as a moving average of 100 bunches after the images and/or video feed has been annotated by any type of image annotation means and the USB will then be automatically segregated from the EFB via artificial intelligence means using any type of mechanical flap or segregation means into a separate conveyor or USB trailer (6) to be brought forth to the beginning of the milling process to go through the sterilization and/or threshing processes all over again based on the signal received from a programme logic controller (10) in the control room of the palm oil mill. The data is inputted to the programme logic controller (10) from the deep learning algorithm (9) in order for the signal to be released from the programme logic controller (TO). The EFB will be segregated into a EFB trailer (7) and discarded separately.

If the percentages of the USB are higher than a pre- determined control range as determined by an individual mill for example at 10%, a signal and alarm will be transmitted out automatically by the programme logic controller (10) based on data received from the deep learning algorithm (9), to adjust parameters of the sterilizing and/or threshing processes accordingly. The sterilization time would need to be increased if the sterilization process is inefficient which results in USB of more than 10% and this would be done by adjusting the speed of the sterilizer (1). The speed for the sterilizer (1) would need to be reduced in order to increase sterilization time which will enable the sterilization or soaking time to be increased and hence making the sterilization process more efficient and effective. The speed of the thresher (3) would also be reduced to allow for a longer threshing period. Further, adjustment to the pressure would also be done to ensure both sterilization and threshing processes are at its optimum level to provide for an efficient separation of oil palm fruitlets from the oil palm FFB.

Images and/or video feed (live stream and also recorded) are continuously provided by the at least one imaging device (8) to the deep learning algorithm (9) to determine whether the bunches are EFB or USB and to calculate USB percentages for every 100 bunches. The percentages will be automatically saved into digital storage spaces and there is no requirement or need for manual recording. The calculated percentages of the USB are automatically saved into digital storage spaces and the images and/or video feed are automatically saved into digital storage spaces. The data will be displayed, and stored in plurality of storage of electronic device, portable mobile devices, cloud computing network or any combination thereof.

“Deep -learning algorithm” for this present invention means machine learning algorithm use for detection, extraction and classification of the images (i.e. USB and EFB) captured by any types of imaging devices (8) of the present invention. Any types or means of deep learning algorithm (9) can be used for this purpose, as long the algorithm is able to identify objects based on a training dataset which is used to train the machine learning algorithm, which is similar to a facial recognition algorithm / software.

The use and application of deep learning algorithm (9) is not used or applied in palm oil mills to-date, specifically to replace the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator) by using data sciences using image processing, machine learning algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (i.e. efficient separation of oil palm fruitlets from the oil palm FFB).

The software which consists of a deep learning algorithm (9) used for this present invention is a You Only Book Once’ (YOFO) which is an object detection system targeted for real-time processing. The YOLO algorithm has been tested with the testing data for 249 samples and the efficiency of the classification is at least 97.2 % for this present invention, Fieure 4 is a confusion matrix which illustrates the results of the deep learning algorithm (9) of the present invention, whereby out of 249 times, the deep learning algorithm only failed to accurately identify the EFB and USB a total of 3 and 4 times respectively. Images captured are extracted for analysis and labelled accordingly using any image labelling software to transform images into computer vision models. Deep learning algorithm (9) has been set to classify using different codes such as 1 for USB and 0 for EFB for classification, localization and detection of the USB and EFB.

Next is the pre-processing and image annotation of the images using any types of image annotation means. The bounding box is the most popular image annotation means used in machine learning and deep learning as the bounding box annotators will outline the object in the box per machine learning requirements. The bounding box annotation means is also the simplest, least time consuming and cheapest method in the industry at the moment.

Purpose of the image annotation is to apply the deep learning algorithm (9) using the computer vision techniques to obtain and classify the images of interest (i.e. USB and EFB) in real time mode for the purposes of the present invention. Evaluation by the deep learning algorithm (9) is to be able to classify USB and EFB via training dataset to train a machine learning algorithm to process the data for machine learning use. For the purposes of the present invention, USB and EFB codes of 1 and 0 have been trained and hence produced very accurate results via the machine learning means. (See Figure 5)

After that the images are then successfully classified into USB and EFB as per Figure 6. Finally, the images are saved in the respective folders with image files for future reference.

Overall accuracy and precision of this system of the present invention is at least 97% or higher.

The sterilized oil palm fruitlets will then proceed to the rest of the milling process, to the digestion process to produce substantially digested palm fruits and pressing the digested palm fruits to produce pressed palm fruits, the pressing of the oil palm fruits produces undiluted crude palm fruit oil (UDCO). UDCO is mixed with dilution water to produce diluted crude palm fruit oil and further clarified, centrifuged, purified and dried to produce crude palm fruit oil, crude palm oil fraction and combinations thereof.

Summary:

In summary, there’s a need to replace the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by an operator, visually by the human eye via data sciences using image processing, machine learning (deep- learning) algorithm and artificial intelligence (to automatically segregate USB from EFB and to automatically adjust parameters of the sterilization and/or threshing processes in the palm oil mill) to provide an efficient means of processing the oil palm FFB (efficient separation of oil palm fruitlets from the oil palm FFB), preferably to produce USB below 10% and to reduce losses from oil being lost from USB being discarded without further processing.

It can be appreciated that the use and application of deep learning algorithm (9) is not used or applied in palm oil mills to-date, specifically to replace the manual means of observing, evaluating and segregating USB from EFB and counting of USB percentages by visual observation of the human eye (by an operator) (4). Therefore, the present invention represents an opportunity for the palm oil industry to move towards digitalisation and automation towards Industry 4.0 to enhance overall operational efficiency of the industry and improve productivity.

It can also be appreciated that the present invention provides continuous, consistent and 100% observation, evaluation and calculation of percentages of USB as conventional means does not allow for all EFB to be inspected 100%. Also, the present invention provides a means to determine the effectiveness of the sterilization and threshing processes and corrective actions can be implemented immediately to have the oil palm fruitlets separated efficiently from the FFB. Corrective measures are sometimes not undertaken via the conventional means, and if corrective measures are taken - does not happen in an immediate manner (specifically in real-time mode) as per present invention.

Further, it can be appreciated that the parameters for the present invention are not obvious for a person skilled in the art and have been determined by the inventors based on numerous trials conducted, observations, discussions with combined expertise and experience in this field, which parameters and/or combination could not be determined without much efforts, testing and/or analysis or by just reviewing prior art documents in this field of interest.

Various modifications to these embodiments as described herein are apparent to those skilled in the art from the description and the accompanying drawings. The description is not intended to be limited to these embodiments as shown with the accompanying drawings but is to provide the broadest scope possible as consistent with the novel and inventive features disclosed. Accordingly, the invention is anticipated to hold on to all other such alternatives, modifications and variations that fall within the scope of the present invention and claims.