Login| Sign Up| Help| Contact|

Patent Searching and Data

Document Type and Number:
WIPO Patent Application WO/2023/234904
Kind Code:
The invention relates to a cost-effective, user-friendly data collection and processing system for electronic-controlled machines, robotic mechanisms, and other industrial equipment used in production. It eliminates the need for different software and hardware, making it suitable for industrial field applications.

Application Number:
Publication Date:
December 07, 2023
Filing Date:
May 31, 2023
Export Citation:
Click for automatic bibliography generation   Help
International Classes:
G05B19/418; G06N20/00; G06T1/00; G06F11/00
Domestic Patent References:
Foreign References:
Attorney, Agent or Firm:
AKKAS, Ahmet (TR)
Download PDF:
CLAIMS An electronic control system for data collection and processing from electronic controlled machines, robotic mechanisms, and devices used in industrial production, and is characterized by;

- At least one camera positioned to monitor the screens of electronic controlled machines, robotic mechanisms, and devices.

- Determination of the start and end times of workers' shifts.

- Collection of machine downtime reason data from machine operators during production, inputting the data into a data collection unit, processing the collected data using artificial intelligence in the information processing unit to perform quality management and machine maintenance analysis.

- Instant recording of processed data into a database using loT and integration with ERP programs.

- Integration with RFID kit for tracking and identification purposes.

- Selection of image processing parameters, such as expansion and erosion options, to determine how images will be processed in each selected section.

- Real-time visualization of the image in black and white by modifying the parameter values.

- Expanding the edges of the image to cover a larger area using the expansion feature.

- Thinning the edges of the image using the erosion feature.

- Finding suitable parameter values by adjusting these features.

- Saving the values if the data is clear and understandable for the program's regular operation.

- Applying the Otsu method to convert the image into a black and white format and eliminate the background.

- Enclosing the white areas within frames and extracting data individually from the image.

- Preparing the data for the implementation of artificial intelligence models. - Constructing an artificial intelligence model with input, feature detection (filter) 3x3, pooling layer, feature detection (filter) 5x5, pooling layer, and output layers.

- Reducing the number of parameters obtained from the pooling layer to prevent overfitting and reduce computational power requirements.

- Saving the coefficients in these layers as matrices for future use after training the model.

- Collecting data from interfaces such as time and user input, RFID, and transferring them to a server.

- Analyzing the collected data for outlier values, including mean, mode, standard deviation, and skewness, to process fault data.

- Calculating the number of days between the occurrence of faults and a reference point.

- Utilizing linear regression machine learning algorithms to establish a mathematical relationship between device deterioration and fault frequency.

- Predicting the number of days until the next fault occurs from the reference point using a mathematical formula.

- Analyzing and reporting the data.

- Storing the reports in a decision support system software platform.



The invention relates to a low-cost, user-friendly data collection and processing system for electronic controlled machines, robotic mechanisms, and devices used in industrial production, without the need for separate software and hardware.


In today's technological age, machines play a significant role not only in various aspects of life but also in the manufacturing industry. Electronic controlled machines, robotic mechanisms, and devices used in production and assembly lines in factories are integrated or used independently. While these machines contribute to speeding up the production and assembly processes, capturing and processing data from these machines in factories, which are often large and consist of separate enclosed areas, poses a significant challenge. Gathering data such as production quantity, working time, downtime, product defects, energy consumption, product type, unit processing times, maintenance downtime (alarms), raw material usage, and operator-related information is essential for productivity analysis, production planning, and reporting. Data collectors commonly used for this purpose are structures specifically designed for collecting data from industrial machines. PLC and SCADA systems are the best-known examples. PLCs can already collect data such as part count, machine status, error codes, temperature, and pressure. However, when a user wants to utilize this product, they are required to use specific software and hardware for the PLC. The cost burden of PLC systems increases when the user only wants to obtain basic data such as part count, work order, operator name, start and end times, and downtime reasons. On the other hand, traditional methods of data collection encounter various problems.

In the known prior art, direct data transfer from industrial sensors to ERP programs is costly, time-consuming, and often requires different software and hardware for each machine, necessitating intervention in the machine's infrastructure. It is not possible to collect data from different brand and model electronic controlled machines, robotic mechanisms, and devices, which involve different software and control mechanisms, with a single data collection system in the known prior art. In our patent application document submitted on December 30, 2019, under application number TR2019/22207, we took the first step in solving this problem with the technique described. However, since the existing system in the known prior art is operator-dependent, it is insufficient to address the identified problems, thus requiring further development.


The invention involves a data collection system that operates in a plug-and- play manner, requiring no separate software or hardware. It incorporates a QR code embedded in a card for capturing worker information using a camera, reading the job quantity information from the machine's display using a camera and a pre-trained image processing algorithm, and capturing downtime reasons through a Human- Machine Interface (HMI). After collecting all these inputs, the system displays and reports this information in an online database.

The system determines the maintenance frequency of machines and performs predictive analysis based on the captured downtime information displayed on the unit's screen, which is selected by the workers using buttons and recorded in the online database by the information processing unit. Additionally, the invention enables the protected data in the database to be accessible for monitoring through the control panel and provides authorized personnel with access to the obtained data. This allows for immediate notification of any disruptions or faults occurring in the machines to the relevant authorities within the company, minimizing operator dependence.

The invention is a low-cost and user-friendly data collection system that can be used for purposes such as gathering data from industrial machines without the need for different software and hardware configurations.


Figure 1 . Installation Flow Diagram View

Figure 2. Main Program Flow Diagram View DETAILED DESCRIPTION OF THE INVENTION

The invention involves a data collection unit equipped with a camera positioned to monitor the screens of industrial machines. It collects production data by capturing the machine screen images using image processing methods. Additionally, it utilizes an RFID kit to determine the start and end times of workers' shifts. The system also incorporates an interface (HMI - Human-Machine Interface) to capture machine downtime reasons from the machine operator. All these inputs are processed by an artificial intelligence-based information processing unit, which enables quality management and machine maintenance analysis. The collected data is immediately stored in a database through the use of loT (Internet of Things) technology and seamlessly integrated with ERP (Enterprise Resource Planning) programs.

During the installation process, after selecting each section, the dimensions for image processing are determined. Options such as dilation and erosion are provided, allowing the user to observe the real-time black-and-white representation of the data on the screen. The dilation feature expands the edges of the image, while the erosion feature thins them. By adjusting these parameters, an optimal value is found for clear data understanding, which is then saved for the program's regular operation. Subsequently, the Otsu method is employed to convert the image to a black-and- white format and remove the background. The white areas are then enclosed in frames, isolating the data individually and making them ready for the application of artificial intelligence models.

The artificial intelligence model consists of input, feature detector (filter) 3x3, pooling layer, feature detector (filter) 5x5, pooling layer, and output. The feature detector layer extracts key features, and the pooling layer reduces the number of parameters, preventing overfitting and reducing processing power requirements. Once trained, the coefficients in these layers are saved as matrices for future use. In the normal operation of the program, these coefficients are applied to processed data obtained from image processing. The values are classified into the closest category, and data from buttons on the interface, RFID, or other tools are collected and transferred to the server. After obtaining fault data, outlier metrics such as mean, mode, standard deviation, and skewness are calculated. The time elapsed from the reference point of the faults is determined, and a linear regression machine learning algorithm is used to identify the mathematical relationship between the increase in equipment wear and the frequency of faults. Based on this mathematical formula, the prediction of the next fault occurrence is estimated in terms of days from the reference point, and appropriate actions are recommended.

Furthermore, the use of time series analysis methods allows for comparisons with other approaches. The model is designed to incorporate reinforcement learning, where the dataset grows and the model approaches accurate results with each iteration. In terms of quality analysis, Shewhart Control Charts are computed using Python libraries to analyze data obtained from the human-machine interface. The quality metrics are established through collaboration with the customer company. The generated reports are then created using PDF libraries within the decision support system software platform based on node.js.

Overall, the invention presents a low-cost and user-friendly data collection system that can seamlessly integrate with industrial machines, enabling quality management, maintenance analysis, and real-time data recording and reporting through loT and ERP integration.