Technical Field:

This invention; handheld terminal and its algorithm for dynamic robot guidance, where simulations of new position, and orientation information according to joint angles are obtained quickly, in any environment, at any time and by any person, so that industrial robots can continue their functions effectively in task or job changes.

State of the Art:

In the current technique, determining the new values of the joint angles in the work line or position changes where new position and orientation information of the serial robots will be needed requires performing non-linear and complex mathematical operations manually or with the help of a computer. Especially if the robots are constantly changing their work line or location, engineers need to allocate a separate time for this work, which leads to an increase in workloads and unnecessary loss of time. Even if complex operations are desired to be performed with the aid of a computer, different algorithms are coded separately, and each result is analyzed on separate platforms after the results are obtained. Afterwards, the appropriate value is tried to be found as a result of inferences or estimations regarding the joint angles obtained. The position information obtained based on estimation and inference has emerged as a result of the optimization of a single value, and even if there are better values, these values need to be revealed with additional calculations. Thus, the operation of the process is delayed and causes loss of time.

Today, the fact that the value that gives the best result in these calculations for robots depends on the inferences of people creates unreliability and creates a disadvantageous situation due to the possibility of affecting the working efficiency of robots. Because in the calculation process, depending on the working principle of the experts, the process steps or the way they reach the result may vary. This situation causes experts to obtain different results for the same process, and also prevents standardization in the institutional structure. Even if the calculations reached by the experts are correct, the robot has to be tested in the field with the new position information and necessary corrections have to be made, and this causes the process to be unnecessarily long. Many inventions have been developed in the literature to overcome such problems.

In the application JP2021122894, the subject titled “Inverse Kinematics Calculation Device and Inverse Kinematics Calculation Method” is discussed. To maske it possible to guarantee positional accuracy and attitude accuracy of a fingertip of a robot arm compared to a conventional configuration. SOLUTION: An inverse kinematics calculation device comprises: an input unit which receives a position and an attitude, and positional accuracy and attitude accuracy of a fingertip of a robot arm having a plurality of joints as input information; a numerical calculator which searches for and updates intermediate variables corresponding to a solution of inverse kinematics calculation for the position and the attitude of the fingertip of the robot arm on the basis of the input information received by the input unit; a joint angle calculator which determines a solution of inverse kinematics calculation for the position and the attitude of the fingertip of the robot arm on the basis of the input information received by the input unit and the intermediate variables updated by the numerical calculator; a checker which performs forward kinematics calculation for the solution determined by the joint angle calculator, and determines whether the input information received by the input unit is satisfied; a convergence determination unit which stops updating by the numerical calculator on the basis of a determination result by the checker; and an output unit which outputs data that indicates a solution determined by the checker to satisfy the input information.

In the application CN108427282, “Inverse kinematics solution method for robot based on teaching learning” is discussed. The invention belongs to the field of robot kinematics solution, and particularly discloses an inverse kinematics solution method for a robot based on teaching learning. The method comprises the following steps of collecting the joint angle of N sets of robots and the Cartesian position and Euler angle of an end effector, and obtaining a data set f; optimizing the data set f to obtain the data setfl; iteratively solving the data set fl to obtain Gaussian mixture model parameters with 2-50 Gaussian models; calculating Bayesian information criterion values corresponding to Gaussian mixture models, drawing a Bayesian information criterion curve, and determining a Gaussian number k according to the curve; solving the Gaussian mixture model with the Gaussian models of which the number is k; using the model parameters to conduct Gaussian regression treatment on the Cartesian position and the Euler angle of the data set fl, and solving an inverse kinematic joint angle value; comparing the joint angle value with the joint angle value in the data set fl, and correcting the number of the Gaussian models to obtain desired solution accuracy. The method has the advantages of fast calculation speed, strong applicability, good real-time control effect and the like.

In the applications discussed above, inventions for obtaining joint angle values with inverse kinematics calculation in robots are discussed. In these methods, the joint angles of the robots are calculated with different techniques. However, in these systems, there are no simulations that include new position, and orientation information according to the joint angles of the robots, and the robots whose joint angles are determined by inverse kinematic calculation cannot be visually animated on a tablet-style handheld terminal. In this case, the verification process causes an extra waste of time and creates unnecessary workload. Even if the verification process is carried out successfully, it is necessary to test the robots in the field with the position information of the new joint and angle values and make the necessary arrangements accordingly. For this reason, it causes the process to be unnecessarily long.

As a result, the hardware that can overcome the disadvantages mentioned above and that performs the same operation in the same way for every situation or every person in order for the robots to continue their functions effectively in task or job changes, and which is embedded on the motherboard integrated into the invention independently of time, place and person. There is a need for a hand-held, small-sized simple device where simulations of new position, position and orientation information can be obtained visually, based on the most suitable joint angle values computed by the invention, calculated using different artificial intelligence algorithms with the unit. Description of the Invention:

This invention; it is about the hand terminal and algorithm for dynamic robot guidance where the simulations created according to the most suitable position and orientation information by calculating the joint angles with the help of artificial intelligence algorithms in the task or work line changes of industrial robot manipulators can be observed.

The invention includes a portable, small-sized handheld terminal device that can be used in any environment, at any time and by any personnel, regardless of time, person, machine, place.

The product subject to the invention generally consists of a touch screen, a battery, a motherboard, and a plastic back cover that protects all these structures. The touch screen that provides data input to the system is created using infrared type touch screen technologies. There are LED and photo sensors on the edge of the screen, and the sensors control the integrity of the LED. Since the product subject to the invention is a handheld terminal device, it must have a portable feature, and the presence of the battery in the invention makes this feature usable. The invention mentioned here is a product and was introduced by integrating hardware software with an embedded numerical design method. The aforementioned invention integrated in an embedded system that is used as a Field- programmable gate array (FPGA) hardware, software, regardless of the number of joints with artificial intelligence-based operations of robots with kinematic calculations are performed within milliseconds and the results on the LCD screen are simulated. The motherboard included in the invention allows different artificial intelligence algorithms to be integrated into a single card and the calculations performed can be obtained using different artificial intelligence techniques in a single environment and analyzed on a single platform. Thanks to the motherboard integrated into the invention, the calculations performed on a single platform and the simulations created in accordance with these calculations are presented to the user. Because of the artificial intelligence-based operations, an infinite number of new values are obtained in the calculations performed and the most appropriate values are proposed by the system. There is also a user interface software in order to transfer the operations performed on the hardware of the inventive system to the user and/or transfer the information that the user wants to be processed on the hardware to the system. In order for this software to work, the user saves the robot DH parameters to the system and the joint angles of the robot, whose task or place of duty is changed by any person at any time or place, are calculated by four different artificial intelligence algorithms by the system. Afterwards, the user can simulate the values calculated on the screen and choose the most suitable one among many new values and visual simulations presented to the user's preference, and then enter this information into the system and run the robot.

With the user interface software developed for the invention, a new addition is made to the defined robot list where reminder information such as the number of joints of the robot and the name of the location of the robot can be included. The DH parameters of the robots that the user will operate using the visual interface are stored in the ROM memory, and after these steps, the robot is simulated, and verification is performed according to the DH parameters added and stored.

In order to realize the main goal of the invention, when a task or a change of duty is detected for a robot registered to the system, the most ideal position and position information of the robot; The hardware unit, which is built using a different and brand new method embedded in the FPGA motherboard, which is one of the components of the invention, performs the reverse kinematics calculation of the robot in parallel with artificial intelligence algorithms and transfers the results to the LCD screen. The results are calculated with these four different artificial intelligence algorithms: firefly (ABO), artificial bee colony (CAP), particle swarm optimization (PSO), quantum particle swarm optimization (QPSO). Artificial intelligence algorithms that take data from RAM memory create a parallel line and enable calculations to be performed quickly.

The motherboard integrated into the invention allows the integration of artificial intelligence algorithms into a single board, as well as allowing the processing steps to be performed much faster since it works entirely with hardware. According to the working algorithm of the system, after the robot information stored in the ROM memory is transferred to the RAM memory, the artificial intelligence algorithms that receive the data from the RAM memory are run by forming a parallel line. Kinematic calculations are performed and then position errors are detected based on the calculations performed with each artificial intelligence algorithm. Meanwhile, with the kinematic calculation of floating-point operations (FPU); Arithmetic logic operations (ALU) are included in the system as interfaces that perform numerical operations related to position error. At the end of the calculations, the joint angles with the least margin of error are saved in the RAM memory and the results are displayed on the screen.

In order to calculate the joint angles with artificial intelligence algorithms, the x, y and z coordinate information of the robot's end element is entered into the system and the results are presented on the screen after the operations are performed in a very short time with the embedded system design on the main board of the system. Robot simulations based on joint angles that give the most accurate results with artificial intelligence algorithms are displayed on the screen.

The structural and characteristic features and all advantages of the product subject to the invention will be understood more clearly thanks to the figures given below and the detailed explanation written by referring to these figures, and therefore the evaluation should be made by considering these figures and detailed explanation.

Description of the Figures:

The invention will be described with reference to the accompanying figures, so that the features of the invention will be more clearly understood and appreciated, but the purpose of this is not to limit the invention to these certain regulations. On the contrary, it is intended to cover all alternatives, changes and equivalences that can be included in the area of the invention defined by the accompanying claims. The details shown should be understood that they are shown only for the purpose of describing the preferred embodiments of the present invention and are presented in order to provide the most convenient and easily understandable description of both the shaping of methods and the rules and conceptual features of the invention. In these drawings. Figure 1 Symbolic view of the parts that make up the invention

Figure 2a Symbolic view of the new robot definition screen in the data entry unit in the user interface software developed for the invention.

Figure 2b Symbolic view of the screen for entering Robot DH parameters in the data entry unit in the user interface software developed for the invention.

Figure 2c Symbolic view of the robot simulation screen in the user interface software developed for the invention.

Figure 3a Symbolic view of the screen for creating a new task in the unit.

Figure 3b Symbolic appearance of the new task simulation screen in the user interface software developed for the invention.

Figure 4 Reverse kinematics calculator block diagram view.

The figures to help understand the present invention are numbered as indicated in the attached image and are given below along with their names.

Description of References:

1- Input Panel

2- LCD Screen

3- Motherboard

10- Data Entry Unit

20- Processing Unit

Description of the Invention:

The invention describes ; The input panel (1), where data is entered to the system from outside, the LCD screen (2) where the results of the operations performed on the motherboard (3) are displayed, the simulation process is performed and the results of the operations performed on the motherboard (3) are displayed, the artificial intelligence algorithms are added to a single card in an integrated manner, making the operations faster. It includes the motherboard (3), which performs the communication in a different way and enables communication with peripheral input/output units (such as keypad, screen) over the server, and the battery, which gives the system the feature of wireless transportation.

The input panel (1) in the hand terminal which is the subject of the invention is a touch screen.

Field-programmable gate array (FPGA) technology is used as an embedded system on the motherboard (3) in the hand terminal which is the subject of the invention.

The inverse kinematics problem of robots is solved, regardless of the number of joints, with the embedded system on the motherboard (3) in the hand terminal which is the subject of the invention.

The LCD screen (2) on the hand terminal which is the subject of the invention simulates the results produced by the motherboard (3) and shows it to the user.

The algorithm of the invention of the hand-held terminal user to the system input panel (1) saved by the DH parameters of the robot, the data input unit (10), changing the location of the robot a new task or task the task that will perform the joint angles by different artificial intelligence algorithms by the system motherboard (3) of the simulation where it is created and is calculated by the processing unit (20) includes.

The algorithm of the inventive hand terminal; It includes the data entry unit (10) where the transactions performed on the hand terminal are displayed to the user or the information that the user wants to be processed on the terminal transfers the information to the system from the outside.

The algorithm of the inventive hand terminal; It contains the data input unit (10) where robot definitions are made to the system over information such as the number of joints, location or section information of the robot. The algorithm of the inventive hand terminal; contains the processing unit (20) that includes artificial intelligence algorithms such as firefly algorithms (FA), artificial bee colony (ABC), particle swarm optimization (PSO), quantum particle swarm optimization (QPSO).

Detailed Description of the Invention:

The basic elements that make up the hand terminal which is the subject of the invention are the input panel (1), the LCD screen (2), the motherboard (3) and the battery.

The features of the above-mentioned elements are as follows:

Input panel (1) with Touch Screen: This part of the invention is used to enter data into the system from outside. It was created with infrared technology and is 7" in size. In infrared technology, there are LED and photo sensors at the edge of the screen. Sensors check LED integrity. Since the LED integrity of the manually touched place is broken, the touched place is determined by the controller.

LCD screen (2): It is used to display the results of the operations performed on the motherboard (3) and to perform the simulation process. It is used to display the result of the operations performed on the motherboard (3).

Motherboard (3): It is the part that makes the invention original. It ensures that artificial intelligence algorithms are integrated into a single card, allowing transactions to take place very quickly. In addition, it provides communication with peripheral input/output units (such as keypad, screen). The invention is an FPGA card that can operate in the 500 MHz frequency range, which incorporates the Artix 7 chip produced with nano technology. The invention can be in an integrated ASIC structure.

Battery: Since the invention resembles a handheld terminal, it has a portable feature. It is the element that provides this feature to be brought to the invention. It is a hardware unit in lithium-ion structure, capable of producing 3.7 V voltage with 5000 mAh power. Its size is in the range of 8 - 10 cm. In addition to all these parts of the inventive system, a user interface software has also been created in order to display the operations performed on the hardware to the user or to transfer the information that the user wants to be processed on the hardware to the system from outside. This interface software works with the following algorithm: a) The user saves the robot DH parameters to the system via the data entry unit (10), b) The joint angles at which the robot, whose task or place of duty will change, will perform its new task in the most perfect way, are calculated by the system with different artificial intelligence algorithms, c) The simulation of these values made by the processing unit (20) is created on the LCD screen (2) and the user looks at these values, d) Applies the most suitable position to the robot

The step-by-step operation of the user interface software developed in order to realize this algorithm in the easiest way independent of users is shown in Figure-2 below.

In Figure 2, a new robot is added to the ROM memory of the invention with the user interface software. Here, the robot is given a name that reminds both how many joints it has and in which section it is located. Adding a robot is done by adding the DH parameters of the robot. The robot is verified by showing the joint simulation of the added robot in Figure 2c.

Figure 3 shows the projection steps performed to find the most ideal position when a new task of the robot registered in the system is defined. Four different artificial intelligence Firefly Algorithm (FA), artificial bee colony (ABC), particle swarm optimization (PSO), quantum particle swarm optimization (QPSO) can be used for this projection.

In Figure 3a, after the x, y and z coordinate information of the robot's end element is entered into the system, the "Calculate" button is used to quickly perform the operations with the embedded system design on the motherboard (3) in the device and the results are displayed. Figure 3b shows the robot simulation according to the results obtained. The core of the invention is the motherboard (3) (FPGA board). The hardware unit embedded in this card, created using a different and brand new method, performs the inverse kinematics calculation of the robot in parallel with four different artificial intelligence methods and transfers the results to the LCD screen (2) very quickly.

The block diagram of the proposed new method is shown in Figure 4. Each unit appearing in this diagram is hardware-created and is called small/medium scale integrated in the digital system literature. The whole method is called large scale integrated (VLSI). Since the method works entirely with hardware, the processes are carried out very quickly and the results are obtained in a very short time.

As can be seen in Figure-4, artificial intelligence algorithms create a parallel line and process the data from the "RAM" memory at the same time. Therefore, parallel computations also speed up the system. The "ROM" in Figure-4 is the hardware unit that keeps the DH parameters of the robots that the user will operate using the visual interface. In order to enter information from outside, a separate hardware unit called “KeyPad” was created in the system. “ALU” and “FPU” are interfaces that perform digital operations on the system. The working algorithm of the system is as follows:

1- Transfer selected robot information from ROM to RAM.

2- Run artificial intelligence algorithms in parallel with the initial information generated in RAM.

3- Perform advanced kinematic calculations with the information obtained from the algorithms.

4- Get the position errors obtained with each algorithm.

5- Save the j oint angles that give the smallest positional error in RAM.

6- Display the results on the LCD (3).

Both the tablet-style handheld terminal, parts of which are given in Figure 1, as well as the hardware unit embedded on the motherboard (3), whose block diagram is given in Figure 4, which performs the calculation operations in a new way, are innovations in the invention. Currently, such calculations in the industry are made by senior authorized engineers. However, according to the working method of each engineer, the process steps or the way to reach the result vary from company to company or from person to person. This situation prevents standardization in the institutional structure. Here, thanks to the invention mentioned here, the same process is carried out in the same way for every situation or every responsible person. Thus, the work line or task change process of the robot is carried out independently of the person or institution.