Description

The present invention relates to a method of positioning a robot with respect to an object, and a system using the same.

In industry, robots are widely employed to perform repetitive tasks. Periodically, a new object is placed in front of the robot for manipulation. However, the accuracy of object placement is generally not high. Therefore, many industrial environments employing robots use visual systems to monitor the placement of the objects and communicate with the robots what the current position of the object is. Consecutively, the robots perform their task. Visual camera's, however, have a difficulty providing accurate information regarding distance and other placement parameters like shifts and rotations. Furthermore, visual camera's are rather sensitive to external disturbances. Therefore, in a number of applications laser sensors are used to determine specific distances between robots and objects.

In this application, a laser sensor is a device that comprises a laser source, a detection unit, a processor and a memory. The laser source provides a laser beam that is directed to the object. The reflected beam is detected by a detection unit and using data stored in the memory, the processor calculates an output signal, which is a measure for the distance between the object and the laser sensor.

Furthermore, in this application a robot is a machine or device that can be programmed to accomplish a variety of tasks and, after being programmed, operates automatically or by remote control. This definition thus not only includes stand-alone robots that can operate in three actions of motions, e.g. in 6 dimensions, but also Computer-Numeric- Control (CNC)-machines.

US-patent 5,799,135 by Fanuc Ltd. discloses a robot controlling method and apparatus that uses a laser sensor. More specifically, the patent relates to a welding robot that welds two work pieces together along a line P-P' with a welding torch. The laser beam generated by the laser sensor is swept over the object surface to detect the two work pieces and determine their orientation with respect to each other. This is established by

registration of the distance between the work pieces at four different times Tl to T4. Consecutively, a three-dimensional position of the points Ql to Q4, i.e. the points "hit" by the laser beam at corresponding times Tl to T4, of which Ql and Q2 reside on one work piece and Q3 and Q4 reside on the other work piece, is calculated. The equations of a straight line Q1-Q2 and a straight line Q3-Q4 are obtained and their corner position QO is calculated as a crossing point of these two straight lines. The starting position P is corrected accordingly, as is the target position P'. Then a welding layer can be formed along the shifted welding line P-P'. The procedure can be repeated for each new welding layer to be formed. However, aforementioned procedure can only be used for a limited number of combinations of structures. The welding line P-P' should be "predictable", i.e. a straight line, as well as its direct surrounding. Furthermore, the orientation of the laser sensor needs to be the same as the orientation of the welding torch.

To position a robot with respect to an object of an arbitrary three-dimensional shape, the present invention provides a method of positioning a robot with respect to an object, the robot comprising a robot arm that is arranged to move in at least three directions and has a predefined spatial relation with a laser sensor, the laser sensor comprising a laser source and a detection unit and being arranged to provide an output signal that represents a distance between the laser sensor and said object and serves as an input signal for a control system for controlling the movement of the robot, the control system comprising a processor and a memory, the method comprising:

- positioning a laser beam, emitted by the laser source, at an initial position Sj in a predetermined region of the object; - adjusting the distance between the laser sensor and the object until a predetermined distance is established and storing a value corresponding to the adjustment in the memory;

- scanning the beam from the initial position in a predetermined direction and registering the variation of the output signal; - storing a registered position Qi of the laser beam within the predetermined region, at which the signal exceeds a predetermined threshold value stored in the memory;

- repeating the positioning, scanning, registering and storing using at least two more initial positions Si on the object, the mutual spatial relations of the at least three initial positions Si being predefined and stored in the memory;

- calculating a reference position Qo based on the registered positions Qi; - calculating a reference frame based on the stored mutual spatial relations between the at least three initial positions Sj, the registered positions Qj, the adjustment values and the at least one predetermined threshold value. By application of this method a reference point Qo and reference frame for each object that is placed in front of the robot can be determined within a limited period of time. After application of this method the robot can be moved to a starting position that can be derived from at least one of the calculated reference position Qo and the reference frame. From this position the robot can execute its execution program.

Preferably, the laser beam is scanned from the initial position Sj over a predetermined distance until a predetermined end position E;. With this approach a large misalignment between the laser sensor and the object is readily detected, since no position Qi is registered.

More preferably, the number of threshold values is larger than one, preferably two, and corresponds to stored positions Qy and Qi _;2 that are used to calculate the registered position Q; for use in the further calculations. By using more threshold values uncertainties raised by changes in the digitalized output signal, for instance caused by height variations, can be minimized.

In an embodiment of the invention, the predefined spatial relation between the laser sensor and the robot arm is measured by performing a 4-points measurement with the laser sensor. This measurement technique allows an accurate determination of a tool center point (TCP) of the laser sensor, such that the laser sensor can be positioned with an arbitrary orientation with respect to a tool accommodated by the robot arm.

In all embodiments, the method, between calculating the reference position Qo and calculating the reference frame, further comprises moving the robot towards the calculated reference position Qo, adjusting the distance between the laser sensor and the object until a predetermined distance is established and storing a reference point adjustment value corresponding to the adjustment in the memory. The calculation of the reference frame is then also based on the reference point adjustment value. As a

result of this additional measurement, the accuracy of the calculated reference frame increases.

The invention further relates to a method of positioning a robot with respect to a mark on an object surface, the robot comprising a robot arm that is arranged to move in at least three directions and has a predefined spatial relation with a laser sensor, the laser sensor comprising a laser source and a detection unit and being arranged to provide an output signal that represents a local absorption coefficient of said object surface and serves as an input signal for a control system for controlling a movement of the robot, the control system comprising a processor and a memory, the method comprising:

- positioning a laser beam, emitted by the laser source, at an initial position S; in a predetermined region of the object surface;

- scanning the laser beam from said initial position in a predetermined direction and registering the output signal during the scanning; - storing a registered position Qi of the laser beam, at which the signal exceeds at least one predetermined threshold value in the memory;

- repeating the positioning, scanning, registering and storing using at least two more initial positions Sj on the object, the mutual spatial relations of the at least three initial positions Si being predefined and stored in the memory; - calculating a mark location Qo based on the stored mutual spatial relations between the at least three initial positions Sj and the stored registered positions

The invention further relates to a computer program for performing when executed by a processor at least one of the aforementioned methods. The invention further relates to a system for positioning a robot with respect to an object comprising:

- a robot comprising a robot arm that is arranged to move in at least three directions and that is capable of accommodating a tool;

- a control system connected to the robot for controlling a movement of the robot arm and/or the accommodated tool;

- a laser sensor having a predefined spatial relation with the robot arm and being connected to the control system, the laser sensor comprising a laser source and a detection unit;

in which the laser sensor provides an output signal that serves as an input signal for the control system, characterized in that the system further comprises an A/D-conversion unit between the laser sensor and the control system for digitalizing the output signal before it is provided to the control system, and the system is arranged to execute at least one of the aforementioned methods.

Preferably the number of predetermined threshold values of the output signal equals two. Furthermore, the laser source preferably generates a laser beam with a wavelength of about 300-5000 nm.

The invention further relates to a control system for controlling a movement of a robot, the robot comprising a robot arm that is arranged to move in at least three directions and that is capable of accommodating a tool, in which the control system comprises a processor and a memory that are arranged to communicate with each other, wherein the control system is arranged to perform at least one of the aforementioned methods.

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference symbols indicate corresponding parts, and in which: Figure 1 schematically shows a system for positioning a robot with respect to an object according to an embodiment of the invention;

Figure 2 schematically shows a flowchart showing a method of positioning a robot with respect to an object according to an embodiment of the present invention; Figures 3a-b schematically show a top view and a side view of an exemplary profile that can be cut out by employing the invention; Figure 4 schematically shows a top view of a square hole that can be manipulated by a robot after positioning the robot with respect to the square hole in accordance with the present invention;

Figures 5a-d show four diagrams to explain the acquisition and processing in the method according to the present invention.

Figure 1 schematically shows a system for positioning a robot 1 with respect to an object 2 according to an embodiment of the invention. The system comprises a robot 1, a control system 3 and a laser sensor 4. The robot 1 comprises a robot arm 5 that is

arranged to move in at least three directions and that is capable of accommodating a tool (not shown). The control system 3 is connected to said robot 1 and controls its movement and/or a movement of the accommodated tool by providing control signals 14. It comprises a memory 6 and a processor 7 that are able to communicate with each other. The connection between the control system 3 and the robot 1 is rigid or detachable. The laser sensor 4 has a predefined spatial relation with the robot arm 5 and is connected to the control system 3. The laser sensor 4 comprises a laser source 8 and a detection unit 9. The laser sensor 4 generates a laser beam 10 with the laser source 8 and directs it towards the object 2. The laser beam may have a wavelength in the visual spectrum of light, between about 300-700 nm, but may also have a larger wavelength, for instance into the "infrared" or "ultrasound" regions, for example between about 700-5000 nm. The generated laser beam 10 reflects on the surface of the object 2, resulting in a reflected beam. The laser sensor 4 then measures a parameter, for instance an intensity, of the reflected laser beam with the detection unit 9. The measured parameter is processed within the laser sensor 4 as known in the art and as a result, an output signal 11 is provided. The provided output signal 11 represents a distance between the laser sensor 4 and the object 2. The output signal 11 serves as an input signal 12 for the control system 3. Before the output signal 11 is provided to the control system 3, the signal is further processed in an A/D-conversion unit 12, that interconnects the control system 3 and the laser sensor 4. The A/D-conversion unit 12 digitalizes the output signal 11, i.e. the output signal 11 is transformed from an analogue signal 11 into a digital signal 13. Preferably, the digitalized signal 13 changes when a threshold value of the output signal 11 is passed. In an embodiment of the invention, more than one, preferably two, threshold values are used in the digitalization of the output signal 11. The control system 3 can be arranged to store relevant parameters regarding the digitalized input signals 13 in memory 6. The stored parameters can be used by processor 7 to calculate a reference frame in which robot 1 should execute instructions it receives from the control system 3.

Figure 2 schematically shows a flowchart 15 showing a method of positioning a robot 1 with respect to an object 2 according to an embodiment of the present invention. Only the method of positioning is explained. Operations following the positioning as well as operations that need to be executed besides the positioning to enable a successful

operation after the positioning of the robot 1 with respect to the object 2 are not explained.

The processing starts under the condition that the initial value of a counter i = 0, the value of which can be stored in memory 6. First, in action 20, the laser sensor 4 is brought in proximity of the object 2, and, in action 21, the laser beam 10 is directed towards an initial position Sj. The exact position of each individual object 2 varies, for instance due to mechanical inconsistencies of a conveyor belt that provides the object 2. Therefore, the initial position Si is located within an anticipated position of a predetermined region of the object 2. If it turns out that the laser beam is positioned outside the predetermined region, and in the following procedure this can be detected, the object 2 needs to be repositioned (not shown) with respect to the robot 1 and the laser sensor 4, and the flowchart 15 is followed from the start again. In this example, four initial positions Si are used, i.e. N = 4. To enable the calculation of a three-dimensional reference frame, in action 22, the distance between the laser sensor 4 and the object 2 at the initial position Sj is adjusted to a predetermined distance by moving the laser sensor 4 relative to the object 2. The difference between the final distance after adjustment and the actual distance before adjustment at this position Sj is stored as an adjustment value in a part of memory 6 of the control system 3.

Consecutively, in action 23, the laser sensor 4 is moved in a predetermined direction over the surface of the object. Preferably the movement is a linear movement, however it may also be possible to sweep the laser beam 10.

During the scanning, the output signal 11 of the laser sensor 4 is registered. The value of the output signal may vary with the distance between the laser sensor 4 and the object 2. The A/D conversion unit employs one or more threshold values, the values of which may also be stored in memory 6 of the control system 3. A position Qi, at which during scanning the output signal 11 starts to exceed one of the one or more threshold values is registered in action 24 in a part of the memory 6. The scanning can be performed over a predetermined distance, i.e. until an end position E; is reached. However, it is also possible to stop with scanning, at the time the signal exceeds the last one of the one or more threshold values.

After finishing the scanning action, in action 25, counter i is increased by ' 1 '. The laser sensor 4 is then moved towards a next initial position S;, of which the spatial relation with the first initial position S; is predetermined and stored in a part of the

memory 6 of the control system 3. Aforementioned sequence of actions 21-25 is repeated until i exceeds the total number of desired positions Qj to register. In this example N equals 4, so when 4 positions Qi have been registered, the laser sensor 4 is not moved towards further initial points Sj. In action 26, the processor 7 calculates a reference point Q ₀ , based on the registered positions Qi and a reference frame, based on both the registered positions Q; and the adjustment values to position the laser sensor 4 at a predetermined distance from the object 2 at each initial position Sj. The accuracy of the calculation of the reference frame can be further enhanced by taking a distance between the object 2 and the laser sensor 4 at the calculated reference point Qo into account. Therefore, before calculation of the reference frame, robot 1 is moved to the calculated reference point Q ₀ . Then, the distance between the laser source 4 and the object 2 is adjusted until a predetermined distance is established. Consecutively, a reference point adjustment value corresponding to the adjustment is stored in a memory, for instance memory 6. The calculation of the reference frame is now not only based on the registered positions Qi and the adjustment values to position the laser sensor 4 at a predetermined distance from the object 2 at each initial positions Si, but also on the stored reference point adjustment value.

Finally, in action 27, the robot 1 can be moved to a starting position for execution of its operational task. The starting position can be the same position as the reference position Qo. However, it may well be a different position that is derived from both the calculated reference position Qo and the calculated reference frame. The shift between the theoretical starting position and the actual derived starting position is then based on the difference between the theoretical reference position and the actual reference position Qo and shifted within the calculated reference frame, a technique well-known by a person skilled in the art.

The invention is now explained by two examples, schematically displayed in figures 3a-b and figure 4 respectively.

Example 1 : Figure 3 a schematically shows a wall 30, for instance a metal wall, comprising a deepened section 31. A cross-sectional view of the same structure 30, 31 along line A-A' in figure 3a is shown in figure 3b. Consider the situation in which the

deepened section 31 needs to be cut out. Preferably, the cut-out operation is performed by a robot 1 that is provided with a cutting tool. A memory 6 of a control system 3 of the robot 1 may comprise stored information regarding the size and shape of the deepened section 31, and the necessary instructions to perform the cutting operation, for example by means of CAD-CAM data. To perform the cutting operation correctly, the position of each wall 30, including its orientation regarding tilt and the like and the orientation of the deepened section 31 therein, needs to be known in advance. In practice, however, each wall is positioned slightly different with respect to the robot 1. Furthermore, not all deepened section 31 are positioned at exactly the same location within each wall 30. To position the robot 1 adequately with respect to the deepened section 31 of the wall 30 aforementioned method 15 in accordance with the present invention can be used. Its application can increase the yield of the cutting operation, i.e. the percentage of cutting operations successfully performed within specified parameters, to a great extent. In this example, the procedure is as follows. First, the laser beam 10 of the laser sensor 4 is directed to a first initial position S ₁ in a predetermined area of the wall 30. The robot 1 is programmed to move the laser beam 10 towards first end position E ₁ . During the movement from Si towards Ei, the detection unit 9 of the laser sensor 4 registers the reflected laser beam 10 of the wall 30. Generally, the intensity of the reflected beam is a measure for the distance between the laser sensor 4 and the wall 30. At a certain position, in this case at position Q ₁ , the distance becomes larger than a predetermined, threshold distance. In this example the threshold distance corresponds to the distance at a position halfway the slope between a higher area B including first initial position Si and a lower area C including first end position Ei. The threshold distance corresponds to a threshold value of the output signal 11 of the laser sensor 4. As a result, digitalized signal 13 changes, and the time and/or position at which this happens, i.e. position Qi is registered in the memory 6 of the control system 3.

Consecutively, the laser beam 10 of the laser sensor 4 is directed to a second initial position S ₂ . The relative position between Si and S ₂ is predetermined and can be stored in the memory 6 of the control system 3. The laser sensor 4 is again scanned in a direction of an end point, i.e. second end point E ₂ . Again, the position at which a threshold value for the output signal 11 of the laser sensor 4 is exceeded, now called position Q ₂ , is stored in the memory 3 of the control system 6.

Similarly as before, the movement of the laser sensor to initial positions S ₃ and S ₄ respectively results in the storage of registered positions Q ₃ and Q ₄ , by moving the laser sensor 4 from the respective initial positions S3 and S ₄ towards the respective end positions E ₃ and E ₄ , and storing the positions at which the output signal 11 of the laser sensor 4 exceeds the predetermined threshold value.

Three-dimensional positions of the positions Q ₁ to Q ₄ can now be derived based on an analogue value of the output signal 11 at these positions, i.e. aforementioned threshold value, and the adjustment values that were used to "focus" the laser sensor 4 at initial positions S ₁ to S ₄ respectively. Next, an equation of a straight line Q ₁ -Q2 is calculated from the position data of the two stored registered positions Q ₁ and Q ₂ .

Similarly, an equation of a straight line Q ₃ -Q ₄ is calculated from the position data of the two stored registered positions Q ₃ and Q ₄ . Then reference position Q ₀ is calculated as being a cross point of these two straight lines. Note that, although this may not appear from figure 2, straight lines Qi-Q ₂ and Q ₃ -Q4 in a three-dimensional space. Furthermore a reference frame can be determined based on the three-dimensional positions of positions Q ₁ to Q ₄ , and the calculated reference position Q ₀ with techniques known in the art. An execution program, either externally provided or downloaded in a memory 6 of the control system 3 can be adapted in view of the determined reference frame and calculated reference position Qo in a way known to someone skilled in the art.

A main advantage of the invention is that it enables an accurate cut out of the desired section 31, even if each wall 30 is positioned in front of the robot 1 in a slightly different manner. Whether the wall 30 is slightly rotated or shifted makes no difference for the end result of the cutting operation.

Example 2: Figure 4 shows a wall 35 that comprises a hole 36. If a robot 1 needs to place something in the hole 36, it may be necessary to know where the boundaries of the hole 36 are located and how the hole 36 is oriented with respect to the robot 1. First, positions Q ₁ to Q ₄ are registered in a substantially similar fashion as described for figure 3 a. However, the mutual spatial relations between the registered positions Q ₁ to Q ₄ are different. Generally, the number, length and position of the "registration scan lines" Si-Ei, i = 1 to N, as well as the mutual spatial relations between the initial positions S, may vary depending on the geometry of the hole 36.

Again, the three-dimensional positions of the positions Q ₁ to Q ₄ can be derived based on the adjustment value that was used to "focus" the laser sensor 4 at initial positions S ₁ to S ₄ respectively. To determine reference position Q ₀ in the case shown in figure 4, first the equations of straight lines Q ₁ -Q ₃ and Q ₂ -Q ₄ are calculated from the position data of the two stored registered positions Q ₁₅ Q ₃ and Q ₂₅ Q ₄ respectively. Consecutively the cross point of these two straight lines is determined as reference point Qo. Furthermore a reference frame can be determined with techniques known in the art based on the three-dimensional positions of positions Q ₁ to Q ₄ , the calculated reference position Qo and stored information regarding the shape and size of the hole. The execution program, either externally provided or downloaded in a memory 6 of the control system 3 of the robot 1 can be adapted in view of the determined reference frame and calculated reference position Q ₀ in a way known to someone skilled in the art.

Figures 5a-d show four diagrams to explain the acquisition and processing of the output signal 11 in an embodiment of the method according to the present invention. Figure 5a shows the height difference between the laser sensor 4 and the object surface as a function of time during a registration line scan between an initial position Sj and an end position Ej. In this case the surface height of the object changes gradually in a way similar to the surface shown at the right side in figure 3b (Example 1).

Figure 5b shows a corresponding output signal 11 of the laser sensor 4 as a function of time during the same line scan. In this case, the value of the output signal 11 increases with increasing distance between the laser sensor 4 and the object surface. Figure 5 c shows a corresponding digitalized output signal 13 of the laser sensor 4 if one threshold value is used. At the time the distance between the laser sensor 4 and the object surface exceeds a certain predetermined distance, the output signal 11 of the laser sensor 4 exceeds a certain predetermined threshold output signal level and the digitalized output signal 13 changes, in this case from 'low' to 'high'.

Figure 5d shows a corresponding digitalized output signal 13 of the laser sensor 4 if two threshold values are used. Again, at the time the distance between the laser sensor 4 and the object surface exceeds a predetermined distance, the digitalized output signal 13 changes, in this case from 'low' to 'high'. However, at the time the distance between the laser sensor 4 and the object surface exceeds a further predetermined

distance, the digitalized output signal 13 changes again. In this case it changes back, i.e. from 'high' to 'low'. Accurately assigning a position Qi based on a single threshold value can be difficult when the height differences of the object surface vary gradually, i.e. a small slope, while the roughness of the object surface is relatively large, i.e. of the same order as the height difference within the desired dimensions of accuracy.

Therefore, in a numerous situations, the use of two threshold values may be preferred. If suitable threshold values are chosen, uncertainties raised by changes in the digitalized output signal 13 caused by height variations around the threshold heights can be minimized, for instance by averaging, as is known to someone skilled in the art.

Aforementioned method works well if the laser sensor is oriented substantially in line with the tool, which is accommodated by the robot arm 5. However, in many situations other orientations of the laser sensor with respect to the tool are desired. Therefore, it is preferred to measure the predefined spatial relation between the laser sensor 4 and the robot 1 or robot arm 5 before positioning the laser beam 10 in action 20. The measuring of the spatial relation preferably comprises a 4-points measurement with the laser sensor 4 on a mark on the robot arm 5 with four different orientations of the laser sensor 4. For instance, if the robot arm 5 is provided with a mark shaped as deepened section 31 in the first example, and the laser sensor at a fixed position sweeps the laser beam 10 with varying angle in a first direction over the structure while the output signal 11 is registered as explained before, again reference points Qi can be registered. However, in this case the calculation is based on the time of the change of the digital signal 13 and the corresponding orientation of the laser sensor 4. The same can be done in a second direction substantially perpendicular to the first direction. The four points that are registered with the corresponding orientations of the laser sensor 4 can serve as an input for the processor 7 to calculate a tool center point (TCP) of the laser sensor 4. Of course, also other types of measurements may be used to determine the TCP of the laser sensor 4 as is known to persons skilled in the art.

While specific embodiments of the invention have been described above, it will be appreciated that the invention may be practiced otherwise than as described. For example, Qi may be stored in a memory outside the control system 3 of the robot 1. Also the calculation of reference position Q ₀ as well as the calculation of a reference

frame can be done in a processor outside the control system 3.

Furthermore, the output signal 11 of the laser sensor 4 may be related to other parameters than the height between the laser sensor 4 and the object surface. The difference in intensity of the reflected signal may for instance be based on local absorption differences at the object surface, e.g. due to different colors. The method is therefore also suitable to find predefined marks on an object surface, and can be employed to verify whether certain marks are present or not.

The invention may take the form of a computer program containing one or more sequences of machine-readable instructions describing the method as disclosed above, or a data storage medium (e.g. semiconductor memory, magnetic or optical disk) having such a computer program stored therein. Such a computer program may be run on the control system 3 of the robot 1, but may also be run on an external terminal that can be connected to the control system 3.

The descriptions above are intended to be illustrative, not limiting. Thus, it will be apparent to one skilled in the art that modifications may be made to the invention as described without departing from the scope of the claims set out below.