NILSSON, Mats (Scheelevägen 27, Tumba, S-147 21, SE)
CLAIMS
1. An arrangement for controlling a movable robot arm (14) for automatically attaching teat cups (15a) of a milking machine (15) to the teats of a milking animal (13) in a milking stall (12), characteri zed in :
a three-dimensional camera arrangement (Ha; Ha, Ha') stationary arranged with respect to the milking stall and directed towards the teats of the milking animal in the milking stall; and
- a visual teat sensing means (lib) arranged on the robot arm; and
- a control device (17) provided, for each of the teats of the milking animal, for determining teat positions from images recorded by the three-dimensional camera arrangement and the visual teat sensing means; and for controlling the movement of the robot arm in order to attach a respective one of the teat cups to the teat of the milking animal in a two- stage procedure, wherein
- during a first stage, said control device is provided for controlling said robot arm to move to an approximate position for attachment of the teat cup to the teat of the milking animal based on a position of the teat as determined from images recorded by the three-dimensional camera arrangement; and
- during a second stage, said control device is provided for controlling said robot arm to fine adjust its position for attachment of the teat cup to the teat of the milking animal based on a position of the teat as determined from images recorded by the visual teat sensing means mounted on the robot arm.
2. The arrangement of claim 1 wherein said three-dimensional camera arrangement comprises two three-dimensional camera systems {lla, lla') .
3. The arrangement of claim 2 wherein said three-dimensional camera systems (lla, lla') are arranged on each side of the milking animal when located in the milking stall.
4. The arrangement of claim 3 wherein the camera system (lla) arranged on the left side of the milking animal is provided to record images of the left teats of the milking animal and the camera system (Ha') arranged on the right side of the milking animal is provided to record images of the right teats of the milking animal .
5. The arrangement of any of claims 1-4 wherein said arrangement is implemented in a rotary milking system, wherein said milking stall is together with other milking stalls arranged on a rotating platform.
6. The arrangement of claim 5 wherein said three-dimensional camera arrangement is arranged to be movable along at least one path (21, 21') such that said three-dimensional camera arrangement is kept stationary with respect to the rotating milking stall when recording images of the teats of the milking animal in the rotating milking stall.
7. The arrangement of claim 5 wherein said three-dimensional camera arrangement is arranged to be movable to one of the other rotating milking stalls subsequent to the recording of images of the teats of the milking animal in the rotating milking stall.
8. The arrangement of any of claims 1-7 wherein said three- dimensional camera arrangement comprises a range camera.
9. The arrangement of any of claims 1-7 wherein said three- dimensional camera arrangement comprises a time of flight camera .
10. A milking system comprising the arrangement of any of claims 1-9.
11. A method for controlling a movable robot arm (14) for automatically attaching teat cups (15a) of a milking machine (15) to the teats of a milking animal (13) in a milking stall (12), characteri zed by the steps of, for each of the teats of the milking animal:
- recording images of the teat of the milking animal in the milking stall by a three-dimensional camera arrangement (lla; 11a, lla') stationary arranged with respect to the milking stall;
- determining a position of the teat from the images recorded by the three-dimensional camera arrangement;
- controlling said robot arm to move to an approximate position for attachment of the teat cup to the teat of the milking animal based on the position of the teat as determined from the images recorded by the three-dimensional camera arrangement;
- recording images by the visual teat sensing means mounted on the robot arm;
- determining a position of the teat from the images recorded by the visual teat sensing means mounted on the robot arm; and
- controlling said robot arm to move from said approximate position to a fine adjusted position for attachment of the teat cup to the teat of the milking animal based on the position of the teat as determined from the images recorded by the visual teat sensing means mounted on the robot arm.
.
12. The method of claim 11 wherein the images of the teats of the milking animal which are recorded by the three-dimensional camera arrangement are recorded by at least one range camera system, such as e.g. a time of flight camera system.
13. The method of claim 11 or 12 wherein the images of the left teats of the milking animal which are recorded by the three- dimensional camera arrangement are recorded by a first camera system arranged on the left side of the milking animal and the images of the right teats of the milking animal which are recorded by the three-dimensional camera arrangement are recorded by a second camera system arranged on the right side of the milking animal. |
ARRANGEMENT AND METHOD FOR CONTROLLING A MOVABLE ROBOT ARM
TECHNICAL FIELD OF THE INVENTION
The present invention generally relates to dairy farm robot milking and to automatic attachment of teat cups related thereto.
DESCRIPTION OF RELATED ART AND BACKGROUND OF THE INVENTION
EP1170987 discloses a method for recognizing and determining the position of at least one teat of a milking animal, which includes the steps: moving a scanning head including a light source to a region containing the teat or teats, capturing at least one image formed by said light, evaluating the image or images so as to determine if each image describes said teat or teats. The scanning head (1) is moved to a fixed initial position (A) in the room which is under the animal and clearly in front of an udder and thereby the teats of all known relevant animals, thereafter the scanning head is moved in determined steps (A - J) under the animal: upwards towards the animal, downwards, backwards towards the udder, upwards, downwards etc. , while carrying out the scanning procedure. The invention also concerns an apparatus carrying out the process and a milking robot including such an apparatus .
A drawback of the approach disclosed in EP1170987 is that the scanning head including the light source has to be moved extensively. Such repeated scanning is time consuming, which renders the method slow. Further, the scanning head is mounted on a robot arm, which has to be moved up and down in order to find the teats of the animal. Such movement of the entire robot arm just for detecting the teats of the animal seems to be inflexible and slows down the process yet further.
High capacity automatic milking requires attachment of teat cups at a very high speed. A goal for tomorrow is to reduce the time for the teat cup attachment procedure to less than a minute . This is hardly achievable by the above described prior art teat detection approach.
SUMMARY OF THE INVENTION
Accordingly, it is an object of the present invention to provide an arrangement and a method for controlling a movable robot arm for automatically attaching teat cups of a milking machine to the teats of a milking animal in a milking stall, which arrangement and method are without the drawback of the prior art and which thus provide for a high speed teat cup attachment as required in tomorrow's high capacity automatic milking systems.
It is a further object of the invention to provide such arrangement and method, which are robust, effective, precise, accurate, reliable, safe, easy to use, and of reasonable cost.
It is still a further object of the invention to provide such arrangement and method, which are capable of obtaining a very high number of correct teat cup attachments in a minimum of time.
These objects among others are, according to the present invention, attained by arrangements and methods as claimed in the appended patent claims .
According to one aspect of the invention there is provided an arrangement for controlling a movable robot arm for automatically attaching teat cups of a milking machine to the teats of a milking animal in a milking stall. The arrangement comprises a three-dimensional camera arrangement stationary arranged with respect to the milking stall; a visual teat sensing means arranged on the robot arm; and a control device provided, for each of the teats of the milking animal, for
determining teat positions from images recorded by the three- dimensional camera arrangement and the visual teat sensing means, and for controlling the movement of the robot arm in order to attach a respective one of the teat cups to the teat of the milking animal in a two-stage procedure. During a first stage the robot arm is controlled to move to an approximate position for attachment of the teat cup to the teat of the milking animal based on a position of the teat as determined from images recorded by the three-dimensional camera arrangement, and during a second stage the robot arm is controlled to fine adjust its position based on a position of the teat as determined from images recorded by the visual teat sensing means mounted on the robot arm.
Preferably, the stationary three-dimensional camera arrangement comprises a range camera such as a time of flight camera.
Yet preferably, the visual teat sensing means arranged on the robot arm comprises a scanning head including a laser source and an image detector.
By such arrangement the control device may control the robot arm very fast and accurately. The three-dimensional measurements and the subsequent processing of the measurement data provide high quality information extremely fast and as a result the robot arm is capable of operating at higher speed.
When an approximate position of the robot arm is reached the image recordings by the visual teat sensing means mounted on the robot arm and subsequent processing of the data provide fine adjusted position data and as result the robot arm can find an exact position at very high speed enabling fast attachment of teat cups. As a result the total milking time will be shorter and the throughput of animals is increased.
In one embodiment the three-dimensional camera arrangement comprises two three-dimensional camera systems arranged on each side of the milking animal when located in the milking stall.
Hereby, images of the left teats of the milking animal can be recorded by the camera system arranged on the left side of the milking animal and images of the right teats of the milking animal can be recorded by the three-dimensional camera arranged on the right side of the milking animal.
According to a further aspect of the invention there is provided a method for controlling a movable robot arm for automatically attaching teat cups of a milking machine to the teats of a milking animal in a milking stall of a milking system provided with an arrangement according to the first aspect of the invention.
According to the method the following steps are performed for each of the teats of the milking animal . Images of the teat of the milking animal in the milking stall are recorded by the three-dimensional camera arrangement stationary arranged with respect to the milking stall. A position of the teat is calculated from images recorded by the three-dimensional camera arrangement. The robot arm is controlled to move to an approximate position for attachment of the teat cup to the teat of the milking animal based on the position of the teat as determined from the images recorded by the three-dimensional camera arrangement. Images are next recorded by the visual teat sensing means mounted on the robot arm and a more accurate position of the teat is calculated from the images recorded by the visual teat sensing means mounted on the robot arm. Finally, the robot arm is controlled to move from the approximate position to a fine adjusted position for attachment of the teat cup to the teat of the milking animal based on the position of the teat as determined from the images recorded by
the visual teat sensing means mounted on the robot arm. Then the teat cup can be attached to the teat of the milking animal.
Further characteristics of the invention, and advantages thereof, will be evident from the following detailed description of preferred embodiments of the present invention given hereinafter and the accompanying Figs. 1-2, which are given by way of illustration only, and are thus not limitative of the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS
Figs. 1-2 display each schematically, in a top view, a milking system including an arrangement for controlling a movable robot arm of the milking system according to a respective embodiment of the present invention.
DETAILED DESCRIPTION OF EMBODIMENTS
In Fig. 1 is shown a milking system, in which an arrangement according to an embodiment of the invention is implemented. The milking system comprises an enclosed milking stall 12, a milking machine 15 with teat cups 15a. A robot arm 14 is provided for automatically attaching the teat cups 15a to the teats of a milking animal 13 prior to milking. A teat cleaning device 16 including brushes and/or a teat cleaning fluid distribution device such as a teat cleaning cup 16a may be provided for cleaning the teats of the milking animal 13 prior to the attachment of the teat cups 15a to the teats of the milking animal 13. The teat cleaning cup 16a may also be automatically attached to the teats of the milking animal 13 e.g. in a sequential order by the robot arm 14.
A processing and control device 17 is provided for processing and controlling of the milking system, and comprises typically a microcomputer, suitable software, and a database of the milking animals served by the milking system. Such database includes
typically information of the milking animals , such as e.g. when the milking animals were milked last time, their milk production, their milk quality, etc. In particular the processing and control deviςe 17 is operatively connected to the milking machine 15 , the robot arm 14, and the teat cleaning device 16 to control the operations thereof.
In order to detect the teats of the milking animal 13 and to determine the positions thereof automatically to provide information so as to enable the processing and control device 17 to control the robot arm 14 to automatically attach the teat cups 15a to the teats of the milking animal 13, a teat detection arrangement 10 is provided, which comprises a three- dimensional camera arrangement 11a stationary arranged with respect to the milking stall 12, a visual teat sensing means lib arranged on the robot arm, and digital image processing means which may be integrated into the processing and control device 17 as illustrated, may be integrated into the three- dimensional camera arrangement 11a and the visual teat sensing means lib, or may be a stand alone unit. The three-dimensional camera arrangement 11a and the visual teat sensing means lib are operatively connected to the processing and control device 17 in order to deliver image data thereto.
The three-dimensional camera arrangement 11a is directed towards the teats of the milking animal 13 in the milking stall 12.
The three-dimensional camera arrangement 11a is preferably a range camera such as a time of flight camera and has a two dimensional array of pixels, each of which is capable of returning time of flight information as well as intensity. A commercially available camera that can be used in the present invention is the SwissRanger SR3000 from Mesa Imaging AG, Switzerland. The SR3000 is a general purpose range three- dimensional real time imaging camera. The camera can easily be
connected to the image processing device via USB 2.0, enabling straightforward measurement of real-time depth maps.
One further advantage of the time of flight camera is that the recorded image contains data sufficient for obtaining absolute position data in a given coordinate system and thus for obtaining absolute size information of objects recorded.
The visual teat sensing means lib, which is mounted on the robot arm 14, is directed forward and possibly slightly upwards .
The visual teat sensing means lib is preferably a scanning head including a laser source and a two-dimensional image capturing device such as the one depicted in EP 1170987 Bl. Other visual teat sensing means lib are disclosed in documents cited in the search report associated with the above patent or documents cited in other patent applications having the same priority as EP 1170987 Bl.
The inventive arrangement operates, for each of the teats of the milking animal 13 in the milking stall 12, in a two-stage procedure as follows. Images of the teat of the milking animal 13 in the milking stall 12 are recorded by the three- dimensional camera arrangement 11a and the image data is continuously forwarded to the processing and control device 17, which determines a position of the teat from the images recorded by the three-dimensional camera arrangement 11a. The processing and control device 17 controls the robot arm to move to an approximate position for attachment of the teat cup 15a to the teat of the milking animal 13 based on the position of the teat as determined from the images recorded by the three- dimensional camera arrangement lla. This is the first stage of operation. The robot arm is moved to an approximate position, which may differ from an exact position by from about ±10 mm to ±30 mm, and preferably about ±20 mm.
Next, images are recorded by the visual teat sensing means lib mounted on the robot arm 13 and the image data is continuously forwarded to the processing and control device 17, which determines a more accurate and updated position of the teat from the images recorded by the visual teat sensing means lib mounted on the robot arm 13. Finally, the processing and control device 17 controls the robot arm 14 to move from the approximate position to a fine adjusted position for attachment of the teat cup 15a to the teat of the milking animal 13 based on the position of the teat as determined from the images recorded by the visual teat sensing means lib mounted on the robot arm 14. This is the second stage of operation.
Then the teat cup can be attached to the teat of the milking animal. The robot arm is moved to a fine adjusted position, which may differ from an exact position by less than 5 mm.
In one embodiment the three-dimensional camera arrangement comprises two three-dimensional camera systems advantageously arranged on each side of the milking animal when located in the milking stall. Images of the left teats of the milking animal are recorded by the camera system arranged on the left side of the milking animal and images of the right teats of the milking animal are recorded by the camera system arranged on the right side of the milking animal. Such embodiment is illustrated in Fig. 2.
The schematically outlined milking system of Fig. 2 is a rotary milking system, wherein the milking stall 12 together with other milking stalls are arranged on a rotating platform 20. Each of the milking stalls is capable of receiving a milking animal and is provided with milking equipment for milking the milking animal therein.
The rotating platform 20 may, for each of the milking animals, be kept still while the milking robot 14 automatically attaches
teat cups to the teats of the milking animal, and is rotated there in between, but preferably the rotating platform 20 is rotated continuously during the attachment of the teat cups and the milking of the milking animals.
The two camera systems lla, 11a' of the three-dimensional camera arrangement can be arranged to be movable along a respective path in the form of a rail 21, 21' or similar mounted e.g. in a ceiling of a building housing the rotary milking system. The movement of two camera systems lla, lla' can be locked to the movement, i.e. rotation, of the rotating platform 20 such that the two camera systems lla, lla' are stationary with respect to the rotating platform 20 when recording images of the teats of the milking animal 13 in the rotating milking stall 20.
Alternatively, the two camera systems lla, lla' are stationary with respect to the rotating platform 20 during all times.
When the teat cups are attached to the teats of the milking animal the two camera systems Ha 7 lla' are moved in an opposite direction with respect to the direction of movement of the rotating platform 20 such that the two camera systems lla, lla' will be aligned for use in another one of the milking stalls, when again the two camera systems lla, lla' are locked to the rotating platform.
It shall be appreciated that the milking system may be provided with more than one robot arm provided with visual teat sensing means as described above, each visual teat sensing means being operatively connected to its own, or to a central, processing and control device.
Preferably the milking system has one common processing and control device and one or two robot arm(s) 14 provided with visual teat sensing means lib as described above. The robot arm
14 is preferably movable along one of the rails 21, 21' or
along a separate path such that it can be moved back and forth between the milking stalls, e.g. corresponding to the movement of the two camera systems 11a, 11a' .
It shall further be appreciated that the two camera systems 11a, 11a' of the three-dimensional camera arrangement can be arranged in other configurations such as behind the milking animal and beside the milking animal, behind the milking animal and in front of the milking animal, under the milking animal and beside the milking animal, etc., and each of the two camera systems can be used for recording images of as many teats of the milking animal as are visible from the camera system.
It shall yet further be appreciated that the inventive arrangement may comprise even more vision systems, which are used for the determination of the teats of the milking animal or for other purposes. Further, the inventive arrangement may be implemented in a variety of different milking systems.