TECHNICAL FIELD OF THE INVENTION

The present invention generally relates to dairy farm robot milking.

DESCRIPTION OF RELATED ART AND BACKGROUND OF THE INVENTION Robot milking and automatic milking systems (AMS) have become increasingly popular during recent years due to its capabilities of eliminating labor; offering milking consistency, increased milking frequency, and herd management; and providing lower stress environment for the milking animals. The approach comprises at least one robot arm, which automatically attaches teat cups of a milking machine to the teats of milking animals. To this end some kind of teat position sensing arrangement has to be applied, by aid of which the robot arm can find the teats of the milking animals. Various two-dimensional or three-dimensional camera arrangements have been proposed as such teat position sensing arrangement. WO 2010/023122 discloses an arrangement for determining positions of the teats of a milking animal in a milking system comprising a milking stall for housing the milking animal during milking, a movable robot arm for automatically attaching teat cups to the teats of the milking animal in the milking stall, and a control device for controlling the movement of the robot arm based on determined positions of the teats of the milking animal. The arrangement comprises a camera directed towards the udder of the milking animal in the milking stall and provided to repeatedly record images of the udder of the milking animal, and processing means provided for repeatedly forming three-dimensional images from the images recorded by the camera; and for detecting the teats of the milking animal and determining their positions in all three spatial dimensions based on the repeatedly formed three-dimensional images. SUMMARY OF THE INVENTION

The present inventors have come to the insight that in an automatic milking system comprising a camera directed towards the udder of the milking animal in the milking stall and provided to repeatedly record images of the udder of the milking animal, there are very little additional adjustments to be made in order to automatically determine a breeding rating.

Estimations of breeding values for dairy breeds in Sweden are based on IMAV (Nordic Cattle Genetic Evaluation). A large number of traits are combined into a breeding index; cows with high index being greater economic benefits to the farmer, and the use of dairy bull sires with high index will give offspring higher economic value. Traits that may go into the index comprises: milk yield and quality, growth, fertility (calving interval), calving (complexity, risk for stillborn calves), udder health (mastitis), other diseases, body, udder (udder shape, teat shape), feet and legs, longevity, milking speed, and temperament. The traits are weighted together on the basis of economic values quantifying the value of a marginal change in the trait. For dairy bull sires, the index also reflects traits of their daughters. Traits such as body, udder (udder shape, teat shape), and feet and legs are typically determined by manual inspection by an experienced herdsman or breeding expert. This is labor intensive and costly and is thus normally only performed at rare occasions. Further, since the trait determination is made by manual inspection, it dependent upon the person making the determination and is thus based on a subjective judgment.

In one aspect a combined teat position and breeding rating determining arrangement is provided for a milking system comprising a milking stall for housing the milking animal during milking, a movable robot arm for automatically attaching teat cups to the teats of the milking animal in the milking stall, and a control device for controlling the movement of the robot arm based on determined positions of the teats of the milking animal. The arrangement comprises a three-dimensional camera, such as e.g. a time-of-flight camera, directed towards the udder of the milking animal in the milking stall and provided to repeatedly record three-dimensional images of the udder of the milking animal prior to milking ; and processing means provided to repeatedly detect the teats of the milking animal and determine their positions based on the repeatedly recorded three-dimensional images. Further, the processing means is arranged to automatically determine the shapes of the teats of the milking animal based on the repeatedly recorded three-dimensional images and to calculate a breeding rating based on the determined shapes of the teats of the milking animal.

Preferably, the processing means is arranged to repeatedly detect the udder including the teats of the milking animal and determine its position based on the repeatedly recorded three-dimensional images, and is arranged to automatically determine the shape of the udder of the milking animal based on the repeatedly recorded three-dimensional images and to calculate a breeding rating based on the determined shape of the udder of the milking animal.

The breeding rating is the breeding rating for that milking animal or for the dairy bull sire which is the father of the milking animal. However, the breeding rating for such a dairy bull sire should of course reflect also other daughters of him.

The processing means may be arranged to automatically determine one or more of the lengths of the teats of the milking animal; the circumferences or thicknesses of the teats of the milking animal; conicities of the teats of the milking animal; the distances between the teats of the milking animal; the orientations of the teats of the milking animal; the orientations of the front teats as compared to the rear teats of the milking animal; the distance between the udder and the floor, on which the milking animal stands; the distance between the udder and the floor for a front portion of the udder as compared to a rear portion of the udder of the milking animal; and the distance between the udder and the floor for a left portion of the udder as compared to a right portion of the udder of the milking animal based on the repeatedly recorded three-dimensional images and to calculate the breeding rating based on the determined parameter(s).

The automated teat position and breeding rating determining arrangement is capable of automatically calculating a breeding rating based on the determined shapes of the teats of the milking animal and optionally on the above determined parameter(s). The breeding rating is determined without labor intensive and costly manual inspection. Further, the breeding rating can be calculated at higher repetition frequency. The three-dimensional camera, which is used for imaging the teats to thereby enable control of the robot arm during teat cup attachment, is also used for the determination of the breeding rating. Thus, the breeding rating can be determined automatically to virtually no additional cost.

The determination of the teat shapes and/or the above parameter(s) and the calculation of the breeding rating may be performed each time the milking animal is milked or somewhat more rarely, such as once a day, once a week, once a month or once a year. The processing means can be arranged to automatically compare the determined shapes and/or the above parameter(s) or the calculated breeding ratings at two different milkings (e.g. consecutive determinations/calculation) and to automatically initiate an alarm if the determined shapes and/or the above parameter(s) differ from one another to more than a threshold extent or the calculated breeding ratings differ from one another to more than a threshold value.

In such instance, the processing means can be arranged, concurrently with the initiation of the alarm, to send instructions to the milking system to not milk the milking animal and/or to take actions in order to give the milking animal treatment. In one embodiment, the three-dimensional camera is directed towards the udder of the milking animal in the milking stall immediately after the milking of the milking animal and is provided to record at least one three-dimensional image of the udder of the milking animal, and the processing means is arranged to automatically determine the post-milking shapes of the teats of the milking animal based on the at least one three-dimensional image recorded immediately after the milking and to calculate a breeding rating based also on the post-milking shapes of the teats of the milking animal. It is quite obvious that the shapes of the teats of the milking animal are different immediately prior to a milking as compared to immediately after the milking. By these provisions, the breeding rating is calculated based on both the pre-milking teat shapes and the post-milking teat shapes. In another embodiment, the processing means may be arranged to automatically receive or collect from the milking system one or more of: the milk yield of the milking animal; the milking frequency of the milking animal; the quality of the milk from the milking animal; the udder filling level of the milking animal, the lactation phase of the milking animal; the health of the milking animal; the udder health of the milking animal, the weight of the milking animal; the age of the milking animal; the condition of the feet and legs of the milking animal; the fertility of the milking animal; and the stillbirth frequency of the milking animal, wherein the processing means may be arranged to automatically calculate the breeding rating also based on this received or collected information.

In yet another embodiment, the three-dimensional camera is movable, is directable towards another area of the milking animal in the milking stall, and is provided to record one or more three-dimensional images of this other area of the milking animal, wherein the processing means may be arranged to automatically determine at least one physical property of the milking animal based on the recorded one or more three-dimensional images of the other area of the milking animal and to automatically calculate the breeding rating also based on the at least one physical property.

The other area of the milking area may be an area including one or more feet and/or one or more legs of the milking animal; and the at least one physical property of the milking animal may be a physical property of the one or more feet and/or the one or more legs.

In another aspect, a combined teat position and breeding rating determining method is provided for a milking system comprising a milking stall for housing the milking animal during milking, a movable robot arm for automatically attaching teat cups to the teats of the milking animal in the milking stall, and a control device for controlling the movement of the robot arm based on determined positions of the teats of the milking animal. A three-dimensional camera is directed towards the teats of the milking animal in the milking stall. Three-dimensional images of the teats of the milking animal are repeatedly recorded by the three-dimensional camera. The teats of the milking animal are detected and their positions are determined repeatedly and automatically based on the repeatedly recorded three-dimensional images. The shapes of the teats of the milking animal are automatically determined based on at least one of the same repeatedly recorded three-dimensional images as used for the teat position determination. Finally, a breeding rating is automatically calculated based on the determined shapes of the teats of the milking animal.

Further characteristics and advantages will be evident from the following detailed description of embodiments given hereinafter and the accompanying Figs. 1-4, which are given by way of illustration only, and are thus not limitative. BRIEF DESCRIPTION OF THE DRAWINGS

Figs. 1-3 display each a schematically outlined milking system including a combined teat position and breeding rating determining arrangement according to a respective embodiment. Figs. 1 and 3 are top views while Fig. 2 is a perspective view. Fig. 4 displays schematically, in a flow diagram, a combined teat position and breeding rating determining method according to an embodiment.

Identical reference numerals are used throughout the Figures to denote identical or similar components, portions, details and the like of the various embodiments.

DETAILED DESCRIPTION OF EMBODIMENTS In Fig. 1 is shown a rotary milking system 3, in which a combined teat position and breeding rating determining arrangement according to an embodiment is implemented. The rotary milking system 3 comprises a plurality of milking stalls 5, which milking animals 8 enter in a sequential order. Each of the milking stalls 5 comprises milking equipment including teat cups that are attached to the teats of the milking animal present in the milking stall prior to milking. For sake of simplicity teat cups 11 are illustrated only for one of the milking stalls 5. The rotary milking system 3 may be of parallel, tandem, or herringbone configuration. In the parallel configuration the longitudinal directions of the milking stalls and of the milking animals therein extend radially (the milking animals stand side by side), in the tandem configuration the longitudinal directions of the milking stalls and of the milking animals therein extend circumferentially (the milking animals stand one after the other), and in the herringbone configuration, which is illustrated in Fig. 1, the longitudinal directions x of the milking stalls and of the milking animals therein extends partly radially, partly circumferentially.

A robot 14 provided with a robot arm 15 is provided for automatically attaching teat cups 11 to the teats of the milking animals 8 present in the milking stalls 5 under the control of a control device 19, which is operatively connected to the milking robot 14. The milking robot 14 is preferably stationary with respect to a rotatable carousel or rotating platform 20 of the rotary milking system 3, which forms the support for the milking stalls 5. Alternatively, the milking robot 14 is movable back and forth in e.g. a circumferential direction.

The rotating platform 20 may, for each of the milking animals, be kept still while the milking robot 14 automatically attaches teat cups 11 to the teats of the milking animal 8, and be rotated there in between. Alternatively, the rotating platform 20 is rotated continuously during the attachment of the teat cups 11 and the milking of the milking animals 8.

A combined teat position and breeding rating determining arrangement comprising a three-dimensional camera 21 and processing means 22 is provided partly to determine positions of the teats of the milking animals 8 present in the milking stalls 5, thereby enabling the robot 14 to move the teat cups 11 to the teats of the milking animals 8 present in the milking stalls 5, partly to determine a breeding rating. The three-dimensional camera 21 may be a time-of-flight camera. A commercially available time-of-flight camera that can be used in the present invention is the SwissRanger SR3000 from Mesa Imaging AG, Switzerland. The SR3000 is a general purpose range three-dimensional real time imaging camera. The three-dimensional camera 21 can easily be connected to a computer via USB 2.0, enabling straightforward measurement of real-time depth maps. Designed for operation under lighting conditions, the compact camera may be offered with a complete visualization software program.

The three-dimensional camera and the technique behind the three-dimensional camera are further disclosed in the publications "An all-solid-state optical range camera for 3D real-time imaging with sub-centimeter depth resolution (SwissRanger™)" and "3D-Imaging in Real-Time with Miniaturized Optical Range Camera" by T. Oggier et al. available at the Mesa Imaging AG Internet site http://www.mesa-imaging.ch/ on December 27, 2007. The contents of the above publications are hereby incorporated by reference. The three-dimensional camera 21 is, for each of the milking animals in the milking stalls, directed towards the udder of the milking animal to repeatedly record three-dimensional images of the udder of the milking animal in real time. The processing means 22 is provided, for each of the milking animals, for repeatedly detecting the teats of the milking animal and determining their positions in all three spatial dimensions by a calculation method based on the repeatedly recorded three-dimensional images of the udder of the milking animal.

The combined teat position and breeding rating determining arrangement thus determines directly the position in all three spatial coordinates for each object point that is found in the pixels of the image recorded. Thus, the coordinates are in principle determined in real time.

Further, the processing means 22 is arranged, for each of the milking animals, to automatically determine the shapes of the teats of the milking animal by a calculation method based on the repeatedly recorded three-dimensional images and to calculate a breeding rating based on the determined shapes of the teats of the milking animal.

The processing means 22 is preferably implemented as a software module in the control device 19, which thus is operatively connected to the three-dimensional camera 21, or in any other device operatively connected to the three-dimensional camera 21 and the control device 19. The processing means 22 may alternatively be integrated in the three-dimensional camera 21. The processing means 22 may automatically determine one or more of: the lengths of the teats of the milking animal 8; the circumferences or thicknesses of the teats of the milking animal 8; conicities of the teats of the milking animal 8; the distances between the teats of the milking animal 8; the orientations of the teats of the milking animal 8; the orientations of the front teats as compared to the rear teats of the milking animal 8; the distance between the udder and the floor, on which the milking animal 8 stands; the distance between the udder and the floor for a front portion of the udder as compared to a rear portion of the udder of the milking animal 8; and the distance between the udder and the floor for a left portion of the udder as compared to a right portion of the udder of the milking animal 8 based on the repeatedly recorded three-dimensional images.

Thereafter, the processing means 22 may calculate the breeding rating based on one or more of: the lengths of the teats of the milking animal 8; the circumferences or thicknesses of the teats of the milking animal 8; the conicities of the teats of the milking animal 8; the distances between the teats of the milking animal 8; the orientations of the teats of the milking animal 8; the orientations of the front teats as compared to the rear teats of the milking animal 8; the distance between the udder and the floor, on which the milking animal 8 stands; the distance between the udder and the floor for the front portion of the udder as compared to the rear portion of the udder of the milking animal 8; and the distance between the udder and the floor for the left portion of the udder as compared to the right portion of the udder of the milking animal 8.

Calculation of the breeding rating based on manual inspection of the above parameters is well known and thus not detailed herein. It may be done in a plurality of manners as disclosed in the field and/or known to a skilled person. Reference is made for example to http ://www.vikinggenetics.com/sv/avel/, especially the article "Genetic evaluation within NAV".

The breeding rating may be calculated repeatedly to follow the rating development over time for a particular milking animal 8. Thus, the three-dimensional camera 21 may be directed towards the udder of the milking animal 8 in the milking stall and provided to repeatedly record three-dimensional images of the udder of the milking animal at a plurality of milkings of the milking animal 8, wherein the processing means 22 may, for each of the plurality of milkings, be arranged to automatically determine the shapes of the teats of the milking animal 8 based on the repeatedly recorded three-dimensional images and to calculate a breeding rating based on the determined shapes of the teats of the milking animal.

The plurality of milkings may or may not be consecutively performed milkings of the milking animal. In the latter case, the milkings may be performed once a day, once a week, once a month, once a year, more rarely, or more frequently.

In one embodiment, the processing means 22 is, for two following ones of the plurality of milkings, arranged to automatically compare the determined shapes of the teats of the milking animal 8 or the calculated breeding ratings; and to automatically initiate an alarm if, for the two following ones of the plurality of milkings, the determined shapes of the teats of the milking animal differ from one another to more than a threshold extent or the calculated breeding ratings differ from one another to more than a threshold value. This may indicate that the milking animal has suffered from a snake bite or that the udder/teats have been injured such as e.g. stepped on.

Concurrently with the initiation of the alarm, the processing means 22 may be arranged to send instructions to the milking system 3 not to milk the milking animal 8.

Alternatively, or additionally, the processing means 22 may be arranged, concurrently with the initiation of the alarm, to send instructions to the milking system 3 to take actions in order to give the milking animal 8 treatment.

Further, the processing means 22 may take further parameters into consideration when calculating the breeding rating for a milking animal 8. For instance, the processing means 22 may be arranged to automatically receive or collect from the milking system 3 one or more of: the milk yield of the milking animal 8; the milking frequency of the milking animal 8; the quality of the milk from the milking animal 8; the udder filling level of the milking animal 8; the lactation phase of the milking animal 8; the health of the milking animal 8; the udder health of the milking animal 8; the weight of the milking animal 8; the age of the milking animal 8; the condition of feet and legs of the milking animal 8; the fertility of the milking animal 8; and the stillbirth frequency of the milking animal 8.

Thereafter, the processing means 22 may automatically calculate the breeding rating also based on one or more of: the milk yield of the milking animal 8; the milking frequency of the milking animal 8; the quality of the milk from the milking animal 8; the udder filling level of the milking animal 8; the lactation phase of the milking animal 8; the health of the milking animal 8; the udder health of the milking animal 8; the weight of the milking animal 8; the age of the milking animal 8; the condition of feet and legs of the milking animal 8; the fertility of the milking animal 8; and the stillbirth frequency of the milking animal 8.

The determination of the shapes of the teats of the milking animal can be made in the following manner. First the milking animal is detected and the position thereof is determined. Based on this information the contour of the back legs and the body therein between is detected and the positions thereof are determined. Based on this information an area of interest (i.e. where the teats most probably are found) is located, and in this area the udder of the milking animal is searched for. When the udder has been detected and located the teats are searched for and located. Finally, when the shapes of the teats are determined, the breeding rating is calculated in dependence thereon.

The three-dimensional camera 21 can be mounted in a fixed position with respect to the different parts of the milking system 3 depending on the application in question. The three-dimensional camera 21 can be located at a fix position with respect to a floor, on which the rotary milking system 3 is installed and with respect to which the rotary platform 20 of the rotary milking system 3 rotates during milking. In such instance, the three-dimensional camera 21 can be mounted on the floor or in the ceiling of the building in which the milking system is installed.

Alternatively, the three-dimensional camera 21 can be mounted on the rotating platform 20 or on the robot arm 15. Still alternatively, the three-dimensional camera 21 can be movable along some path. In one version, for instance, the three-dimensional camera 21 can be movable between a first position or area, in which the three-dimensional camera 21 is located during the recordings of the three-dimensional images, and a second position, in which the three-dimensional camera 21 is located there in between, that is, when being idle.

In one embodiment, the three-dimensional camera 21 is movable, is directable towards another area of the milking animal 8 in the milking stall 5, and is provided to record one or more three-dimensional images of the another area of the milking animal 8, wherein the processing means 22 determines automatically at least one physical property of the milking animal 8 based on the recorded one or more three-dimensional images of the another area of the milking animal 8 and calculate the breeding rating also based on the at least one determined physical property. The another area of the milking animal 8 may be an area including one or more feet and/or one or more legs, whereas the at least one physical property of the milking animal 8 may be a physical property of the one or more feet and/or the one or more legs, such as the straightness of the hind legs.

The three-dimensional camera 21 can further be located, for each of the milking animals, in various positions and orientations during the recordings of the three- dimensional images of the teats of the milking animal 8. For instance, it may be located behind the udder of the milking animal 8, at the side of the udder of the milking animal 8, or below the udder of the milking animal 8.

Still further, two or more three-dimensional cameras can be connected to the processing means 22 in order to provide further detailed three-dimensional information of the positions and orientations of the teats of the milking animal 8.

For instance, several of the three-dimensional cameras seem to be usable if they are not mounted fixedly on the robot arm since at least one teat would possibly be obscured at each instant. Further, one or several three-dimensional cameras arranged stationary on the floor or on the rotating platform 20 could be combined with one three-dimensional camera mounted on the robot arm 15. It shall further be appreciated that the combined teat position and breeding rating determining arrangement may be used as a multifunctional detection arrangement for detecting any of the following : (i) presence of a milking animal 8 in a milking stall 5, (ii) the behavior of a milking animal 8, (iii) the activity of a milking animal 8, (iv) the body shape of a milking animal 8, (v) an incorrect teat cup attachment, (vi) a teat cup kick-off, (vii) presence of an obstacle in the working area of the milking robot 14, (viii) a parameter related to the operation of the milking robot 14, and (ix) a parameter related to the operation of the three- dimensional camera 21. With reference next to Fig. 2, the combined teat position and breeding rating determining arrangement is implemented in a voluntary milking system or station 3 comprising an enclosure having an inlet gate 4 and an outlet gate 5, which are both capable of being opened automatically. The front end of the milking station 3 is denoted by 3a, the back end is denoted by 3b, the sides are denoted by 3c and 3d.

The milking station 3 comprises further an automatic milking machine (not explicitly illustrated) including teat cups 11 connected to an end unit by means of milk lines (only the portions attached to the teat cups 11 are shown in Fig. 2). The milking station further includes a milking robot 14 having a movable robot arm 15 provided with a gripper. The milking robot 14 is arranged to automatically apply the teat cups 11 of the milking machine to the teats of a milking animal 8 present in the milking station 3 prior to milking. In Fig. 2 three of the teat cups 11 are arranged in a teat cup rack or magazine 16, whereas the fourth one is held by the gripper of the robot arm 15. Typically, a teat cleaning device including e.g. a teat cleaning cup 24 or brushes 25 may be provided for cleaning the teats of the milking animal 8 prior to milking.

Further, the milking station 3 comprises an identification device (not illustrated) provided to identify a milking animal approaching the milking station 3, and a control device 19, which is responsible for controlling of the milking system, which inter alia includes the initiation of various activities in connection with the milking such as e.g. opening and closing of the gates 4 and 5, and control of the milking machine and its handling device 14

The combined teat position and breeding rating determining arrangement comprises a three-dimensional camera 21 arranged to repeatedly record three- dimensional images of the udder of the milking animal in real time and processing means e.g. implemented in the control device 19. The three-dimensional camera 21 is mounted on the movable robot arm 15 of the milking robot 14.

The processing mean is arranged to repeatedly detect the teats of the milking animal 8 and determine their positions based on the repeatedly recorded three- dimensional images and to automatically determine the shapes of the teats of the milking animal 8 based on the repeatedly recorded three-dimensional images and to calculate a breeding rating based on the determined shapes of the teats of the milking animal 8.

In other respects, the embodiment of Fig. 2 may comprise any of the features or functionalities as disclosed with reference to the embodiment of Fig. 1.

Fig. 3 illustrates a further embodiment wherein the combined teat position and breeding rating determining arrangement is implemented in a voluntary milking system of the above kind. Here, two three-dimensional cameras 21 are mounted in fixed positions with respect to the milking station 3. One three-dimensional camera 21 is arranged to record three-dimensional images of the udder of a milking animal from behind and one three-dimensional camera 21 is arranged to record three-dimensional images of the udder of the milking animal from a side.

The processing means, e.g. implemented in the control device 19, to which the three-dimensional cameras 21 are operatively connected, is provided for determining the positions of the teats of the milking animal in all three spatial dimensions relative the milking station 3 based on three-dimensional images recorded by the two three-dimensional cameras. Further, the processing means is arranged to automatically determine the shapes of the teats of the milking animal based on the repeatedly recorded three-dimensional images and to calculate a breeding rating based on the determined shapes of the teats of the milking animal. It shall be appreciated that the combined teat position and breeding rating determining arrangements disclosed above may be implemented in virtually any kind of milking system where the teat positions need to be found automatically.

Fig. 4 displays schematically, in a flow diagram, a combined teat position and breeding rating determining method according to an embodiment. The method is implemented in a milking system comprising a milking stall for housing a milking animal during milking, a movable robot arm for automatically attaching teat cups to the teats of the milking animal in the milking stall, and a control device for controlling the movement of the robot arm based on determined positions of the teats of the milking animal.

A three-dimensional camera 21, such as e.g. a time-of-flight camera, is, in a step 41, directed towards the teats of the milking animal in the milking stall, and three- dimensional images of the teats of the milking animal is, in a step 42, repeatedly recorded by the three-dimensional camera 21. The teats of the milking animal are, in steps 43 and 44, repeatedly and automatically detected and their positions determined based on the repeatedly recorded three-dimensional images. Preferably, the steps 43 and 44 are performed by digital image processing software. These steps are performed in order to enable the control device to control the robot arm to attach the teat cups to the teats of the milking animal . Next, the shapes of the teats of the milking animal are, in a step 45, automatically determined based on the repeatedly recorded three-dimensional images. Finally, a breeding rating is, in a step 46, automatically calculated based on the determined shapes of the teats of the milking animal . Preferably, the steps 45 and 46 are performed by digital image processing software. It shall be appreciated that the breeding rating may be calculated also based on a number of other parameters, e.g. as being disclosed above. Each of the features disclosed above may be independently incorporated into the teat position and breeding rating determining method displayed in Fig. 4.