Field of the invention

The present invention relates generally to automated milking and to visual detection related thereto. In particular, the invention relates to a camera test arrangement, device and method for determining the performance of a camera arranged for use in an automated milking system.

Background of the invention

In automated milking, detection devices such as cameras or optical sensors are used for determining e.g. the position of a milking animal in a milking stall and the location of her teats. Such information is needed for guiding an automated milking apparatus, for example to guide a robot arm arranged to attach teat cups to the teats of the animal.

As an example, laser-based optical triangulation sensors may be used to image the teats and to derive their position in three dimensions. In particular, light emitted by a laser is projected into the field of view and the light as reflected from the teats is imaged on a camera arranged in the projection plane. The movement of the robot arm can then be controlled in response to the calculated positions so as to find the teats for teat cup attachments.

It is rather costly for a farmer not to be able to use his milking system to maximum extent, for example due to a detection system that requires excessive time for locating the teats of the animals or that fails altogether to locate them. It is thus important to have a well functioning camera or other detection system in the automated milking system. One reason for difficulties in locating the teats is shortcomings of the camera. For example, the camera should be kept clean so as to be able to make accurate and fast position determinations. However, the environment in which the camera is located is non-clean, and the camera is thus exposed to dirt. It is difficult to completely avoid having dirt settling on the camera. Further, the camera may also be damaged by physical contact with the milking animal, its performance thereby being degraded.

A service engineer, or possibly the farmer himself, may inspect the cameras at regular intervals, but this can generally not be done sufficiently often at reasonable costs. Even if such inspections were to be done, only some shortcomings of the camera would be detected. It is difficult, if not impossible, for a human eye to, for example, detect infinitesimal flaws of a camera.

In view of the above, it would be desirable to provide reliable means for ensuring proper functioning of cameras used in the automated milking system.

Summary of the invention

It is an object of the invention to provide arrangements and methods for determining the performance of a camera arranged in an automated milking system, and in particular for ensuring that the cameras function properly.

It is another object of the invention to provide means for fast detection of failing cameras and cameras having degraded performance. In particular, is an object of the invention to provide such means that can provide diagnosing of the cameras as often as desired, at reasonable cost and with high reliability. It is yet another object of the invention to provide arrangements and methods for determining the transparency of the viewing glass of a camera arranged in an automated milking system.

These objects, among others, are achieved by a camera test arrangement for determining the performance of a camera arranged for use in an automated milking system. The camera test arrangement comprises a camera test device with means for providing a repeatable test environment for the camera. The camera test arrangement further comprises image processing means for processing data obtained from the camera when it is arranged in the camera test device and for determining the performance of the camera. The invention provides reliable yet non-expensive means for ensuring proper functioning of the cameras and thereby assuring good image quality. A farmer is able to quickly find out when a camera is non-functioning or having degraded performance. He may then take proper action and avoid costly interruptions of the milking system. Further, the camera performance testing can be performed as often as desired, for example in dependence on the extent to which the camera is exposed to dirt .

In accordance with an embodiment of the invention, the means for providing a repeatable test environment comprises means for receiving the camera into a predetermined position for imaging a predetermined view. Such means could for example comprise rails, slots or a shape precisely following the shape of the camera. The camera test device can thus be implemented in a flexible manner, designed to suit a particular environment .

In accordance with another embodiment, the means for providing a repeatable test environment comprises a lightproof housing. This is an efficient and reliable means for providing an environment with non-changing light conditions.

In accordance with yet another embodiment, the camera test device comprises n reference objects, preferably sticks, arranged in a predetermined pattern within the camera test device. Reliable means for providing data that can be analyzed by the image processing means is thus provided. Such a solution is also cost-efficient in that software used for determining teat positions can be used also for analyzing the performance of the camera. In an embodiment, the predetermined pattern is symmetrical and comprises a number of rows comprising reference stick positions laterally displaced. The pattern can be designed so as to suit the particular user, for example regarding accuracy.

In accordance with still another embodiment, the performance of the camera is defined in terms of degree of transparency of a viewing glass of the camera. The image processing means is able to determine lowered transparency of the viewing glass by identifying cracks, dirt, and/or calcareous deposits. This is an important feature in that the viewing glass is very exposed to dirt and is often damaged e.g. by cows lowering the performance of the camera. Such lowered performance of the camera and its ability to locate the teats can quickly be identified.

In a further embodiment, data from different parts of the viewing glass can be treated differently by the image processing means. If it is determined that there is a crack in a corner of the viewing glass, data originating from that part of the viewing glass can be disregarded.

Further embodiments of the inventive camera test arrangement are set out in the dependent claims. The invention further relates to a corresponding camera test device and methods, whereby corresponding advantages are obtained.

Brief description of the drawings

Figure 1 illustrates an exemplary milking system in which the present invention may be implemented.

Figures 2a and 2b illustrates in a side view and view from above, respectively, an embodiment of the camera test arrangement in accordance with the invention.

Figure 3 illustrates a flow chart over steps of the method in accordance with the invention.

Detailed description of the invention

Figure 1 illustrates an exemplary milking system or station 1 in which the present invention may be implemented. The milking system 1 is arranged for voluntary milking of freely walking animals such as cows, i.e. the cows enter the milking system 1 in order to be milked on a voluntary basis. The milking system 1 comprises a number of conventional parts, for example one or more milking stalls 2, teat cups 5 connected to an end unit by means of milk lines (only the portions attached to the teat cups 11 are shown in Fig. 1) , teat cup rack or magazine 8, etc. Since such parts are not essential for the invention they are not described further herein. Another exemplary milking system in which the present invention may be implemented is a rotary system, but any type of automated milking system may benefit from the present invention. The milking stalls 2 may be arranged in any suitable manner, for example in herringbone, parallel or tandem configurations.

The milking system 1 further comprises a milking robot or automatic handling device 3 including a processing and control device 6 and a robot arm 4 provided with a gripper. The milking robot 3 is arranged to, among other things, automatically apply the teat cups 5 to the teats of the cow 10 present in the milking stall 2 prior to milking.

The processing and control device 6 of the milking robot 3 is responsible for processing and controlling of various actions in the milking system, which inter alia includes the initiation of various activities in connection with the milking such as e.g. opening and closing of gates, cleaning of teats, attaching teat cups, initiating milking. The processing and control device 6 comprises typically a microcomputer, suitable software, and a database including information about each of the cows milked by the milking robot 3, such as e.g. when the respective cow was milked last time, when the cow was fed last time, the milk production of the cow, the health of the cow, etc.

A visual detection means, in the following a camera 7, is provided on or in connection to the robot arm 4. The camera 7 is preferably rotatably mounted on the robot arm 4 so as to be able to detect the positions of the teats of the cow 10, but may also be mounted stationary with respect to the milking station. Additional visual detection means or cameras may also be provided, but for simplicity only a single camera is illustrated in the figure.

The camera 7 comprises a housing having sidewalls, a rear wall and a front wall. The front wall comprises a viewing opening covered by a viewing glass 7a, into which light from the environment being photographed enters. The viewing glass 7a is thus made of a material transmissive for light. The camera 7 further comprises other conventional parts, for example lens systems and optical sensors. The viewing glass 7a, only schematically indicated in figure 1, is the part of the camera 7 that is most sensitive to dirt and damages when considering the ability of the camera 7 to locate the teats. For example, a crack in the viewing glass 7a may result in errors of refraction and therefore difficulties to locate the teats. Besides dirt and damages, the performance of the viewing glass 7a can be lowered by calcareous deposits. Many dairy farms use water from wells that are highly calcareous and when the viewing glass 7a of the camera 7 is cleaned, calcareous deposits may result during the course of time. Such calcareous deposits also lowers the performance of the camera 7 and the problem is increased when cleaning the viewing glass 7a.

Image processing means 9 is further provided for calculating the positions of the teats of the milking animal 10. The teat positions may, for example, be determined in all three spatial dimensions by a calculation method based on repeatedly recorded images of the udder of the milking animal. The image processing means 9 is preferably implemented as a software module in the control device 6, which thus is operatively connected to the camera I ₁ or in any other device operatively connected to the camera 7 and the control device 6. The image processing means 9 may alternatively be a separate processing device or be part of the camera 7.

The camera 7 may be any known type of camera, comprising means for determining the positions of the teats of the cow. The camera 7 comprises one or more light sources 13 or is arranged to work with an external light source. The light source 13, e.g. a laser source, emits light towards the udder of the cow. The emitted light is then reflected by the udder and the teats, and travels back to the camera 7. The light as reflected is registered in the camera 7. The distance to the teat reflecting the light can thereby be calculated. Any type of range-finding camera can be used, including a three- dimensional camera, e.g. a time-of-flight camera.

The detection of the positions of the teats could be performed by means of triangulation, mentioned briefly in the introductory part of the present application. Triangulation relies on known positions of the camera 1 , the light source 13 and their respective orientations, in order to determine the distance to the teats . The camera 7 conveys data to the image processing means 9, wherein the data is analysed in order to obtain a so-called contour map of the udder, in which the teats are recognizable. The contour map is a representation of the udder by means of which the teats and thus their positions can be recognized. In order to obtain reliable detection of the positions, a reliable contour map is required. Therefore, it is important that the camera 7 is functioning properly.

In accordance with the invention, a camera test arrangement 20 is provided for testing, or diagnosing, the camera 7, regarding its functionality. The camera test arrangement 20 comprises the earlier mentioned image processing means 9.

The camera test arrangement 20 further comprises a camera test device 11, designed so as to provide a constant, i.e. non- changing, environment for the camera 7 when docked to it. The environment is designed to give repeatable and comparable conditions for the camera 7 and could for example be a lightproof box, having a construction not letting any external light pass through.

Further, in order to provide repeatable conditions, the camera test device 11 also comprises means for forcing the camera 7 into the same position every time the test is to be performed. For example, the camera test device 11 may be designed so as to precisely follow the external shape of the camera 7. That is, so that the camera 7 precisely fits into the camera test device 11. Alternatively, the camera test device 11 comprises guidance means such as a rail or slot into which some part of the camera 7 is guided, thereby guiding the camera 7 into the same position each time. In the solution indicated in figures 2a and 2b, the camera 7 is pushed into the camera test device 11 until it reaches a pair of stops 14. Alternatively, gripping means for gripping and placing the camera 7 in the same position each time is provided. In the case of a stationary camera, gripping means for gripping and placing the camera test device each time in the same position with respect to the camera may be provided. A person skilled in the art may provide further solutions for docking the camera 7 to the camera test device 11 or the camera test device 11 to the camera 7 at the same place each time, so as to provide a repeatable environment for the camera 7.

Figure 2a illustrates the camera test arrangement 20 in accordance with the invention. A dashed line A illustrates schematically the coupling to the image processing means 9. In figure 2a the camera 7 is shown, in a side view, as being in its predetermined position within the camera test device 11. Light beams Bl and B2, indicated in the figure, illustrates light emitted from the light source 13, and passed through the viewing glass 7a.

The camera test device 11 comprises n reference sticks 12a, 12b,..., 12i,..., 12n used for determining the performance of the camera 7, as will be described in the following. The number of reference sticks can be suitably chosen, depending on size of camera, desired accuracy etc.

In figure 2a, the camera test device 11 is shown comprising a lightproof housing 15, providing a constant light environment for the camera 7. The housing 15 comprises an opening to which the camera 7 and/or the viewing glass 7a can be docked, for example inserted into.

In figure 2b, the camera test device 11 and the camera 7 are shown in a view from above. The reference sticks 12a, 12b,..., 121,..., 12n are indicated by dashed lines.

The reference sticks 12b,..., 12i,..., 12n are placed in a predetermined, preferably symmetrical, pattern in the X-Y- plane, as indicated in figures 2a and 2b. The distances between the reference sticks 12a, 12b,.,., 12i,..., 12n and the camera 7, when the camera 7 is in its predetermined position within the camera test device 11, are therefore known. Further, the widths of the reference sticks are known and also the number n of reference sticks. This known data can then be used for analyzing the actual data obtained when the camera 7 is utilized within the camera test device 11, in order to determine the performance of the camera 7.

For example, the distance to a certain reference stick 12i can be compared to a distance measured by the camera 7. The light beams Bl, B2 transmitted by the light source 13 are reflected by the reference sticks whereby the distance can be measured. The image processing means 9, possibly in cooperation with the control device 6, is/are arranged to analyse the data. For example, the transparency of the viewing glass 7a and the performance of the light source 13 can be determined, or the sharpness of the images taken by the camera 7. Based upon the transparency, conclusions can be drawn regarding, for example, the need to clean the camera 7.

The image processing means 9 receives data from the camera 7, and is able to receive or retrieve reference data from a database. The reference data can be any kind of data relating to the performance of the camera, for example expected distance values or reference images. The image processing means 9 is then able to analyze the results obtained when the camera 7 is utilized within the camera test device 11, for example comparing the measured data to the stored reference data. The reference data could be data obtained before the camera 7 was taken into use in the milking system, i.e. when the camera was new.

The same image processing means 9, and in particular the same software, as is used for determining the positions of the teats described earlier can be used for determining the performance of the camera 7.

As an example, the predetermined pattern into which the reference sticks are placed may comprise reference stick positions arranged in a number of rows that are displaced laterally in relation to each other, as is shown in figure 2b, The reference sticks are placed so that it is possible to obtain data originating from each part of the image. That is, one or more of the reference sticks can reflect the light as let through from different parts of the viewing glass 7a.

Historical data for the camera 7, i.e. data obtained from different camera test occasions can be stored in the control device 6, or elsewhere. If the data indicates a deteriorating performance of the camera 7, the farmer may take appropriate action, for example perform service of the camera. Threshold values may be used for identifying such problems with the camera performance.

In an embodiment, if the data indicates that there is a problem, the camera 7 may be arranged to clean its viewing glass 7a. Such cleaning may be performed automatically by moving the camera 7 to a cleaning device. After performed cleaning, the test may be repeated in order to determine whether the problem remains. If the problem remains, an alarm may be set off alerting the farmer about it. Similarly, the camera 7 may be arranged to scrape off deposits on the viewing glass 7a.

Preferably, data from each reference stick is analyzed individually. Thereby the transparency of the viewing glass 7a at a location corresponding to the placement of the specific reference stick can be determined. That is, the glass transparency of different parts of the viewing glass area can be determined. Further, depending on the degree of deterioration of the glass transparency or depending on which part of the viewing glass 7a that has a lowered transparency, different actions may be taken. For example, if it is determined that a crucial part of the viewing glass 7a has lowered transparency immediate action is called upon, e.g. by means of an alarm. If however it is determined that a less important part of the viewing glass 7a has lowered performance, and that the camera 7 may be continued to be used temporarily, then the farmer may only be notified about a coming need for service .

In particular, the image processing means comprises means, e.g. algorithms, for determining lowered transparency of the viewing glass 7a by identifying cracks, dirt, and/or calcareous deposits etc. Further, the image processing means comprises algorithms for disregarding data received from parts that have lowered performance. That is, data from different parts of the viewing glass can be treated differently by the image processing means. For example, if it is determined that there is a crack in a corner of the viewing glass 7a, data originating from that part of the viewing glass 7a can be disregarded.

The camera test arrangement 20 may be used at regular intervals, or when problems occur. For example, if there are recurrent incidents of difficulties to locate the teats of the milking animals, a camera test may be performed.

The camera test device 11 may be fixedly arranged within the milking stall 2, or integrated therein, at a position so that the camera 7 can be moved to it and to the predetermined test position within the camera test device 11. Alternatively, the camera test device 11 is a movable device that can be moved to different milking stalls 2 when needed.

In the above description, the camera 7 is described as being utilized for detecting the teat positions. However, there could be a need to determine the position of other parts of the cow and the invention may be used for such needs also.

Figure 3 illustrates steps of the method in accordance with the invention. The method 30 for determining the performance of the camera 7 comprises the first step of docking the camera 7 to the camera test device 11. The docking 31 can be done in any suitable manner. For example, the camera 7 may be moved to the camera test device 11 by controlling the robot arm 4. Thereafter any of the earlier mentioned guidance means for guiding the camera 7 into the predetermined position might be used. The camera 7 is then used 32 within the camera test device 11 in order to obtain image data. The data obtained from using the camera 7 is then analysed 33, for example compared 33 with expected data. The expected data are stored in or retrieved by the image processing means from a suitable database, for example stored in the control device 6 controlling the milking system 1. Finally, the performance of the camera 7 is determined 34 based on the step of analyzing 33.

The step of using 32 the camera 7 comprises the steps of emitting light from the light source 13 into the camera test device 11 and registering in the camera 7 light as reflected by the reference sticks 12a, 12b,..., 12n that are arranged in the camera test device 11. The data registered is then transmitted to image processing means 9 responsible for analyzing the data,

The steps 31 - 34 of the method 30 are preferably performed automatically.

Additional steps can be included, for example the step of cleaning the camera viewing glass 7a, preferably automatically, when a problem has been identified. Another example comprises the step of scraping off calcareous deposits from the viewing glass 7a.