HAMMERSLAND, Helge (Stølshaugen 5, Amatveit, N-5262, NO)
| CLAIMS
1. Method for an automatic classification of fish under water, comprising the following steps: to place a camera system for taking pictures of fish which are led passed the camera system, to analyse each picture to find the contour of the fish, and to divide the contour of the fish into a number of segments for analysis of each segment, characterised in: that the camera system is being placed under water, and to analyse transitions in each segment expressed by the strength of the transition, such as degree of darkness and direction, in an analysis unit, to describe characteristic features of the fish such as fins, eyes, stripes along the fish, pattern, etc., based on the analysis of the transitions, and to compare the characteristic features with data of fish in a knowledge database for classification offish, with help of discriminatory analysis.
2. Method according to claim 1, characterised in that the camera system is being placed in a trawl or the like which is pulled by a vessel, and that the analysis unit is arranged next to the camera system or onboard the vessel.
3. Method according to claims 1 or2, characterised in that after the contour of the fish is detected, the longitudinal axis of the fish is estimated, whereupon the object is normalised so that the longitudinal axis is approximately horizontal and the head of the fish turns to the left.
4. Method according to claim 3, characterised by generating a picture of the fish for displaying transitions of the object and to describe said transitions with the help of strength, such as degree of darkness and direction.
5. Method according to claims 3-4, characterised in that each fish is divided up into a number of equal segments, independent of the size and species offish, and to analyse transitions such as strength and direction within each segment for the formation of said characteristics.
6. System for automatic classification of fish under water, comprising a camera system arranged to take pictures of fish that pass, and an analysis unit arranged to receive and to analyse each picture to find the contour of the fish, and to divide up the contour of the fish into a number of segments to analyse each segment, characterised in that the camera system is arranged under water and that the analysis unit is arranged to analyse transitions in each segment expressed by the strength of the transitions such as degree of darkness and direction, thereafter to describe the characteristic features of the fish such as fins, eyes, stripes along the fish, pattern, etc., based on the analysis of the transitions and to compare the characteristic features with data of fish in a knowledge database for classification of fish with the help of discriminatory analysis, and to display the result in a user interface.
7. System according to claim 6, characterised in that the camera system is arranged onboard a trawl or the like which is pulled by a vessel and that the analysis unit is arranged next to the camera system or onboard the vessel.
8. System according to claim 7, characterised in that said trawl or the like comprises means to guide the fish passed the camera.
9. System according to claims 7 or 8, characterised in that said trawl or the like comprises a sorting system to sort fish that enter the trawl and that the analysis unit is connected to and arranged to control the sorting system, whereby unwanted fish or other marine organisms can be taken out of the trawl. |
Method and system for automatic classification of fish under water
The present invention relates to a method and a system for automatic classification of fish under water, where a camera system is arranged to take pictures of fish that are led passed the camera system, analysing each picture to find the contour of the fish and to divide the contour of the fish into a number of segments and analyse of each segment.
The present invention is a further development of an automatic classification system of fish, where the system determines the fish species and size with the help of picture treatment, and is considered for use onboard ocean research vessels and commercial fishing boats.
A known system comprises a light box and a camera system that, for example, is placed over a conveyor belt. In addition, software has been developed which continuously takes pictures of the conveyor belt to detect when fish is coming. A picture of the fish is then taken, which is analysed to find the size and species of the fish. The system has, for example, a capacity of about 8000 fish per hour at a belt speed of 1.5 meters per second.
Classification of fish can, for example, be based on a method which was developed by Norval Strachan at Aberdeen University in 1993 (described, for example, in WO 9409920 A1). The method involves analysis of each picture to find the contours of the fish, divide the fish into a certain number of segments and analyse the colour in each segment. The result of the analysis is compared to corresponding analysis of known fish, and forms the basis for the classification. Tests show that it is possible to obtain a degree of recognition of 98-100%, and an accuracy of the length measurement of about 1%.
As the system is based on colour analysis, it is dependent on controlled light conditions. This is achieved with the help of a light box that makes it possible to control the amount of light and the spectrum. It will not be possible to use this method for classification of fish without having full control of the light, and it is therefore not used for classification of fish in water.
From NO 167182, a method and device are known for classification of fish, where analysis of grey tones in a picture of a fish is made to decide the
species. Limit values such as area, length, breadth, nose angle, etc., are determined, and thus the fish species. A camera takes pictures and sends them to a unit that determines the limiting values, based on, for one thing, a summation of distances between mid-points to get a value that represents the length of the fish, and to determine the mid-point of the summed distance. Furthermore, the sum of distances between pixel positions in the picture is determined, and also a number of other summations. This solution is also dependent on controlled light conditions and cannot be used under water.
The inventor has worked with this problem for a while, and the result is a new method for classification of fish which is not dependent on constant light conditions. The analysis is initially made of the same picture as before, but the analytical method is different. Instead of measuring colours one looks at transitions or edges in the picture that are described with the help of the strength and direction of the transition. The transitions will thereby describe characteristic features such as fins, eyes, stripes along the fish, pattern and the like. The result of the analysis will, in the same way as before, be compared to corresponding analysis of known fish to classify the fish. Tests have been carried out with the new method and promising results have been obtained.
According to the invention the camera system is placed under water, for example, in or adjoining a trawl which is pulled behind a vessel. Known solutions which have been pointed out are fish sorting machines that are used on land or onboard a vessel where both the camera system and analytical equipment are located at the same place out of the water. As the known methods are, in the main, based on colour analysis, they are dependent on controlled light conditions. This is achieved with the help of a light box which makes it possible to control the amount of light and the spectrum. It will not be possible to use the known methods for classification of fish without having full control of the light and therefore they cannot be used for the classification of fish in water.
With the present invention such a light box is not used to control the light. To avoid the problem of controlling the light conditions, the analysis is carried out, as mentioned, on basically the same picture as previously, but the analytical method is different. Instead of measuring colour one looks at the transitions in the picture which are described with the help of the strength and direction of the
transitions. The transitions will thereby describe characteristic features such as fins, eyes, stripes along the fish, pattern and the like.
The analysis starts with a search for transitions. If there is a transition this is described by, for example, a line that follows the transition and a numerical value that gives the strength of the transition. The line can, in principal, be regarded as a vector (without in reality being a vector). Because much computer power is required to work with vectors, the software can describe the form of the line in a simplified way which makes it easy to store in the database. The form can be, for example, a straight line (with direction), an arch or a circle. If, for example, the eye of the fish is in the segment, a dominating circle will, in all probability, be obtained. For example, if a mackerel is to be analysed, it will be possible to obtain a number of horizontal lines in the segment whereupon the characteristic features can be compared with data from fish in a knowledge database for classification of fish, with the help of discriminatory analysis. The result can thereafter be displayed in a user interface.
From document WO9409920 A1 (Strachan) it can clearly be seen that colour and light intensity form the basis for the classification of the fish. The method in WO9409920 A1 involves analysing each picture to find the contours of the fish, divide the fish into a certain number of segments and analyse the colour in each segment. The result of the analysis is compared with corresponding analysis of known fish, and forms the basis for the classification. It is, as mentioned above, contrary to the present solution where it is not the colours that are analysed.
It is consequently an object of the present invention to provide a solution which it will be possible to use for classification of fish under water and which is not dependent on constant light conditions. Application areas which may be relevant are sorting the fish according to species and size in an entrapment situation (for example, in a trawl). It can also be possible that such a system can be used by ocean researchers so that they can classify fish which are caught in a trawl without the fish being taken on board but swimming freely out the end of the trawl.
The present method and the system shall be able to be used for recognising all commercial fish species and also fish which is gutted and headless.
The above mentioned objects are reached by a method as described in claim 1 , in that the camera system is placed under water to analyse transitions in each segment expressed by the strength of the transitions, such as degree of darkness and direction in an analysis unit, to describe characteristic features of the fish, such as fins, eyes, stripes along the fish, pattern, etc., based on the analysis of the transitions, and to compare the characteristic features of the fish with data from fish in a knowledge database for classification of fish with the help of discriminatory analysis.
Alternative embodiments of the methods are characterised by the dependent claims 2-5.
The camera system can be placed in a trawl or the like, which is pulled by a vessel and the analysis unit can be arranged next to the camera system or onboard the vessel.
After the contour of the fish has been detected, the longitudinal axis of the fish is estimated, whereupon the object is normalised so that the longitudinal axis is approximately horizontal and the head of the fish is preferably turned to the left. A picture of the fish is thereafter generated for display of transitions/edges of the object, and to describe said transitions with the help of strength, such as degree of darkness and direction.
Each fish is preferably divided into a number of equal segments independent of the size and species of fish, and to analyse transitions, such as strength and direction, within each segment for the formation of said characteristic. Thereafter, the characteristics of the analysed fish are compared with characteristics of known fish with the help of discriminatory analysis.
Said objects are also reached by a system described in claim 6 in that the camera system is arranged under water, and that the analysis unit is arranged to analyse transitions in each segment expressed by the strength of the transitions, such as degree of darkness and direction, thereafter to describe the characteristic features of the fish such as fins, eyes, stripes along the fish, pattern, etc., based on the analysis of the transitions, and to compare the characteristic features with data of fish in a knowledge database for
classification of fish with the help of discriminatory analysis, and to display the result in a user interface.
Alternative embodiments of the system are characterised by the dependent claims 7-8. The camera system can be arranged in a trawl or the like which is pulled by a vessel and that the analysis unit can be arranged next to the camera system or onboard the vessel. Said trawl or the like can comprise means to guide the fish passed the camera.
Furthermore, said trawl or the like can comprise a sorting system to sort fish which come into the trawl and the analysis unit can be connected with the sorting system, whereby unwanted fish or other marine organisms can be taken out of the trawl.
The invention shall now be described in more detail with the help of the enclosed figures, in which:
Figure 1 shows an example of a picture of a cod generated according to the present method.
Figure 2 shows an example of division of a cod into segments according to the present method.
Figure 3 shows, corresponding to figure 1 , an example of a smaller catfish generated according the present method.
Figure 4 shows, corresponding to figures 1 and 3, an example of a capelin generated according to the present method.
Figure 5 shows an example of one of the application areas of the present invention under water.
The system according to the present invention comprises a camera system which is placed next to an area with fish that are moved passed. The camera can be monitoring the fish continuously and a high resolution measuring device can also be arranged there to maintain that the camera is synchronised with the movement/speed of the fish that are moved passed. An analysis unit is connected to the camera where the analysis unit is arranged to receive pictures taken by the camera. Furthermore, the analysis unit is arranged to analyse each picture to find the contour of the fish and to divide the contour of the fish into a number of segments to analyse each segment. The analysis unit analyses transitions or edges in each segment expressed by the strength and direction of the transitions or the edges, thereafter to describe characteristic
features of the fish such as fins, eyes, stripes along the fish, pattern, etc., based on the analysis of the transitions or the edges. These characteristic features are thereafter compared with data of fish in a knowledge database for classification of fish.
In one embodiment (not shown) the system can be arranged, more or less, gathered together, i.e. that the analysis unit is arranged near to the camera or that the analysis unit and the camera are integrated with each other. The knowledge database can be included in the analysis unit or a connection to an external database can be provided. Figure 5 shows an application of a classification system under water. A research vessel 100 pulls a trawl 110 where the rear part comprises, at least, parts of a classification system based on the above mentioned principle. When the fish has passed through the classification system it is released into the sea again. This makes it possible for ocean scientists to classify fish in the ocean without taking them onboard. The result can be sent to the vessel 100, through a cable, wirelessly with the help of acoustics or be stored in the classification system until this is taken on board. Thus the camera can be placed in the trawl, and also means to guide the fish passed the camera can be placed here, while the analysis unit can be, for example, placed onboard the vessel. Alternatively the camera can be placed in the trawl but in such a way that the fish which are to be caught in the trawl are led passed the camera. The system and the method can thus also be used in commercial fishing.
In connection with commercial fishing, the present system will thereby be able to give information to the captain of the ship in approximately true time about which fish that are caught and also the size of the fish. In this connection, a sorting system can also be arranged in the trawl which, based on the information from the analysis unit, sorts out the fish which shall be caught and which fish or other marine organisms shall be released from the trawl. The sorting system can be any system that is arranged for remote sorting. Thereby, the possibility to know which fish the trawl contains is provided, even before the trawl is hauled up, something which also can, amongst other things, lead to an increased catch for each hauling up of the trawl. Even with today's equipment setting out and hauling up of a trawl is a considerable job, and the present invention can thereby, to a considerable extent, contribute to an increased fishing of a correct and permitted species and thereby give an increased revenue.
Figure 1 shows a software-generated picture of a cod which is analysed to bring out the transitions/edges of the object. The transitions can be described with strength, (shown here by degree of darkness) and direction.
First the contour of the fish is detected during transport of the fish passing the camera system that takes pictures of the fish that passes, thereafter each picture is analysed to find the contours of the fish and the longitudinal axis of the fish is estimated. Thereafter, the object is normalised so that the longitudinal axis is approximately horizontal and the head of the fish turns, for example, to the left. The fish is then divided into a number of segments (as shown in figure 2). The number of segments is initially the same for all fish sizes. A software programme in the analysis unit analyses transitions within each segment, such as strength and direction, and this will together with the form of the fish make the basis for a description/characteristic of each individual fish, where the description/characteristic can, for example, comprise about one hundred parameters. The description/characteristic of the fish is compared with the description/characteristic of known fish with the help of discriminatory analysis. The fish is classified as the species that gets the highest score. If the score value is too low the fish is classified as "unknown".
The figures 3 and 4 show examples corresponding to figure 1 , but with small catfish and capelin, respectively. Figure 4 with the capelin shows the generated picture before normalisation and rotation.
As can be seen in figure 2, each fish will have unique transitions in each segment and by analysing these transitions or edges in the picture, which are described with the help of the strength and direction of the transition, characteristic features such as fins, eyes, stripes around the fish, pattern and the like, will thereby be given. The result of the analysis can be compared, in the same way as before, with corresponding analysis of known fish to classify the fish.
The method can also be used for classification of fish in tanks or the like, on land or onboard vessels. It will make it possible to simplify the technical equipment which is required or is used in addition to colour analysis to achieve a better result with difficult species.