DEVICE AND METHOD FOR MEASURING THE SPEED OF AN OBJ ECT MOVING IN A RESTRICTED ENVIRONMENT

The invention relates to a device and a method for measuring the speed of an object moving in a restricted environment.

Speed measurement is data used in many fields and in particular on roads or during sports competitions such as, for example, swimming.

Very often the speed measurement is accompanied by the installation of sensors on the moving device whose speed is desired to be measured.

However, in the applications mentioned above, it is difficult to equip sportsmen or vehicles with a sensor. For sportsmen, the presence of a sensor can be awkward depending on the movements carried out or dangerous in the event of falls, during football or rugby matches for example. Equipping vehicles does not present the same problems, but it is difficult to envisage in the case where vehicle speed measurements are carried out by surprise during speed checks, since the vehicles are obviously not equipped with specific sensors for these purposes.

The speed measurement of a sportsman can be used for information purposes during the retransmission of the sports event for spectators or for television viewers and also for teaching purposes in order to improve performance.

The invention proposes a measurement of the speed of an object moving in a restricted environment not necessitating any measurement means on the moving object.

For this purpose, the invention relates to a device for measuring the speed of at least one object moving in a restricted environment comprising: - video capturing means whose field of capture covers the environment,

- calibration means making the granularity of the captured images correspond with the real distance in the environment,

- means of spotting a distinctive sign of the moving object.

According to the invention, the device comprises means of identifying the position in the environment of the distinctive sign on the captured video at a first time and at a second time, the calibration means being able to convert the movement of the distinctive sign on the video between the first time and the second time into a distance travelled in the environment, the device comprises means of calculating the speed of the object over the distance travelled between the first time and the second time.

According to a preferred embodiment, the device comprises means of retrieving the dimensions of the restricted environment and the distance between the environment and the video capture means.

According to a preferred embodiment, the calibration means make the granularity of the captured images correspond with the real distance according to intrinsic parameters of the capture means, and according to the position of the capture means with respect to the absolute reference system of the environment.

According to a preferred embodiment, the capture means comprise a fixed camera placed above the environment.

According to a preferred embodiment, the device comprises means of determining the direction in which the moving object is moving, the means of identifying the position of the distinctive sign seeking the position on the video according to the direction.

Advantageously, the distinctive object is the barycentre of a zone of the moving object.

Preferably, the moving object is a swimmer and the environment is a swimming pool.

Advantageously, the device comprises means of identifying the different lanes of the swimming pool.

According to another embodiment, the moving object is a motor car and the environment is a road.

According to another aspect, the invention relates to a method of measuring the speed of at least one object moving in a restricted environment comprising: - a video capture step in which the capture field covers the said environment,

- a calibration step making the granularity of the captured images correspond with the real distance in the said environment,

- a step of spotting a distinctive sign of the said moving object.

According to the invention, the method comprises a step of identification of the position in the environment of the distinctive sign on the captured video at a first time and at a second time, the said calibration step converting the said movement of the said distinctive sign on the said video between the first time and the second time into a distance travelled in the said environment, the said method comprising a step of calculating the speed of the said object over the said distance travelled between the first time and the second time.

The invention will be better understood and illustrated by means of examples of embodiment and advantageous application that are in no way limiting and given with reference to the appended figures in which:

- Figure 1 shows an example of a device according to a preferred embodiment of the invention,

- Figure 2 shows an example environment according to the preferred embodiment of the invention,

- Figure 3 shows a preferred embodiment of a method according to the invention,

- Figure 4 shows another aspect of the invention.

The modules shown are functional units, which may or may not correspond to physically distinguishable units. For example, these modules, or some of them, can be grouped in a single component or can consist of functions of the same software. On the contrary, some modules can possibly consist of separate physical entities.

Figure 1 shows a first embodiment in which a swimmer is moving in a swimming pool 3 and it is desired to know his instantaneous speed.

A camera 1 is above the pool. The camera is centred over the swimming pool such that its field of action covers the whole of the pool. Its optical axis is therefore centred on the swimming pool. This camera is connected to a processing device 2 by the intermediary of a communications network of the Ethernet type if it is a camera provided with an IP interface. This communication can be different in other embodiments.

The camera centred over the pool is able to capture the video of different swimmers moving in the pool. The camera is therefore placed at a sufficient height to be able to cover the whole of the swimming pool.

The processing device 2 knows the characteristics of the swimming pool and in particular its width and its length.

The processing device 2 calibrates the camera 1 in order to establish a correspondence between the distance measurements in the pool 3 and the distance measurements in the captured images.

The calibration step consists in estimating the extrinsic and intrinsic parameters of the camera.

The extrinsic parameters express the 3D position of the camera with respect to the absolute reference system of the scene. They break down into a rotation matrix and a translation matrix:

- R is a rotation matrix between the absolute reference system of the scene and the local reference system of the camera,

- 1 represents the translation vector between the origin of the absolute reference system of the scene and the local reference system of the camera.

The intrinsic parameters contain the information on the internal data of the camera. In the hypothesis of a pure perspective projection without non-linear distortions, there are five intrinsic parameters: the ratio between the sizes of the pixels in x and in y and the focal length, the angle between the axes of the image reference system, and the coordinates of the projection of the optical centre in the image.

In order to estimate the intrinsic and extrinsic parameters of the camera, metrical data coming from the scene can be used (size of the swimming pool for example). This metrical data associated with the position of the 3D points is expressed in the absolute reference system associated with to the scene. This reference system can any system whatsoever, it will advantageously be possible to choose it such that the centre of the reference system is a corner of the swimming pool, the X and Y axes corresponding to the sides of the swimming pool (length and width) and the Z axis being taken such that the reference system thus formed is directly orthonormal (Z axis perpendicular to the water surface). An initial or prior manual step of making points in the image correspond with known 3D points in this reference system then allows the calculation of the intrinsic and extrinsic parameters of the camera.

Knowing the intrinsic and extrinsic parameters, it is possible to know the position in the image of any 3D point whose coordinates are known in the absolute reference system. Conversely, it is not possible to know the 3D position of a point in the image but only the equation of the 3D straight line upon which this point is situated (for a point of the flat image p, all of the 3D points belonging the straight line passing through the image point p and the optical centre are projected at p). This equation is as follows:

where the parameters a, b, c, d, e, f, g and h are known.

A hypothesis must therefore be added here regarding the a priori knowledge of the 3D surface over which the object is moving. In most cases, the planar hypothesis is sufficient (surface of water, football pitch, etc.). It is however possible, although complex to use, to consider non-planar surfaces, the main difficulty being in the estimation of the 3D parametric equations of the surface.

Having positioned the absolute reference system on a corner of the swimming pool with the Z axis perpendicular to the plane, the equation of the water surface is written simply as Z=O. If a point p of the image is considered, knowing the extrinsic and intrinsic parameters, it is possible to calculate the 3D coordinates of a straight line upon which is located the corresponding 3D point in the scene. On now adding the hypothesis that this point is part of the water surface (a valid hypothesis in the case of a swimmer moving in the water surface), the 3D point is determined by the intersection of the water surface (of equation Z=O) with the 3D line of projection.

Taking the equation of the straight line of projection and introducing the equation Z=O, a system of two equations with two unknowns X and Y is obtained again which can be solved in order to evaluate, in the absolute reference system of the scene, the coordinates X and Y of the 3D point corresponding to the point p in the image. In the rest of the description the vector of the X and Y coordinates associated with the point p in the absolute reference system of the scene will be called pos(t).

The calibration step is also possibly followed by a step of detection of the lanes in the swimming pool, in such a way as to then restrict a swimmer's search zone during the estimation of speed. The more the search zone is limited, the faster and more real-time will be the calculation of speed, which is important in the case where the calculated speed represents an instantaneous speed to be displayed in real time, for example 25 times per second.

The detection of the lanes is carried out by gradient based image processing, or by the detection of straight contours, of straight lines, or by a technique based on the contrast of the lane markings, black in colour, using

mathematical morphology tools, by straight filtering or by using the top hat, a known mathematical morphology tool which makes it possible to extract fine and contrasted details of an image.

Once the segmentation of the pool into lanes has been carried out, the processing device 2 restricts the search zone of a swimmer to his lane.

The processing device 2 then receives images from the camera 1 in real time. It then analyses the video data at all times in order to make the distance d2 on the video correspond with the distance d1 actually travelled by the swimmer. These measurements can be made at very short intervals in order to measure the instantaneous speed.

The processing device 2 previously notes a distinctive sign of the swimmer whose speed it analyses. This distinctive sign can be detected prior to the start of the swimming while the swimmer is waiting for the start of the race on the starting point. Each swimmer is therefore spotted by this distinctive sign and the distance travelled by the swimmer is the distance travelled by this distinctive sign. It is possible for example to choose the swimmer's swimming cap. This makes it possible to distinguish between two different swimmers.

The distinctive sign is preferably an element of the swimmer which remains close to the surface of the water and parallel with it.

This distinctive sign is initialized manually by a man-machine interface on the processing device. The operator therefore simply clicks on the distinctive sign and the processing device 2 analyses this distinctive sign in such away as to extract from it the characteristics necessary for its subsequent recognition. The necessary characteristics include colour, texture and shape.

In other embodiments, the distinctive sign can be recognized automatically by the processing device which then chooses a distinctive sign for each swimmer in each lane.

The movement of the distinctive sign from one image to another can also be envisaged using object following methods in a series of images known to those skilled in the art.

When the processing device 2 has obtained the distance d1 on the basis of the distance d2 and calibration data, it calculates the speed of the swimmer at each desired time t by means of the following formula:

Instantaneous speed = (Pos (t+dt) - Pos (t)) / dt

where (Pos (t+dt) - Pos (t)) represents the distance d2 travelled between the times t and t+dt, the swimmer moving in a plane and not in a 3D surface.

Figure 2 shows a search zone in which the processing device 2 will search for the swimmer. The swimmer, whose speed is to be measured, is in the lane L4. The pool comprises seven lanes L1 to L7.

From the first measurements made, the processing device 2 analyses the direction of movement of the swimmer. This advantageously allows it to reduce the search zone of the distinctive sign in the image and therefore to save time in the calculation of the instantaneous speed.

When the swimmer arrives at the end of the lane, the search zone situated upstream of the swimmer's movement no longer contains the distinctive sign and the search zone such as situated in Figure 2 is not usable. The search zone is then widened in both directions, in front of and behind the swimmer. This search zone can be restricted to the lane in which the swimmer is moving.

Similarly, when the swimmer arrives at the end of the lane, either the race is finished, or the swimmer will disappear under the water for the time it takes to make his turnaround. The processing device cannot therefore spot the distinctive sign at that time. It will therefore successively analyse all of the images without being able to generate the instantaneous speed until the distinctive sign can again be spotted. The processing device 2 displays the instantaneous speed in real time. Thus all of the spectators in the swimming pool can see the value of this speed.

The processing device 2 also transmits the instantaneous speed with the video as associated metadata. In this way, during the transmission of the

video, the instantaneous speed thus measured can be displayed simultaneously with the images. This transmission of the speed is carried out by the intermediary of metadata. The video data is transmitted directly with the associated speed as metadata to broadcast means, for example television means. The video data and the metadata are also recorded on a hard disk or on any other permanent storage medium for subsequent use, for example by the swimmers and the trainers during the display of the race in order to assess and to understand their performance.

Figure 3 shows a method according to the invention, used by the processing device 2 and by the camera 1.

In a first step E1 , a representative video of the restricted environment in which the swimmer is moving is captured. This video makes it possible to calibrate the camera, during a step E2, and thus to make a distance d1 in the pool correspond to a distance d2 on the video as mentioned previously.

In a step E3, the operator indicates on an image of the video the distinctive sign which will make it possible to distinguish the moving object on the video. In a step E4, this moving object is characterized in such a way as to spot it in the following images. This moving object is characterized by the features of the moving object as mentioned previously.

Then, in a step E5, the capture of the moving object can be started when it is desired to start the speed measurements of the moving object. In a step E6, the speed calculations are started, for example when the race has started. In a step E7, starting from the first speed calculations, the calculated speed is displayed and/or transmitted to a remote device to be recorded or broadcast.

Figure 4 shows another aspect of the invention in which the camera is mobile. The camera is slaved to the speed of the swimmer. The camera is provided with speed measuring equipment. In this case, the instantaneous speed of the camera is simply calculated by conventional speed calculation devices. The speed measurement is thus transmitted to the processing device 2.

In other embodiments, the distinctive sign is the barycentre of a recognizable zone of the image. This advantageously makes it possible to obtain real value movements providing more accurate instantaneous speed measurements.

In other embodiments, it is possible to measure the speed of several swimmers simultaneously. In order to do this, each swimmer is identified by the processing device 2 in his lane. The processing device spots a distinctive sign for each swimmer, for example the swimming cap. As the lanes are usually separated by floats whose colour can be noted, the search device can easily isolate each swimmer.

In other embodiments, it is possible to have several cameras above the pool in order to cover the whole of the pool if a single camera is not sufficient. The cameras can therefore be placed at different angles to cover the pool. Making the distances d1 and d2 correspond is then carried out by combining the parameters of the different cameras and their positions with respect to the pool.

Another preferred embodiment relates to the measurement of the speed of a vehicle on a road. This can also be the case during a Formula 1 grand prix for example or during speed checks on roads carried out by the authorities.

In this case, a video camera is placed above the road and observes the movement of a vehicle in a zone whose extent is restricted, for example a 300m length of road. The camera can be mounted high, for example suspended from a bridge.

As in the preceding embodiment described in Figures 1 to 4, the camera is connected to a processing device which calibrates the camera and makes it possible to establish correspondence between the distance measured on the video and the real distance travelled by the vehicle.

The functioning described for Figures 1 to 4 is also valid for this embodiment. The distinctive sign is for example the barycentre of the vehicle.