IMAGE ANALYSIS

This invention relates to a method of analysing an image of a road along which a vehicle is travelling and to related apparatus.

Lane Departure Warning (LDW) systems, other automotive guidance and collision mitigation systems frequently use optical video cameras to capture images of a road ahead of a vehicle. Analysis of the captured images allows such systems to determine where road or lane markings are and possibly to warn a driver of the vehicle when they are about to inadvertently leave a lane. Such a system is disclosed in European Patent Application Publication number 1 530 186 A2.

However, it is often difficult for such systems to differentiate between road markings and other objects in the field of view of the camera. Lane markings generally appear to stretch forwards and upwards towards infinity, thereby appearing as lines rising up to the horizon.

Unfortunately, so do such objects as trees, lampposts and even the vertical sides of other vehicles.

According to a first aspect of the invention, there is provided a method of analysing an image of a road along which a vehicle is travelling, comprising:

capturing an image of the road with an image capture apparatus on the vehicle and having a field of view, the image capture apparatus having a first area of the field of view over which the image capture apparatus is responsive preferentially to light of a first polarisation and a second area of the field of view over which the image capture apparatus has a different response to polarised light;

determining the change in intensity of a feature in the captured image between the first and second areas; and

determining from the change in intensity of the feature a parameter of the feature.

By image of a road, we mean an image of a portion of road or of a scene including a road, which may include one or more vehicles or obstacles.

By feature we may mean objects such as vehicles, street furniture, lane markings etc.

Objects of different orientations relative to the vehicle will primarily reflect light of differing polarisations. Typically, horizontal objects such as road markings will reflect mostly horizontally polarised light, whilst the vertical objects such as the vertical sides of other vehicles will reflect mostly vertically polarised light.

Therefore, by measuring a difference in intensity of the light reflected from the feature from areas of the field of view having different responses to polarised light, information relating to the orientation of the feature can be determined. Accordingly, the parameter is typically the orientation of the feature.

This information is particularly useful when determining whether a feature in the captured image is a lane marking or not, as these are generally horizontal. The step of determining the orientation of a feature in the captured image may include the step of determining in which direction the light reflected from the feature is polarised and determining from that direction the orientation of the feature.

The response of the second area may not discriminate between differently polarised light. However, it is preferable that the second area is preferentially responsive to light of a second polarisation different from the first. Typically, the first and second polarisations will be linear polarisations at right angles to each other. This is simpler, as it means the relative intensities of the first and second areas can be compared rather than relying on the absolute difference between an area having a preferential response to polarised light and one that does not.

Preferably, the first area is preferentially responsive to horizontally polarised light and the second is preferentially responsive to vertically polarised light. There is preferably a residual area of the field of view that is not preferentially responsive to light of any particular polarisation, or is preferentially responsive to a light of a third, different polarisation.

The first and second areas may comprise a pair of parallel strips of the field of view, each strip forming one of the first and second areas. The strips may be spaced apart in the field of view, but are preferably close to one another: Where the strips have a width, the distance between the strips may be less than 5, or 2, strip widths and is preferably one or about one strip width. Having the areas spaced apart but closely so means that the differences in intensity over small features can be discerned.

Where the first and second polarisations are linear, one of the strips is preferably aligned with, or at right angles to, the direction of polarisation of the first or second polarisations. The strips may be, for example, horizontal or vertical.

The step of capturing the image may comprise viewing the image through a filter, wherein sections of the filter have differing responses to polarised light corresponding to the first and second areas of the field of

view. This is simple approach, which allows a simple monochrome camera to be used as the image capture apparatus. The sections of the filter having different responses may comprise polarising filters.

The first and second areas may each comprise two pairs of parallel strips as described above, where the pairs are spaced apart. The pairs of strips may be spaced apart by significantly more than the distance between the strips of each pair. Preferably, each pair is situated in a respective half of the field of view; this is especially useful where the strips are vertical because, when driving in a lane, lane markings tend to occur on either side of the vehicle.

The field of view of the image capture apparatus may be provided with a third area being preferentially responsive to a third polarisation being different to the first or second polarisations. This third polarisation may be a circular polarisation, or may be a linear polarisation, typically at an angle of 45 degrees to the first or second polarisations.

The first, second and possible third areas may have different colour responses. This may aid a monochrome image capture apparatus determine the colour of lane markings, as they will have different responses as they pass through the field of view.

The captured image may be that ahead of the vehicle, typically through a front windscreen of the vehicle.

The method may comprise the step of using the determined orientation of the feature in the captured image to determine whether the feature is a road marking. The method may then comprise, where it is determined that a feature is a road marking, providing the position of the lane marking to an automotive guidance system such as a lane departure

warning system or automatic cruise control system. This allows for reliable identification of road markings from features in the captured image that are not road markings.

According to a second aspect of the invention, there is provided a sensing apparatus for a vehicle, comprising a image capture apparatus having a field of view, the image capture apparatus having a first area of the field of view over which it is responsive preferentially to light of a first polarisation and a second area of the field of view over which over which has a different response to polarised light; and a processor arranged to, in use, take an image of a road along which the vehicle is travelling captured by the image capture apparatus and to determine the position of objects in the image, and to determine the change in intensity for those objects between the first and second areas and from that change their orientation.

The apparatus may be arranged to, in use, carry out the method of the first aspect of the invention.

The processor may be arranged to determine, from the change in intensity, the orientation of the objects relative to the vehicle in the captured image. The processor may also be arranged to determine in which direction the light reflected from the object is polarised and determine from that direction the orientation of the object.

The response of the second area may not discriminate between differently polarised light. However, it is preferable that the second area is preferentially responsive to light of a second polarisation different from the first. Typically, the first and second polarisations will be linear polarisations at right angles to each other. This is simpler, as it means the relative intensities of the first and second areas can be compared

rather than relying on the absolute difference between an area having a preferential response to polarised light and one that does not.

Preferably, the first area is preferentially responsive to horizontally polarised light and the second is preferentially responsive to vertically polarised light. There is preferably a residual area of the field of view that is not preferentially responsive to light of any particular polarisation, or is preferentially responsive to a light of a third, different polarisation.

The first and second areas may comprise a pair of parallel strips of the field of view, each strip forming one of the first and second areas. The strips may be spaced apart in the field of view, but are preferably close to one another. Where the strips have a width, the distance between the strips may be less than 5, or 2, strip widths and is preferably one or about one strip width. Having the areas spaced apart but closely so means that the changes in intensity over small features can be discerned.

Where the first and second polarisations are linear, one of the strips is preferably aligned with, or at right angles to, the direction of polarisation of the first or second polarisations. The strips may be, for example, horizontal or vertical.

The image capture apparatus may comprise a filter, wherein sections of the filter have differing responses to polarised light corresponding to the first and second areas of the field of view. This is simple approach, which allows a simple monochrome camera to be used as the image capture apparatus. The sections of the filter having different responses may comprise polarising filters.

As the image capture apparatus will typically comprise a lens, the filter may be provided in front of the lens (that is, between the lens and the

image to be viewed) . Alternatively, the filter could be provided on or in the lens.

As an alternative to the filter, a part of the image capture apparatus which is sensitive to light may be differently sensitive to polarised light in the manner required. However, this is more complex than a simple filter.

The first and second areas may each comprise two pairs of parallel strips each as described above, where the pairs are spaced apart. The pairs of strips may be spaced apart by significantly more than the distance between the strips of each pair. Preferably, each pair is situated in a respective half of the field of view; this is especially useful where the strips are vertical because, when driving in a lane, lane markings tend to occur on either side of the vehicle.

The field of view of the image capture apparatus may be provided with a third area being preferentially responsive to a third polarisation being different to the first or second polarisations. This third polarisation may be a circular polarisation, or may be a linear polarisation, typically at an angle of 45 degrees to the first or second polarisations.

The first, second and possible third areas may have different colour responses. This may aid a monochrome image capture apparatus determine the colour of lane markings, as they will have different responses as they pass through the field of view.

Preferably, the apparatus is arranged to be fitted to a vehicle, typically such that the image capture apparatus is positioned, in use, so as to capture a view through a front windscreen of the vehicle.

The processor may be arranged to, in use, use the determined orientation of the object in the captured image to determine whether the object is a road marking. The sensing apparatus may comprise part of an automotive guidance system such as a lane departure warning system or automatic cruise control system.

According to a third aspect of the invention, there is provided a method of analysing images of a road along which a vehicle is moving, comprising:

capturing a first and a second image of a road with an image capture apparatus on the vehicle and having a field of view, the image capture apparatus having a first area of the field of view over which the image capture apparatus has a first response and a second area of the field of view over which the image capture apparatus has a different response, the first and second images being taken at different times and the vehicle having moved along the road between the times when first and second images are captured; and

comparing the first and second images to determine the difference between first and second areas over different areas of a feature in the captured images.

This method takes advantage of the fact that, as a vehicle moves, objects viewed from the vehicle will appear to move past the vehicle. Having differing areas in the field of view therefore allows the method to scan objects as they pass through the differing areas in the field of view. An image of the feature can possibly be built up, providing information over a range of positions over the feature.

One or more further images may be taken with the image capture apparatus as the vehicle moves along the road and compared to the first or second images or other of the further images. This can provide further information on the feature at differing points over the feature.

The method is particularly useful where the first area only occupies a small fraction of the field of view. Preferably, the first area only occupies less than half, a quarter or a tenth of the field of view. This allows the remainder of the field of view - typically the second area - to be used with a "standard" response.

The first area may be arranged so that objects passing to one side of the vehicle as the vehicle passes the objects pass through the first area. This may be achieved by the first area extending from one side of the field of view to the other. Accordingly, the first area conveniently comprises a strip extending from the top of the field of view to the bottom, or between opposite horizontal sides. The first area may be a horizontal or vertical strip. Where the strip is a vertical strip, it may be positioned towards the left or right sides of the field of. view, in order to allow objects on that side of the vehicle to be scanned.

The first and second responses may be differing responses to different colours, polarisations or combinations thereof. Multiple first areas may be provided, having different responses. In a preferred embodiment, the method also comprises the steps of the method of the first aspect of the invention; the two methods may advantageously be combined in order to scan features in the display to determine their orientation scanned over their extent in the field of view.

According to a fourth aspect of the invention, there is provided a sensing apparatus for a vehicle comprising a image capture apparatus having a

field of view, the image capture apparatus having a first area of the field of view over which it has a first response and a second area of the field of view over which it has a different response; and a processor arranged to, in use, capture first and second images from the image capture apparatus at different times as the vehicle moves along the road and to compare the first and second images to determine the difference between first and second areas over different areas of a feature in the captured images.

Preferably, the apparatus is arranged so as to, in use, carry out the method of the third aspect of the invention.

As with the previous aspect, having differing areas in the field of view therefore allows the apparatus to scan objects as they pass through the differing areas in the field of view.

Advantageously, the first area only occupies a small fraction of the field of view. Preferably, the first area only occupies less than half, a quarter or a tenth of the field of view. This allows the remainder of the field of view - typically the second area - to be used with a "standard" response.

The first area may be arranged so that objects passing to one side of the vehicle pass through the first area as the vehicle, in use and in motion, passes the objects. This may be achieved by the first area extending from one side of the field of view to the other. Accordingly, the first area conveniently comprises a strip extending from the top of the field of view to the bottom, or between opposite horizontal sides. The first area may be a horizontal or vertical strip. Where the strip is a vertical strip, it may be positioned towards the left or right sides of the field of view, in order to allow objects on that side of the vehicle to be scanned.

The first and second responses may be differing responses to different colours, polarisations or combinations thereof. Multiple first areas may be provided, having different responses. In a preferred embodiment, apparatus also falls within the second aspect of the invention; the two apparatus may advantageously be combined in order to scan features in the display to determine their orientation scanned over their extent in the field of view.

The image capture apparatus may comprise a filter, wherein sections of the filter have differing responses corresponding to the first and second areas of the field of view. This is simple approach, which allows a simple monochrome camera to be used as the image capture apparatus.

The sections of the filter having different responses may comprise polarising or coloured filters.

According to a fifth aspect of the invention, there is provided a vehicle fitted with the sensing apparatus of the second or fourth aspects of the invention.

There now follows, by way of example only, description of embodiments of the invention, described with reference to the accompanying drawings, in which:

Figure 1 shows a car fitted with a sensing apparatus according to an embodiment of the invention;

Figure 2 shows an image captured with the sensing apparatus of Figure 1, together with the intensities measured along lines A-A and B-B; and

Figure 3 shows an image captured with a sensing apparatus according to a further embodiment of the invention, together with the intensities measured along lines C-C, D-D, E-E and F-F.

Figure 1 of the accompanying drawings shows a car 100 fitted with a sensing apparatus according to an embodiment of the present invention. The sensing apparatus comprises a monochrome video camera 102 - an image capture apparatus - fitted behind the car windscreen 101 so as to be able to view the road ahead. A filter 104 is fitted in front of the camera so as to change its response to polarised light as set out below. A processor 103 receives the output signal from the video camera 102 in order to process captured images as will also be described below.

A sample image captured by the video camera 102 can be seen in Figure 2 of the accompanying drawings. This shows the road 105 ahead of the car 101, road markings 106 on either side of the road 105 and a large truck 107 travelling along the road 105 in front of the car 101.

The filter 104 generally provides little barrier to light passing through it to the camera 102 except over four vertical strips 1, 2, 3, 4 shown in the field of view 10 of the camera 102. These are in two pairs 5, 6, each pair being situated in a respective half of the field of view 10. Each strip of each pair 5, 6 comprises a polarising filter, with the left-hand strip 1,

3 of each pair being horizontally polarised (and hence preferentially allowing vertically polarised light through) and the right-hand strip 2, 4 being vertically polarised (and hence preferentially allowing horizontally polarised light through) .

This has the effect that vertically and horizontally orientated objects can be differentiated. The light reflected from a vertical object, such as the vertical side of truck 107, will be mostly vertically polarised, whereas the

light reflected by horizontal objects such as road markings 106 will be mostly horizontally polarised.

The effect can be seen in the intensity plots in Figure 2 of the accompanying drawings. The image of the truck 107 falls over the left- hand pair of strips 5. Along example trace A-A and trace B-B, the image of the truck 107, on the left-hand side of the trace, shows a greater response over left-hand strip 1 than on right-hand strip 2. This indicates that more vertically polarised that horizontally polarised light is being reflected by this object. The processor 103 would therefore be able to determine that the object in the image representing the truck 107 was a vertically-orientated object.

On the other hand, the road markings 106 give a different response where they cross the right-hand pair of strips 6. Where the road marking 106 crosses the left hand strip 3 of the pair 6 (on trace A-A) the intensity is lower than where the same marking crosses the right hand strip 4 (on trace B-B) . Accordingly, this indicates that more horizontally polarised light is being reflected and so that the image is of a horizontally- orientated object. The processor can therefore identify the image as being one of a horizontal object and therefore as a possible lane marking, which of course it is.

A further embodiment of the invention can be seen in Figure 3 of the accompanying drawings. This embodiment uses the same arrangement of camera 102, filter 104 and processor 103 as shown in Figure 1 of the accompanying drawings, but the filter 104 is different to that of the first embodiment.

Instead of vertical polarised strips 1, 2, 3, 4, the filter is provided with a single pair 15 of horizontal polarised strips 11, 12. The top strip 11 is

horizontally polarised, in that it preferentially allows vertically polarised light to pass through it, whereas the bottom strip 12 is vertically polarised, in that it preferentially allows horizontally polarised light to pass through it.

The effect of the strips is, however, the same. The image shown in the field of view 10 of the camera 102 is similar to that of the first embodiment. The image shows the road 105 ahead of the car 101, road markings 106 to either side of the road 105 and a truck 107 travelling along the road ahead of the car 101. Also shown is a vertical lamppost 108.

As the strips 11, 12 are positioned across the middle of the field of view 10, they are well positioned to cover most features of interest. Traces C- C and D-D show, as before, the behaviour of the intensity of light reflected from vertical surfaces, namely the lamppost 108 and truck 107 respectively. In each case, more light is reflected through the top, horizontally polarised strip 11, indicating that the light is more vertically polarised than horizontally. The processor 103 can therefore determine that these are vertical objects.

Traces E-E and F-F demonstrate the behaviour of the intensity of the light reflected from road markings 106. A greater intensity is passed through bottom strip 12, thereby indicating more horizontally polarised light being reflected. The processor 103 can therefore determine the road markings 106 are a horizontal object, and therefore determine that they indeed may be road markings.

The provision of strips of differing response to light to the remainder of the field of view 10 in both embodiments positioned across the field of view is useful, as it allows the motion of the car 101 along the road 105

to be used to scan objects across the strips. As shown in Figures 2 and 3 of the accompanying drawings, as the car passes objects on the road, they will appear to be moving away from the vanishing point of the road 110 as depicted by arrows 109.

As viewed by the video camera 102, objects between the pairs 5, 6 of strips in Figure 2, or above the pair 15 of strips in Figure 3, will move through the strips as the car 101 passes them. This allows an image of such objects when viewed through each strip 1, 2, 3, 4; 5, 6 to be built up by the processor, using successive images captured from the video camera 102. This can allow a more reliable determination of object's orientation and could even allow more definite determination of the limits of adjacent objects that otherwise would look similar.