The invention relates to a driver assisting system for a motor vehicle, comprising an imaging means adapted to capture images from a region surrounding the motor vehicle, a processing means adapted to process image data provided by said imaging means and to detect objects in said vehicle surrounding re ^¬ gion, and a light source arrangement comprising a light source for generating an optical figure in the vehicle surrounding region through a light beam and being controllable by said electronic control means to change the direction and/or shape of said light beam. Furthermore, the invention relates to a corresponding driver assisting method.

US 6 281 806 Bl discloses a vision enhancement system where a processor determines whether a particular object sensed in or near the current path of the vehicle requires alerting of the driver. If so, the processor outputs a control signal to ef ^¬ fect illumination of the sensed object. The source of illumi ^¬ nation is controlled so as to continuously direct light onto the identified object, even as the vehicle and/or object move simultaneously with respect to one another. However, misalignments between the imaging means and the source of illumination may occur reducing the system performance or, in the worst case, failing to direct the driver attention to the correct position .

The object of the invention is to provide a driver assisting system and method having a high system performance. The invention solves this object with the features of the in ^¬ dependent claims. According to the invention an offset between the position of the optical figure and the reference position is determined. Since the light source has been controlled to illuminate the reference position before, this corresponds to determining an offset of the light source from a set position. In this manner, misalignments between the imaging means and the light source can be determined effectively. An essential point of the invention is to control the light source in order to illuminate the reference position before determining the offset. In this manner several sources of systematic error can be eliminated. The accuracy of the illumination of the de ^¬ tected objects is much higher, leading to a higher recognition probability of the object by driver and/or by the object de ^¬ tection means. Therefore, the driver assisting system performance is increased and the confidence of the driver in the sys ^¬ tem is large.

The invention advantageously enables to compensate for mis ^¬ alignments between the imaging means and the light source. Preferably, a compensation of the vehicle motion and/or the predicted object motion is taken into account prior to the de ^¬ termination of the offset between the imaging means and the light source. Vehicle yaw rate and speed measured by separate sensors are preferably used to improve system performance since there is a latency between the signal of the detected object in frame N and the illuminated object in frame N+m. The light source is controlled so as to continuously direct light onto the reference position, even as the vehicle and/or the reference position move simultaneously with respect to one an ^¬ other . The reference position is preferably related to the position of a detected object. In particular, the reference position corresponds to the position of a detected object, preferably a detected object to be illuminated by the light source arrange ^¬ ment in order to avoid a collision, like pedestrians, vehicles and/or traffic signs. Preferably a static object like a speed sign may be used as defining the reference position in order not to have to compensate for the object motion. Typically the reference position may be the center of an object bounding box of the detected object.

Preferably the processing means is adapted to find a modeled light distribution or brightness distribution. The light dis ^¬ tribution is preferably two-dimensional and may be a normal distribution. The modeled light distribution preferably has independent standard deviations in two perpendicular direc ^¬ tions. Preferably the modeled light distribution has a pre ^¬ determined form or shape. A typical light distribution may be oval or elliptical. The parameters of the light distribution may be varied within reasonable predetermined ranges.

The model distribution is preferably searched in the captured images only in a pre-defined search region around the refer ^¬ ence position, because a large deviation of the light figure from the reference position is unlikely. Expediently, the search region is significantly smaller than the area of the captured images. For example, the area of the search region may be less than 20%, preferably less than 10% of the area of the captured images. This saves a lot of processing time.

In a preferred embodiment the processing means is adapted to extract characteristic properties of the optical figure, or of the modeled light distribution representing the optical fig- ure . In particular the center of a detected light figure or light distribution is preferably extracted. The offset accord ^¬ ing to the invention can then be determined as the deviation of the light distribution center to the reference position.

It may be useful to extract other characteristic properties of a detected optical figure, or of a modeled light distribution representing the optical figure. This allows to perform a ver ^¬ ification that a determined optical figure originates from the vehicle light source, by proving whether said extracted char ^¬ acteristic properties fulfill predetermined conditions. Such verification may be advantageous because in the region sur ^¬ rounding a motor vehicle multiple other light sources from other vehicles or buildings may occur which could otherwise lead to false determinations of light figures.

Preferably the characteristic properties comprise at least one spatial property of said optical figure, preferably two spa ^¬ tial properties in two dimensions, for example the width, length and/or height of the optical figure. For example, at least one standard deviation of a modeled light distribution found in the acquired images may be extracted. Preferably the characteristic properties comprise two perpendicular spatial extensions of said optical figure. This allows to calculate an aspect ratio which may be constant independent from the dis ^¬ tance of the optical figure. Other characteristic properties comprise the shape of the optical figure, which for example may be oval, and/or the distance between the optical figure and the reference position. If this distance exceeds a prede ^¬ termined threshold, it is unlikely that the determined optical figure originates from the own vehicle light source. According to embodiments of the invention, the processing means may be adapted to control the light source in order to impose a temporal variation and/or a characteristic movement of said light beam. This leads to a kind of characteristic "fingerprint" of the optical figure which can easily and re ^¬ liably be detected by the image processing means. In this man ^¬ ner a detected optical figure can be easily and reliably veri ^¬ fied as generated by the vehicle light source. In particular in case of a visible light beam the temporal variation and/or a characteristic movement are so short or small that they are not perceivable by the driver and do not disturb him. For ex ^¬ ample, the light beam may be shut off for one or more very short periods of time well below 1 second, preferably below 0.5 s.

In the following the invention shall be illustrated on the ba ^¬ sis of preferred embodiments with reference to the accompany ^¬ ing drawings, wherein:

Fig. 1 shows a schematic diagram of a driver assisting system for a motor vehicle;

Fig. 2 shows an example of a scene in front of a motor vehi ^¬ cle; and

Fig. 3 shows a schematic diagram of a driver assisting system for a motor vehicle according to another embodiment .

A driver assisting system 10 is mounted in a motor vehicle and comprises an imaging means 11 for detecting images of a region surrounding the motor vehicle, for example a region in front of the motor vehicle. Preferably the imaging means 11 is an infrared device and more preferably comprises one or more op ^¬ tical and/or infrared imaging devices, in particular cameras 12a, 12b, where infrared covers near IR with wavelengths below 5 microns and/or far IR with wavelengths beyond 5 microns. Preferably the imaging means 11 comprises two cameras 12a, 12b forming a stereo imaging means 11; alternatively only one cam ^¬ era forming a mono imaging means can be used. Each camera 12a, 12b is provided to convert incident infrared radiation or vis ^¬ ible light transmitted through the camera 12a, 12b into an electrical signal containing image information of the detected scene. The driver assisting system 10 may in particular be a night vision system.

The cameras 12a, 12b may be coupled to an image pre-processor 13 adapted to control the capture of images by the cameras 12a, 12b, warp pairs of left/right images into alignment and create disparity images, which per se is known in the art. The image pre-processor 13 may be realized by a dedicated hardware circuit. Alternatively the pre-processor 13, or part of its functions, can be realized in the processing means 14. The processing means 14 expediently has access to a memory means 15.

The pre-processed image data is then provided to a processing means 14 where further image and data processing is carried out by corresponding software. In particular, the image and data processing in the processing means 14 comprises the fol ^¬ lowing functions: detection, identification and classification of possible objects in front of the motor vehicle, such as other motor vehicles, pedestrians, bicyclists or large ani ^¬ mals, traffic signs; tracking over time the position of iden ^¬ tified object candidates in the detected images; activation or control of vehicle safety means 17, 18, 19 ... depending on the result of the object detection and tracking processing. The vehicle safety means 17, 18, 19, ... may comprise a warning means 17 adapted to provide a collision warning to the driver by suitable optical, acoustical and/or haptical warning sig ^¬ nals; display means 18 for displaying information relating to an identified object; one or more restraint systems 19 such as occupant airbags or safety belt tensioners; pedestrian air- bags, hood lifters and the like; and/or dynamic vehicle con ^¬ trol systems such as brakes.

The electronic processing means 14 is preferably programmed or programmable and may comprise a microprocessor or micro ^¬ controller. The image pre-processor 13, the electronic processing means 14 and the memory means 15 are preferably real ^¬ ized in an on-board electronic control unit (ECU) and may be connected to the cameras 12a, 12b and to the light source 20a, 20b preferably via a separate cable or alternatively via a ve ^¬ hicle data bus. The ECU may be integrated with other devices in another embodiment, such as an imaging means 11 or a light source 20a, 20b. All steps from imaging, image pre-processing, image processing to activation or control of safety means 17, 18, 19 ... are performed automatically and continuously during driving in real time.

The driver assisting system 10 comprises a light source arrangement 24. In general, the light source arrangement 24 may comprise at least one visible light source and/or at least one infrared light source. In the Figures, the light source ar ^¬ rangement 24 comprises at least one light source 20a, 20b adapted to generate a visible light beam 21 in order to illu ^¬ minate the vehicle environment. The light beam 21 generates a light figure 33, in particular a light spot, when falling on a surface of the environment, for example the road. The light sources 20a, 20b may in particular be the headlights of the vehicle; however, fog lights and/or separate search lights for example are also possible. In the embodiment of Fig. 1, the right head light 20b is electrically controllable by the proc ^¬ essing means 14 to actively change the direction and/or shape of the light beam 21 in particular through an electric drive, like an electric motor. In another embodiment both headlights 20a, 20b may be controllable in this manner either synchro ^¬ nously or independent from each other.

The offset determination an adjustment procedure is explained using the exemplary images 30a, 30b and 30c shown in Fig. 2.

Image 30a shows a scene in front of a motor vehicle, wherein a pedestrian 31 is detected by the processing means 14. The de ^¬ tection is indicated by an object bounding box 32.

When an object 31 is detected, the processing means 14 may control the light source 20b to change the direction and/or shape of the light beam 21 in order to continuously illuminate the detected object 31 during driving. In other words, the light source 20b is aligned to the expected direction of the detected object 31. This is done by using the position of the object bounding box 32 and taking into account speed, yaw, pitch and/or roll of the vehicle as measured by independent sensors, and a time delay between detection and directing the light beam on the detected object 31. This is shown in image 30b where the centre of the light figure 33 generated by the light beam 21 only approximately coincides with the centre of the bounding box 32. The directed illumination of the detected object 31 by the light beam 21 makes the driver aware of the detected object 31 and enables him to react properly. Thus, the light source 20b is directed on basis of the posi ^¬ tion data provided by the imaging means 11. Furthermore the direction of the light source 20b may be compensated for the detected object's 31 trajectory, the vehicle velocity and/or the vehicle yaw-rate, pitch and/or roll. For different practi ^¬ cal reasons, however, a perfect alignment between the light figure 33 and the object bounding box 32 is hard to achieve, and misalignments between the light figure 33 and the object bounding box 32 may occur, like it is shown in image 30b. In image 30b of Fig. 2 the light figure 33 generated by the light beam 21 is not accurately aligned with the detected object, in this case the detected pedestrian 31, leading to an offset 35 between the light figure 33 and the object bounding box 32.

The processing means 14 is adapted to detect the light figure 33 by image processing. This can be done for example by searching and fitting an oval light distribution with two independent standard deviations preferably within a well-defined region 36 around the object bounding box. Alternatively, the light distribution is searched for in the whole image and it is proved afterwards whether the distance between the deter ^¬ mined light figure and the object bounding box lies within reasonable limits. Although a circular light distribution of the light figure 33 is shown in Fig. 2 for simplicity, the light distribution is generally oval or elliptical. The stan ^¬ dard deviations in two perpendicular directions are preferably varied within reasonable limits. Alternatively, the standard deviations are arbitrarily varied and it is proved afterwards whether the determined standard deviations are within reason ^¬ able limits. It is also possible to have fixed standard devia ^¬ tions corresponding to expected extensions of the light figure 33. When the light figure 33 originating from the light beam 21 has been found in the search region 36, the center point of the light figure 33 is determined and the offset 35 is calcu ^¬ lated as the difference in x and y direction between the cen ^¬ ter point of the light figure 33 and center point of the ob ^¬ ject bounding box 32. In a subsequent step the processing means 14 controls the light source 20b to move towards the ob ^¬ ject bounding box 32 by an amount effectively bringing the offset 35 to zero. This ideal state where the light figure 33 exactly illuminates the detected object 31 continuously during driving is shown as image 30c in Figure 2.

In another embodiment, shown in Fig. 3, an auxiliary light source 22 is provided which is electrically controllable by the processing means 14 to actively change the direction and/or shape of an auxiliary optical beam 23 in particular through an electric drive, like an electric motor. The auxil ^¬ iary light source 22 may be arranged within a headlight 20, for example in the right headlight 20b as in Fig. 3, but this is not necessarily the case. Furthermore, a plurality of aux ^¬ iliary light sources 22 may be provided, for example an auxil ^¬ iary light source 22 for each headlight 20. The light source

22 may comprise at least one laser. Preferably the auxiliary light source 22 generates an optical beam 23. The optical beam

23 generates an optical figure 34 when falling on a surface of the environment, for example the road. Preferably the optical beam 23 essentially lies outside the visible spectrum such that the optical beam 23 and the optical figure 34 do not ir ^¬ ritate the driver. In a preferred embodiment the auxiliary light source 22 is an infrared light source, comprising for example at least one infrared laser. The auxiliary optical figure 34 generated by the auxiliary light source 22 is detectable by the imaging means 11 after the optical beam 23 has been directed to the reference posi ^¬ tion. The auxiliary light source 22 can be specifically adapted to generate a smaller and brighter optical figure 34 having a characteristic pattern which makes it much easier detectable as compared to the light figure 33 of the headlight 20b. Consequently the optical figure 34 generated by the aux ^¬ iliary light source 22 is used for the determination of the offset 35 instead of the light figure 33 generated by the head light 20b. It is possible, however, to use the detection of the light figure 33 and the detection of the auxiliary optical figure 34 in combination. For example, the detection of the light figure 33 may be used as a confirmation of a detection of the optical figure 34. In summary, by using an auxiliary light source 22 the reliability of the system 10 can be en ^¬ hanced .

The auxiliary optical figure 34 preferably has a characteris ^¬ tic pattern, for example a spot, a line, a circle, a cross and/or a plurality of these in an arbitrary combination and/or arrangement such that the auxiliary optical figure 34 can be unambiguously assigned as generated by the auxiliary light source 22 in the manner of a "fingerprint".

The auxiliary light source 22 may be mechanically or electri ^¬ cally coupled to the head light 20b such that both light beams 21, 23 can be moved synchronously. In the case of a mechanical coupling one electric drive may be sufficient to change the direction and/or shape of both beams 21, 23.

However, the auxiliary light source 22 and the head light 20b may also be adjustable independent from each other, in par- ticular through individual electric drives. In such an embodi ^¬ ment the auxiliary light source 22 may be used "in the back ^¬ ground" to calibrate the system 10 without the driver noticing it, where the offsets 35 determined by the auxiliary light source 22 are used to adjust the head light 20b exactly to a detected object 31 only if necessary, in particular if a certain risk of collision with the detected object 31 is determined .