The invention relates to a method for presenting an image overlay element in an image of an environmental region of a motor vehicle. The image is displayed on a display surface of the motor vehicle. Furthermore, at least one object of the environmental region is imaged in the image. The invention also relates to a driver assistance system for a motor vehicle as well as to a motor vehicle with a driver assistance system.

Methods for presenting an image overlay element in an image of an environmental region of a motor vehicle are known from the prior art. Thus, in US 2012/0224060 A1 , a method is described, in which the image overlay element is displayed on a head-up display. The position of the image overlay element in an image displayed on the head-up display is determined depending on a line of sight of the driver to the environmental region. As the image overlay element, navigation information is presented by a line with an arrow.

It is the object of the invention to provide a method, a driver assistance system as well as a motor vehicle, by which or in which an image overlay element can be vividly presented in an image of an environmental region of a motor vehicle.

According to the invention, this object is solved by a method, by a driver assistance system as well as by a motor vehicle having the features according to the respective independent claims.

In a method according to the invention, an image overlay element is presented in an image of an environmental region of a motor vehicle. The image is captured by means of a camera system including at least one camera of the motor vehicle. Furthermore, the image is displayed on a display surface of the motor vehicle and at least one object of the environmental region is imaged in the image. An essential idea of the invention is to be considered in that a shape of a surface of the object is determined and the presentation of the image overlay element is performed locally specifically associated with the object, and in particular the presentation of the shape of the image overlay element is in particular geometrically adapted to the shape, in particular presentation, of the surface of the object in the image. By the method according to the invention, it becomes possible that the image overlay element is realistically and thus vividly presented overlaid in the image. Hereby, a vivid augmented reality (AR) can be generated. One understands the computerized extension of the reality perception by the augmented reality.

The shape of the surface of the object is in particular present in three dimensions. The three dimensions of the object are for example provided by a sensor of the motor vehicle and/or by an external data source such as for example a geo-database. The 3D information or three-dimensional information can then for example be linked to the image. Thus, the image overlay element can for example be presented in bent or curved manner if the shape of the surface of the object is also formed bent or curved. By the three- dimensional information of the environmental region, the image overlay element can for example be presented such that it is at least partially covered by an object located closer to the motor vehicle. The vivid presentation of the image overlay element can thereby be further increased. In the shape adaptation of the image overlay element, in particular a size and a geometry of the image overlay element are varied adapted to object.

Furthermore, the image overlay element is presented locally specifically associated with the object. This means that the image overlay element is preferably presented on the object to provide an optically attractive augmented reality.

In particular, the image overlay element is characterized as such an element, which is linked to the object with respect to its presented information content. This means that object-related information is presented by the image overlay element.

The adaptation of the shape of the object-linked image overlay element is preferably effected depending on the number of the spatial dimensions, with which the object is displayed on a two-dimensional display panel of the display surface. Furthermore, depending on situation, a two-dimensional or three-dimensional presentation of the image overlay element on the object is effected if the object is presented with the third dimension and thus with spatial depth and therefore also in perspective manner. Depending on situation means that the shape of the image overlay element is adapted depending on the spatial dimension number of the area of the object, on which the image overlay element is displayed. The shape describes a variation of the number of the spatial dimensions and/or a degree of distortion of a basic geometry of the image overlay element. The presentation of the image overlay element depending on situation is dependent on the angle between the viewing direction, in particular of the camera, to the object and thus can also dynamically change between the two-dimensional and the three-dimensional presentation.

In particular, it is provided that the shape of the surface of the object is determined as a three-dimensional, in particular fine-mesh, dot matrix. By the three-dimensional dot matrix, a digital surface model can be provided for the object and/or the environmental region. The digital surface model describes the shape of the surface of the object with three dimensions, thus, a depth value is present in the image in addition to a column coordinate and a row coordinate. Additionally or alternatively, the shape of the surface of the object can also be described by parameters of a mathematical area model. The surface of the object is the outermost envelope of the object.

Preferably, it is provided that the presentation of the shape of the surface of the object in the image is adjusted depending on a view of the object changed by a movement of the motor vehicle and/or a movement of the object to the changed view and the image overlay element is dynamically adapted to the adjusted presentation of the shape of the surface. Thus, it is preferably intended that the image overlay element is adapted to the shape of the surface of the object such that the presentation of the image overlay element corresponds to the view of the driver to the surface of the object. Since the shape of the surface of the object can continuously change by the movement of the motor vehicle, the image overlay element is also dynamically and/or continuously adapted to the shape of the surface of the object. The vivid presentation of the image overlay element in the image can therefore also be obtained after the movement of the motor vehicle.

Furthermore, it is preferably provided that the image overlay element is motion-coupled to the object and is presented at the same location on the object carried along with it upon movement of the object, wherein depending on a direction variation and/or size variation, the direction and/or the size of the image overlay element are adapted to the

corresponding variation of the object. Thus, upon movement of the object, the image overlay element is preferably also moved in the same manner as the object. The image overlay element is preferably also adapted if the size of the image overlay element and/or the view to the surface of the object change. By the co-adaptation of the image overlay element, the vivid presentation of the image overlay element and thus of the augmented reality can also be provided upon movement of the motor vehicle and/or the object.

Furthermore, it can be provided that upon movement of the object, a presentation associated with a relative movement between the object and the image overlay element presented thereon is performed and the image overlay element is shape-specifically adapted to an overlay area of the object, on which the object is then overlaid presented. Thus, the image overlay element is for example shifted in the overlay area of the object such that the presentation of the image overlay element in the image can be perceived by a user in vivid manner. To this, the object and the image overlay element are moved to each other with the relative movement. The relative movement can occur if the object and/or the motor vehicle move and thus the view of the user to the object or the surface of the object is changed.

Furthermore, it is preferably provided that information about the object is presented on the object in the image by the image overlay element. The image overlay element can for example be formed as a symbol and/or a letter and/or a number. The information about the object can for example be provided by a geo-database of the environmental region and/or be acquired by environmental sensors of the motor vehicle and also be provided. Thus, a navigation indication, in particular a directional arrow, can for example be displayed as the symbol, and for example a distance to a next exit can be presented as the number. The distance to the next exit can for example also be determined based on the 3D information acquired by the environmental sensor and/or be determined from the geo-database in combination with the position of the motor vehicle, which is for example determined by a GNSS (global navigation satellite system) receiver. Thereby, the augmented reality can be provided in more significant manner.

As an embodiment of the invention, it can be provided that the object is characterized as a roadway in the environmental region and the image overlay element is presented on the roadway as a navigation indication for the driver of the motor vehicle. Thus, it can be provided that the image overlay element is adapted to the surface of the roadway. Now, if the roadway is for example ascending because it goes uphill, thus, this can be recognized based on the three-dimensional information about the object, and the presentation of the image overlay element can be accordingly adapted with respect to the orientation and configuration. The same applies if the roadway for example extends in wavy manner. The image overlay element can then be adapted to the wavy extension of the roadway and also have a wavy shape. Furthermore, the image overlay element can also be presented on the surface of the roadway in the image such that another object, which is for example located on the roadway, is not covered by the image overlay element. The other object on the roadway can for example also be provided based on the 3D information or the three- dimensional information. Thus, if the image overlay element is presented such that overlapping with an object independent of the information to be presented is avoided, thus, the image overlay element can be more vividly presented in the image. Thus, it is in particular provided that the image overlay element is exclusively displayed on the object itself and not on an object extraneous with respect to the information of the image overlay element.

In a further embodiment, it is preferably provided that the object is characterized as a traffic participant and information about the traffic participant, in particular a velocity and/or a distance from the traffic participant to the motor vehicle, is presented by the image overlay element on the traffic participant in the image. The information about the traffic participant can for example be acquired by an environmental sensor of the motor vehicle and be provided for presenting in the form of the image overlay element. For a user, it is advantageous to get presented the information about the traffic participant directly on the traffic participant himself. Thus, confusion of the information with another object can be avoided. Furthermore, the information can thereby be fast perceived. In addition, thereby, the safety of the motor vehicle can be increased.

Furthermore, it is in particular provided that the image overlay element is presented completely within a contour of the object. The contour of the object is in particular an outer boundary of the object. Thus, it is in particular provided that the image overlay element is not presented outside of the contour of the object. Thus, the image overlay element is preferably completely on the object. A user or viewer of the image with the image overlay element or the augmented reality can therefore perform a simple association of the information of the image overlay element with the object. Thus, the augmented reality can thereby be more vividly generated. Additionally or alternatively, the contour of the object can also be determined with the aid of the three-dimensional information. Thus, the information can for example be determined by methods of image processing in a two- dimensional image, but the contour can be simpler and more precisely determined depending on situation by the three-dimensional information.

Preferably, it is provided that the shape of the surface of the object in the environmental region is determined by means of a geo-database. The geo-database can for example be held available in the motor vehicle or else additionally or alternatively be provided from a source external to vehicle, for example restricted to the current environmental region or unrestricted. Thus, a digital surface model, which can for example be generated by air- based laser scanning, can for example be provided by the geo-database. The position of the motor vehicle in the digital surface model can for example be determined by means of a global navigation satellite system. In this case, the 3D information is overlaid by the digital surface model and in particular the image. The overlay can also be effected in the form of an ortho-projection. The geo-database is advantageous in that the surface of static objects, in particular buildings, can be provided in particular completely and/or overlapping with respect to the local environmental region - thus in ample manner.

In a further embodiment, it is preferably provided that the shape of the surface of the object in the environmental region is determined by means of an environmental sensor of the motor vehicle, in particular a radar sensor and/or a lidar sensor and/or a laser scanner and/or an ultrasonic sensor and/or a stereo camera system. By the environmental sensor, the three-dimensional information can also be determined of temporary objects in the environmental region. The temporary object can for example be a moving traffic participant. A fusion of the three-dimensional information of multiple environmental sensors is also advantageous. The three-dimensional information can thereby be more precisely provided.

Furthermore, it is in particular provided that a relative position and/or a relative orientation and/or a distance of the object in the environmental region to the motor vehicle are determined by means of an environmental sensor of the motor vehicle. Thus, by means of the environmental sensor, it can be determined how the relative orientation or the relative direction of the object in the environmental region to the motor vehicle is. Thus, a direction of movement or direction of travel of the object can for example be inferred. Thus, it can for example be assumed that a vehicle as the object does not laterally travel without executing a forward or rearward movement. By the relative position and/or the relative orientation, it can thus be determined how relevant the information is to the driver of the motor vehicle, and thus a selection of image overlay elements potentially to be displayed can be made based on this criterion. The distance can then for example be presented in the form of the image overlay element on the object.

Preferably, the image overlay element is presented by alpha blending known per se on the object in the image. The alpha blending is a technology in the image or video processing, in which various images are superimposed to an overall image, wherein the alpha channel is also taken into account besides the color information. Therein, an alpha value of the alpha channel of a pixel is a measure of the transparency or opacity of the image overlay element at the location of the pixel in the image.

Furthermore, it can be provided that a position of the motor vehicle is determined depending on a method of odometry based on the shape of the surface of the object and/or depending on a global navigation satellite system. Thus, the position of the motor vehicle can for example be first coarsely determined by a global navigation satellite system (GNSS) and be improved or pursued based on the method of odometry if reception of the global navigation satellite system is temporarily restricted. By the method of odometry, the position of the motor vehicle can also be improved based on the shape of the surface of the object, in particular of multiple objects in the environmental region. Thus, the information presented on the object in the form of the image overlay element can be precisely effected.

In a further embodiment, it can be provided that the object is characterized as a building and/or a parking space and/or a traffic sign and/or a light signal system and/or a roadway damage of a roadway and/or a speed bump in the environmental region. Thus, the object can be variously present in the environmental region, however, preferably such that the object is relevant to a user. Thus, relevant objects for the user are for example objects potentially having a potential of risk in the environmental region. However, furthermore, these are for example also objects pointing to infrastructure in the environmental region, such as for example restaurants and/or hotels and/or gas stations and/or traffic signs.

The invention also relates to a driver assistance system for a motor vehicle with a camera system and a display surface, wherein the driver assistance system is adapted to perform a method according to the invention. To this, the driver assistance system can also include at least one evaluation unit formed for performing the method steps.

In a further embodiment, the driver assistance system preferably includes a head-up display (HUD) as the display surface. The head-up display is a display system, in which the user can keep his head posture or viewing direction because the information is projected into his field of view. The head-up display is advantageous in that the image overlay element can be simply projected into the image without a step being required to incorporate the image overlay element in the image or to newly "render" the image with the image overlay element.

A motor vehicle according to the invention, in particular a passenger car, includes a driver assistance system according to the invention or an advantageous implementation thereof.

The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the driver assistance system according to the invention as well as to the motor vehicle according to the invention.

Further features of the invention are apparent from the claims, the figures and the description of figures. The features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations or alone, without departing from the scope of the invention. Thus, implementations are also to be considered as encompassed and disclosed by the invention, which are not explicitly shown in the figures and explained, but arise from and can be generated by separated feature combinations from the explained implementations. Implementations and feature combinations are also to be considered as disclosed, which thus do not have all of the features of an originally formulated independent claim.

Below, embodiments of the invention are explained in more detail based on schematic drawings.

There show:

Fig. 1 in schematic plan view an embodiment of a motor vehicle according to the invention with a driver assistance system;

Fig. 2 a schematic illustration of an image of an environmental region of a motor vehicle with a known overlay scenario of an image overlay element;

Fig. 3 a schematic illustration of an image of an environmental region of a motor vehicle with an example of a first image overlay element presented according to the invention and a second image overlay element presented according to the invention;

Fig. 4 a schematic illustration analogously to Fig. 3, wherein the motor vehicle has moved compared to Fig. 3 and the second image overlay element is at least partially covered by an object; Fig. 5 a schematic illustration analogously to Fig. 4, wherein the motor vehicle has further moved;

Fig. 6 a schematic illustration analogously to Fig. 5, wherein the second image overlay element extends over the corner of a house and is accordingly adapted; and

Fig. 7 a schematic illustration of an image of an environmental region of the motor vehicle, wherein a parking lot with a parking space is shown and the parking space is marked with a third image overlay element and with a fourth image overlay element.

In the Fig., identical or functionally identical elements are provided with the same reference characters.

In Fig. 1 , a plan view of a motor vehicle 1 with a driver assistance system 2 or a driver assistance device according to an embodiment of the invention is schematically illustrated. The driver assistance system 2 includes a camera system 3 and a display surface 4. The display surface 4 can for example be formed as an LCD (liquid crystal display) screen or a head-up display. The arrangement of the display surface 4 is variously possible, however, preferably such that a user, in particular the driver of the motor vehicle 1 , has an unobstructed view to the display surface 4. In the embodiment, the camera system 3 includes a camera 5 disposed on a front 6 of the motor vehicle 1 . However, the arrangement of the camera 5 is variously possible, however, preferably such that a partial area of an environmental region 7 around the motor vehicle 1 can be captured.

Furthermore, the driver assistance system 2 includes an evaluation unit 8, which can also be variously disposed on the motor vehicle 1 . The evaluation unit 8 can be formed as a separate unit to the camera system 3 or else be integrated in the camera 5 or the display surface 4.

The camera 5 can be a CMOS (complementary metal-oxide-semiconductor) camera or else a CCD (charge-coupled device) camera. In particular, the camera 5 is a video camera, which continuously provides an image sequence of images. Fig. 2 shows a known image 9 with a known overlay scenario of an image overlay element 10. In the known image 9, the image overlay element 10 is presented as a navigation indication. The image overlay element 10 shows an arrow bent right, which is to signal turning right. The image overlay element 10 is presented without considering three- dimensional information of the environmental region 7 in the known image 9. In the known image 9, an object 1 1 in the form of a truck is imaged. Because three-dimensional information about a shape of a surface of the object 1 1 is not available of the object 1 1 , the image overlay element 10 cannot be adapted to the shape of the surface of the object 1 1 in the known image 9. Thus, the image overlay element 10 overlays multiple objects, namely the object 1 1 and a roadway 12. The presentation of the image overlay element 10, which is overlaid on the object 1 1 and the roadway 12, results in a little vivid augmented reality.

Fig. 3 shows an image 13 provided according to an embodiment of the invention. The image 13 is for example captured by the camera system 3. The image 13 shows the environmental region 7 and includes three-dimensional information. This means that a depth value also exists of the image 13 in addition to a column coordinate and a row coordinate. Thus, it can thereby for example be determined how far an object in the image 13 is away from the motor vehicle 1 . In the image 13, a first object 14 is shown as a truck. Furthermore, a second object 15 is shown as a building. A first image overlay element 16 is adapted to a shape of a surface of the first object 14 and overlaid. The first image overlay element 16 is disposed on a rear wall 17 of the first object 14, thus of the truck. In particular, it is provided that the first image overlay element 16 is disposed completely within an outer contour of the first object 14. To this, the image overlay element 16 is individually adapted to the size and position and orientation of the object 14 in its shape. If the rear wall 17 was curved, thus had different depth values, thus, the first image overlay element 16 would also be curved because it would be adapted to the rear wall 17. The second object 15 is partially overlaid by a second image overlay element 18. According to the embodiment, the second object 15 is a building with a hotel. For this reason, the second image overlay element 18 shows a symbol with an "H" to point to the presence of a hotel. By the three-dimensional information for example provided by an environmental sensor 19 of the motor vehicle 1 , a shape of a surface 20 of the second object 15 can be provided. The second image overlay element 18 can be accordingly adapted to the shape of the surface 20 of the second object 15. By the shape of the surface 20 of the second object 15, it is for example also known how high the building or the second object 15 is. The second image overlay element 18 can therefore be precisely presented on the surface 20 of the second object 15. The description and the schematic illustration are focused on the surface 20 of the second object 15. However, the description of the surface 20 analogously also applies to other objects in the environmental region (7).

Fig. 4 shows the image 13 with the environmental region 7, wherein the motor vehicle 1 has moved on on a roadway 21 in the environmental region 7 compared to Fig. 3. The first image overlay element 16 is motion-coupled to the first object 14 and is accordingly presented at the same location of the first object 14. The first image overlay element 16 is composed of letters and numbers according to the embodiment to represent a distance from the first object 14 to the motor vehicle 1 and thereby to inform the driver of the motor vehicle 1 about the distance. In Fig. 4, the second object 15 is overlaid or partially covered by a light signal system post 22. Thereby, the second image overlay element 18 is also partially covered. By the three-dimensional information, it can be ascertained that the light signal system post 22 is in a lower distance to the motor vehicle 1 than the second object 15. Thus, it can be differentiated between foreground and background in the image 13. Similarly, based on the three-dimensional information of the second object 15 and the light signal system post 22, it can be precisely determined which area of the surface 20 of the second object 15 is covered by the light signal system post 22 or by a surface of the light signal system post 22. The second image overlay element 18 can therefore be presented such that it looks as if it would be disposed behind the light signal system post 22.

Thereby, a vivid 3D effect can be generated and the augmented reality appears more realistic.

Fig. 5 shows the image 13 analogously to Fig. 4, wherein the motor vehicle 1 has moved on on the roadway 21 . The first image overlay element 16 is further presented at the same location on the first object 14. In addition, the second image overlay element 18 is presented on the second object 15. The second image overlay element 18 is presented in an overlay area 23 of the second object 15. The overlay area 23 is in particular defined via the outermost contour of the respective object 14, 15. In this overlay area 23, the second image overlay element 18 is adapted to the new location or to the changed position of the second object 15 in the image 13 upon movement of the second object 15. Thus, a relative movement between the second object 15 and the second image overlay element 18 is performed.

Furthermore, the first object 14 and/or the second object 15 are presented larger in the image 13 according to Fig. 5 than according to Fig. 4 by the approach of the motor vehicle 1 , and thus a size of the respective image overlay element 16, 18 is adapted to a size of the respective object 14, 15 and/or the distance of the respective object 14, 15. Thus, the image overlay elements 16, 18 are adapted to the size variation of the objects 14, 15 by a movement of the motor vehicle 1 and/or of the respective object 14, 15.

Fig. 6 shows the image 13 analogously to Fig. 5, wherein the motor vehicle 1 has further approached to the first object 14 and the second object 15. The second image overlay element 18 is accordingly shifted in the overlay area 23 of the second object 15. The second image overlay element 18 is adapted to the corner of the building or of the second object 15. Thus, the second image overlay element 18 is partially presented on a left wall 24 of the second object 15 and partially on a right wall 25 of the second object 15. The second image overlay element 18 is therefore presented curved to vividly represent an angle deviating from 180°between the left wall 24 and the right wall 25 in the image 13. Thus, the image overlay element 18 is virtually presented overlaid on the walls 24, 25 also on the wall transition in overlapping manner and thus projected thereon in clinging manner such that high closeness to reality is achieved.

For example, a navigation indication can also be presented in the image 13 as the image overlay element 16, 18. The navigation indication can for example also include distances to objects 14, 15, wherein the distance can for example be determined based on the 3D information. For example, a road grade of the roadway 12 can also be taken into account in presenting the navigation indication. Thus, the roadway 12 can for example ascend or descend and/or be tilted transversely to the roadway longitudinal axis of the roadway 12.

Furthermore, information about buildings in the environmental region 7 can be provided by the image overlay element 16, 18 in the image 13. Thus, the position with respect to the motor vehicle 1 and the buildings can for example be determined based on geo- databases in connection with a global navigation satellite system. Buildings located in the vicinity of the motor vehicle 1 can for example be presented based on their function with a corresponding image overlay element 16, 18 in the image 13. Thus, a hospital and/or a school and/or a library and/or a museum can for example be identified by the image overlay element 16, 18 as the building. The image overlay element 16, 18 can then again be adapted to the surface of the respective building and thus be vividly presented in the image 13.

Additionally or alternatively, the image 13 with the 3D information can for example be presented with a bowl view. In the bowl view, for example, multiple images 13 are combined to generate a panoramic image of the environmental region 7. Overlapping areas between the various images 13, which are used for the bowl view, can for example be more vividly presented with the image overlay element 16, 18 due to the 3D

information.

Fig. 7 shows a further schematic illustration of the image 13 from another scene. The image 13 shows the environmental region 7 according to Fig. 7, which is characterized by a parking lot 26. On the parking lot 26, there is a third object 27 and a fourth object 28. The third object 27 and the fourth object 28 are each formed as parking spaces of the parking lot 26. The third object 27 shows a normal parking spot, which is marked by a third image overlay element 29 in the image 13. The third image overlay element 29 displays a conventional P for a car space or a parking space. The third image overlay element 29 can be precisely oriented between the vehicles in the image 13, which bound the parking space or the third object 27 to the left and the right side. The presentation of the third image overlay element 29 can therefore be effected on the respective object 27, 28 such that a unique association of the image overlay element 29 with the third object 27 is effected. A fourth image overlay element 30 is associated with the fourth object 28. The fourth image overlay element 30 is characterized as a symbol with a wheelchair. The fourth image overlay element 30 thus marks a disabled parking space in the image 13. Here too, it is provided that the position of the fourth image overlay element 30 is adapted to the disabled parking space or the fourth object 28 and thus unique marking of the fourth object 28 is effected for a user of the driver assistance system 2 on the display surface 4.

Additionally or alternatively, multiple of the cameras 5 can be provided, by which then multiple of the images 13 of the environmental region 7 are provided. Thus, the image overlay element 16, 18, 29, 30 can for example be shifted from one image 13 into another image 13 if the camera 5 providing the respective image 13 is no longer capable of capturing the respective object 14, 15, 27, 28 due to the movement of the motor vehicle 1 or the respective object 14, 15, 27, 28. The image overlay element 16, 18, 29, 30 can therefore be passed from one image into the other if the respective object 14, 15, 27, 28 is no longer present in the image, in which the image overlay element 16, 18, 29, 30 was originally presented.