Field

The present application relates to a camera for a vehicle, a method for operating a camera and a system with a camera and a controller.

Background

Devices for the execution of control tasks in vehicles are referred to as electronic control units (ECU). Electronic control units in vehicles, in particular motor vehicles, can have at least one processing unit, memory, interfaces and possibly other components that are required for processing input signals with input data and generating control signals with output data. The interfaces are used to receive the input signals or to output the control signals. Electronic control units for driving function operation, both for advanced driver assistance systems (ADAS) and for autonomous or semi-autonomous driving, can receive sensor data from various sensors including cameras as input data.

With advanced driver assistance systems and autonomous or semi-autonomous systems being more and more integrated in automotive vehicles, the number of cameras installed in vehicles is raising significantly. For autonomous or semi-au- tonomous systems, the cameras can output image information which can be input into an electronic control unit to support driving operation, like lane monitoring, sign identification, collision avoidance etc.

There are one or more front cameras mainly used for computer vision, which are used for lane monitoring, sign identification, collision avoidance etc. There are one or more parking cameras, mainly used for human vision, used to display what stands directly behind the vehicle in order to support the driver for parking. There is at least one Camera Monitor System (CMS) mainly used for human vision, used to display the view of the back side of the vehicle, to replace side mirrors. There is a Surround View System (SVS) mainly used for computer vision, used to monitor the whole surrounding environment of the vehicle. There might also be cameras at the interior of the vehicle, which can be used to capture a driver or a passenger. They can be used for human vision and/or computer vision like e. g. driver authorization, gaze detection etc.

Human vision relates to the visual sense of human beings and their ability translate light passing through the eyes to images. Images for human vision in a vehicle might for example be shown on a display for a human being to see.

Computer vision is a field of artificial intelligence that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human visual system can do. The image data can take many forms, such as video sequences, views from multiple cameras, etc. Sub-domains of computer vision in the area of automotive include e. g. scene reconstruction, object detection, object recognition etc.

In US10609148 a smart car with a number of sensors is described. The sensors include a camera with a lens receiving liquid to change a focal length of the lens.

In US2014036084 a vehicular camera having a lens assembly, a housing and an image sensor is described. The lens assembly comprises an adjustable lens, whose radius of curvature can be controlled via a suitable voltage supplied to the lens.

In US2014340510 a vision system for a vehicle with multiple cameras is described. At least one of the cameras may have a tunable lens, capable of adapting the focal length of the lens.

In US2015156383 a vision system of a vehicle includes a camera disposed at a vehicle and having a field of view. The camera includes an imager and a lens, with the lens having a liquid optic. A control is operable to vary a voltage at the liquid optic to adjust the liquid optic to adjust at least one of (I) a tilt of the lens and (II) a focal length of the lens.

In US2016021312 an around view monitoring system for vehicles is described. The around view monitoring system and an active safety system use a camera in common. The camera may include a liquid lens having both the narrow-angle characteristic and the wide-angle characteristic. The liquid lens may be controlled to have the wide-angle characteristic while the around view monitoring system is operating, and the liquid lens may be controlled to have the narrow-angle characteristic while the active safety system is operating.

In WO2012145313 a vehicular camera with a lens assembly comprising a group of lenses is introduced. The lens assembly comprises a lens that is capable of changing its shape to adjust its focal distance to adjust the effective focal length of the group of lenses in the lens assembly.

In US2019306430 a camera module is disclosed which comprises a liquid lens which is electronically adjustable to perform an autofocus function or an optical image stabilization function.

Summary

A camera for a vehicle comprises an assembly of optical elements and has a cone of view with an angle and direction. The assembly of optical elements comprises a first liquid cell, to which first voltage is applicable thereby adjusting the magnitude of the angle of the cone of view of the camera, to which the assembly of optical elements belongs. The first liquid cell is an optical lens, comprising at least one liquid. The first liquid cell has an electrically variable focal length. Associated with variable focal lengths of the first liquid cell is the variability of the angle of the cone of view of the assembly of optical elements, to which the first liquid cell belongs. Therefore, by applying a first voltage to the first liquid cell, the width of the field of view of the camera can be changed. This allows to make the camera adjustable between a wider field of view and a narrower field of view. The first liquid cell allows for the focal length of the camera to be adapted by applying a first voltage to the first liquid cell. The change of focal length corresponds to a change in the field of view of the camera. The assembly of optical elements further comprises a second liquid cell which second voltage is applicable thereby adjusting the direction of the cone of view of the camera which the assembly of optical elements belongs. The second liquid cell is an optical lens, comprising at least one liquid. It has an electrically variable optical axis. The variability of the optical axis corresponds to the direction of the cone of view of that assembly of optical elements, to which the second liquid cell belongs.

A liquid cell with adjustable angle of cone of view and/or direction of cone of view is sometimes also named a liquid lens with adjustable angle of cone of view and/or direction of cone of view. The angle of the cone of view refers to the opening angle of the cone with the tip of the cone being the vertex of the angle.

A liquid cell may for example be a two-phase liquid cell, also named liquid lens. A two-phase liquid lens may use two immiscible liquids (e. g. aqueous and oil) of as close to the same density as possible, but with different refractive indices and electrical properties, in a chamber. In embodiments, an electric voltage may control the electric field, which changes the curvature of the polar liquid (aqueous, e. g. water). As a result, the lens is caused to change its focal length. This allows focusing or zooming to be realized. In embodiments, four radially arranged electrodes may enable a dosed wedge-like distortion or displacement of the polar liquid. In other examples, a single-phase liquid cell, also called single-phase liquid lens in e. g. MEMS technology (MEMS: Micro Electro Mechanical System) may be used. MEMS may be used to deform a transparent membrane and thereby give a liquid volume a different shape that has the desired refractive effect.

In other examples a liquid crystal lens may be used as a liquid cell. A tunable liquid crystal lens may be obtained by sandwiching a liquid crystal layer between two electrodes. Tunability may be obtained by the application of an electric voltage to the electrodes.

Examples of liquid cells described throughout this application may refer to any liquid lens technology that allows to adjust the angle of cone of view and/or the direction of cone of view, e. g. liquid lens, liquid crystal lens and any other technology involving liquid lenses. The assembly of optical elements comprises a succession of optical elements, also called stack. The optical elements may for example comprise one or more lenses, one or more prisms or other optical elements. The assembly of optical elements is therefore sometimes referred to as stack of lenses.

In an embodiment, the second liquid cell is located at an end of the succession facing an imager of the camera. The second liquid cell may act like a prism or a Fresnel prism in order to deflect the light passing the assembly of optical elements. The second liquid cell serves to reorient the camera's cone of vision. The second liquid cell enables doing this without the need of a mechanical movement of the assembly of optical elements. The second liquid cell therefore allows for an object in the cone of view of the camera to be moved from for example a border region to a centre region of the cone of view.

In an embodiment, the assembly of optical elements comprises a succession of lenses, wherein an eyepiece lens is located in between the first liquid cell and the second liquid cell. The eyepiece lens or ocular lens is a type of lens that is usually located closest to the point where the image is actually created, in the case of the camera the image. The eyepiece lens may determine the back focal distance, so where the image will be formed. Its purpose is to magnify the image. In this embodiment, the eyepiece lens is located before the second liquid cell, so that the image enlarged by the eyepiece lens can be deflected onto the imager by the second liquid cell.

In an embodiment, the succession of lenses comprises an entrance lens facing the cone of view of the camera and at least one corrector lens. The corrector lens is located in between the entrance lens and the first liquid cell. The entrance lens serves the purpose of determining the field of view of the assembly of optical elements. The corrector lens or the corrector lenses serve the purpose of compensating for different aberrations of the camera. More specifically the corrector lens or the corrector lenses serve the purpose of compensating for the curvature of field or similar.

By adjusting the first and second voltage it is therefore possible to adjust the width of the field of view, as well as the direction of the field of view of the camera. By doing this, it is possible to realise different purposes with only one properly adjusted camera. For example, the camera may serve as an SBS camera, replacing the two wing mirror cameras, and it may serve as a CMS camera. The purpose of the camera will then depend on the first and second voltage, applied to the first and second liquid cell.

In an example of the camera the magnitude of the angle of the cone of view is adjustable between just above 0° and around 210°, in particular between around 30° and around 195°. The direction of the cone of view of the camera may also be adjusted up to 90° in both directions. The broader angle of cone of view (with corresponding shorter focal distance) can for example be used for a camera which serves a near field purpose, like for example for parking tasks. The camera with a narrower cone of view (with corresponding longer focal distance) can for example be used to support driving functions.

Additionally, the angle of the cone of view will may be adjustable to a value between 20° and 40°, in particular around 30°. Alternatively or additionally, the angle of the cone of view may be adjusted to a value between 110° and 130°, in particular around 120°. Alternatively or additionally, the angle of the cone of view may be adjusted to a value between 170° and 210°, in particularly around 195°. This will allow for a camera to serve as a camera with three designated widths of field of view. This may for example be a front/rear far field camera, a front/rear near field camera, a rear traffic camera or side camera with different angles.

In an embodiment, the magnitude of the angle of the cone of view is adjustable to a value between 60° and 80°, in particular around 70°. This particular angle of the cone of view might for example be used for side, front or rear camera use.

In an embodiment, the image data output by the camera is suitable for computer vision and/or human vision. This means that the camera might serve as a camera of which generated image data is displayed to a user in the vehicle and or it is usable as a camera, of which the image data output is input to an electronic control unit of the vehicle to be used for driving control functions.

In an embodiment, the camera is configured to be mounted to a vehicle with a cone of view exterior to the vehicle. In this embodiment, the camera may be configured to be operated as a surround view camera or as a camera producing output usable by a system supporting driving operation. It might also be used as a parking camera or as a camera replacing the side mirrors of the vehicle. Usage depends on the adjustments that are done to the first and second liquid cells of the assembly of optical elements of the camera.

In another embodiment, the camera is configured to be mounted to a vehicle with a cone of view interior to the vehicle. The camera may be configured to be used as a camera for capturing a driver and/or one or more passengers, depending on the adjustments to the first and second liquid cell of the assembly of optical elements. Capturing a driver or a passenger of the vehicle may be done by one and the same camera, just by adjusting the voltage applied to the first and second liquid cells. In a method for operating any of the cameras described above, one or more input signals are received, and a first voltage is applied to the first liquid cell, thereby adjusting the magnitude of the angle of the cone of view of the camera. The first voltage depends on the one or more input signals. A second voltage is applied to the second liquid cell, thereby adjusting the direction of the cone of view of the camera. The second voltage depends on the one or more input signals. This method allows the camera to be adapted to different users, as described above.

A system that is configured to perform the method described above comprises any of the cameras described above and a controller. The controller may be comprised in an electronic control unit of the vehicle.

Brief Description of the Figures

Embodiments will now be described with reference to the attached drawing figures by way of example only. Like reference numerals are used to refer to like elements throughout. The illustrations are not necessarily drawn to scale.

Figs, la, lb schematically illustrate an assembly of optical elements with one liquid cell.

Figs. 2a, 2b schematically illustrate an assembly of optical elements with two liquid cells.

Figs. 3a, 3b, 3c schematically illustrate objects in a cone of view of a camera.

Fig. 4 schematically illustrates a camera and an electronic control unit.

Fig. 5 schematically illustrates a vehicle with two cameras.

Fig. 6 schematically illustrates a vehicle with several cameras.

Detailed Description

Fig. la shows an assembly of optical elements 20 and an imager 22 of a camera 100. The assembly of optical elements 20 comprises a succession of optical elements, e. g. lenses, a so-called stack. At one end of the stack an entrance lens 14 is located facing the cone of view 24 of the camera 100. The entrance lens 14 determines the cone of view of the assembly of optical elements 20. In an embodiment the entrance lens may be a wide angle lens, like a fisheye or a pano- morph lens, giving a very wide angle of view.

Following the entrance lens 14 there are two corrector lenses 16. The corrector lenses 16 serve to compensate for aberrations. The aberrations include imperfections in the image formation by the optical system of the camera 100. The optical system, also called optics, of the camera 100 comprises, for example, the assembly of optical elements 20.

The two corrector lenses 16 are followed by a first liquid cell 10. To the first liquid cell 10 a first voltage VI can be applied thereby adjusting the width of the field of view of the liquid cell 10. Connected to the width of the field of view is the focal distance of the lens. Both, the field of view and focal distance of the liquid cell 10 can be adjusted by applying a first voltage VI to the first liquid cell 10. The first liquid cell 10 is followed by an eyepiece lens 18. The eyepiece lens 18 defines the back focal distance of the assembly of optical elements 20. It serves to enlarge the image that is created on the imager 22. In the embodiment shown in Fig. la the eyepiece lens is located at the end of the stack of lenses adjacent to the imager 22.

Fig. lb shows the same setup as Fig. la. Here, the first voltage has been applied to the first liquid cell 10 thereby adjusting the width of the cone of view 24 of the camera 100. This is illustrated in Fig. lb, where the width of the cone of view 24 is smaller than in Fig. la. This means that a narrower portion of the environment of the camera 100 will be reflected into the assembly of optical elements 20 and transformed into an image by the imager 22. Therefore, in the assembly of optical elements 20, as shown in Fig. la and Fig. lb, the magnitude of the angle of the cone of view 24 of the camera 100 can be adjusted by applying a first voltage VI to the first liquid cell 10 or by changing first voltage VI applied to the first liquid cell 10.

The elements of the camera 100 shown in figures 2a and 2b are the same as those shown in figures la and lb. A second liquid cell 12 has been added to the assembly of optical elements 20 in Fig. 2a and Fig. 2b. In the assembly of optical elements 20 shown in Fig. 2a, the same first voltage VI is applied to the first liquid cell 10 as in Fig. lb. Therefore, the magnitude of the angle of the cone of view 24 is the same in Fig. lb as it is in Fig. 2a. The second liquid cell 12 has been added in between the eyepiece lens 18 and the imager 22. The second liquid cell 12 acts like a prism or Fresnel prism. As can be seen in Fig. 2a, the direction of the light beam passing through the second liquid cell 12 is deflected and not all of the light passing through the second liquid cell 12 can be captured by the imager 22. Then, in Fig. 2b, a second voltage V2 is applied to the second liquid cell 12 or the second voltage V2 applied to the second liquid cell 12 is changed in Fig. 2b compared to Fig. 2a. The effect of the second voltage V2 applied to the second liquid cell in Fig. 2b is to deflect the light that is passing through the second liquid cell 12. This has the effect that the direction of the cone of view 24 of the camera 100 is changed. Now, as can be seen in Fig. 2b the light that is passing through the second liquid cell 12 reaches the imager 22. The imager 22 will create an image corresponding to the cone of view 24 is shown in Fig. 2b. Notably, the width of the cone of view is influenced by the first voltage VI applied to the first liquid cell 10 and the direction of the cone of view 24 is influenced by the second voltage V2 applied to the second liquid cell 12. The second liquid cell 12 therefore acts as a prism or Fresnel prism within the assembly of optical elements 20.

Fig. 3a shows an image which is for example created by the camera 100 as shown in Fig. la. In the embodiment shown in Fig. 3a, the camera 100 might for example be mounted on a vehicle EGO with a cone of view outside of the vehicle EGO. Within the cone of view 24 of the camera 100 is an object 26, which is a vehicle 26 close to the vehicle EGO.

As shown in Fig. 3a the angle of view is broad in this example and corresponds to an angle of the cone of view 24 with a large magnitude. An object O has been identified on the edge of the cone of view 24. The identification has been marked by drawing a box around object O. The identification of the object O may for example be done by an image processing software running on another controller C2 (Fig. 4).

By introducing the second liquid cell 12 into the assembly of optical elements 20, the direction of the cone of view 24 can be changed to be centred on the object O, as for example shown in Fig. 3b. This could for example be done by a set up as shown in figures 2a or 2b, except that for the view of Fig. 3b, the first voltage VI applied to the first liquid cell 10 is such that the magnitude of the cone of view 24 is still large and the view wide. The situation of Fig. 3b is thus, that the viewing angle is still wide and that a second voltage V2 has been applied to the second liquid cell 12 so that the object O has been moved to the centre of the cone of view 24. In Fig. 3c, now a first voltage VI is applied to the first liquid cell 10 - or the first voltage applied to the first liquid cell 10 has been changed- so that the magnitude of the angle of the cone of view 24 is arranged to a smaller one. This means that the field of view of the camera 100 is now smaller and the object O is not only centred but also appears larger and closer to the camera 100.

The embodiment shown in the previous figures illustrates how one and the same camera 100 can be used for different purposes, with the broader angle of view are narrower angle of view and changing the direction of the cone of view, without the need of mechanically moving the camera. This increases the robustness of the set up and reduces the cost.

In Fig. 4 a system including the camera 100 and an electronic control unit ECU is shown. The camera 100 comprises the assembly of optical elements 20 and the imager 22. The camera 100 comprises other camera electronics 34, like for example, further data storage. The electronic control unit ECU comprises a controller Cl which is configured to control the first and second voltage VI, V2 that is to be applied to the first and second liquid cell 10, 12 of the assembly of optical elements 20. Another controller C2 of the electronic control unit ECU controls the functioning of the camera 100. For example, it receives input signals relating to the intended use of the camera 100. It communicates with the controller Cl about the first and second voltage VI, V2 to be applied to the first and second liquid cell 10, 12. The other controller C2 might also perform the image processing on the image data received from the camera 100. The electronic control unit ECU comprises other electronics 32, like for example the input and output interfaces, storage and/or other processing units.

Fig. 5 shows an embodiment of the vehicle EGO. Two cameras 100 are mounted to the vehicle EGO. Each of the cameras 100 has an adjustable cone of view 24. The adjustments can be done as described before in connection with figures 1 to 4. In the embodiment shown in Fig. 5, one camera each is located at each side of the vehicle EGO. The two cameras 100 located at both sides of the vehicle EGO can for example be used to replace side wing mirrors of the vehicle EGO. For such a use that cone of view could be around e. g. 120° or for example be around e. g. 70°. In this mode of operation, the cameras 100 mounted at the side of the vehicle could for example be used as a Camera Monitor System for mainly human vision to display the back of the vehicle to replace the side mirrors. When supporting parking operations the angle of the cone of view 24 of the two cameras 100 located on both sides of the vehicle EGO could be around e. g. 195°. In this mode of operation, the cameras 100 mounted at the side of the vehicle could for example be used as a Surround View System for mainly computer vision to monitor the whole surrounding environment of the vehicle EGO.

In an embodiment, the electronic control unit ECU may have all the vehicle information such as odometry, gear, etc. This information may be used by the electronic control unit ECU to identify whether the vehicle EGO is in a normal drive or parking manoeuvre state. While in a normal driving scenario the electronic control unit ECU may keep one or more of the cameras 100 configured to Camera Monitor System, and once the electronic control unit ECU identifies that the driver is starting a parking manoeuvre, the electronic control unit ECU would trigger in parallel the controller Cl to adjust the angle and direction of the cone of view 24 of the one or more cameras 100 and put in Surround View System mode. Once the driver leaves the parking and starts to drive again, the electronic control unit ECU would trigger the controller Cl one more time to adjust the angle and direction of the cone of view 24 of the one or more cameras 100 to act as a Camera Monitor System. This brings the possibility of using one camera 100 placed each side view mirror, which work as Surround View System and Camera Monitor System depending on the request.

Fig. 6 shows another embodiment of the vehicle EGO. Four cameras 100 are mounted to the vehicle EGO. Each of the cameras 100 has an adjustable cone of view 24. The adjustments can be done as described before in connection with figures 1 to 4. In the embodiment shown in Fig. 5, one camera 100 is located at the front of the vehicle EGO, another one at the back of the vehicle EGO and one camera each at each side of the vehicle EGO. The two cameras 100 may operate as described in connection with Fig. 5 above. The front camera 100 located on the left side of Fig. 6, may for example serve as a front camera for assisting driving functions with the cone of view 24, which is narrow (around e. g. 30°) and has a focal length, which is large. The use for supporting driving operations could for example be for computer vision. Another possible use of such a front camera could be as a camera with a broad cone of view 24 (around e. g. 195°) and corresponding shorter focal length for the purpose of for example supporting parking operations. The use for supporting parking operations could for example be computer vision and human vision. Computer vision for parking operations could for example be used for an automated parking function. Human vision for parking operations could for example be used to show the front of the car on a display to support the driver who is performing the parking operation himself. The camera 100 located at that back of the vehicle EGO on the right side in Fig. 6, may have a similar use as camera 100 at the front and be similarly adjustable in its use.

In an embodiment, the electronic control unit ECU may have all the vehicle information such as odometry, gear, etc. This information may be used by the electronic control unit ECU to identify whether the vehicle EGO is in a normal drive or parking manoeuvre state. Upon detecting such scenarios, the electronic control unit ECU may operate the side cameras 100 as described in connection with Fig. 5.

While in a normal driving scenario the electronic control unit ECU may keep the front and/or the rear camera 100 configured to a far view camera supporting the driving operation, and once the electronic control unit ECU identifies that the driver is starting a parking manoeuvre, the electronic control unit ECU would trigger in parallel the controller Cl to adjust the angle and direction of the cone of view 24 of the front and/or rear camera 100 and put in a mode adapted for parking operation, e. g. Surround View System mode. Once the driver leaves the parking manoeuvre and starts to drive again, the electronic control unit ECU would trigger the controller Cl one more time to adjust the angle and direction of the cone of view 24 of the front and/or rear camera 100.