Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
A WIDE-ANGLE COMPOSITE MULTI-CAMERA SYSTEM AND METHOD FOR IN-AUTOMOTIVE VEHICLE RECORDING AND/OR VIEWING
Document Type and Number:
WIPO Patent Application WO/2021/050005
Kind Code:
A1
Abstract:
A method and multi-camera system for recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle. The method comprises the steps of disposing a first camera and one or more additional cameras inside the automotive vehicle at or near a windscreen of the automotive vehicle; and recording and/or displaying a composite view comprising a forward view and at least one of two additional views; wherein the forward view is captured in a forward direction through the windscreen using the first camera; and wherein the two additional views are captured through at least the windscreen of the automotive vehicle using one or more additional cameras, each additional view being in a view direction at a horizontal angle relative to the forward direction and wherein the respective view directions of the additional views are different from each other.

Inventors:
BREESE MARK (SG)
Application Number:
PCT/SG2020/050523
Publication Date:
March 18, 2021
Filing Date:
September 10, 2020
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
NAT UNIV SINGAPORE (SG)
International Classes:
B60R1/00; B60R11/04; G02B5/04
Foreign References:
GB2537390A2016-10-19
DE102015210870A12017-01-19
US20100321497A12010-12-23
US20080025571A12008-01-31
US20180095206A12018-04-05
Attorney, Agent or Firm:
VIERING, JENTSCHURA & PARTNER LLP (SG)
Download PDF:
Claims:
CLAIMS

1. A method of recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, the method comprising the steps of: disposing a first camera and one or more additional cameras inside the automotive vehicle at or near a windscreen of the automotive vehicle; and recording and/or displaying a composite view comprising a forward view and at least one of two additional views; wherein the forward view is captured in a forward direction through the windscreen using the first camera; and wherein the two additional views are captured through at least the windscreen of the automotive vehicle using one or more additional cameras, each additional view being in a view direction at a horizontal angle relative to the forward direction and wherein the respective view directions of the additional views are different from each other.

2. The method of claim 1, wherein the two additional views are chosen to overlap with respective vertical edge regions on respective sides of the front view.

3. The method of claims 1 or 2, wherein the view directions of the additional views are symmetric relative to the forward direction.

4. The method of any one of the preceding claims, wherein the additional views are captured through the windscreen and respective side windows of the automotive vehicle.

5. The method of any one of the preceding claims, wherein the view directions of the additional views are perpendicular to the forward direction.

6. The method of any one of the preceding claims, wherein capturing the two additional views comprises using a filter to suppress capturing of light reflected from a surface of the windscreen.

7. The method of claim 6, wherein the filter comprises a polarization filter.

8. The method of claim 7, wherein the polarization filter comprises a linear polarization filter.

9. The method of claim 8, comprising disposing the linear polarization filter with its polarization axis substantially perpendicular to the surface of the windscreen.

10. The method of any one of the preceding claims, comprising limiting the additional views such that a front passenger and/or driver are not captured.

11. The method of any one of the preceding claims, comprising covering any reflective or light-coloured surfaces inside the automotive vehicle and within the composite view with anti- reflective covers and/or coatings.

12. The method of any one of the preceding claims, wherein capturing the additional views comprises disposing two prisms on a first additional camera for diverting a field of view of the one additional camera in the forward direction into the two additional views in the respective view directions.

13. The method of any one of the preceding claims, wherein capturing the additional views comprises using first and second additional cameras, respectively.

14. The method of any one of the preceding claims, comprising applying vertical shift image processing on one or more of the captured front and additional views relative to the other(s) for reducing discontinuities in the composite image.

15. The method of any one of the preceding claims, comprising adjusting the position and/or angle of one or more of the first and additional cameras relative to the other(s) for reducing discontinuities in the composite image.

16. The method of any one of the preceding claims, comprising applying angular correction image processing.

17. The method of any one of the preceding claims, comprising placing the one or more additional cameras at a distance from the windscreen chosen for reducing the amount of light reflected from the windscreen being captured by the one or more additional cameras.

18. A multi-camera system for recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, the multi-camera system comprising: a first camera and one or more additional cameras configured to be disposed inside the automotive vehicle at or near a windscreen of the automotive vehicle; a recorder for recording, and/or a display for displaying, a composite view comprising a forward view and at least one of two additional views; wherein the first camera is configured to capture the forward view in a forward direction through the windscreen; and wherein the one or more additional cameras are configured to capture the two additional views at least the windscreen of the automotive vehicle, each additional view being in a view direction at a horizontal angle relative to the forward direction and wherein the respective view directions of the additional views are different from each other.

19. The multi-camera system of claim 18, wherein the one or more additional cameras are configured such that the two additional views overlap with respective vertical edge regions on respective sides of the front view.

20. The multi-camera system of claims 18 or 19, wherein the one or more additional cameras are configured such that the view directions of the additional views be symmetric relative to the forward direction.

21. The multi-camera system of any one of claims 18 to 20, wherein the one or more additional cameras are configured such that the additional views are captured through the windscreen and respective side windows of the automotive vehicle.

22. The multi-camera system of any one of claims 18 to 21, wherein the one or more additional cameras are configured such that the view directions of the additional views are perpendicular to the forward direction.

23. The multi-camera system of any one of claims 18 to 22, comprising a filter to suppress capturing of light reflected from a surface of the windscreen by the one or more additional cameras.

24. The multi-camera system of claim 23, wherein the filter comprises a polarization filter.

25. The multi-camera system of claim 24, wherein the polarization filter comprises a linear polarization filter.

26. The multi-camera system of claim 25, wherein the linear polarization filter is disposed with its polarization axis substantially perpendicular to the surface of the windscreen.

27. The multi-camera system of any one of claims 18 to 26, wherein the one or more additional cameras are configured to limit the additional views such that a front passenger and/or driver are not captured.

28. The multi-camera system of any one of claims 18 to 27, comprising anti-reflective covers and/or coatings for reflective or light-coloured surfaces inside the automotive vehicle and within the composite view.

29. The multi-camera system of any one of claims 18 to 28, comprising two prisms on a first additional camera for diverting a field of view of the one additional camera in the forward direction into the two additional views in the respective view directions.

30. The multi-camera system of any one of claims 18 to 28, comprising first and second additional cameras for capturing the additional views, respectively.

31. The multi-camera system of any one of claims 18 to 30, wherein the recorder and/or the display is configured to apply vertical shift image processing on one or more of the captured front view and additional views relative to the other(s) for reducing discontinuities in the composite image.

32. The multi-camera system of any one of claims 18 to 31, wherein one or more of the first and additional cameras are configured to adjust the position and/or angle thereof relative to the other(s) for reducing discontinuities in the composite image.

33. The multi-camera system of any one of claims 18 to 32, wherein the recorder and/or display are configured to apply angular correction image processing.

34. The multi-camera system of any one of claims 18 to 33, wherein the one or more additional cameras are placed at a distance from the windscreen chosen for reducing the amount of light reflected from the windscreen being captured by the one or more additional cameras.

Description:
A WIDE-ANGLE COMPOSITE MULTI-CAMERA SYSTEM AND METHOD FOR IN AUTOMOTIVE VEHICLE RECORDING AND/OR VIEWING

FIELD OF INVENTION

The present invention relates broadly to a wide-angle composite multi-camera system and method for in- automotive vehicle recording and/or viewing.

BACKGROUND

Any mention and/or discussion of prior art throughout the specification should not be considered, in any way, as an admission that this prior art is well known or forms part of common general knowledge in the field.

Most (and in some countries, all) new cars are fitted with a camera which records a forward view through the front windscreen, and a second camera to record a view through the rear window, as illustrated figure 1. These cameras are of great use in situations such as accident investigation where they provide video evidence of the circumstances leading to the accident. Both cameras are typically connected to a single recording device which is capable of saving two or more simultaneous video streams.

Each camera ideally has a wide angle so that a greater field of view can be recorded; 100° is standard though larger FOVs (field of view) are available with inherent geometric distortions and chromatic aberrations. The camera FOV is usually quoted as the diagonal range, from the left-most lower corner to the right- most upper comer. The horizontal FOV, i.e. FOV(H) is smaller than the diagonal value; e.g. for a 4:3 horizontal to vertical aspect ratio it is 80% of the diagonal FOV. Thus the FOV(H) typically ranges between 80 to 130° and the FOV(V) is typically 60° or more, e.g. 70°. A wider FOV(V) is not usually needed since one only need to view a relatively narrow range of upwards and downwards angles.

The FOV(H) of the forward or rear camera in figure 1 is adequate for recording and providing evidence for accidents which happen straight in front or directly behind the vehicle, but is of little help in other situations such as side impacts. These involve other vehicles or road users approaching perpendicular to the car direction which are not recorded in the geometry in figure 1. The ability to record a much wider horizontal angles in both the forward and backwards directions would therefore be of great use.

One existing solution is to embed cameras in the car exterior side walls, so that each points outwards and records a range of side views. There are multi-camera systems available which do use a range of cameras located at different parts of the car exterior, pointing in different directions, to provide a 360° surround view. Such systems tend to be expensive and as yet are not widely implemented. There are other types of camera systems which are capable of viewing and recording over very wide angles, such as fisheye lenses, which can view over angles of 180° of more, in both horizontal and vertical directions. However, these have severe imaging aberrations and geometric distortions at large angles, and also since only a FOV(V) of about 60° is generally required, then at least half of the fisheye recorded angular cone is not useful, leaving the useful part being recorded in a limited number of pixels, producing low resolution views. Despite these limitations there are camera devices which operate two back-to-back fisheye lenses, producing a combined FOV over all angles, from which any wide angular range might be extracted, however much of the recorded FOV suffers from the above problems. Also, the video processing usually renders the stitched video with significant latency. A further issue is that if this back-to-back fisheye device was implemented as car recording camera, located in the front camera position in figure 1, the backwards facing camera would cover the driver and passengers, leading to privacy issues.

WO2019027369A1 describes how a wide angle horizontal FOV(H) of 180° or slightly more can be formed from three cameras, typically each with a narrow FOV(H) of typically 60°, pointing in the same direction by using small, reflective prisms positioned above the outer two cameras. The use of prism optics allows a very low profile device to form a wide composite FOV(H).

While the same approach as described in WO2019027369A1 is equally valid to use three narrow FOV(H) cameras as a car recording camera, one generally cannot use lenses with a much wider FOV(H) (e.g. about 60°) as the prisms required become large. Furthermore, for use as a car recording camera, located inside the vehicle, there is not the same imperative to maintain a low profile for the overall device architecture used as a car recording camera so one can indeed put up with cameras pointing in different directions.

Embodiments of the present invention seek to address at least one of the above problems.

SUMMARY

In accordance with a first aspect of the present invention, there is provided a method of recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, the method comprising the steps of: disposing a first camera and one or more additional cameras inside the automotive vehicle at or near a windscreen of the automotive vehicle; recording and/or displaying a composite view comprising a forward view and at least one of two additional views; wherein the forward view is captured in a forward direction through the windscreen using the first camera; and wherein the two additional views are captured through at least the windscreen of the automotive vehicle using one or more additional cameras, each additional view being in a view direction at a horizontal angle relative to the forward direction and wherein the respective view directions of the additional views are different from each other.

In accordance with a second aspect of the present invention, there is provided a multi-camera system for recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, the multi-camera system comprising: a first camera and one or more additional cameras configured to be disposed inside the automotive vehicle at or near a windscreen of the automotive vehicle; a recorder for recording, and/or a display for displaying, a composite view comprising a forward view and at least one of two additional views; wherein the first camera is configured to capture the forward view in a forward direction through the windscreen; and wherein the one or more additional cameras are configured to capture the two additional views through at least the windscreen of the automotive vehicle, each additional view being in a view direction at a horizontal angle relative to the forward direction and wherein the respective view directions of the additional views are different from each other.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be better understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which:

Figure 1 shows a schematic plan view illustrating use of front and rear cameras inside an automotive vehicle.

Figure 2 shows a schematic of combining three wide angle cameras with the same vertical alignment (altitude) but different horizontal (azimuthal) alignments such that their horizontal fields of view slightly overlap and combine to give a very wide horizontal angle, according to an example embodiment.

Figure 3 shows a schematic of a wide angle car camera solution according to an example embodiment, featuring a three camera system at the front windscreen which is capable of providing 270° (±135°) or more of viewing angle in the horizontal direction.

Figure 4A shows a plan view schematic and photograph of a three camera system according to an example embodiment.

Figure 4B shows a side view schematic and photograph of the three camera system of Figure 4A. Figure 5A shows photos of a composite FOV(H) of 270° for a three camera system according to an example embodiment located at the front windscreen, with white diagonal supports covered in black (non-reflective) material.

In Figure 5B shows photos of a composite FOV(H) of 270° for a three camera system according to an example embodiment located at the front windscreen, with portions of the black material removed for comparison.

Figure 6A shows a schematic illustrating light reflected from a flat glass surface being partially polarized, with most reflected light being polarized parallel to the reflective surface.

Figure 6B shows a plot of the reflected intensity of light from a glass surface for components polarized parallel and perpendicular to the surface versus incident angle Q to the surface normal.

Figure 7A shows a schematic illustrating a three camera system with linear polarizers placed over the two side cameras and rotated so that they only transmit light which is polarized perpendicular to the windscreen glass surface, according to an example embodiment.

Figure 7B shows a view from the left-side camera of a three camera system without polarizer, according to an example embodiment, with no anti-reflective material used to cover the white diagonal support(s) of the windscreen.

Figure 7C shows the same view as in Figure 7B, under the same conditions, using a three camera system with a suitably- aligned linear polarizer over the left-side camera, according to an example embodiment, for comparison.

Figure 8A shows a composite view of a three camera system according to an example embodiment.

Figure 8B shows a composite view of a three camera system with according to an example embodiment, with barrel distortion correction and cylindrical projections correction, for comparison.

Figure 9A shows a schematic plan view illustrating possible rear blind spots in a three camera system according to an example embodiment, when a 180° rear camera is used.

Figure 9B shows schematics illustrating a dual prism camera for use in a multi-camera system according to an example embodiment.

Figure 10A shows a plan view of a two camera system according to an example embodiment.

Figure 10B shows respective side views (staggered for better clarity) of the two camera system of Figure 10A.

Figure 11 shows a schematic illustrating a three camera system positioned in front of the rear view mirror, according to an example embodiment. Figure 12 shows a schematic illustrating a three camera system according to an example embodiment may be dimensioned so that the side cameras are aligned with or close to the edges of the rear view mirror.

Figure 13 A shows a schematic drawing illustrating side and central cameras being located close to the windscreen, according to an example embodiment.

Figure 13B shows a schematic drawing illustrating the side cameras being located further away from the windscreen compared to Figure 13 A, according to an example embodiment.

Figure 14 shows a flow-chart illustrating a method of recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, according to an example embodiment.

Figure 15 shows a schematic illustrating a multi-camera system for recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, according to an example embodiment.

DETAILED DESCRIPTION

Embodiments of the present invention provide a device and method suitable for producing a composite, wide angle view of typically 270°, primarily for use as a wide angle recording camera located at or near the front windscreen inside an automotive vehicle such as a car, and preferably at or near the centre of the windscreen and/or cabin. While larger composite viewing angles up to 300° or more are possible, they are nor preferred in order to preserve passenger privacy.

Embodiments of the present invention can provide a device and method suitable for combining three cameras, each with a wide FOV(H) of typically 90 to 110° (though different values can equally be used), so as to produce a composite wide angle view of typically 270°. Such devices according to example embodiments can be used as a wide angle recording camera located at the front windscreen. Figure 2 shows a schematic of combining three wide angle cameras with the same vertical alignment (altitude) but different horizontal (azimuthal) alignments such that their horizontal fields of view 201-203 slightly overlap and combine to give a very wide horizontal angle. It is noted that not all three camera views need to be combined at all times. Instead, the central view 202 may be combined with only one of the views 201, 203, for example responsive to different steering actions by the driver.

In figure 2 a small angular overlap, ±10° in this example, is used to preferably ensure that there are no blind spots. Larger composite viewing angles up to 300° or more are possible but not preferred in order to preserve passenger privacy.

Figure 3 shows a schematic of a wide angle car camera solution according to an example embodiment, featuring a three camera system 300 at the front windscreen 301 which is capable of providing 270° (±135°) or more of viewing angle Q in the horizontal direction. In this example embodiment, a single camera 302 with a standard FOV of 90° is located at the rear window.

Specifically, figure 3 shows the three camera system 300 according to an example embodiment providing a composite FOV(H) of 270° when located at the front of the car. The forward facing camera 304 (azimuthal angle of 0°) records a standard front-facing view 306 along the forward direction (Q = 0°), and the two side cameras 308, 310 (with viewing direction centred at azimuthal angles of ±90°, respectively) allow the full view through the front windscreen 301 to be recorded (in conjunction with the forward facing camera 304), along with the view through both front, side windows 312, 314 (here +135° to +90°, and -90° to -135°). Preferably, no view of the front passenger 316 and driver 318 is recorded.

For a single camera which views through the windscreen over a FOV(H) of, for example, 120°, the small viewing angle Q of light passing through the windscreen (maximum Q of ±60° relative to the forward direction) means that there are no strong reflections. However, for the three camera device 300 according to the example embodiment described here, which involves viewing through the glass windscreen 301 at very large angles in figure 3, there can be significant problems due to unwanted scattering and reflection of light from the windscreen. This can pose a disadvantage on a sunny day where strong sunlight can generate intense reflections from the windscreen 301 at large viewing angles, which enter the side cameras 308, 310. However, even on overcast days it can still be a problem. This problem can advantageously be minimized using suitably-oriented linear polarizers placed over the outer two cameras 308, 310. Some embodiments of the present invention exploit the fact that light reflected from a smooth glass surface is mostly linearly polarized parallel to the surface (see also figure 6A). Thus there are strong reflections of linearly polarized light parallel to the windscreen. The linear polarizers over the cameras 308, 310 are oriented so that they preferentially allow light which is polarized perpendicular to the windscreen, such that they preferentially reduce the intensity of unwanted reflections.

Camera alignment and display considerations according to example embodiments

Video stitching, as is understood in the art, can provide a seamless, high quality all-around view and is performed in some multi-camera systems, providing wide angle composite views. However, it is computationally intensive to implement in real time and challenging to implement perfectly without constraints on camera geometry and orientation. In embodiments of the present invention it is assumed that the, for example, three separate camera video streams are normally viewed in a “raw data”, unstitched mode, though they may also be stitched off line if required. In the “raw data” mode the three adjacent views are displayed in a row with a small angular overlap, as illustrated in figure 2. In this mode the wide angle composite view is advantageously observed without any further intensive computation, though minor processing may be used to compensate for camera distortions (e.g. barrel distortions) inherent in wide viewing angles, and different geometric projections used where necessary, e.g. cylindrical projection. This viewing mode according to such example embodiments is thus robust in not having any stitching artefacts, is fast to record and display and is low cost in not requiring intensive computation facilities. Furthermore, for video stitching of multi-camera devices, all cameras should ideally radiate away from a common origin otherwise parallax errors make the stitching process more difficult and more likely to produce artefacts.

Removing the need for all cameras to have a common origin of alignment according to example embodiments in which camera video streams are viewed in a “raw data”, unstitched mode advantageously allows a more flexible device configuration in which the cameras can be mounted at different locations with respect to each other (this point is further elaborated on below with reference to Figure 13). Figure 4A shows plan view and Figure 4B a side view schematics and photographs of a three camera system 400 according to an example embodiment. Central camera is number 1, outer cameras are numbers 2 and 3. The size and shape of the three camera system 400 is thus similar to that presently used as a recording device using a single camera located at the front windscreen. While the effect of even small parallax errors is significant for stitching, embodiments of the present invention exploit that they produce minimal problem in the intuitive understanding of motion between different fields of view in a “raw data” mode.

A further advantage of multi-camera solutions according to example embodiments to provide a composite wide angle view is that even if the sun during the daytime (or headlights at night) is shining directly at any angle into one field of view, the recording quality provided by the other two cameras is unaffected, even if the affected camera suffers from lower quality and image darkness owing to saturation by the bright incident light. In other words, good quality images can advantageously still be recorded from the other camera(s).

Figure 5 shows photos of a composite FOV(H) of 270° for a three camera system according to an example embodiment located at the front windscreen. In Figure 5A, white diagonal supports 500, 502 are covered in black (non-reflective) material. In Figure 5B, portions (white arrows) of the black material have been removed for comparison, resulting in significant reflection into the side cameras (black arrows) from the uncovered portions of the supports 500, 502.

Specifically, figure 5A shows the composite FOV of 270° for the three camera system located at the front windscreen (compare figure 3). The diagonal supports 500, 502 between the front windscreen and side windows obscure a small range of viewing angles, leading to small blind spots, but overall figure 5A demonstrates the wide viewing angle with high clarity on all cameras. Figure 5 A also demonstrates that the front passenger and driver are not included in the recording; this is a deliberate choice of limiting the FOV to angles so that all car passengers/driver are not recorded according to preferred embodiments.

As discussed above with reference to figure 3, one limitation of this wide angle recording geometry in strong sunlight is light reflected off any surfaces and into one of the side cameras. In figure 5A, the white diagonal supports 500, 502 at the edges of the front windscreen have been covered in black, non-reflective material. Even though intense sun light is shining brightly through the driver side window and front windscreen, there is little problem caused by residual light scattering into any camera except for minimal scattering into the right-side camera from other in-car surfaces, as indicated by the black outline arrow. The importance of covering the white diagonal supports and generally minimizing unwanted reflections into any of the cameras is demonstrated in figure 5B. This was recorded under the same conditions as figure 5 A, except that the portions of black material has been removed from both diagonal supports (see white outline arrows). Now a high intensity of reflected light scattering is observed (see black outline arrows), caused by the uncovered portions of the supports 500, 502 being white in colour and therefore strongly reflecting light back onto the windscreen and into the side- viewing camera.

Thus, while covering up or coating any reflective surface mitigates the effect of unwanted reflections, it is may not be an ideal solution in that it requires the car interior to be fitted with non-reflective material.

Advantageously, as mentioned above, some embodiments of the present invention minimize the effect of unwanted reflections on the images provided by the side cameras by exploiting the fact that light 600 reflected from a flat glass surface 602 tends to be polarized, with the reflected light 604 being polarized parallel to the reflective surface 602, as illustrated in figure 6A. Figure 6B shows a plot of the reflected intensity of light from a glass surface for components polarized parallel and perpendicular to the surface versus incident angle Q to the surface normal. Note that Q is essentially the same as the previously-defined camera viewing angle in figure 1. Clearly the reflected light intensity increases sharply for large incident angles and so for large camera viewing angles. The reflected component of light which is polarized parallel to the surface is stronger than that which is polarized perpendicular to the surface. However, since both are present then a linear polarizer which is oriented so as to block light polarized parallel to the surface will still transmit the orthogonal component which is polarized perpendicular to the surface. The effect of a linear polarizer in such example embodiments is therefore to minimize reflections.

Specifically, to make use of this, linear polarizers 700, 702 are placed over the two side cameras 2, 3 and rotated so that they only transmit light which is polarized perpendicular to the glass surface (compare also figure 6A) according to example embodiments, as illustrated in figure 7A, for reducing unwanted reflections. Figure 7B shows a similar view from the left-side camera as in figure 5A. No anti-reflective material is used to cover the white diagonal support(s) 704 and because of this, and also because of reflected light from other surfaces, there is a large patch of white reflection, indicated by the black outline arrow. Figure 7C shows the same view, under the same conditions, but with a suitably-aligned linear polarizer over the left side camera, resulting in a reduced intensity of the patch of reflected light since most of the polarized component is not transmitted to the camera.

It is noted that there are a number of existing front-viewing dashcams incorporating with a polarizer to reduce reflections in the forward viewing direction. This helps to reduce glare and reflections from the dashboard when the sun is shining directly in front. However, the required polarizer orientation to minimize reflections in the forward direction is different to that required to minimize reflections at wide viewing angles according to example embodiment. Specifically, for such existing very wide angle front camera which views the full extent of the windscreen, a polarizer which is oriented to reduce reflections in the forward direction has little beneficial effect on the strong reflections at wide viewing angles and may even strongly degrade these parts of the view by preferentially transmitting intense polarized reflected light from these regions.

In the compact, composite three camera device 400 in figure 4 each of the three cameras 1, 2, and 3 is aligned so that its central ray is parallel to the road surface, i.e. the cameras have different azimuthal or view direction alignments but have the same altitude (to use an astronomical notation of angle). This can advantageously ensure that features moving between separate camera views are intuitive to understand.

Figure 8A shows discontinuities at the boundaries between adjacent views 801-803 from the three cameras of a three camera system according to an example embodiment (see e.g. within the white circle) and curved features (e.g. see arrow). As shown in figure 8B, both of these issues can be fixed in software according to example embodiments. Specifically, there may be discontinuities at the boundaries of each camera view, see figure 8A, white outline circle, where the bonnet of the car and also the road are at different heights across the two adjacent FOV. This is caused by small misalignments of the outer cameras in height or angle, or by features located at different distances within each FOV. The can result in features jumping in vertical position when moving horizontally between camera FOVs, leading to non-intuitive viewing. However, this can be remedied in example embodiments either by small adjustments in the position or angle of each camera, or in software by vertically shifting each FOV with respect to each other, which does not require extensive image processing, as will be appreciated by a person skilled in the art. Figure 8B shows the views 911-913 where this has been fixed by vertical shifts of the FOV of the outer cameras, indicated as views 801* and 803*.

Similarly, in relation to the streetlight indicated by the black outline arrow in view 802 in figure 8A, which appears curved, this may also be readily fixed in software according to example embodiments, see view 802* in figure 8B, by appropriate use of angular correction software, including for barrel distortion and/or cylindrical projections, where appropriate. Again, such correction processing, if desired, does not require extensive image processing, as will be appreciated by a person skilled in the art

Accordingly, in an example embodiment three individual wide angle views can be joined together in a “raw data” mode, with minimal software processing used to correct any larger distortion or discontinuity, if desired, and suppression of reflections caused by intense sunlight.

Dual prism camera system according to example embodiments

There may be instances where a larger FOV(H) at the rear than that shown in figure 1 is desirable, for example up to 180°. This can reduce the blind spots, as illustrated in Figure 9A. Rear viewing/recording angles larger than this are usually unnecessary at the rear as they are blocked by the rear supporting columns.

Figure 9B describes a dual prism camera for use according to example embodiments which allows two adjacent cameras which point in the same direction, for example, each with a FOV(H) of 90°, to form a composite wide angle FOV(H) of 180° using two cameras 900, 902. Two right-angle prisms 904, 906 are placed over one camera 902 so that it allows two separate FOV from (+45° to +90°) 908, and (-45° to -90°) 910 to be recorded on the sensor. Light from angles greater than ±90° may enter the prism e.g. 906 but it is not recorded on the camera sensor because it either is outside the FOV and also may not fall on the sensor. However, this can be adjusted by e.g. using cameras with larger FOV and/or by tilting the prism(s) 904, 906. The FOV 912 of the other camera 900 is from (+45° to -45°), so the sum of the two FOV(H) extends over 180° in the configuration shown in figure 9B.

Furthermore, if for example each camera has a FOV(H) of 130° then the same approach may be used to generate a composite FOV(H) of 260° in a two camera front view system for wide angle front view according to an example embodiment, and so be used for the same purpose as the three camera device described above, since it is typically cheaper to use two rather than three cameras to do so where possible.

Figure 10A shows a plan view and Figure 10B shows respective side views (staggered for better clarity) of two camera system 1000 according to an example embodiment of the present invention in which a dual prism 1002, 1004 camera 2 in combination with camera 1 provides a combined FOV(H) of 260°, for locating at the front of the car. In this embodiment, the prims 1002, 1004 are slightly tilted to obtain the desired FOV on the camera 2 sensor. To minimize the effect of unwanted reflections at wide viewing angles according to preferred embodiments, linear polarizers 1006, 1008 are placed over the prism entrance faces.

Rear View Mirror considerations according to example embodiments

For the very wide angles available with a device according to example embodiments of the present invention located at the front windscreen, its location relative to the rear view mirror can be important. A single camera recording device is usually positioned in front of the rear view mirror - this is fine since the FOV is much less than 180° and so the rear view mirror cannot obscure the wide angle forward view. However, for the wide angle composite view of 270° according to an example embodiment, if a three camera system 1100 according to an example embodiment is positioned in front of the rear view mirror 1102, then its side and backwards views are obscured by the rear view mirror 1102, as indicated in figure 11. Instead, the three camera system 1100 according to an example embodiment is preferably positioned low enough relative to the rear view mirror 1102.

Alternatively, a three camera system 1200 according to an example embodiment may be dimensioned so that the side cameras 2 and 3 are aligned with or close to the edges of the rear view mirror 1202, as illustrated in figure 12.

A further refinement incorporates a three camera system according to example embodiments into the rear mirror 1200.

The rear view mirror 1200 may be configured as a display device, such as available for single dashboard cameras and/or camera assisted rear view mirrors. The wide horizontal extent and limited vertical extent of rear view mirrors can be ideal for displaying the composite wide angle view from the three cameras (compare figure 2). Figures 13 A and B show the effect of locating the two side cameras further away from the windscreen, according to some example embodiments. The camera arrangement in figure 13A according to one example embodiment is compared with a different arrangement in figure 13B according to another example embodiment, whereby the outer two side cameras 2, 3 are located at the back of the unit 1300 such that they are further away from the windscreen. Hence in the embodiment on figure 13B reflected light off the windscreen from the diagonal sidebars enters the side cameras 2, 3 with a smaller angle Q and so with a lower intensity (compare also figure 6B, which illustrates the overall reduced reflectivity for smaller angles, irrespective of polarization). It is noted that in modifications of the embodiments shown in figures 13 A and B, polarizers may still be used at least for the side cameras 2, 3 to further reduce the reflected intensity captured by filtering out the component with a polarization parallel to the surface of the windscreen. It is further noted that in the embodiment shown in figure 13B, the side cameras 2, 3 are moved by distances of the order of a few cm, e.g. 4 cm, which has little effect on the FOVs of the side cameras through the side windows, which preferably ensures that the passenger/driver remain outside the FOV s for privacy issues, as described above with reference to figure 3.

Figure 14 shows a flow-chart 1400 illustrating a method of recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, according to an example embodiment. At step 1402, a first camera and one or more additional cameras are disposed inside the automotive vehicle at or near a windscreen of the automotive vehicle. At step 1404, a composite view comprising a forward view and at least one of two additional views is recorded and/or displayed, wherein the forward view is captured in a forward direction through the windscreen using the first camera; and wherein the two additional views are captured through at least the windscreen of the automotive vehicle using one or more additional cameras, each additional view being in a view direction at a horizontal angle relative to the forward direction and wherein the respective view directions of the additional views are different from each other.

The two additional views may be chosen to overlap with respective vertical edge regions on respective sides of the front view.

The view directions of the additional views may be symmetric relative to the forward direction.

The additional views may be captured through the windscreen and respective side windows of the automotive vehicle.

The view directions of the additional views may be perpendicular to the forward direction.

Capturing the two additional views preferably comprises using a filter to suppress capturing of light reflected from a surface of the windscreen. The filter may comprise a polarization filter. The polarization filter may comprise a linear polarization filter. The method may comprise disposing the linear polarization filter with its polarization axis substantially perpendicular to the surface of the windscreen. The method may comprise limiting the additional views such that a front passenger and/or driver are not captured.

The method may comprise covering any reflective or light-coloured surfaces inside the automotive vehicle and within the composite view with anti-reflective covers and/or coatings.

Capturing the additional views may comprise disposing two prisms on a first additional camera for diverting a field of view of the one additional camera in the forward direction into the two additional views in the respective view directions.

Capturing the additional views may comprise using first and second additional cameras, respectively.

The method may comprise applying vertical shift image processing on one or more of the captured front and additional views relative to the other(s) for reducing discontinuities in the composite image.

The method may comprise adjusting the position and/or angle of one or more of the first and additional cameras relative to the other(s) for reducing discontinuities in the composite image.

The method may comprise applying angular correction image processing.

The method may comprise placing the one or more additional cameras at a distance from the windscreen chosen for reducing the amount of light reflected from the windscreen being captured by the one or more additional cameras.

Figure 15 shows a schematic drawing illustrating a multi-camera system 1500 for recording and/or viewing a wide horizontal angle view including a forward facing view relative to an automotive vehicle, according to an example embodiment. The multi-camera system 1500 comprises: a first camera 1502 and one or more additional cameras 1503, 1504 configured to be disposed inside the automotive vehicle at or near a windscreen of the automotive vehicle; a recorder 1506 for recording, and/or a display 1508 for displaying, a composite view comprising a forward view and at least one of two additional views; wherein the first camera 1502 is configured to capture the forward view in a forward direction through the windscreen; and wherein the one or more additional cameras 1503, 1504 are configured to capture the two additional views at least the windscreen of the automotive vehicle, each additional view being in a view direction at a horizontal angle relative to the forward direction and wherein the respective view directions of the additional views are different from each other.

The one or more additional cameras 1503, 1504 may be configured such that the two additional views overlap with respective vertical edge regions on respective sides of the front view. The one or more additional cameras 1503, 1504 may be configured such that the view directions of the additional views be symmetric relative to the forward direction.

The one or more additional cameras 1503, 1504 may be configured such that the additional views are captured through the windscreen and respective side windows of the automotive vehicle.

The one or more additional cameras 1503 1504 may be configured such that the view directions of the additional views are perpendicular to the forward direction.

The system 1500 may comprise a filter to suppress capturing of light reflected from a surface of the windscreen by the one or more additional cameras. The filter may comprise a polarization filter. The polarization filter may comprise a linear polarization filter. The linear polarization filter may be disposed with its polarization axis substantially perpendicular to the surface of the windscreen.

The one or more additional cameras 1503, 1504 may be configured to limit the additional views such that a front passenger and/or driver are not captured.

The system 1500 may comprise anti-reflective covers and/or coatings for reflective or light- coloured surfaces inside the automotive vehicle and within the composite view.

The system 1500 may comprise two prisms on a first additional camera 1503 for diverting a field of view of the first additional camera 1503 in the forward direction into the two additional views in the respective view directions.

The system 1500 may comprise first and second additional cameras 1503, 1504 for capturing the additional views, respectively.

The recorder 1506 and/or the display 1508 may be configured to apply vertical shift image processing on one or more of the captured front view and additional views relative to the other(s) for reducing discontinuities in the composite image.

One or more of the first and additional cameras 1502-1504 may be configured to adjust the position and/or angle thereof relative to the other(s) for reducing discontinuities in the composite image.

The recorder 1506 and/or display 1508 may be configured to apply angular correction image processing.

The one or more additional cameras 1502, 1504 may be placed at a distance from the windscreen chosen for reducing the amount of light reflected from the windscreen being captured by the one or more additional cameras.

Embodiments of the present invention can have one or more of the following features and associated benefits/advantages:

Aspects of the systems and methods, such as the systems and methods for image/video data processing and/or recording and/or displaying, described herein may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (PLDs), such as field programmable gate arrays (FPGAs), programmable array logic (PAL) devices, electrically programmable logic and memory devices and standard cell -based devices, as well as application specific integrated circuits (ASICs). Some other possibilities for implementing aspects of the system include: microcontrollers with memory (such as electronically erasable programmable read only memory (EEPROM)), embedded microprocessors, firmware, software, etc. Furthermore, aspects of the system may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types. Of course the underlying device technologies may be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (MOSFET) technologies like complementary metal-oxide semiconductor (CMOS), bipolar technologies like emitter-coupled logic (ECL), polymer technologies (e.g., silicon- conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, etc.

The above description of illustrated embodiments of the systems and methods is not intended to be exhaustive or to limit the systems and methods to the precise forms disclosed. While specific embodiments of, and examples for, the systems components and methods are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the systems, components and methods, as those skilled in the relevant art will recognize. The teachings of the systems and methods provided herein can be applied to other processing systems and methods, not only for the systems and methods described above.

The elements and acts of the various embodiments described above can be combined to provide further embodiments. These and other changes can be made to the systems and methods in light of the above detailed description. In general, in the following claims, the terms used should not be construed to limit the systems and methods to the specific embodiments disclosed in the specification and the claims, but should be construed to include all processing systems that operate under the claims. Accordingly, the systems and methods are not limited by the disclosure, but instead the scope of the systems and methods is to be determined entirely by the claims.

Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise," "comprising," and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in a sense of "including, but not limited to." Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words "herein," "hereunder," "above," "below," and words of similar import refer to this application as a whole and not to any particular portions of this application. When the word "or" is used in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.