Technical domain

[0001] The present disclosure relates to displays, more particularly to a light- field projector for projecting an image, such as 3D displays providing correct monocular depth cues. In particular, the present disclosure relates to near-eye light-field projector for virtual and augmented reality goggles and glasses, and virtual and augmented reality applications.

Related art

[0002] Today's light-field displays use, among other concepts, sequential projection of a set of "always-in-focus" images through a set of spatially disparate virtual pinholes. This approach does not allow to reconstruct the wavefront of a virtual pixel because the image components are projected in a time sequence while the wavefront reconstruction requires interference and therefore coincidence of wave front components. Consequently, the perceived resolution is limited by the resolution or individual light-field components delivered to the imaging sensor, e.g. into an eye. The resolution is determined by the effective numerical aperture of the pinhole and by the difference between the focal plane of the image of the image source, i.e. the plane of the image pixels, and the plane where the imaging optics - usually a human eye - is focused. Hence, the expression "always in focus" is approximative as the optical path of light-field components have always finite aperture which, when it is large, carries finite depth of field of the imagery, or, when it is small, it reduces the resolution of the image source due to diffraction on an aperture. [0003] The same problem is present in light-field systems based on emissive displays and lens arrays. The individual pixels are decoupled and do not provide mutually coherent beams with mutually coupled phases which does not allow their interference while the finite aperture of the lenses in the lens arrays controls the resolution and depth of field of individual light-field components.

[0004] Light-field imagery is typically constructed by a plurality of images that are projected through spatially disparate pupils in the optical path. It is achieved in various ways including sequential light-field projection or by splitting and collimating light beams from an emissive display by a lens array.

[0005] These optical systems are apparently displaying virtual pixels in a different focal plane than that of the flat image source itself. The element which allows to promote the apparent focusing on a virtual pixel over real pixels on the image source is the content itself together with the large depth of field of each light-field component that is secured by a small pupil through which each light-field component is passing compared to the combined aperture of all individual spatially disparate sub-pupils. The finite aperture of each sub-pupil, on the other hand, limits the effective resolution of light-field imagery especially in locations far from the focal plane of the image of the image source itself.

Summary

[0006] The present disclosure concerns a light-field projector for projecting an image, the light-field projector comprising: an image source comprising an image generating device including a plurality of pixel components generating a plurality of modulated light beams; an imaging optical element projecting an image of the plurality of modulated light beams to a light-source image plane; projection optics configured to project the modulated light beams such as to define an eye-box and to form projector pixel images; the projection optics being further configured to form the projector pixel images at a projector image plane and projector virtual pixel image at a shifted plane. The imaging optical element is configured such that the position of the projector image plane coincides with the shifted plane.

[0007] Moreover, the image source can be controllable such as to adjust spatial distribution of the pixel components forming the image component to be projected by the light-field projector. The controllable image source allows for the modulated light beams to construct the intended light-field output.

[0008] The light-field projector disclosed herein exceeds the effective resolution limits of the known light-field projection systems.

[0009] The light-field projector allows the modulated light beams to preserve the qualities for the light-field created by known light-field projection systems, and, moreover, it allows controlling which focal plane coincides with the image plane of the image source, where the image resolution is limited only by the image source resolution and apertures in individual optical paths of each light- field component.

[0010] The disclosed light-field projector is more robust than conventional projection systems imitating focal depth purely by dynamic adaptation of focal distance of a flat image achieved by the action of varifocal elements based on eye-tracking information. These conventional systems do not generate a viewer-independent light-field, but mimic what an eye should see depending on its actual focus. [0011] In the invention disclosed here, the viewer-independent light-field feature is present at all times and provides correct accommodation cues without any additional action. The varifocal mechanism in the disclosed light- field projector allows on-demand promotion of resolution at desired focal plane in the light-field scene.

[0012] The light-field projector can be used for near-eye light-field displays for virtual and augmented reality goggles and glasses, and virtual and augmented reality applications.

Short description of the drawings

[0013] Exemplar embodiments of the invention are disclosed in the description and illustrated by the drawings in which:

Fig. 1 shows a light-field projector comprising a light-field image source, an imaging optical element and projection optics.

Fig. 2 illustrates the light-field projector comprising the imaging optical element according to an embodiment and wherein the light-field image source comprises an array of time-sequentially activatable point-lights;

Fig 3a illustrates the light-field projector wherein the light-field image source comprises an emissive display;

Fig 3b illustrates the light-field projector of Fig 3a comprising the imaging optical element according to an embodiment;

Fig 3c illustrates the light-field projector of Fig 3a comprising the imaging optical element according to another embodiment; and Fig. 4 shows the light-field projector where the light-field image source comprises a coherent light-source illuminating a reflective or transmissive and diffractive display, according to an embodiment.

Examples of embodiments

[0014] Fig. 1 shows a light-field projector comprising an image source 1. The image source comprises a light-source 10 comprising an array of time- sequentially activatable point-lights (also called pin- lights), each point-light generating a light beam 110. In Fig. 1, two point-lights 10a and 10b are shown. The light-field projector further comprises a spatial light modulator (SLM) 20a comprising a plurality of pixel components 15a, 15b, each modulating a light beam component 111 reflected and diffracted on the pixels such as 15a and 15b.

[0015] In the example of Fig. 1, pixel component 15a is active for point-light 10a, and pixel component 15b is active for point-light 10b. In practice however, the light-source 10 can comprise any number, for example tens or hundreds of spatially disparate point-lights 10a, 10b that are illuminated in a time-sequence. The spatial disparity can be only apparent, generated by a single light-source or smaller array of light-sources that is multiplicated by another active element, such as a beam steering mirror or a phase spatial light-modulator. The function of the light-source array can be also substituted by an active phase array. The SLM 20a can comprise complex image patterns having thousands or millions of active pixel components 15a, 15b.

[0016] The image source 1 further comprises an imaging optical element 70 projecting an image of the light source 10 to a light-source image plane 34 of the pin-light source. The imaging optical element 70 can further collimate the modulated light beams 111.

[0017] The light-field image source 1 can further comprise a collimating element 50 collimating the light beams 110 on the SLM 20a. Here, the combination of the first and second optical elements 50, 70 projects the image of the light source 10 to a light-source image plane 34 of the pin-light source.

[0018] In one aspect, the light-source image plane 34 can comprise a passive or active Fourier filter. The Fourier filter can be formed from an array of pin-holes or an array of optical shutters. The array of optical shutters may comprise a fast modulator such as DMD, FLCOS, a phase change material shutter, or any other device acting as optical shutters. The light-source image plane 34 can be considered as the entrance pupil where a fully constructed light-field enters a projection optics.

[0019] The light-field projector further comprises projection optics 2 configured to project the modulated light beams 111 such as to define an eye-box 121.

The eye-box 121 can be defined along a projection axis 170 of the light-field projector, but more generally, the eye-box 121 is defined in the direction of the projected modulated light beams 111. More particularly, projection optics 2 can comprise a first projection element 32 configured to project the modulated light beams 111 such as to form a projector image plane 115, and such as to form projector pixel images 16a, 16b of the pixel components 15a and 15b, respectively, in the projector image plane 115. Fig. 1 shows two projector pixel images 16a and 16b of the pixel components 15a, 15b, respectively. [0020] Projection optics 2 can further comprise a second projection element 40 configured to project the image of the projector image plane 115, to any chosen location in the direction of the projected modulated light beams 111. The second projection element 40 can comprise an eye-piece or a combiner. In the example of Fig. 1, the image of the projector image plane 115 is located at a "neutral" position such that an eye-box projector image, i.e. the image of the pixel components 15a, 15b as seen from the eye-box 121, is at infinity. In Fig. 1, the eye-box projector image is not shown but its position at infinity is indicated by the collimated modulated light beams 111.

[0021] In Fig. 1, the two exemplary modulated light beams 111 from the pixel 15a and 15b intersect at a projector virtual pixel image 26a that differs from the projector image plane 115. The projector virtual pixel image 26a corresponds to an image of a virtual pixel in the light-field projector before the second projection element 40. Consequently, an eye-box virtual pixel image 24 is formed by the apparent intersection of the pixel beams as seen from the eye- box 121. The eye-box virtual pixel image 24 is located at a different distance than the eye-box projector image plane (not shown) which is in infinity in this example.

[0022] The light-field projector can be configured to project images to an imaging optic device, such as a camera or the eye 90 of a viewer as shown in Fig.1. The exit pupil of the light-field projector may coincide with an entrance pupil of the imaging optic device, for example with the eye pupil 130 of a viewer's eye 90. [0023] When the light-field projector projects images towards the eye 90 of a viewer, the lens 130 of the eye 90 is focused on the location of the eye-box virtual pixel image 24. Thus, the lens 130 projects the modulated light beams 111 at the same location, here a sensor virtual pixel image 27a near the eye's retina 92. Note that the light-field projector may as well project images towards a camera. A lens of the camera would then project the modulated light beams 111 at a sensor virtual pixel image 27a near a camera's sensor.

[0024] Sensor pixel images 17a, 17b of the pixel components 15 are formed before the retina 92 (or sensor) at a different distance than the sensor virtual pixel image 27a. The sensor pixel images 17a, 17b correspond to a second image of, respectively, the pixel component 15a and 15b in the viewer's eye 90 or in an imaging optic device.

[0025] In some aspects, the light-field projector may also be configured to project a content or react to content-related instructions in order to promote the resolution of virtual objects of interest, for instance of a virtual text at a certain distance.

[0026] Fig. 2 illustrates a light-field projector, according to an embodiment, wherein the imaging optical element 70 is configured to move the position of the projector image plane 115 such that the position of the projector image plane 115 can coincide with the position of the projector pixel image 16a, 16b.

[0027] In the example of Fig. 2, the imaging optical element 70 comprises a lens that is movable. In the present embodiment, the lens 70 is moved towards the SLM 20a. In comparison with Fig. 1, the projector image plane 115 is shifted to a shifted plane 114 where the projector pixel image 16a, 16b coincides with the projector virtual pixel image 26a. In other words, the projector pixel image 16a,

16b is located where the modulated light beams 111 from the pixel 15a and 15b intersect.

[0028] Other configurations of the imaging optical element 70 allowing to move the position of the projector image plane 115 to the shifted plane 114 are possible. For example, the imaging optical element 70 can comprise an optical element having a variable focal length, such as an optical element comprising an element with variable geometry, a medium whose refractive index is variable, an electro-optic material, a birefringent element with polarization rotating element such as a liquid crystal or other elements doing the same.

[0029] Moreover, the SLM 20a can be controlled such as to adjust the spatial distribution of the pixel components 15a, 5b forming the image component to be projected by the light-field projector. As can be seen in Fig. 2, the pixel components 15a and 15b are distributed differently (closer) in the SLM 20a compared to the configuration of Fig. 1. The adjustment of the active pixel components 15a, 15b forming the image component allows constructing the intended light-field output and the coincidence of the projector pixel image 16a, 16b with the projector virtual pixel image 26a.

[0030] As shown in Fig. 2, the eye-box virtual pixel image 24 is formed by the apparent intersection of the modulated light beams 111 as seen from the eye- box 121. The location of the eye-box virtual pixel image 24 coincides with that of the eye-box projector images 18a, 18b.

[0031] When the light-field projector projects images towards a viewer eye 90, the lens 130 of the eye 90 is focused on the location of the eye-box virtual pixel image 24. The lens 130 projects the modulated light beams 111 at the same location, here a sensor virtual pixel image 27a near the eye's retina 92. Sensor pixel images 17a, 17b of the pixel components 15 are also formed near the eye's retina 92 and coincide with the sensor virtual pixel image 27a.

[0032] The perceived resolution of the virtual imagery in the focal distance of interest, i.e. in the distance of the eye-box virtual pixel image 24 is essentially limited only by the resolution of the image source on the SLM 20a.

[0033] In one aspect, the light-source image plane 34 can comprise a Fourier filter especially when a highly diffractive SLM 20a is used. Here, the light- source image plane 34 can be shifted along the projection axis 170, towards the SLM 20a, in order to make the position of the projector image plane 115 coincide with the position of the projector pixel image 16a, 16b.

[0034] Fig. 3a shows the light-field projector, according to another embodiment wherein the image source 1 comprises an emissive display 20b. Here, the light- field projector is based on decomposition of an emissive display into modulated light beams 111 with collimated pixel beams by a lens array. The emissive display 20b can comprise a liquid crystal display (LCD), organic light- emitting diode (OLED), microLED, or other suitable displays. The pixel components 15a, 15b of a virtual pixel 24 are directly emitted by the emissive display 20b and are collimated by the imaging optical element 70 located in the light-source image plane 34 or entrance pupil. Here, the imaging optical element 70 comprises an array of collimating optical elements, such as an array of lenses. The imaging optical element 70 can further act as a Fourier filter and determine the pinhole apertures corresponding to the individual modulated light beams 111 for each pixel components 15a, 15b.

[0035] Similarly to the light-field projector configuration of Figs. 1 and 2, projection optics 2 can comprise the first projection element 32 and the second projection element 40.

[0036] In Fig 3a, the lens 130 of the viewer's eye 90 is focused on the projector image plane 115 in a neutral position, such that the position of an eye-box projector image, the image of the pixel components 15a, 15b as seen from the eye-box 121, is at infinity. The eye-box projector image (the image of the projector SLM plane ) is not shown in Fig. 3a but its position at infinity is indicated by the collimated modulated light beams 111 entering the eye-box 121.

[0037] The projector virtual pixel image 26a and the sensor virtual pixel image 27a correspond to the intersection of the modulated light beams 111 from the pixel 15a and 15b, and the projector virtual pixel image 26b and sensor virtual pixel image 27b correspond to the intersection of the modulated light beams 111 from the pixel 15c and 15d. The projector pixel images 16a, 16b, 16c and 16d correspond to the images (focal points) of the modulated light beams 111 in the projector image plane 115 from the pixel 15a, 15b, 15c and 15d, respectively. The sensor pixel images 17a, 17b, 17c and 17d correspond to the a second image of, respectively, the pixel component 15a, 15b, 15c and 15d, respectively, in the viewer's eye 90 or in an imaging optic device.

[0038] In the configuration of Fig. 3a, none of the projector virtual pixel images 26a, 26b coincides with the projector pixel images 16a, 16b, 16c and 16c, and none of the sensor virtual pixel images 27a, 27b coincides with the sensor pixel images 17a, 17b, 17c and 17d.

[0039] Fig. 3b illustrates the light-field projector of Fig 3a wherein the imaging optical element 70 has been shifted from its "neutral" position shown in Fig. 3a towards the emissive display 20b to a position such as to shift the projector image plane 115 to the shifted plane 114 closer to the emissive display 20b. Moreover, the emissive display 20b is controlled such as to adjust the active pixel components 15 forming the image component to be projected by the light-field projector. As can be seen in Fig. 3b, the pixel components 15a and 15b are distributed differently (closer from each other) in the emissive display 20b compared to the pixel components 15c and 15d.

[0040] As a result, the projector pixel images 16a, 16b coincide with the projector virtual pixel image 26a. When the light-field projector cooperates with a viewer the eye 90, the lens 130 projects the modulated light beams 111 at the sensor virtual pixel image 27a near the eye's retina 92 and the sensor pixel images 17a, 17b are formed near the eye's retina 92 and coincide with the sensor virtual pixel image 27a.

[0041] Since the pixel components 15c and 15d are not adjusted in the emissive display 20b, the projector pixel images 16c, 16d do not coincide with the projector virtual pixel image 26a and the sensor pixel images 17a, 17b do not coincide with the sensor virtual pixel image 27b.

[0042] Again, the imaging optical element 70 can comprise a varifocal optical element configured to move the position of the projector image plane 115 such that the position of the projector image plane 115 can coincide with the position of the projector pixel image 16a, 16b.

[0043] Fig. 3c shows the light-field projector from Fig 3a wherein the imaging optical element 70 has been shifted from its "neutral" position shown in Fig. 3a closer to the emissive display 20b to a position such as to shift the projector image plane 115 to the shifted plane 114 farther from the emissive display 20b. Moreover, the emissive display 20b is controlled such as to adjust the active pixel components 15a-15d forming the image component to be projected by the light-field projector. As can be seen in Fig. 3b, the pixel components 15c and 15d are distributed differently (spaced farther apart) in the emissive display 20b compared to the pixel components 15a and 15b.

[0044] As a result, the projector pixel images 16a, 16b coincide with the projector virtual pixel image 26a. When the light-field projector projects images towards a viewer's eye 90, the lens 130 projects the modulated light beams 111 at the sensor virtual pixel image 27a near the eye's retina 92 and the sensor pixel images 17c, 17d are formed near the eye's retina 92 and coincide with the sensor virtual pixel image 27b. Since the pixel components 15a and 15b are not adjusted in the emissive display 20b, the projector pixel images 16a, 16b do not coincide with the projector virtual pixel image 26a and the sensor pixel images 17a, 17b do not coincide with the sensor virtual pixel image 27a.

[0045] Fig. 4 shows the light-field projector according to another embodiment where the image source 1 comprises a coherent monochromatic light-source 11 projecting mutually coherent light beams illuminating a transmissive or reflective and diffractive display 20b, such as in the configurations of Figs. 3a and 3b. In the implementation Fig. 4, the light-field projector functions as a wavefront shaping projection system.

[0046] For instance, the coherent monochromatic light-source 11 can comprise a laser. The display 20b can be transmissive or reflective modulating amplitude and possibly phase of the incident light. The display 20b acts as an SLM and diffracts the light beam of the coherent light-source 11 at each pixel and widens the light beam before each of the lenses of the lens array at the light-source image plane 34. The light beam of the coherent light-source 11 can be optionally collimated by a collimating element 50 and modulated at the display 20b acting as an SLM.

[0047] The imaging optical element 70 comprises an array of collimating optical elements, such as an array of lenses. The imaging optical element 70 can further act as a Fourier filter and determine the pinhole apertures corresponding to the individual light-field component images for each pixel components 15a, 15b. The Fourier filtering is however not needed in this embodiment.

[0048] The modulated light beams 111 exiting the image source 1 through the different lenses of the imaging optical element 70, and possibly the light-source image plane 34, are mutually coherent. The modulated light beams 111 are thus able to interfere at the points of their incidence where they construct a wavefront composed of multiple wavefront components. It is then possible to reconstruct a wavefront of an arbitrary virtual pixel when the light-field components coincide in the vicinity or exactly at the sensor of an imaging optic device, for example the retina 92 of an eye 90. [0049] The wavefront construction is illustrated in the zoomed-in part of Fig. 4, where arrows represent the wave vectors and lines the wavefronts of individual light-field components. The plane waves are considered only for illustration. In reality, each component is approximately spherical with mutually disparate origins. The coherent light-field components construct a spherical wave which has an apparent origin in a virtual point source, representing the virtual pixel.

Reference Numbers and Symbols

1 image source

2 light-field projection optics

10 light source

10a, 10b point-light

11 coherent monochromatic light-source

15a-15d pixel component

16a-16d projector pixel image

17a-17d sensor pixel image

18a, 18b eye-box projector image

20a spatial light modulator

20b display

24 eye-box virtual pixel image

26a projector virtual pixel image

26b projector virtual pixel image

27a sensor virtual pixel image

27b sensor virtual pixel image

32 first projection element, lens

34 light-source image plane, Fourier filter

40 second projection element

50 collimating element

70 imaging optical element

90 imaging optic device, eye

92 sensor, retina

110 light beam

111 modulated light beam

114 shifted plane

115 projector image plane

121 eye-box

130 eye pupil, lens