BACKGROUND

Field of the Disclosure

The present disclosure relates generally to stereoscopic near-eye displays, and more particularly to providing visual cues for depth perception in stereoscopic near-eye displays.

Description of the Related Art

In a conventional stereoscopic near-eye virtual reality (VR) display system, two- dimensional (2D) VR imagery is focused at a fixed depth from the user's eyes, whereas the depth of the objects in the 2D VR imagery may vary. This results in conflicting information between the vergence cues and accommodation cues utilized by the human visual system. This conflict, frequently referred to as the "accommodation-vergence conflict," often can lead to eye fatigue, headache, or other user discomfort. Moreover, the 2D VR imagery typically is rendered at only one fixed focal plane, and thus the user typically cannot choose to accommodate on a different focal plane as the user otherwise would be able to in an actual 3D environment. This inability to change focal planes with the appropriate

corresponding visual cues negatively impacts the user's sense of immersion in the virtual environment represented by the VR imagery.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings. The use of the same reference symbols in different drawings indicates similar or identical items.

FIG. 1 is diagram illustrating a near-eye display system utilizing a modulated varifocal liquid membrane lens synchronized with the display of focal ly-rendered imagery to provide a perceived sense of multiple concurrent focal planes in displayed VR imagery in accordance with some embodiments.

FIG. 2 is a diagram illustrating example waveforms representing a dual-focal- depth implementation of the near-eye display system of FIG. 1 in accordance with some embodiments.

FIG. 3 is a diagram illustrating the two perceived focal lengths obtained by the implementation of FIG. 2 in accordance with some embodiments.

FIG. 4 is a diagram illustrating example waveforms representing a quad-focal- depth implementation of the near-eye display system of FIG. 1 in accordance with some embodiments.

FIG. 5 is a diagram illustrating the four perceived focal lengths obtained by the implementation of FIG. 4 in accordance with some embodiments.

FIG. 6 is a diagram illustrating a hardware implementation of the near-eye display system of FIG. 1 in accordance with some embodiments.

FIG. 7 is a block diagram illustrating a method for synchronized modulation of a varifocal liquid membrane lens assembly to provide multiple concurrent perceived focal planes in accordance with some embodiments.

DETAILED DESCRIPTION

FIGs. 1 -7 illustrate techniques for resolving the accommodation-vergence conflict often found in stereoscopic near-eye VR displays through the modulation of a pair of liquid membrane lenses through which a user views VR image content displayed on a near-eye display assembly. The pair of liquid membrane lenses are modulated so as to cyclically and continuously vary the focal power exhibited by the liquid membrane lenses. Further, in some embodiments, the sequence of display frames representing the VR image content are focally rendered such that the sequence alternates or iterates between frames rendered for different discrete focal lengths of a set of focal lengths. The modulation of the liquid membrane lenses may be synchronized to the display frame clock signal used to control the timing of the display of the display frames of the sequence. Thus, the varying focal length of the lens system employing the liquid membrane lenses is synchronized to the focal planes or focal lengths for which the corresponding frames were rendered. As the frames may be displayed at a rate greater than the persistence-of-vision threshold, multiple frames rendered for different focal lengths are perceived by the user as a single image having multiple concurrent focal planes in the resulting 3D VR content. These multiple focal planes provide more effective focus cues for the user's eyes, and thus facilitate an enhanced sense of immersion in the VR scene.

FIG. 1 illustrates an example near-eye display system 100 utilizing synchronized modulation of a varifocal liquid membrane lens assembly synchronized with display of correlated focally rendered display frames in accordance with at least one embodiment. The near-eye display system 100 comprises a display assembly 102 and a lens assembly 104 mounted on or near a user's head 106 (e.g., as a head-mounted display (HMD)). The display assembly 102 comprises one or more display panels, with each display panel mounted at a corresponding fixed distance from the eyes 108 of the user. In the illustrated embodiment, the display assembly 102 comprises a single display panel 1 10 that is logically divided into separate left and right regions, with the left region used to display VR imagery for the left eye of the user, and the right region used to display VR imagery for the right eye of the user. In other embodiments, the display assembly 102 may comprise two display panels 1 10, one for each eye 108. In yet other implementations, the display assembly 102 may be implemented using more than two display panels 1 10.

The lens assembly 104 comprises a set of lenses or other optical elements for each eye 108 of the user. Each set of lenses or other optical elements includes a liquid membrane lens 1 12 disposed between the display assembly 102 and the corresponding eye 108 of the user and thus facing the display panel 1 10. The set further may include one or more fixed focal power lenses aligned with the liquid membrane lens 1 12, with the one or more fixed focal powered lenses providing most of the focusing power of the set, while the liquid membrane lens 1 12 modulates the focus by a relatively small amount (e.g., about 1 to 2 diopters). Although FIG. 1 illustrates only one liquid membrane lens 1 12 for one eye 108 for ease of illustration, it should be appreciated that another liquid membrane lens 1 12 would be provided for the other eye 108 and would be implemented and operated in the same manner described herein.

As the name suggests, a liquid membrane lens employs a membrane containing one or more liquids (fluid) at an aperture of the lens, and shape, size, or other physical characteristic of the membrane is modified through application of a current or a voltage. This modification to the membrane or fluid(s) contained therein results in a corresponding change in the focal power of the liquid membrane lens. This change may be implemented using an electro-optical approach in which the membrane contains two fluids with different refractive indices (e.g., oil and water), and thus forming a diopter, and the radius of curvature of the meniscus between the two fluids is changed through application of an electrostatic field, thereby resulting in a corresponding change in the focal length of the lens. In an electro-optical implementation, a current is applied to a ring surrounding the membrane, which causes the ring to constrict or expand based on the magnitude of the current, which in turn deforms the membrane and thus causing a change in the focal power. Alternatively, a voice bobbin may be used to press against a periphery of the membrane responsive to application of current or voltage, and the resulting deformation in the membrane modifying the focal power of the lens.

In a conventional application of liquid membrane lenses, the current applied to the lens is changed as relatively large discrete steps to step change the focal power of the lens from one focal length to another focal length. However, when a liquid lens membrane is subjected to this discrete change, its physical properties are such that the liquid lens membrane exhibits a dampening or settling effect, which typically results in the liquid lens membrane having a relatively long switching time (e.g., 16 milliseconds (ms)) that generally is unsuitable for VR applications.

However, when the liquid membrane lens is controlled so as to continuously change the focal length (that is, without large step changes) via application of a sinusoidal signal or other similar continuously varying waveform, while the damping effect is still present the liquid membrane lens 1 12 responds with a focal modulation that has the same frequency and is temporally phase shifted from the driving signal. As such, this is readily compensated for by phase-shifting a driving signal for the liquid membrane lens 1 12 so that the liquid membrane lens 1 12 and display rendering are appropriately synchronized. In particular, the

membrane/fluid configuration of a liquid membrane lens causes the liquid membrane lens to exhibit a resonant frequency, and when modulated at this resonant frequency the liquid membrane lens is particularly efficient in changing its focal power. To illustrate, the EL-16-40-TC liquid membrane lens available from Optotune Switzerland AG exhibits a resonant frequency of approximately 400 hertz at a driving signal cycling between -50 milliamps (mA) and +50 mA, and when driven at this resonant frequency, exhibits an ability to change focal states in 1 .3 ms. This ability, coupled with the use of a low-persistence display system in which the pixels of each frame are illuminated for only a short time compared to the varifocal modulation period of the liquid membrane lens 1 12, enables the creation or perception of a nearly constant focal plane for each frame rendered and displayed.

The near-eye display system 100 utilizes this fast focal power switching exhibited by the liquid membrane lens 1 12 when modulated to enable the lens assembly 104 to cyclically vary the overall focal power of the lens assembly 104 while providing focally-rendered imagery synced to the change in the focal power. To this end, the system 100 includes a lens driver 1 14 to generate at a driving signal 1 16 (also denoted herein as "driving signal S") that comprises a continuously varying cyclical/periodic signal having at least one non-zero frequency

component. This driving signal 1 16 is input to each of the liquid membrane lenses 1 12, which in turn causes each liquid membrane lens 1 12 to vary its focal power accordingly. Typically, there is an approximately linear relationship between the amperage of the driving signal 1 16 and the focal power exhibited by the liquid membrane lens 1 12, and thus the focal power of the liquid membrane lens 1 12 approximately follows the amplitude of the driving signal 1 16.

Concurrently, a rendering component 1 18 renders, for each eye of the user, a corresponding sequence 121 of display frames for display by the display assembly 102 via a display driver 122 of the system 100, wherein the sequence 121 of display frames comprising 3D VR image content to be displayed. The rendering component 1 18 employs focal-based rendering such that each rendered frame is rendered for a corresponding focal length of a set of specified focal lengths, and such that the sequence 121 of display frames cyclically sequences through the set of focal lengths. For each display frame of the sequence 121 , the display driver 122 controls the display assembly 102 to display the display frame in accordance with a frame clock signal 124 that sets the frame rate for the display assembly 102. As described in greater detail herein, the driving signal 1 16 is synchronized to the frame clock signal 124 such that frame clock signal 124 has a frequency that is an integer multiple of the frequency of the driving signal 1 16 so that there is a fixed phase relationship between the two signals and such when a display frame rendered for a particular focal plane is displayed, the liquid membrane lens 1 12 exhibits a focal power correlating to that focal plane at that time. As a result, the instantaneous focal power of the liquid membrane lenses 1 12 and the rendering and display of VR imagery at the display assembly 102 are synchronized such that the display frames displayed at the display assembly 102 are rendered so as to cycle through a set of different focal planes/lengths, and the liquid membrane lenses 1 12 of the lens assembly 104 are modulated so as to provide, at the time of display of each display frame, a focal power correlated to the focal plane/length for which the display frame was rendered.

FIG. 2 illustrates an example of this synchronized operation between the focal- plane-rendered display frames and the periodic modulation of the focal power of the liquid membrane lenses 1 12 of the near-eye display system. FIG. 2 in particular shows a set of waveforms, including a varifocal power waveform 201 , a frame display waveform 202, and a perceived depth waveform 203. The varifocal power waveform 201 represents the focal power of the liquid membrane lens 1 12 over time. In this example, the driving signal 1 16 output by the lens driver 1 14 is composed of a sinusoidal signal having a frequency f1 , and due to the

approximately linear relationship between the driving signal 1 16 and the focal power response of the liquid membrane lens 1 12, the resulting modulation of the focal power of the liquid membrane lens 1 12 has a sinusoidal form with the frequency f1 . That is, the amplitude and frequency of the varifocal power waveform 201 exhibited by the liquid membrane lens 1 12 substantially tracks the amplitude and frequency of the driving signal 1 16 input to the liquid membrane lens 1 12. As noted above, a liquid membrane lens typically operates

considerably more efficiently when modulated at its resonant frequency, or a harmonic thereof (that is, integer multiple of the resonant frequency). Accordingly, to conserve power consumption, in at least one embodiment a liquid membrane lens having a harmonic frequency at or near the frequency f1 may be selected or manufactured for use as the liquid membrane lens 1 12. To illustrate, there typically is a relatively simple relationship between the membrane properties and geometry and the fundamental resonant frequency of a liquid membrane lens or other vibrating circular membranes, as represented in the expression: where f _r represents the fundamental resonant frequency, T represents the tension of the membrane in Newtons/meter, σ represents the density of the membrane in kilograms per square meter, and D represents the diameter of the membrane in meters. Based on this expression, a liquid membrane lens may be selected or manufactured with the appropriate tension, diameter, and density to provide a resonant frequency near or equal to the intended frequency of modulation of the liquid membrane lens 1 12.

The frame display waveform 202 represents the timing of the display of the sequence 121 of display frames at a frequency f2 (e.g., 60 hertz (Hz), 120 Hz, 240 Hz, etc.). In at least one embodiment the display assembly 102 employs one or more display panels 1 10 that provide low persistence display of display frames. In such low-persistence display panels, the duration at which the display panel emits light for a particular display frame is substantially less than the overall period between display frames. To illustrate, for a 120 Hz frame rate, a display frame is displayed every 8.3 ms; however, rather than displaying the display frame for most or all of the 8.3 ms duration, in a low-persistence display panel the display frame may be displayed for only a portion of the 8.3 ms duration, such as, for example, only 4.0 ms. The manner in which the display of a frame is controlled depends on the particular technology of the display panel. In backlighted display panels, such as liquid crystal display (LCD)-based and light emitting diode (LED)-based display panels, the actual duration of display of a frame is controlled by the activation of a backlight which is used to provide the light source for the pixels of the display panel. In organic LED (OLED)-based display panels, each pixel serves as its own separate light source, and thus the duration of display of a frame is controlled by the activation of the pixels themselves. In the particular example of FIG. 2, the frame display waveform 202 represents the activation state of a backlight of the display panel 1 10, such that when the waveform 202 is "high" or at state 1 , the backlight is activated and thus a frame is being displayed, and when at "low" or state "0" the backlight is deactivated and thus a frame is not being displayed. A similar waveform would be exhibited for the activation of pixels for each display frame in an OLED-based implementation of the display panel 1 10. As noted above, this low persistence of illumination for the pixels of any given frame being displayed results in a frame being illuminated for a short time compared to the period or duration of the cycle in the focal power of the liquid membrane lens 1 12 as it is modulated as shown by waveform 201 , results in the focal power of the liquid membrane lens 1 12 being effectively nearly constant for the duration of the display of the frame, and thus providing an effectively constant overall focal power for the lens assembly 104 for the duration of the display of that frame. In at least one embodiment, the display frame rate (that is, the frequency f2 of the frame display waveform 202) is an integer multiple of the frequency f1 of the varifocal power waveform 201 (that is, f2 = N x f1 , where N is an integer greater than one), and thus there is a fixed phase relationship between the waveforms 201 , 202. This fixed phase relationship, in effect, synchronizes the instantaneous focal power of the liquid membrane lens 1 12 to the timing of the display of display frame such that display frames are displayed at the same instantaneous focal power(s) in each cycle. To illustrate, in the depicted example the waveforms 201 , 202 are synchronized such that each display frame is actively displayed either at a point in the cycle where the focal power of the liquid membrane lens 1 12 is at its maximum focal power (+2 diopters in this example) or at a point in the cycle where the focal power at its minimum focal power (-2 diopters in this example). Thus, if numbering each displayed frame in the waveform 202 from left to right starting at 1 , frames 1 , 3, 5, 7, and 9 are actively displayed at the maximum peak of the varifocal power waveform 201 , whereas frames 2, 4, 6, 8, and 10 are actively displayed at the minimum peak of the varifocal power waveform 201 . It should be appreciated that while this particular configuration provides for the display frames to be actively displayed during the turning points, or zero-derivative points, in the varifocal power waveform 201 as there is a larger margin for timing error at these points, this particular phase relationship is not required. To illustrate, the phase relationship between the waveforms 201 , 202 may be configured such that frames 1 , 3, 5, 7, and 9 are actively displayed when the varifocal power waveform is at +1 diopters, while frames 2, 4, 6, 8, and 10 are actively displayed when the varifocal power waveform is at -1 diopters.

The impact of this synchronization on the user is represented by the perceived depth waveform 203, which illustrates the focal length perceived by the user for each displayed frame when viewed through the liquid membrane lens 1 12. As shown, for frames 1 , 3, 5, 7, and 9, the perceived focal length is +2 diopters, and for frames 2, 4, 6, 8, and 10, the perceived focal length is -2 diopters.

Accordingly, the rendering component 1 18 leverages this alternating perceived focal length to render each frame in the sequence 121 for a focal plane correlating to the perceived focal length that will occur at the time the frame is displayed and viewed by the user through the liquid membrane lens 1 12. To illustrate, assuming that the liquid membrane lens 1 12 is the only lens impacting focal length, frames 1 , 3, 5, 7, and 9 are rendered based on a focal plane "A" that corresponds to the perceived focal length of +2 diopters, whereas frames 2, 4, 6, 8, and 10 are rendered based on focal plane "B" that corresponds to the perceived focal length of -2 diopters. As a result, the sequence 121 of frames repeatedly alternates between a frame rendered for the focal plane "A" and a frame rendered for the focal plane "B". Consequently, due to the persistence-of-vision effect of the human vision system, the user will concurrently perceive two distinct focal planes with the corresponding appropriate focal cues, and thus provide an improved immersive experience over conventional near-eye display systems that provide only a single perceived focal plane.

This perceived concurrent dual focal length result is depicted more fully by FIG. 3, which shows a side view of the lens assembly 104 and display panel 1 10 of the near-eye display system 100 implementing the configuration represented by the waveforms 201 -203 of FIG. 2. In this example, the lens assembly 104 includes a fixed focal power lens 302 in line with the liquid membrane lens 1 12, with the fixed focal power lens 302 providing much of the overall focusing power of the lens assembly 104 so as to minimize aberrations. As shown in FIG. 3, the display panel 1 10 is at a fixed distance from the eye 108 and thus, when the liquid membrane lens 1 12 is not being modulated, is perceived as being at a fixed focal plane (designated focal plane "0" in FIG. 3) by the user. However, when the liquid membrane lens 1 12 is modulated as shown in the varifocal power waveform 201 and the sequence 121 of frames is focally-rendered and displayed within the persistence-of-vision threshold as shown with reference to the frame display waveform 202, two concurrent focal lengths, or focal planes, are perceived by the user. To illustrate, as noted above, frame 1 of the sequence 121 is rendered based on a focal plane A (which accounts for the fixed focal power of the lens 302 as well), whereas frame 2 of the sequence is rendered based on a focal plane B. Thus, when frame 1 is displayed, the liquid membrane lens 1 12 exhibits a focal power correlated to the focal plane A, and when frame 2 is subsequently displayed, the liquid membrane lens 1 12 exhibits a focal power correlated to the focal plane B. However, due to visual persistence, frames 1 and 2 are perceived as effectively displayed concurrently, and as each was rendered at, and perceived at, a different focal plane, the user in effect perceives the displayed imagery has having two focal planes, A and B.

Although FIGs. 2 and 3 illustrate an example implementation whereby the modulation of the liquid membrane lenses 1 12 and corresponding synchronous display of a focally-rendered sequence 121 of display frames results in user perception of two concurrent focal planes, similar techniques may be used to provide more than two concurrent focal planes. To illustrate, FIGs. 4 and 5 depict an example implementation that provides four perceived concurrent focal planes. Although the principles described herein may be utilized to provide more than four focal planes, the human eye has a typical accommodation range of 0-4 diopters and a 1 diopter depth-of-focus (DOF), and thus 2-4 discrete concurrent focal planes should be sufficient to yield a realistic varifocal display. FIG. 4 depicts a set of waveforms, including a varifocal power waveform 401 , a frame display waveform 402, and a perceived focal depth waveform 403. As with waveform 201 of FIG. 2, the varifocal power waveform 401 represents the cyclically and continuously varying focal power of the liquid membrane lens 1 12 over time. In this example, the driving signal 1 16 output by the lens driver 1 14 is composed of two sinusoidal signals of different amplitudes and different frequencies (e.g., f1 and 2*f1 in this example), with the resulting combination of these two sinusoidal signals modulating the liquid membrane lens 1 12 to have an absolute maximum focal power 404, an absolute minimum focal power 406, a local maximum focal power 408, and a local minimum focal power 410 for each cycle of the resulting waveform. As noted above, the liquid membrane lens 1 12 may be selected or designed so that the frequency f1 is at or near the resonant frequency of the liquid membrane lens, or a harmonic thereof. The frame display waveform 202 represents the timing of the display of the sequence 121 of display frames at a frequency f2, which in this example is four times the frequency f1 (f2 = 4 x f1 , N=4) and with the illustrated fixed phase relationship.

As a result, the waveforms 401 , 402 are synchronized so as to have a fixed phase relationship so that each display frame of the sequence 121 is actively displayed at one of the maximums or minimums of the cyclical focal power of the liquid membrane lens 1 12. In the particular example, if numbering each displayed frame in the waveform 402 from left to right starting at 1 , frames 1 , 5, 9, 13, and 17 are actively displayed at the absolute maximum focal power 404 of their respective cycles of the waveform 402, frames 2, 6, 10, 14, and 18 are actively displayed at the local minimum focal power 410 of their respective cycles of the waveform 402, frames 3, 7, 1 1 , 15, and 19 are actively displayed at the local maximum focal power 408 of their respective cycles of the waveform 402, and frames 4, 8, 12, 16, and 20 are actively displayed at the absolute minimum focal power 406 of their respective cycles of the waveform 402. As similarly noted above, while the example of FIG. 4 provides for the display frames to be actively displayed during the turning points, or zero-derivative points, in the varifocal power waveform 401 , this particular phase relationship is not required. As shown by the perceived focal length waveform 403, as a result of this synchronization the focal power perceived by the user through the liquid membrane lens 1 12 cycles from approximately +3 diopters, to -1 diopter, to +1 diopter, to -1 diopter as the sequence 121 of display frames progresses.

Accordingly, the rendering component 1 18 renders the display frames of the sequence 121 based on a select one of four different focal planes, with the particular focal plane selected for a display frame corresponding to the perceived focal power at the time that that frame is to be displayed. To illustrate, assuming that the liquid membrane lens 1 12 is the only lens impacting focal length, frames 1 , 5, 9, 13, and 17 are rendered based on a focal plane "A" that corresponds to the perceived focal length of +3 diopters, frames 2, 6, 10, 14, and 18 are rendered based on a focal plane "B" that corresponds to the perceived focal length of -1 diopters, frames 3, 7, 1 1 , 15, and 19 are rendered based on a focal plane "C" that corresponds to the perceived focal length of +1 diopters, and frames 4, 8, 12, 16, and 20 are rendered based on a focal plane "D" that corresponds to the perceived focal length of -3 diopters. As a result, the sequence 121 of frames repeatedly sequences between a frame rendered for the focal plane "A" a frame rendered for the focal plane "B", a frame rendered for the focal plane "C", and a frame rendered for the focal plane "D." If these four frames in each sequence are displayed within the persistence threshold of the human vision system, the user will concurrently perceive four distinct focal planes with the corresponding appropriate focal cues, with the improved immersive experience that entails.

FIG. 5 shows this perceived concurrent quad focal plane arrangement in more detail. In particular, FIG. 5 shows a side view of the lens assembly 104 and display panel 1 10 of the near-eye display system 100 implementing the configuration represented by the waveforms 401 -403 of FIG. 4. In this example, the lens assembly 104 includes a fixed focal power lens 502 in line with the liquid membrane lens 1 12. While the display panel 1 10 is at a fixed distance from the eye 108 (i.e., at focal plane "0"), when the liquid membrane lens 1 12 is modulated as shown in the varifocal power waveform 401 and the sequence 121 of frames is focally-rendered and displayed as shown with reference to the frame display waveform 402, four concurrent focal lengths, or focal planes, are perceived by the user. To illustrate, as noted above, frame 1 of the sequence 121 is rendered based on the focal plane A, frame 2 is rendered based on the focal plane B, frame 3 is rendered based on the focal plane C, and frame 4 is rendered based on the focal plane D. Thus, when frames 1 -4 are displayed in sequence, with the liquid membrane lens 1 12 providing the correlated focal power at the time of display of each frame, frames 1 -4 are effectively perceived by the user as displayed concurrently, and displayed at their respective focal lengths, and thus the user perceives frames 1 -4 as a single image with four discrete focal planes A, B, C, and D.

FIG. 6 illustrates an example hardware configuration 600 of the near-eye display system 100 in accordance with at least one embodiment. The hardware configuration 600 includes the display assembly 102, the lens assembly 104, the lens driver 1 14, the display driver 122, an application processor 602, a graphics processing unit (GPU) 604, and a system memory 606. The lens assembly 104 includes a liquid membrane lens for the left eye (denoted "liquid membrane lens 1 12-1 ") and a liquid membrane lens for the right eye (denoted "liquid membrane lens 1 12-2"), as well as one or more fixed focal power lenses (not shown) for each eye. In this example, the display assembly 102 includes a single display panel 1 10 used to display left-eye imagery and right-eye imagery concurrently.

As a general operational overview, the application processor 602 executes a VR/AR application 608 (stored in, for example, the system memory 606) to provide VR/AR functionality for a user. As part of this process, the GPU 604 executes a focal rendering routine 610 (which may be part of the VR/AR application 608 or a separate routine) to focally render, for each eye, a sequence of display frames (e.g., sequence 121 , FIG. 1 ) display at the display panel 1 10, with the paired sequences of images representing a 3D VR or AR scene when viewed by the user through the liquid membrane lenses 1 12-1 , 1 12-2. The display driver 122 operates to display left-right paired display frames in sequence in a low-persistence mode such that each left-right pair of display frames is displayed for only a relatively small fraction of the frame rate period. In parallel, the lens driver 1 14 drives the liquid membrane lenses 1 12-1 , 1 12-2 using the driving signal 1 16 so as to continuously and cyclically modulate the focal power of the liquid membrane lenses 1 12-1 , 1 12-2 in synchronization with the display of the paired sequence of frames such that the instantaneous focal power exhibited by the lenses 1 12-1 , 1 12-2 at the time of display of a left-right pair of display frames is correlated to the focal plane for which the left-right pair of display frames was rendered. To this end, the lens driver 1 14 includes an input to receive the frame clock 124 used by the display driver 122 for timing of display of display frames at the display panel 1 10, an input to receive control information 612, and an output to provide the driving signal 1 16. The control information 612 includes information pertaining to the parameters for generating the driving signal 1 16, such as the frequency of each sinusoidal component to be incorporated into the driving signal 1 16, the amplitude of each sinusoidal component, the phase offset to be implemented between these sinusoidal components, or the phase offset to be implemented between the frame clock 124 and the driving signal 1 16, and the like.

In the depicted example, the lens driver 1 14 further includes a frequency control component 614, a plurality of signal generators, such as signal generators 616,

618, and 620, a signal combiner 622, and an offset component 624. As explained above, the driving signal 1 16 may be composed of a one or more sinusoids of frequency f1 , and in such instances the signal generators may comprise sine wave generators that may use the frame clock 124 as a feed or reference signal. In the event that the driving signal 1 16 is composed of only a single sinusoid, the frequency control component 614 controls a single one of the signal generators, e.g., signal generator 616, to generate a sinewave signal having the frequency f1 and the desired amplitude (denoted signal "S" in FIG. 6). This single sinewave signal is passed through to the offset component 624, which delays the sinewave signal by a particular delay period specified by the frequency control component 614 so as to introduce the desired phase relationship between the sinewave signal and the frame clock 124, and the resulting delayed sinewave signal is output as the driving signal 1 16.

In the event that the driving signal 1 16 is composed of multiple sinusoids at frequency f1 but with different phase offsets, the frequency control component 614 controls a number of the signal generators equal to the number of sinusoids in the driving signal to generate a corresponding sinewave signal having the frequency f1 , an indicated amplitude, and an indicated phase offset, and the resulting sinewave signals are combined by the signal combiner 622 and the resulting signal (denoted S in FIG. 6) is input to the offset component 624, which introduces the appropriate delay into the signal so as to implement the intended phase relationship between the driving signal 1 16 and the frame clock 124. The resulting phase-offset signal is then output as the driving signal 1 16.

FIG. 7 illustrates an example method 700 for focal rendering of display frames and synchronized modulation of liquid membrane lenses for providing VR imagery with apparent multiple concurrent focal planes for improved user immersion in accordance with at least one embodiment. For ease of illustration, the method

700 is described in the example context of the hardware configuration 600 of FIG. 6. The method 700 initiates at block 702, wherein the VR/AR application 608 determines the number N of perceived concurrent focal planes to be implemented for the near-eye display system 100. This number may be fixed, or may be selected based on various considerations, such as the processing power of the

GPU 215, the complexity of the VR world to be rendered, the focal power range of the lens assembly 104, and the like. As discussed above, two to four focal planes (N=2, 3, or 4) typically is sufficient to provide realistic immersion. Initialization further may include setting a temporary variable X to 1 . With the system initialized, the hardware configuration 600 performs two processes in parallel: focal-plane-based rendering of paired sequences of display frames; and synchronized modulation of the liquid membrane lenses 1 12-1 , 1 12- 2. For the focal rendering process, at block 704 the VR/AR application 608 instructs the focal rendering routine 610 to render a display frame to represent specified VR or AR image content, and in response the focal rendering routine

610 manipulates the GPU 604 to render a display frame based on a focal plane X using any of a variety of focal rendering techniques, with X being the temporary variable initialized at block 702 and representing the index of a corresponding one of the set of N focal planes implemented by the system 100. The process of block 704 is repeated twice for each iteration, once for each eye, so as to generate a stereoscopic pair of display frames. At block 706, the display driver 122 drives the display panel 1 10 to concurrently display the pair of display frames using the timing provided by the frame clock 124. As noted above, the display driver 122 may implement a low-persistence display whereby the display frames are actively displayed for only a portion of the corresponding period of the frame clock 124, and thus enabling the persistence-of-vision effect. The variable X is then shifted to the index of the next focal plane in the set of N focal planes (e.g., X = X MOD N), and the process of blocks 704 and 706 is repeated for the next focal plane. The result of iterations of blocks 704 and 706 is a sequence of display frames that repeatedly cycles through the N different focal planes.

In parallel, at block 708 the lens driver 1 14 generates a cyclical, continuously varying driving signal 1 16 having a frequency that is an integer divisor of the frame rate of the frame clock 124 and which has a fixed phase relationship with the frame clock 124 such that when the liquid membrane lenses 1 12-1 , 1 12-2 are driven with the driving signal 1 16, the liquid membrane lenses 1 12-1 , 1 12-2 exhibit a continuously modulated focal power that is synchronized to the display of the focally-rendered display frames such that when each stereoscopic pair of display frames is displayed, the lenses 1 12-1 , 1 12-2 exhibit a focal power correlated to the focal plane for which the pair of display frames was rendered. This focal-plane specific rendering, coupled with the persistence-of-vision effect, results in the user perceiving two or more display frames displayed in series as being displayed as a single image with multiple focal planes, and when this effect is repeated again and again throughout the sequence of display frames representing the VR imagery of a 3D VR scene, the user is more fully immersed in the VR scene due to the improve focal cues provided by the multiple concurrent focal planes perceived by the user to be present in the displayed imagery. In some embodiments, certain aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software comprises one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

A computer readable storage medium may include any storage medium, or combination of storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer readable storage medium may be embedded in the computing system (e.g. , system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such

modifications are intended to be included within the scope of the present disclosure. Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter.

Accordingly, the protection sought herein is as set forth in the claims below.