EYE TRACKED FOVEAL DISPLAY BY CONTROLLED ILLUMINATION FIELD OF THE INVENTION The present invention is related to a viewing apparatus. More specifically, the present invention is related to a viewing apparatus which displays a foveal inset image on the image on the screen about the fixation point so a viewer's fovea sees the foveal image while the rest of the eye sees the image.

BACKGROUND OF THE INVENTION The human eye has a high resolution area called the fovea, which is responsible for the perception of color and fine detail using specialized retinal receptors called cones.

The foveal region subtends a visual angle of about a degree of arc. Our perception of the world is therefore time- multiplexed, in that we scan this one degree patch around to fill in the details in which we are interested, and assume that these details don't change while we aren't actively perceiving them with our fovea. Our non-foveal receptors (rods) can detect movement very well ; they function to guide our eyes so we can more accurately perceive anything which may have changed since we last looked at it with our fovea.

Because the foveal region is so narrow, and the highest resolution displays are so expensive, many people have speculated about and experimented with displays which attempt to present a high resolution"inset"image which can move inside of a lower resolution"surround"image. Ideally,

this high-resolution inset would perfectly follow the movement of the eye so that fovea always perceives it; the image shown by the inset would change to match its current position so the fovea can see the correct image for its position. The lower resolution surround image would either be dimmed or blocked for the area of the surround which is temporarily used for the foveal inset, so as not to distract from it.

There are several technical challenges in producing such a foveal display system. The system must accurately perceive where the viewer is looking, it must compute the correct foveal image for the foveal position, and last but not least, it must move the foveal image to the correct position fast enough for the system to be useful. Faster graphics engines and better gaze tracking systems now available make the first two problems tractable. Using a mechanical means (such as a mirror) to move the foveal inset in a projection system may be feasible with the fastest galvanometers available, but this is expensive and likely to be troublesome given the time response needed.

SUMMARY OF THE INVENTION The present invention pertains to an apparatus for viewing. The apparatus comprises a screen. The apparatus comprises means for detecting a fixation point of a viewer's eyes on an image on the screen. The apparatus comprises means for displaying a foveal inset image on the image on the

screen about the fixation point so a viewer's fovea sees the foveal image while the rest of the eye sees the image.

The present invention pertains to a method for viewing. The method comprises the steps of detecting a fixation point of a viewer's eyes on an image on a screen.

There is the step of displaying a foveal inset image on the image on the screen about the fixation point so the viewers fovea sees the foveal image while the rest of the eye sees the image.

BRIEF DESCRIPTION OF THE DRAWINGS In the accompanying drawings, the preferred embodiment of the invention and preferred methods of practicing the invention are illustrated in which: Figure 1 shows a lens array arrangement.

Figure 2 is a schematic representation of an apparatus of the present invention.

DETAILED DESCRIPTION Referring now to the drawings wherein like reference numerals refer to similar or identical parts throughout the several views, and more specifically to figure 2 thereof, there is shown an apparatus for viewing. The apparatus comprises a screen. The apparatus comprises means for detecting a fixation point 12 of a viewer's eyes 11 on an

image on the screen. The apparatus comprises means for displaying a foveal inset image on the image on the screen about the fixation point 12 so a viewer's fovea sees the foveal image while the rest of the eye sees the image.

Preferably, the displaying means includes a surround image projector 5 which produces the image on the screen. The display means preferably includes a computer 9 in communication with the surround image projector 5 and the detecting means which receives the fixation point 12 and causes the foveal inset image whose center is closest to the fixation point 12 to be displayed on the image on the screen.

Preferably, the displaying means includes light sources 1 which produce the foveal inset image at the direction of the computer 9.

The display means preferably includes an illuminating switching system in communication with the light sources 1 and the computer 9 which turns on the light sources 1 at the direction of the computer 9. Preferably, the displaying means includes a condensing lens 2 disposed between the light sources 1 and the screens screen. The displaying means preferably includes an array of projection lenses 4 disposed between the condensing lens 2 and the screen, the condensing lens 2 imaging the light from the light sources 1 into a corresponding lens of the array.

Preferably, the displaying means includes an LCD 3 in communication with the computer 9 which sends an image for the foveal region image to the LCD 3, the LCD 3 disposed

between the array and the screen, light from the light sources 1 which is imaged by the condensing lens 2 passes through the LCD 3. The computer 9 preferably causes the surround image projector 5 to black out the region of the image on the screen that corresponds to where the foveal inset image is displayed on the screen. Preferably, the displaying means includes a beam splitter/combiner 6 disposed between the array and the screen through which the foveal region image passes and from which the image from the surround image projector 5 is reflected. The detecting means preferably includes an eye-tracking system 8. Preferably, the light sources 1 are LEDs.

The present invention pertains to a method for viewing. The method comprises the steps of detecting a fixation point 12 of a viewer's eyes 11 on an image on a screen. There is the step of displaying a foveal inset image on the image on the screen about the fixation point 12 so the viewers fovea sees the foveal image while the rest of the eye sees the image.

Preferably, there is the step of blacking out the region of the image on the screen that corresponds to where the foveal inset image is displayed on the screen. There is preferably the step of updating where the foveal inset image is displayed on the screen when the detected fixation point 12 moves closer to a center of a different foveal display region 13. Preferably, the detecting step includes the step of detecting a fixation point 12 of the viewer's eyes 11 looking at the screen with an eye tracking system.

The displaying step preferably includes the step of sending a location of the fixation point 12 to a computer 9.

Preferably, there is the step of activating by the computer 9 the foveal display region 13 whose center is the closest to the fixation point 12. There is preferably the step of directing by the computer 9 an illuminator switching system 10 to turn on an LED which is imaged by a condensing lens 2 onto a corresponding lens of an array of projection lenses 4.

Preferably, there is the step of sending an appropriate image for the foveal region to an LCD 3.

In the operation of the preferred embodiment, a projection display"moves"the image by changing the illumination of a spatial light modulator (cf LCD panel).

This can be accomplished by sharing the same LCD 3 panel as a common element of different optical systems-each of which has a different light source (LED or lamp) and corresponding projection lens. As shown in figure 2, each optical path forms a projected image of the LCD 3 at different position.

If the lens pitch is half the width of the LCD 3 panel, the images of adjacent paths would overlap by about 50%. In this embodiment, only one LED would be illuminated at a time-the one corresponding to the closest image to the foveal region.

Allowing the areas to overlap means that perfect gaze tracking is not required, and dithering between adjacent regions at the boundary can be reduced.

Bear in mind that an actual system would presumably use a 2D array of LEDs and projection lenses to allow the foveal inset to"move"vertically as well as horizontally.

For example, in the initial prototype, 10 lenses are used in a"honeycomb"format as shown in Fig. 1.

Given a LCD resolution of 1280 x 1024, this would give an effective resolution of: x 3 = 3840 by 3072 The effective resolution of the system, which determines the field of view seen by an observer at a distance which takes full advantage of viewer's foveal resolution, is entirely a function of the LCD 3 resolution and the number of lenses. If a 4,5, 4 lens array is used (for wide-screen) the final resolution would by 4x and 3y, or 5120 by 3072. Given a simplified foveal acuity model of 60 pixels per degree, we can obtain an optimal field of view of (5120/60) x (3072/60) or about 85 degrees horizontal by 51 degrees vertical.

It should also be noted that the inevitable distortions will change with the differing optical paths, and needs to be compensated for in a calibration step. Also, the lower resolution surround image can be provided by a separate optical system which might be an identical LCD 3 with a shorter focal length lens. The surround and foveal paths could be combined by a beamsplitter or might just overlap at the projection surface with appropriate keystone correction.

The surround image can be dynamically updated to blank out the area of the surround with edge blending. It

will be less necessary to make the blending"perfect"than to eliminate any high spatial frequency components (sharp edges) or rapid flickering at the boundary between the inset and surround images, which might be picked up by the viewers' peripheral vision. One method to minimize this transition is to selectively decimate and low-pass filter the part of the inset window which is furthest away from the fixation point 12, so that it matches the lower resolution of the surround window.

Details about the optical system are as follows: The overlap (at a reasonable projection distance » than the projection lens focal length) is entirely a function of the projection lens array pitch to LCD 3 size. If the lens pitch is half the LCD 3 diameter, 50% overlap is achieved, enabling the foveal region to be centered at a point close to fixation point 12. Thus, the lenses must be small enough to fit this spacing. The light from each LED is imaged into the corresponding projection lens by the condenser lens just upstream of the LCD 3. Each LED image must not bleed into adjacent lenses or crosstalk will occur.

The condenser lens must therefore match the etandue (size * angular extent) of the LEDs and the projection lenses, and the size of the LCD 3. An ideal LED would not only be bright but have the narrowest possible angular spread, so that as much light a possible is imaged by the condenser onto the correct projection lens.

Parts List: 1. Light sources (LEDs) 2. Condensing Lens 3. Foveal image LCD 4. Array of Projection Lenses 5. Surround Image Projector 6. Beam Splitter/Combiner 7. (Surround image region of) Projection Screen 8. Eye-tracking system 9. Computer 10. Illuminator Switching system 11. Viewer's eyes 12. Fixation point of viewer 13. Foveal display region of projection screen Operational Description: Overview : Eye tracking system 8 detects the fixation point 12 of the viewer's eyes 11 looking at the screen, and sends the location of the fixation point 12 to computer 9. The foveal display region 13 whose center is the closest to the fixation point 12 is activated by the computer 9, which directs the illuminator switching system 10 to turn on the LED which is imaged by condensing lens 2 onto the corresponding lens of array 4. Simultaneously, the computer 9 sends the appropriate image for the foveal region to LCD 3 and blacks out the corresponding region projected by surround image projector 5.

The viewer's fovea sees the foveal image 13 while the rest of the eye sees the surround image 7, and system updates when the detected fixation point 12 moves closer the center of a different foveal display region 13.

Detailed : For the following, we refer to the coordinates of the image as x and y, the coordinates of the center of each foveal inset image (i, j) as xi and yj (where i and j are the column and row of a given inset image). I and j also refer to the corresponding LED and projection lens elements which produce foveal inset image (i, j) 1. Eye tracking system 8 detects the fixation point 12 of the viewer's eyes 11 looking at the screen.

2. The coordinates (xe, ye) of the fixation point 12 are sent to the computer 9.

3. The computer 9 determines which foveal inset image is closest to the fixation point 12 by determining which values of i and j yield the minimum result for (distance2) = (xi-xe) 2 + (yi-ye) 2 4a. The foveal inset image 13 whose center is the closest to the fixation point 12 is activated by the computer 9, which directs the illuminator switching system 10 to turn on the LED (-i,-j) which is imaged by condensing

lens 2 onto the corresponding lens (i, j) of array 4.

4b. At approximately the same time as 4a, the computer 9 sends the high resolution foveal inset image for the foveal region to LCD 3 and blacks out the corresponding region projected by surround image projector 5. For example, in a display which uses a projection lens array pitch of half the LCD size along each axis, there is exactly 50% overlap.

Therefore the blacked-out region is defined as (xi- (x/(i+1))) to (xi + (x/(i+1))) horizontally, and (yj- (y/(j+1)) to (yj + (y/(j+1))) vertically.

5. The viewer's fovea and nearby areas of the retina sees the foveal image 13 while the rest of the retina sees the surround image 7.

6. The system updates when the detected fixation point 12 moves closer to the center of a different foveal display region 13 as determined by the method of step 3.

Although the invention has been described in detail in the foregoing embodiments for the purpose of illustration, it is to be understood that such detail is solely for that purpose and that variations can be made therein by those skilled in the art without departing from the spirit and scope of the invention except as it may be described by the following claims.

APPENDIX References, all of which are incorporated by reference herein: US 4,479, 784 Eye line-of-sight responsive wide angle visual system (Mallinson et al.) US 6,351, 335 Extremely high resolution foveated display (Perlin)