Remote interaction system based on light guiding layer with improved detector signal

The present invention relates to a user input apparatus for detecting direction and/or input position of a light beam incident on the apparatus from a light beam generating device.

To improve user interactivity between users and computers, screen displays have been introduced where the user interacts by directing a light beam from a light source onto the screen.

By pointing on the screen by means of light, a user may interact with the computer by illuminating icons, pictures, words or other visual objects on the computer screen. This technique is used in connection with, for example, multimedia information kiosks, educational centers, vending machines, video games, PCs, etc. As compared with a touch screen, physical contact with the display is avoided, and a user interacting with the screen will not block an audience's view thereof. WO2005/029395 discloses a screen having a light guiding layer for detecting the coordinates of a laser beam incident on the screen from an input device controlled by a user. The coordinates are related to icons, data commands etc, and the light guiding layer has an optical structure arranged to confine a fraction of the laser beam in the layer when the beam is incident on the layer from the exterior of the apparatus, such that the beam is guided by total internal reflection (TIR) to the sides of the layer/screen. The layer has on its edges light detecting means arranged to detect the light that is confined in the layer, and the position of the detected light is related to the coordinates of the laser beam. The layer further transmits light through the layer when the light is incident on the layer from the interior of the screen. The coordinates, or x-y position of the laser point on the light guiding layer, are used for interactivity. Additional user flexibility is achieved when the input angles (θ, φ) of the laser light are used as input parameters. Laser beam landing position and incidence angle can both be determined by trapping part of the laser light with a microstructure

consisting of, for example, pyramids, such that the light is guided through TIR to the side of the layer, where it generates detector signals.

The angle θ of the laser beam to the normal of the plane of the light guiding layer is proportional to the angles α and β in the plane of the light guiding layer (α is proportional to the vertical projection θ sin φ, and β is proportional to the horizontal projection θ cos φ). The four beam input parameters x, y, θ, φ can be deduced from four detector signals xl, x2, yl and y2 generated at the four sides of the light guiding layer. A problem with the prior art is that four position sensitive detectors (PSDs) positioned at the four sides of the light guiding layer are needed to generate the four signals necessary for extracting the input position and angles.

Furthermore, due to non-linearities, the relation between detector signals and beam input is not always unambiguous since the four detector signals may be generated by more than one combination of input angles and positions.

Yet another problem is that information may be missing or wrong when the laser beam is near the edge or a corner of the layer. For example may one or more of the four signal beams either land or the wrong detector, e.g. the y signal beam may land on an x- detector due to the angle α, or may land on the correct detector but at the wrong position, due to total internal reflection at the side.

A further problem is that, depending on the incident angle of the incoming light, light coupled into the light guiding layer may propagate only to one edge of the layer on which no detector is present, such that the position of a light spot cannot be determined.

It is an object of the present invention to provide an improvement of the above techniques and prior art.

A particular object is to provide a user input apparatus for more reliably detecting direction and/or input position of a light beam incident on the apparatus from a light beam generating device.

Another object is to detect signals preferably at only two sides of the input apparatus.

Yet another object is to provide a user input apparatus that offers lower production costs both in respect of material and assembly.

Another object is to provide a user input apparatus that is optimized to accept a wide range of angles of light beams incident on the apparatus, while light detection means, or PSDs, are preferably arranged on only two sides of the light guiding layer.

These objects are achieved by a user input apparatus according to claim 1. Preferred embodiments are defined by the dependant claims.

By arranging a first array of optical structures and at least a second array of optical structures, the light beam may be directed in the light guiding layer in at least four directions towards the light detecting means. This results in four detection signals, and since the arrays are different corresponding to a fifth variable, the position and direction of the light beam may be derived by applying geometrical rules. In more detail, the first array of optical structures directs, according to a known pattern, the light beam differently than the at least second array of optical structures. It should be noted that if the difference between the arrays is small, the geometrical rules may be linearized making it possible to calculate the input position and angle based only on the four detection signals. The first and the at least second array of optical structures may comprise pyramidally shaped structures.

Preferably the pyramidally shaped structures have four sides respectively for directing light towards four sides of the light guiding layer/screen.

Preferably the optical structures of the first array are rotated in relation to the optical structures of the at least second array.

This provides a fix angular value indicating the difference between the structures of the two arrays, which is needed for determining the input position and angle of the light beam. Of course, said angular value may be omitted if the calculation of input coordinates and angles are linearized. The axis of rotation may be a normal direction of the light guiding layer, and/or an axis parallel to the plane of the light guiding layer.

Said axes of rotation, in combination or not, provides for a very efficient manufacturing method for creating the difference between the structures of the arrays. In another embodiment the optical structures of the first array may have different top angles than the top angles of the optical structures of the at least second array. Again a fix value indicating the difference between the structures of the two arrays, or the four directions of the reflected light beam, is provided. This embodiment also provides a cost efficient manufacturing process of the different structures of the layer.

The first array of optical structures and the at least second array of optical structures may also be integrated in a common structure, and in one embodiment the common structure comprise a pyramidally shaped structure having at least two top angles.

Preferably the light guiding layer is configured to canalize the directed light in directions parallel to a plane of the light guiding layer.

This configuration provides a very efficient way of entrapping and transferring the light to the light detecting means.

Preferably two of said at least four directions are substantially orthogonal to two other directions of said at least four directions. Preferably the light detecting means comprise a first light detecting means for generating a first xl and a second x2 detection signal, and a second light detecting means for generating a third yl and a fourth y2 detection signal.

Again, this saves cost in the manufacture of the light guiding layer and its associated display unit. Preferably, the first xl and third yl signals are generated by the light beam directed by at least one optical structure of the first array of optical structures, and the second x2 and fourth y2 signals are generated by the light beam directed by at least one optical structure of the at least second array of optical structures.

Preferably the apparatus comprises processing means for determining the direction and input position of the light beam incident on the apparatus, based on said four signals xl, x2, x3, x4. Of course, the fix variable representing the difference between the arrays/structures may be used for determining the beam input position and angle, when linearization is not desired.

Preferably, the top angle of the pyramidally shaped structures and refractive index of the light guiding layer are selected, for maximizing the acceptable direction of the light beam incident on the apparatus, by the fulfilling the conditions that: light incident on the light guiding layer from air has an arbitrary angle, light hit two opposite sides of a pyramidally shaped structure, incident light undergo TIR on the two opposite sides of the pyramidally shaped structure, and incident light undergo TIR on the light guiding layer after TIR on the two opposite sides of the pyramidally shaped structure.

This optimization is advantageous since it allows for the manufacturer of the apparatus to properly select material and geometrical shapes for the constituting parts.

The above-described selection is advantageous since it offers a solution where two light detecting means are sufficient, located on two non-opposite sides of the light guiding layer, for detecting the coordinates of the light spot as well as the light beam incident angle. This saves the cost of two additional detection means and a more complex electronics, as well as reduces the space that would have been required for mounting the additional detection means.

In further detail, the top angles of the pyramidally shaped structures are preferably between 50-70°, and the refractive index of the light guiding layer preferably exceeds 2λ/3. Retro -reflectors may be arranged opposite the light detecting means, which provides a possibility to increase the acceptable direction of the light beam incident on the apparatus.

Retro reflectors facilitate a wider range of acceptable directions of the light beam, since they reflect light back to light detecting means. Again, costs are reduced as compared with a solution utilizing detection means at all sides of the light guiding layer, since reflectors are cheaper than light detecting means.

According to another aspect of the invention, the light guiding layer is arranged on the exterior of a display of a TV, computer, mobile phone, personal digital assistant, electronic agenda, navigation device or projector screen. According to yet another aspect of the invention, a display unit comprising the inventive apparatus is provided.

The display unit according to the invention has the same advantages as the apparatus according to the invention.

Embodiments of the present invention will now be described, by way of example, with reference to the accompanying schematic drawings, in which

Fig. 1 illustrates a schematic front view of a user input apparatus according to the invention, Fig. 2 illustrates a schematic lay-out of light propagation in a light guiding layer having two arrays of optical structures,

Figs. 3a-3b illustrate a front view and a side view of a light guiding layer according to a first embodiment of the invention,

Fig. 4 illustrates a front view of a light guiding layer according to a second embodiment of the invention,

Fig. 5 illustrates a front view of a light-guiding layer according to a third embodiment of the invention, Figs. 6a-6b illustrate a front view and a side view of a light guiding layer according to a fourth embodiment of the invention,

Fig. 7 illustrates a schematic front view of a user input apparatus having retro- reflectors,

Figs. 8a-8e illustrate cross-sectional views of five embodiments of a light guiding layer according to the invention,

Fig. 9 illustrates a schematic lay-out of light propagation in a light guiding layer having two pairs of offset light detecting means, and

Figs. 10a- 10b illustrate cross sectional views of two embodiments of offset light detecting means.

Fig. 1 illustrates a user input apparatus 108 for detecting direction and/or input position of a light beam 103 incident on the apparatus 108 from a light beam generating device 105. It should be noted that the light beam-generating device 105 is held in front of the apparatus 108 in a different plane. A light guiding layer 101 is arranged on the exterior of the apparatus 108, and preferably the apparatus 108 is a display monitor of a computer (not shown), but may also be display of a TV or a mobile phone, or may be a projector screen.

With the light beam generating device 105 a point of light 109 is generated on the face of the light guiding layer 101, and on two preferably orthogonal sides of the apparatus 108, two respective pixellated light detecting means 106a, 106b are arranged, capable of detecting the position of at least two light spots. When the light beam 103 hits the light guiding layer 101, the light beam is reflected inside the layer 101 and propagated in different directions towards the edges of the layer 101, which involves the detecting means 106a, 106b being hit by the light generated by the light beam generating device 105. A processing unit 107 is connected to the detecting means 106a, 106b, and determines the point of light 109 and incident angel of the light beam 103, based on the signals generated by the detecting means 106a, 106b when the means 106a, 106b are hit by the light.

Fig. 2 illustrates a schematic lay-out of light propagation in a light guiding layer. A point of light 209 is incident on the layer and hit two different arrays of optical

structures, which structures will be described later. The light is reflected from the optical structures in (at least) four directions respectively, as shown in the figure. Since the arrays are different, or more precisely, since the optical structures of the arrays are different (i.e. two different optical structures), light hit two points of the side 211 of the layer 101 representing the x-axis, and light hit two points of the side 210 of the layer 101 representing the y-axis. At said sides 210, 211 of the layer 101, light detecting means are arranged as discussed above. How the direction and/or input position of a light beam directed at the point of light 209 are determined, is described below.

As can be seen in Fig. 2, the light beam is directed in eight directions, since the structures are pyramidally shaped as further described below. However may the structures be configured to direct light only in four directions, towards two light detecting means.

Fig. 3a illustrates a light guiding layer 301 comprising a first array 320a of optical structures 302a and a second array 320b of optical structures 302b. A light beam 303 incident on the layer 301 is generated by a user input apparatus 305 and is reflected from at least one of respective optical structures 302a, 302b in at least four directions 304a, 304b. The reflected light is confined in the layer 301 and propagates towards the edges of the layer 301 in a known manner, and as schematically shown in Fig 3b.

The optical structures 302a, 302b are four- sided pyramids no longer having their faces aligned with the x- and y- axes. The first optical structures 302a of the first array 320a (half the pyramids of the layer) are rotated at a small angle -γ with respect to an axis parallel to the normal of the plane of the layer 301, while the second optical structures 302b of the second array 320b (the other half of the pyramids) are rotated at γ with respect to an axis parallel to the normal of the plane of the layer. This results in said splitting of the light beam as illustrated in Fig. 2, where the effect of said angles -γ and γ is indicated. Since the structures 302a, 302b are four- sided, the beam is in practice directed in eight directions.

The axes of rotations parallel to normal of the layer preferably pass the tip of respective pyramidally shaped structure (symmetrical rotation), but may of course be offset from the structure. In general, the structures all have a respective axis of rotation, and the normal of the plane of the layer defines their direction. Fig. 4 illustrates a light guiding layer 401 comprising a second embodiment of optical structures 402a, 402b, wherein the structures have the form of four-sided pyramids. The structures are rotated with respect of an axis parallel to the plane of the layer 401, and the difference between the structures results in small angles having an effect corresponding to the effect of the angles -γ and γ of Fig. 2.

In this second embodiment the axes of rotations parallel the plane of the layer are preferably located above the tip of respective pyramidally shaped structure, but may also be offset from the structure. In general, the structures all have a respective axis of rotation, and the axis parallel to the plane of the layer defines their direction. Fig. 5 illustrates a light guiding layer 501 comprising a third embodiment of optical structures 502a, 502b having the form of four-sided pyramids, wherein the structures have different top angles. For example, the optical structures have the same height but the second structure 502b has a greater base area than the base area of the first structure 502a. The difference between the structures results in small angles having an effect corresponding to the effect of the angles -γ and γ of Fig. 2.

The angles in the three embodiments above may of course have different absolute values.

In yet another embodiment the pyramidal structure has at least four sides, such as a pyramid with a hexagonal base, and may be rotated according to above in a manner corresponding to the four-sided structure.

Figs. 6a and 6b illustrate a light guiding layer 601 comprising a fourth embodiment of optical structures 602a, 602b having the form of common four-sided pyramids with blunt tops. In other words, the structures are integrated forming a pyramid having two different top angles λi and λ ₂ . In Fig. 6b, λ ₂ is greater than λ _{l s} but of course λ ₂ may be smaller than λi. The difference between the structures again results in small angles having an effect corresponding to the effect of the angles -γ and γ of Fig. 2.

When observing Fig. 2, it is clear that: (x 1 -x)/y = tan(β-γ), (x2-x)/y = tan(β+γ) (yl-y)/x = tan(α-γ), (y2-y)/x = tan(α+γ)

For small angles the above equations may be linearized, i.e. a first order approximation can be made, such that: x = δy/2γ, y = δx/2γ α = (y _m -y)/ ^χ , β = ( ^χ _m - ^χ )/y

where δx = x2-xl, x _m = (xl+x2)/2 and where δy = y2-yl, y _m = (yl+y2)/2

Hence it is demonstrated that the two position coordinates and the two beam angles can be directly derived from two light detectors instead of four. Test have also verified that linearization may successfully be employed, i.e. the first order approximation of the full equations renders satisfactory results in practice. The direction of the laser beam is given by the vector (x, y, z) = (-sin θ cos φ, - sin θ sin φ, -cos θ), and when the beam enters the light guiding plate having refractive index n, known geometrical optics are applied for determining θ and φ from α, β and n. Hence the direction of the light beam is determined.

Turning to Figs. 9, 10a, and 10b, it is illustrated how information about the direction of the light beams can be obtained directly and unambiguously by using two pairs of light detecting means that are off-set by a distance as indicated in the drawings 10a and 10b. The reference numerals 910, 911, 914 and 916 represent the light guide interface side of respective light detecting means.

E.g., from the two measured positions xl and x2, the direction β can be reconstructed, while α is derived from yl and y2. If the light beams propagate in air as in Fig. 9, the Snellius-refraction that takes place at the light guiding layer/air-interface, amplifies the positional shift. If the light beams propagate in an extended light guide as in Fig. 10b, the positional shift is smaller but the construction is more rugged. Since relevant refractive indexes are known, α and β and in turn θ and φ are derived from xl, x2, yl and y2 by applying known geometrical optics.

Alternatively, two light detecting means are used of which the first is semitransparent.

Hence another aspect of the invention is a user input apparatus for detecting direction and/or input position of a light beam incident on the apparatus from a light beam generating device, said apparatus comprising a light guiding layer configured to direct at least a fraction of the light beam towards light detecting means, wherein said light detecting means comprise two pairs of light detectors arranged on a respective side of the light guiding layer.

The detectors of a pair may be mutually offset in a direction parallel to the plane of the light guiding layer, and preferably the detectors of both pairs are offset. Of course, the detectors may be integrated in one unit, as long as offset surfaces, facing the light guiding layer, are present.

Air or any other suitable substrate may be present between at least one of the detectors 1017, 1018, and/or the light guiding layer 1001 may comprise a part 1019 extending to at least one of the offset detectors 1018.

According to another aspect, it is concluded that two light detection means are sufficient to determine the x-y coordinate of the light spot and/or incident angle of light beam, assuming that both of these detection means receive enough light, such that the generated signal by the incident light is large enough to be detected.

Incoming light with only a small incident angle with respect to the normal of the light guiding layer comprising four-sided pyramids having a top angle of 90°, is coupled into the light guiding layer via TIR independent of which face of the pyramid it reflects from. However, when the incident angle becomes too large, light will only be coupled into the light guiding layer via TIR on one face of the pyramid, whereas only a small part of the light that hits the other (opposite) face of the pyramid is coupled into the light guiding layer via (regular) reflection. The regular reflection of the edges of the pyramids is only small, which is inherent to the required "invisibility" of the screen, i.e. the front-of-screen performance of the display may not be distorted by the substrate of the light guiding layer.

The light, coupled into the light guiding layer via TIR on only one pyramid face, propagates only in one direction, which is similar to the in-plane propagation direction of the incoming light. The light coupled into the light guiding layer via (regular) reflection on the other (opposite) pyramid face propagates in the opposite direction. As the amount of light coupled into the light guiding layer via (regular) reflection is only small compared to the light coupled into the light guiding layer via TIR, the majority of light coupled into the light guiding layer propagates into the direction it had before being coupled in.

In case the light coupled into the light guiding layer via TIR, i.e. the majority of the incoupled light propagates to an edge of the light guiding layer on which no detector is present, the light is lost without detection. In that case, the light coupled into the light guiding layer via (regular) reflection, i.e. the minority of the incoupled light, propagates to the edge of the light guiding layer comprising the detector, but is generally, and as experienced in practice, too low to cause a detectable signal. Although the initial amount of light coupled into the light guiding layer via (regular) reflection may be high enough to cause a detectable signal, light is lost while propagating to the edge of the light guiding layer. Hence, the problem becomes more severe with increasing screen size.

Consequently, a system consisting of two detectors of which one is located on the x-edge and one on the y-edge, and the substrate of the light guiding layer with optical

structures, cannot determine the x-y coordinate of the light spot and the light beam incident angle for all incoming angles.

Figs. 8a-8e disclose further embodiments of were use of only those specific combinations of the pyramid top angle λ is proposed, the refractive index n _g of the layer in which the pyramids arc located and the refractive index tWity of the medium in the pyramid cavities, e.g. air, for which light incident on the light guiding layer 803 with an angle θ in air, with I θ I < β, where β = acceptance angle, can be coupled into the light guiding layer by TIR via all four faces of the pyramid in order to allow detection of the light-spot position and orientation with only two detectors irrespective of the azimuthal angle φ of the incident light. The acceptance angle β for which light can be coupled into the light guiding layer by TIR is determined by first requirements, that (i) incident light is able to hit all pyramid faces, (ii) incident light undergoes TIR from all 4 faces of the pyramid, and (iii) incident light undergoes TIR on the top and bottom planes of the light guiding layer after TIR on the pyramid faces.

It has been shown that these requirements are only fulfilled for specific combinations of λ, n _g and n _cavity . In addition, it can be shown that for certain combinations of λ, n _g and n _cavit y the acceptance angle β can be optimized, i.e. made as large as possible. It is advantageous to use a high refractive index layer in which the pyramids are located; at least n _g should exceed 2/V3 to allow TIR, and n _g is preferably larger than 1.3. In case, the medium in the pyramid cavities is air, i.e. tWity ⁼ 1, the use of pyramid top angles between 50° and 70°, preferably close to 60°, is advantageous for an optimal acceptance angle of incident light.

In another embodiment of the invention retro -reflectors are arranged on the edges of the light guiding layer on which no position sensitive detectors are positioned, in order to reflect the light back into the light guiding layer in the same direction it came from. This method also allows detection of the light-spot position and orientation with only two detectors irrespective of the azimuthal angle φ of the incident light. In this case, the acceptance angle β for which light can be coupled into the detector such that it can be detected is determined by second requirements, that (i) incident light is able to hit at least one face of each pair of two opposite faces of a pyramid,

(ii) incident light undergoes TIR from at least one face of each pair of two opposite faces of a pyramid, and

(iii) incident light undergoes TIR on the top and bottom planes of the light guiding layer after TIR from a pyramid face.

Fig. 7 illustrates a light guiding layer having retro-reflectors 712 arranged opposite light detecting means 706. A processing unit 707 calculates an input position and angle of a light beam.

Mathematics known in the art are applied to the first requirements according to above for optimizing and give the relationship between the acceptance angle β and the pyramid top-angle λ and refractive index n _g .

Calculations clearly show that only for specific combinations of the pyramid top-angle a and refractive index n _g of the substrate, the acceptance angle β is such (i.e. β > 0) that detection of the light-spot position (and orientation) with only two detectors is possible irrespective of the azimuthal angle φ of the incident light.

The requirement above applies for all structures illustrated by Figs. 8a-8e.

It is noticed that an anti-reflective coating or other non-diffuse layers may be incorporated on top of the substrate, i.e. on the opposite side of the substrate than the side on which the pyramids are located. Moreover, additional non-diffuse layers may be incorporated between the substrate and the layer comprising the pyramids.

The pyramid cavities may be filled with the same medium as the adjacent medium, or may be filled with a medium that differs from that of the adjacent medium. The refractive index tWity of the medium in the pyramid cavities should be lower than the refractive index n _g of the layer comprising the pyramids. The adjacent medium may be the same as the layer comprising the pyramids, and in a particular embodiment, the adjacent medium may be an adhesive material, including, but not limited to, a pressure sensitive adhesive layer. The adjacent medium may consist of multiple layers.