BACKGROUND OF THE INVENTION

Field of the invention

This invention relates to computer pointing devices, in particular to such devices as used in portable computers.

Background and prior art

The presently used pointing devices like the mouse, the joystick or trackball use mechanical moving parts to translate the operator's hand movements into the movements of the computer cursor. However, in small and portable computers like the laptop computers the space on the keyboard is inadequate for mounting of such devices which are of dimensions suitable for easy and unrestrained actuation. During the operation the operator's hand often covers the keyboard or and essential part of it and the said pointing device is too small compared to the hand to precisely control the movement. In computers and control panels designed for operation under adverse conditions (water, dust, body liquids, etc.) it may be hard to make the system tight and protected. Other solutions include pointing devices to be used at the screen, like the light pen. Such a device has the disadvantage of being a separate additional device, sometimes hard to use on a tilted screen. Yet other pointing devices, the digitisers, have been invented and are in use in precision digitalization and include acoustic tablets, where two orthogonal microphones detect the relative position of a sound source (e.g. an electrical discharge at the tip of a pen-like device) based on the time elapsed for the sound to reach the respective microphones. In all cases known to us is the pointing device a movable or separate device, often requiring the possibility and space to lean the hand and arm against a fixed object in order to closely control the movements of the hand controlling the said pointing device.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide for a computer pointing device which enables the control of the cursor movements by activation of a device containing no moving parts and thus overcoming the problems of the present small or separate pointing device actuators mentioned above. The pointing or control device shall be arranged tight and protected from being impaired by water, dust, body liquids etc. within a housing.

The invention is characterized by the features of claim 1. Advantageous embodiments of the invention are described in the remaining claims.

The apparatus according to the invention enables the control of the cursor movements by activation of devices without moving parts and thus overcomes the problems of the present small pointing device actuators in the following way:

The movement of the object may be detected and conveyed to the computer memory by an optical movement detection device at the bottom of a recess on the computer housing or a set of ultrasonic transceivers at the same place. These devices may be mounted onto a protrusion from the said computer housing as well. In both cases the detectors use waves (mechanical or electromagnetic) to detect the movement of, for example, the operator's hand in front of the device as illustrated in figure 1.

The optical system contains light transmitters and receivers that can detect movements of objects moving in front of them. This detection is designed in such a way as to particularly detect the movement of the operator's palm leaned against the said recess. The transmitter's and the receiver's directivity characteristics are focussed and intersect at the level of the palm surface as it is moved in front of it. The transmitter-receiver sets are arranged orthogonally so that each of the sets can detect one (Cartesian) coordinate of the movement. Other angular arrangements have a lower sensitivity. The movement detection is done by the calculation of the cross correlation of the detected light levels at pairs of

light receivers which are a known distance apart, and detection of the time delay corresponding to the speed of the object passing in front of the detectors. The crosscorelation function has an extreme at the delay which corresponds to the time needed for the two light reflection variation functions to match. They are detected by the said light receivers. The velocity of the movement is the ratio of the distance between the detectors and the characteristic delay at which an extreme occurs in the cross correlation function. Thus calculated velocity can be used to steer the cursor on the computer screen.

The two sets of optical movement detectors may be positioned in a recess in the computer housing in order to define the sensitivity distance and to protect the detectors. The confirmation of an action, once the appropriate icon has been pointed at, is done by additional buttons on the keyboard.

As a further example there exists the possibility to detect the movements of the operator's palm in front of a predetermined area on the computer by the use of ultrasonic waves. The detectors in this case are ultrasonic transducers capable of transmitting ultrasonic waves into the air and receiving ultrasonic waves reflected from a reflective object in front of them (e.g. operator's palm as in figure 1). Two sets of the transducers detect waves at a space angle. This is achieved by tilting the axes of their directivity functions, preferably in such a way that the projections of the axes onto the surface where they are mounted are at 90°, i.e. their azimuthal angles are at 90°. Other angles may be used as well, but they yield a lower movement detection sensitivity. Their elevation angles are normally less than 60°. The piezoelectric transducers are used for Doppler measurement of movements of the objects in front of them.

If continuous wave Doppler technique is used the said transducers are pairs of transmission-reception transducers. In this case the orientation of the split between the two transducers, determines the axis in which the movement is measured in addition to the orientation obtained by the axis tilting. The directivity characteristics of the transducers within a pair must be chosen such as to overlap only within a short distance in order to

yield an axial resolution. The said distance must be at a depth at which the operator's palm lays while operating the pointing device.

If pulsed wave Doppler system is used, each of the said transducers acts both as transmitter and receiver with ultrasound pulse length and time delay determining the space resolution and measurement position respectively. The distance determined by the said range-gate must be at the distance of the operator's palm when moving the cursor. Length of the ultrasound pulse determines the height to which the palm must be lifted in order not to activate the pointing device. In both of the said cases, the movement of the hand above the sensitive area on the computer keyboard yields two Doppler signals which describe the movement of the hand in two dimensions and which can be transformed into the steering signal for the movement of the cursor on the computer GUI (graphic user interface) screen. In both cases, the movement function is not activated if d e palm is not moved within the detecting area, i.e. a predetermined height above the pointing device. This height being determined by the range-gate delay and length of the ultrasound pulse for pulsed Doppler device and by the overlapping area for continuous wave method.

The electronic circuits use the measured Doppler velocity component data by vector summing them and converting these data into signals used for the change of the cursor address, i.e. position of the cursor on the screen.The function of the mouse or trackball buttons, i.e. the confirmation of an action, once the appropriate icon has been pointed at, is done by additional buttons on the keyboard.

The result of the outlined procedure yields an original method for driving the cursor in a Graphic User Interface (GUI).

BRIEF DESCRIPTION OF FIGURES

Fig. 1 is a perspective view of the operator's hand leaning against the pointing device from Fig. 1 and moving in order to move the cursor on the computer screen.

Fig. 2 is a perspective view of the right lower corner of a typical computer keyboard with a recess where the pointing device that detects hand movements is positioned.

Fig. 3 is a schematic perspective drawing of an optical setup for movement detection.

Fig. 4 is a schematic perspective drawing of the two optical setups as per Fig. 3, each of them used for detection of movement in its own plane.

Fig. 5 is a top projection drawing of the optical movement detector of Fig. 4, outlining the actual arrangement of the light transmitters and receivers.

Fig. 6 is a block diagram of the circuitry used to calculate the actual velocity components.

Fig. 7 is a block diagram of the electronic circuitry which can use data from two circuits per Fig. 6.

Fig. 8 is a perspective illustration of a part of the keyboard with the right, bottom end keys and die ultrasound pointing device built into the right corner of the housing.

Fig. 9 illustrates in more detail the setup of the positioning and aiming of the ultrasound transmitter-receivers within the pointing device.

Fig. 10 illustrates the device of Fig. 9 in three projections in order to clarify the preferred geometrical arrangement.

Fig. 11 is a block diagram of electronic circuits which can utilize the data obtained from devices described in Fig. 8 to 10 in such a way as to vector sum the hand

(palm) movement velocity data and form a composite velocity data which is used for cursor movement on the computer screen (GUI).

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the embodiment of Fig. 1, the utilising of computer pointing device without moving parts is demonstrated. The pointing device described herein is based on the detection of the amount and direction of the movement of the operator's palm 5 in front of the here described pointing device, being mounted on the edge of die computer housing 1 at the keyboard 2.

Referring to Fig. 2, one sees that our pointing device has no moving parts at all. It is positioned at the right side of d e computer keyboard for right handed operators and it can be mounted at the left side for the left handed operators. The position of the said pointing device is such tiiat it can be accessed wid out obstructing the view of or access to the keyboard. The pointing device consists of a recess 7 in the housing 1, in this case shown as triangular, although it may have any suitable form (e.g. round, square, etc.). At the bottom 6 of the recess 7 there are mounted linear light transmitters 11 and 14 at right angles to each other, and light receivers 21,27,... in pairs which are parallel to the said light transmitters.

The top of the recess is covered with a light-transparent cover (not shown, e.g. glass sheet) in order to protect the movement detectors and keep the distance of the palm surface down to the bottom 6 constant. The glass cover must be made of hard material resistant to scratching and other damage. Referring to Fig. 3, the principle of movement detection in one direction is illustrated in some detail.

It consists of a linear light source 11 with a lens 12 focussing the sheet-like light field 13 at a position where it intersects the focus points 36 and 37 of d e light receivers 22 and 23.

All the mentioned devices are mounted at the bottom of the recess 7 of Fig. 2. The movement is detected in the plane crossing points 36 and 37. Receivers 21 and 23 serve transmitter 11 while as shown in Fig. 4 receivers 25 and 27 serve transmitter 14.

The light generating source 11 can be any homogeneous light generator (e.g. incandescent, gas discharge or semiconductor LEDs). The essentially cylindrical lens 12 concentrates the light into a sheet like beam, i.e. basically an illumination plane 13, tilted at an angle to d e device bottom 6. The device contains two light receivers 21 and 23 with focussing lenses 22 and 24 which generate narrow directivity characteristics with axes 31 and 32, respectively crossing (piercing) the light plane 13 at the same height at crossing points 36 and 37, respectively and along a parallel to the linear light source 11. This arrangement essentially detects only the light reflecting objects travelling at the height of points 36 and 37. If me reflective object travels in direction along the line connecting points 36 and 37, me light changing patterns in each of the receivers 21 and 23 will be equal, but delayed in time. The time delay depends on me distance between points 36 and 37 and velocity of the movement. It can be shown (see, for example in: A.Papoulis: Probability, Random Variables and Stochastic Processes; McGraw Hill Inc., 1965.) that the cross correlation and cross covariance functions taken between die two detected light variation functions exhibit an extreme (maximum or minimum) at the delay equal to die delay needed for the same feature to reach from one to the other light detector. The light sensitive devices 21 and 23 can be phototransistors or other devices known to detect light and generate proportional electrical signals. The lenses 22 and 23 may be of a simple focussing shape, e.g. plan-convex glass lenses.

Since we wish to detect and measure the velocity of the palm movement in all directions of a plane, the device outiined in Fig. 4 shows an arrangement of two devices as per Fig. 3 arranged at an angle between their light generators. The optimum angle is 90°, but odier angles, except for angle 0° are essentially feasible as well. In this case the two light generators 11 and 14 witii lenses 12 and 15, respectively yield two light planes 13 and 16, respectively which cross along a line 35. The two sets of light receivers 21,23 and 25,27

which have directivity characteristics with axes 31,34 and 32,33, respectively cross the said light planes at essentially die same height from bottom 6. Movement of an object parallel to the light source 11 yields one component of the velocity of the light reflecting object, while die movement parallel to the light source 14 yields the orthogonal velocity. If die said points where the said light receiver axes and the said light planes cross are near enough, diere is an increased probability that at oblique movements, die system can recognise the same features of the operators palm since they can not change abruptly. On die odier hand, if die same feature appears on both of the paired receivers, the said characteristic delay equals zero and thus can automatically be excluded, since me only odier solution is trivial and impossible, i.e. infinite velocity of the hand movement.

Referring to Fig. 5 we see a projection of the light emitting-receiving arrangement at the bottom 6 of the device, which further clarifies the explanation of Fig. 4. The method for calculation of the velocity along one pair of the light receivers can be realised by using electronic circuits as in the block diagram of Fig. 6. The received light variations from the two receivers are fed to a cross correlation circuit. The output of the cross correlation circuit is taken to die maximum (or extreme) detection circuit (for example by detection of an abrupt change of the first derivative), which detects die maximum of the thus calculated cross correlation function and die delay time at which this occurs. Division of the light axes distance by die said delay equals the movement velocity in the specified direction.

Fig. 6 is a block diagram of the circuitry used to calculate the actual velocity components by cross correlating die received light from the two receivers along one axis (in this example X axis), detection of die characteristic delay to d e characteristic discontinuity on die cross correlation curve and division of d e distance between die two receivers by the said characteristic delay value.

Referring to Fig. 7, each of the results of one-dimension measurements as per Fig. 6 and 3 for the X- and die Y-direction are fed to a vector summing circuit which

trigonometrically takes into account the angle between die single dimension devices. When die angle is 90° Pythagorean theorem suffices. The resulting signal is used to steer the cursor on the computer screen.

Referring to Fig. 8, a further embodiment having no moving parts at all is shown. An ultrasonic pointing device is positioned at die right side of die computer keyboard for right handed operators and it can be mounted at me left side for the left handed operators. The position of die said pointing device is such diat it can be accessed without obstructing the view of or access to the keyboard. The ultrasonic pointing device is arranged in a recess 7 of the housing 1, in this case shown as round, although it may have any suitable form (e.g. triangular, square, etc.). At the bottom of the recess two ultrasonic transmitter-receivers 101 and 102 are mounted.

The pointing device has a depdi 9 defined such as to fix the distance between die transmitting-receiving transducers when d e operator actuates the device by moving die palm in front of it. Two ultrasound transmitting-receiving devices 101 and 102 are mounted widiin the pointing device housing.

The transceivers can eidier operate as pulse systems or as continuous wave systems. A pulse system operates by transmitting short pulses of ultrasound and receiving them, when reflected, with d e same transducer. The continuous wave transducer is, in fact, a split transducer where one part of it (the form can vary, but simple geometrical divisions are preferred) continuously transmits ultrasound and the otitier part continuously receives the reflected ultrasound. In both cases is die difference of the transmitted and received ultrasound frequency a measure of the velocity of the reflector (hand) movement and can be evaluated using die Doppler formulae. The depdi 9 of the case 7 is chosen such as to enable die operator to lean the palm against die top of the case 7 and dius stay with the palm at die appropriate distance from me transducers as shall be explained when discussing

Fig. 8,9 and lO.The mode of actuating the Doppler measuring devices by movement of me hand 5 in front of the said ultrasonic devices without touching diem is illustrated in Fig.

1. The hand can, however, be leaned against me computer housing 1 since the depdi of die device described in the comment of Fig. 8 keeps die palm surface at die appropriate distance from the said ultrasonic transducers.

Referring to Fig. 9 we can explain the geometrical relations and die method of operation in more detail.

Ultrasonic transceivers 101 and 102 are mounted at me bottom 6 of the pointing device (illustrated here as a shallow round cup with d e upper rim 106, altiiough any other suitable form (elliptic, triangular, square, etc.) may be used as well. The two ultrasonic transceivers can operate as pulse wave transducers, in which case they are essentially single, normally piezoelectric, transducers, or as continuous wave transducers, in which case they are composed of a continually transmitting and a continually receiving part. In both cases they have beam tilting devices 103 and 104 mounted at d eir front surfaces in order to tilt their respective beams 111 and 112 at appropriate elevation angles φ and T. The projections of die said beam axes at the bottom 6 are under an angle o. which shall preferably be near to the right angle. The beams cross within a well defined area 115 in space at a position so chosen that die palm of the operator's hand moves in mis particular area while activating the system. Thus, die components of die movements of me palm can be detected by Doppler methods using die transceivers 101 and 102 and associated electronic circuits. In order to achieve a practically applicable directivity and narrowness of the main beam lobe, me dominant surface dimension of die transceiver face must be much larger than die wavelengdi at d e frequency used, e.g. ten times larger than die said wavelength.

Fig. 10 shows the device from Fig. 8 and 9 in diree projections, illustrating the preferred geometrical relations of the different components of the system. The elevation tilting of the beams is, in mis illustration, secured with beam tilting devices - lenses 103 and 104, but it is understood that die beam can be tilted by mounting me transceiver transducers proper at some desired elevation and azimudial angle.

The block diagram of electronic circuits which can use the data obtained by reflection of ultrasonic waves from the palm of the operator is illustrated in Fig. 11. The transceivers are connected to Doppler shift measurement devices known in die art of electronics. Alternatively, one can use time domain calculations, e.g. autocorrelation calculations to assess the said velocity. These Doppler shift measurement devices yield components of the movements of the operator's palm and are vector summed, subtracting elevational movements which appear simultaneously on bom 101 and 102 transducers. This vector sum of the palm movement velocities parallel to the bottom (6 in Fig. 9 and 10) are then used to generate the cursor address change in die computer memory. The frequencies used must be of d e order of magnitude of a few hundred kHz, so that the wavelengd is of die order of magnitude of about a millimeter. The advantage of using the pulse mode Doppler device is diat the distance between die said transducer surface and d e sensing volume 115 can be additionally secured by d e pulse-time-gating. Its disadvantage is a more complicated electronic system compared to die continuous wave mode, which, however, has a less well defined sensitivity area 115.

The overall result is the movement of me cursor on the screen widiout the use of moving parts.

A compact computer pointing device has been described widiout moving parts which detects die movement of objects including operator's hand by means of various waves. In an ultrasound embodiment it consists of two ultrasound transmitter-receivers mounted at an appropriate position on die computer housing or keyboard which can sense die direction and velocity of moving objects in front of them using me Doppler effect. In an optical embodiment it consists of two optical transmitter-receivers 11/21; 14/27 mounted at an appropriate position on the computer housing or keyboard which can sense die direction and velocity of moving objects in front of them using light waves and die cross correlation calculations. The movement can be exerted by die operator moving the palm of his hand in front of these transceivers in die desired direction and at die desired speed. The, thus detected, movement is measured and utilized as velocity vectors for generation of the

signals for the screen cursor movement. The system is particularly suited for compact computers where space sparing is important.