Display device and pointing device The invention relates to a display device comprising means for displaying a cursor on a display screen while displaying an object on said screen, a pointing device comprising an actuator having first and second ends, a force sensor arranged at the second end and adapted to detect a force applied to the first end of the actuator so as to provide an electrical signal indicative of the magnitude and the direction of the force applied to the first end of the actuator, and a translator for translating the electrical signal into a speed and a direction of a cursor movement across the displayed object, the pointing device also using usable for performing a scrolling action through the displayed object.

The invention also relates to a pointing device comprising an actuator having first and second ends, a force sensor for detecting a force applied to the first end of the actuator so as to provide an electrical force signal indicative of the force applied and of the direction of the force applied to the first end of the actuator.

A display device of the type described in the opening paragraph is known from International Patent Application publication WO 91/20072. The cited application discloses an electronic book in which a cursor is moved within a range of free movement by means of a joystick. The joystick may be forced beyond the range of free movement to send special signals that will cause the book to page or scroll forwards or backwards.

Although the joystick in the known display device can be used for movement of the cursor as well as for scrolling problems still remain. Only a very simple form of scrolling can be performed and a transition from scrolling to cursor movement and vice versa is cumbersome, since the cursor must be moved outside the range of free movement and, consequently, will be at the edge of the screen after scrolling has ended and must be moved back to a position inside the range of free movement.

It is an object of the invention to provide a display device of the type described in the opening paragraph which allows for a fast and easy transition between cursor movement and scrolling action as well as for an increase of the scrolling capabilities.

To this end, the display device in accordance with the invention is characterized in that the display device comprises a mode selector for selecting a first and a second mode of operation of the pointing device, wherein - in the first mode of operation the electrical signal is translated into a speed and a direction of the cursor movement, and - in the second mode of operation the electrical signal is translated into a main direction of scrolling and a speed of scrolling.

Different types of user interfaces require different pointing solutions. However, in multi- application solutions such as in a TV it will be more and more convenient if different user interfaces are controlled by one and the same pointing device and in an easy manner with an easy transition between different modes of operation.

In the device in accordance with the invention the pointing device allows for direct 360 degree cursor control in the first mode, while allowing scrolling action along main directions in the second mode. The scrolling speed is controllable as well as the scrolling direction in the second mode. Activating the mode selector enables easy and direct transition back and forth between the modes. The same actuator and the same signal are used for the cursor movement as well as for scrolling, thus allowing for a fast and easy transition as well as a simple design of the pointing device.

One possible way of doing this is to provide a position on the screen in which the cursor could be positioned and then activate a mode transition. Such an embodiment would provide easy transition between the modes, control of the speed of scrolling and a smooth and easy change of the direction of scrolling. In preferred embodiments the mode selector is, however, arranged such that mode selection is independent of the position of the cursor on the screen. This allows even faster and easier transition between the different modes, since the cursor can remain in the same position and, consequently, after the scrolling action has ended the cursor will be positioned on the screen, and not at the edge of the screen.

Transition between the modes is preferably activated by a mode selector (for instance, a thumb knob) on the pointing device itself since this allows for fast transition, which can be implemented by one hand, although in embodiments the transition between the modes may be performed by means of a switching element separate from the pointing device.

Preferably in such cases the pointing device and the switching element are arranged to be operable by one hand. An example of such an arrangement is a relatively small pointing device arranged on the keyboard near or in between the keys of a keyboard, whereas, when the pointing device is activated, the space bar (or any other key but, because of their relatively large size, preferably the space bar or the enter key) operates as the switching device between the first and second mode, so that the pointing device can be operated by a finger and switching can be performed by tapping the space bar or enter key with the thumb (or other finger). The actuator upon which pressure is exerted may be a joystick or a pressure pad or a part or element of such.

These and further aspects of the invention will be explained in greater detail by way of example and with reference to the accompanying drawings, in which Fig. 1 illustrates a display device in accordance with the invention, Fig. 2 illustrates schematically a pointing device, Fig. 3 illustrates schematically a pressure sensor actuator of a pointing device, Fig. 4 illustrates the operation of the pointing device in two modes, and Fig. 5 illustrates an arrangement having three main directions.

The Figures are not drawn to scale. In general, corresponding reference numerals refer to corresponding parts.

Fig. 1 illustrates a display device 1 in accordance with the invention. In this exemplary embodiment the display device 1 comprises a monitor 2 with a substantially rectangular display screen 3. The main directions (vertical and horizontal) of the display screen are schematically indicated by arrows on the display screen 3. The device comprises a CPU (central processing unit) 4, a keyboard 5 and a joystick 6. The joystick 6 (forming a pointing device) comprises an arm 7 (forming in this example an actuator) upon which a knob 8 is formed. The joystick also comprises a base 9 in which four pressure-sensitive resistors are integrated. The joystick can be moved in two perpendicular directions, indicated

in Figure 1 by crossed arrows above the joystick. A miniature version 14 of the joystick is also present on the keyboard 5. This joy stick is also movable in two directions indicated by arrows next to the joystick. The joystick 6 is linked to the CPU via a line 10; the keyboard is linked to the CPU via a line 11 and the CPU is linked to the monitor via a line 12. The joystick 6 is operable in two modes. Switching between the modes is done by pressing the knob 8. The miniature joystick 14 on the keyboard 5 is also operable in two modes.

Switching between the two modes is done by pressing the space bar 13. The arrow on the keyboard indicates a preferred position of the joystick vis-a-vis the space bar (or more in general a switching key). Preferably, the distance between the joystick 14 and the space bar 13 (or more in general a switching key) is between 5 and 10 cm. The switching key may for instance also be formed by an enter or shift key. In this example lines are shown for communication between the keyboard and/or the joystick an/or the monitor and the CPU; such communication may, however, also take place in wireless manner.

Figure 2 illustrates a joystick. The arm 7 is provided with a switching knob 8.

Pressure-sensitive resistors are provided in the base 9.

Figure 3 illustrates schematically an arrangement for a force-sensitive resistor (FSR) device in the base 9 of the joystick. The basic FSR device comprises four resistors (RN, Rs, RE and Rw) with resistance values that are a function (usually a decreasing function) of the pressure exerted on the device in the respective N (North), E (East), W (West) and S (South) area. Moving the arm 7 puts pressure on the various resistors RN, Rs, RE and Rw.

The different resistor values (and thereby the force on the various resistors) are measured continuously or regularly using an A-to-D conversion device. Using said resistance values a force vector is construed indicating both the direction and the magnitude (strength) of the force applied. Figure 4 illustrates this vector by way of an arrow 41. Seen in up (U), down (D), left (L) and right (R) direction, the main direction of the arrow 41 is the up (U=in this example North (N)) direction. The main direction is a preset direction for which the projection on said direction of the vector representing the magnitude and angle of the applied force is the largest. In the arrangement shown in Figure 4 there are two main directions, the up-down and the left-right direction. The direction of scrolling is given by the main direction (in this case upwards) and the speed of scrolling is given by the length of the projection on the up-down axis, i. e. the component of the applied force in the upward direction. The main direction and amplitude are represented in Figure 4 by an arrow 42. A translator (which may be part of the CPU as indicated by the reference 4a in Figure 1) translates the electrical signal (s) from the pointing device into a speed and direction of the cursor movement (in the

cursor moving mode) or a main direction and speed of scrolling in the scrolling mode. Such a translator may take any form suitable for the function, i. e. it may be piece of hardware or a program or part of a program (i. e. software). The translator may be part of or provided near the CPU or part of or provided near the pointing device or keyboard.

Switching between the two modes enables fast and easy movement of the cursor as well as scrolling while using one and the same pointing device and the same parts of said device in the same manner and using one and the same signal. Compared to the known device several advantages are obtained. In the known device scrolling is possible by using the joystick, but the speed of scrolling cannot be controlled. In the known device upward scrolling and downward scrolling cannot be performed immediately after each other, since the cursor must be moved from beyond one end of the area of free movement to beyond an opposite end of the area of free movement and inevitably will shift the cursor in the object.

In the present invention upward and downward (as well as left-right scrolling) can be performed immediately following each other. It is also possible to perform up-down scrolling immediately followed by left-right scrolling without having to use the cursor movement mode in between said scrolling actions. Finding a particular page or part of an object <BR> <BR> (program, list etc. ) can thus be done much more rapidly and accurately. Also, speed control is performed in the same manner (and by the same hand or finger) in the cursor movement mode as it is done in the scrolling movement mode. This renders the transition between the modes simple and easy, since the user does not have to switch from one type of control or (part of a) device or hand or finger to another. The main directions in the present invention may seem a restriction in scrolling movement, but in practice scrolling is almost always performed along main directions.

Preferably there are two main directions corresponding to directions along the axes of symmetry of the display screen. In embodiments a third, diagonal, axis may be added.

Figure 5 illustrates a pointing device having three main directions. In the areas 51 and 52 the main direction is up-right; in the areas 53,54, 55 and 56 the main direction is diagonal and in the areas 57 and 58 the main direction is left-right. When the force vector is within these areas it is projected on the respective main direction, which is the closest main direction within these areas.

One possible way of switching between modes is to provide a position on the screen in which the cursor could be positioned and to activate a mode transition subsequently. Such an embodiment would provide easy transition between the modes, control of the speed of scrolling and a smooth and easy change of the direction of scrolling.

In preferred embodiments, as in the examples shown, the mode selector is, however, arranged such that mode selection is independent of the position of the cursor on the screen. This allows faster and easier transition between the different modes, since the cursor can remain in the same position when switching between modes. This offers the advantage that before and after scrolling the position of the cursor will be in a position of the screen the user would prefer. In practice the user will probably keep the cursor near the center of the displayed image or near the top. In practice a user often moves the cursor and scrolls so as to position the cursor in a particular desired position in an object or document. In comparison with the known device, in which after scrolling the cursor finds itself at an edge or even outside the screen, the average distance between the cursor immediately after scrolling and the desired position will be less and, therefore, the time required to position the cursor correctly is reduced.

In the examples shown, representing preferred embodiments, the pointing device is positioned in relation to the switching element such that both can be simultaneously operated with one hand (as on the keyboard) or the pointing device itself comprises the switching element. The pressure pointing device may also be or comprise a pressure pad.

Returning now to Figure 1 the user would operate the pointing device 14 on the keyboard 5 as a cursor control device by manipulating, using one or several fingers, the actuator (in this case the small arm of the micro-joystick). The finger movement is translated into pressure on the pressure sensors beneath, in this example on the resistance values of the FSR sensor beneath. Normally a given finger position leads to a given combination of resistance values (RN, RE, Rs, Rw), which in turn will be translated by soft ware into: in the cursor movement mode: a direction (angle) and speed in the scrolling control mode: a scrolling direction (up-left, down or right) and a scrolling speed.

It will be clear that within the framework of the invention many variations are possible.

Various types of pointing devices may be used. Examples include force- sensitive resistors FSR as shown in the example, which form preferred embodiments, because vector summation is easy on such devices, but also devices using strain gauges, micro- joysticks with capacitive sensors etc.

There are numerous applications for the new scrolling mode with variable speed. Hereinafter, some examples will be given:

scrolling up and down through long documents/lists with a variable speed; for instance, the harder the actuator is pushed in the"down"direction, the faster scrolling down through the document will be, increasing settings such as volume, color or image brightness in a convenient way; for instance, on a TV remote control, the harder the actuator is pressed in the "right"direction, the faster the on-screen slider of the volume will move to louder; the transverse direction can then be used to scroll through the various settings, selecting characters on a virtual on-screen keyboard. The selection would take place by"jumping"up, down, left or right from one character to the neighboring one.

When pressing the corresponding direction harder on the actuator knob, the speed at which the character selection moves across the virtual keyboard is increased. This means that the selection of characters that are relatively far away from one another on the keypad becomes more convenient, selecting different spooling/winding speeds in devices such as tapedecks, CD players, DVD players, VCRs. For instance, in a DVD player, the harder the actuator is pressed to the left, the faster the"rewind"of the video images will be.

The invention can be used in a wide variety of devices, that is in short in all devices with cursor control: keyboards, remote controls, PDAS, notebook computers, gaming devices, information kiosks, PC peripherals.

The scope of the invention is not limited to the embodiments explicitly disclosed. The invention is embodied in each novel characteristic and each combination of characteristics. Any reference signs do not limit the scope of the claims. The word "comprising"does not exclude the presence of other actuators than those listed in a claim.

Use of the word"a"or"an"preceding an actuator does not exclude the presence of a plurality of such actuators unless otherwise indicated.

In short the invention can be described as follows: a display device comprises a pointing device such as a joystick. Pressure applied to the arm of the joystick provides an electrical signal indicative of the direction and magnitude of the force applied. The pointing device may be switched between two modes, that is, a cursor movement mode and a scrolling mode. In the cursor movement mode of operation the electrical signal is translated into speed and direction of the cursor movement and in the scrolling mode of operation the electrical signal is translated into a main direction of scrolling and a speed of scrolling. Switching is preferably performed by means of a mode selector (8, 13).