Field of the invention

The present invention relates to a system for translating visual movements of an object into tactile stimulations of a user, comprising an IR camera adapted to capture a series of images, an image processing unit adapted to receive the series of images from the camera and to generate one or more signals based on the received images, and a receptor stimulating device comprising a two dimensional matrix having a plurality of elements adapted to, upon activation, stimulate at least one tactile receptor of a part of the user, and the receptor stimulating device is adapted to receive the signals from the image processing unit, and to activate one or more of the elements based on the received signals.

Background of the invention

Persons that are blind or have impaired vision are able to read and visualize their surroundings using a detailed sense of touch. For reading, the Braille display is used, whereby patterns of raised dots pressed on paper represent Braille characters. Tactile receptors under the skin of the fingers allow the user to feel the different characters. Many computer based systems have been developed, whereby objects are translated into signals that can be received by tactile receptors.

For example DE 197 11 125 and DE 42 41 937 disclose systems that allow a blind person to experience the intensity of colors. A grid pattern of elements is used to send signals to the skin of a person. These elements may be pins moving from and to the skin surface, or electrical elements sending a current to the skin. Colored images from a camera are processed in a computer device to produce signals for an element, whereby the signals have different intensities.

GB 2523355 discloses a system to aid a blind person to learn musical notations. A camera records musical notes on paper and translates these notes to elements positioned under a sole of a foot. The elements are transducers and positioned on specific spots under the foot, such that each note has its own position under the foot. Although this system works well for learning musical notes, it cannot be used for quick reading of musical notes while playing a song. US 8,154,392 discloses a system comprising a touch inducer pad that is in contact with a sole of a foot. The inducer pad has a grid pattern of electro vibrating inducers as elements that are wirelessly connected using a low radiofrequency for transmission. The system is adapted for armed forces in combat, whereby each individual soldier has a gate code wearer's address selector with its own frequency. The system is adapted to send simple signals, such as directions to the soldiers. The source of the system may be equipped with a camera in combination with software manipulation to "visualize" outlines of an object placed in front of the camera.

US 7,352,356 discloses a system to tactile simulate a virtual displayed image using a high density set of points (e.g. movable pressure responsive pins) at a finger or foot of a blind person. During scanning, the image is translated into signals of the pins in the matrix in order to visualize the scanned image. A pattern of the image is caused to move across a surface of the skin by selective activation and deactivation of the pins in a matric of pins. The pressure response is caused using a fluid in a microelectromechanical system (MEMS) comprising an array of microvalve actuators.

Apart from systems that can visualize static objects, systems for recoding moving objects have been developed. Some of the challenges in the development of these systems is the filtering of "noise" such as movement of other objects in the surrounding. This is especially a problem when the moving object is small in a background, where larger objects move at the same time. This filtering can be done by complex computer programs. However, this slows down the translation process to the elements. Another challenge is the translation of fast moving objects. Although software for tracking a recorded small object exists, translation of such fast movement to the elements is difficult to realize in real time, i.e. without delay.

Different types of elements have been developed to overcome this problem. Electro elements that give an electric impulse have a good response time, but are unpleasant for the user and the signal produced is not always clear or strong enough. Vibration elements sometimes suffer the same problem. The use of pressure responsive pins as elements provide clear signaling to the tactile receptor. However, the time needed to move the pins, e.g. by using a fluid in MEMS comprising an array of micro valve actuators may introduce a latency that prohibits sensing the tactile signals in real time, i.e. without delay.

US2009/0326604 discloses a system for managing brain functions and improving sensory perception. An electro-tactile screen of elements is placed in contact with the tongue of a person by using an individual molded retainer to hold the screen. The elements are preferably wireless connected to a computer device or image processing unit, which sends output signals to the elements. The image processing unit receives signals from e.g. an IR camera placed on the head of a blind person. The system allows a blind person to visualize a ball and catch it. The image processing unit creates a complex multidimensional electro tactile image similar to visual imagery. The stimulation by the element may be an electro-tactile stimulation, a vibro- tactile simulation or the input may be audio input, visual input or a temperature depending input using heat and cold for signaling. Because the invention in US2009/0326604 is aimed at improving impaired brain functions, such as balance, the document is not aimed at filtering background movement "noise", such that small fast moving objects can be visualized or sensed by a blind person without delay. I n this system, the where and when precision is more important than the speed of a movement. Further, using complex algorithms in an image processing unit slows down the translation of the movement to the elements. I nfrared ca meras have been used to record moving objects. A problem with infrared cameras may be that these cameras have a latency in their recording. This causes a delay in translation and makes it difficult to translate a moving object to an element without delay.

A blind m usician that wishes to play in an orchestra needs to read or sense musical notes or sense instructions from a conductor in order to be able to play together with the others. Especially for playing music together with others, significant delays in translation time from signa l source to tactile receptor are not acceptable. There is a need for the translating visual movements into tactile stimulations without delay in time. The existing systems do not allow a blind person to play in a concert, because the present systems do not allow approximate real time translation from the movement of a baton of the conductor to a tactile signal.

Object and summary of the invention

It is an object of the present invention to at least partly overcome the above problems, and to provide an improved system for translating visual movements using a tactile stimulation.

This object is achieved by a system as defined in claim 1.

The system comprises an I R camera adapted to capture a series of images and an image processing unit, such as a computer. This unit is adapted to receive the series of images from the camera and to generate one or more signals based on the received images. The system also comprises a receptor stimulating device comprising a two dimensional matrix of a plurality of elements, which extend along an X-axis and a Y-axis. This device is adapted to, upon activation, stimulate at least one tactile receptor of a part of the user, and the receptor stimulating device is adapted to receive the signa ls from the image processing unit, and to activate one or more of the elements based on the received signals.

The system further comprises a light emitting device designed to be attached to the object and configured to emit light of a wavelength larger than 700 nm. The IR camera is adapted to capture a series of images of the light emitting device. The light emitting device may be a moving device. The I R camera is disposed at a distance between 0.5 and 20 meters, or 1 to 10 meters, from the light emitting device.

The image processing unit is adapted to determine two dimensional positions of the light emitting device in a defined 2D coordinate system based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device, a nd the receptor stim ulating device is ada pted to activate at least one of the elements of the matrix corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals, such that translation of the movement of the object to the elements is realize in real time. The light emitting device may be attached to a moving object, such as a stick/baton of a conductor. The system allows for rapid recording of a movement by the IR camera. Background "noise" in the form of movements of other lightened objects that do not need to be recorded, are prevented from being recorded by using the light emitting device that emits light at a wavelength larger than 700 nm, i.e. a wavelength in the IR spectrum. The system can thus discriminate between light from the light emitting device and light from other sources in the background of the device.

The system according to the invention allows translation of visual movement into element activation almost without delay, i.e. practically in real time. This makes it for example possible for a blind musician to play together with other musicians in an orchestra. The system may be used for many other applications, such as teaching.

In an embodiment, the IR camera has a resolution of at least 1665 x 1088. In another embodiment, the IR camera has a resolution of at least 640 x 480. IR cameras may have a latency in the recoding. By using an IR camera having at least a resolution of at least 640 x 480, or a latency of 20 milliseconds or less the translation of the recoded image to the receptor stimulating device can be achieved without delay. In one embodiment, the IR camera has a latency between 1 and 20 milliseconds.

In a further embodiment, the IR camera has a pixel size between 3 and 7 micrometers. The quality of the camera is important for the translation of the movement. The better the quality the quicker the translation of the movement of the object to the receptor stimulating device.

In an embodiment, the IR camera operates at a wavelength between 1000 and 14000 nm. The wavelength may be between 690 and 1050 nm when a diode is used as the light emitting device. The wavelength may be between 1000 and 14000 nm if a temporized (heated or cooled) light emitting device is used.

In a further embodiment, the distance between the IR camera and the light emitting device is between 1 and 5 meters.

In another embodiment, the IR camera is a thermal imaging camera. A thermal imaging camera is believed to be less sensitive to light in a background of the light emitting device, which improves the accuracy of translation of the movement of the object.

According to an embodiment of the invention, the light emitting device comprises at least one light emitting diode. Light emitting diodes (LED) are small and can easily be attached to an object, such as the baton of a conductor. Further, LEDs are electrically efficient. A ring of diodes may be used as a light emitting source. The diodes can be stacked in a housing or casing. This way, the IR camera perceives the diodes as one and the same light spot. This increases the intensity of the light and improves the distinction by the camera between light from the light emitting device and light from the backgraound.

In an embodiment, the light emitting device comprises at least one temporized element. The temporized element, may be any kind of element that can be heated or cooled. In one embodiment, the temporized element is a resistor. In another embodiment, the temporized element is a peltier element. These temporized elements are simple and robust and readily available. The elements can be made in small sized that can be attached to a moving object. The temporized elements allow for a quick translation of visual movements of the object to which the element is attached.

In an embodiment, the receptor stimulating device is adapted to keep the elements activated for a defined time period after the position has been received. For example, the time period is between 0.1ms and Is. Thus, several of the elements can be activated simultaneously and thereby forming a pattern, which can be recognized by the user. The pattern of activated elements corresponds to the movements of the light emitting device and accordingly to the movements of the object.

In a further embodiment, the receptor stimulating device comprises a pushing member adapted to move each individual element to and from the tactile receptor, depending on the signal from the image processing unit. The pushing member allows all elements to be signaled individually in sequence. This gives the user a tactile replication of the exact movement of the light emitting device. In another embodiment, the receptor stimulating device comprises two linear electromagnetic motors adapted to move the pushing member along an X-axis and a Y-axis of the matrix. The motors are advantageously used in the receptor stimulating device, because they can operate at high speed, such as at least 5 m/s. This allows movement of the writing ball along the X- or Y-axis at a speed of e.g. 5 m/s and thus prevents delay in signaling the movement to the elements. The receptor stimulating device is preferably robust and comprises only a few parts. The material is light such as aluminum. The receptor stimulating device can be used by any individual, i.e. the device does not need to be fitted or molded before it can be used. This reduces cost and improves ease and flexibility in use of the device. In an alternative embodiment, the receptor stimulating device comprises two linear ultrasonic piezo motors adapted to move the pushing member along an X-axis and a Y-axis of the matrix.

In one embodiment, the pushing member is a writing ball in a holder. The round surface of the ball allows for a fluid movement along pins. I n another embodiment, the elements are pressure responsive pins movable along a Z-axis, which extends substantially perpendicular to the X- and Y-axis. When movable pressure responsive pins are used as elements, the pins will be pushed up by the writing ball one by one.

The receptor stimulating device ca n be provided in different ways depending on the application of the system and the technique available. The present invention is not limited to a specific receptor stimulating device. The elements in the matrix may be stimulated by a motor adapted to move the pushing member along an X-axis and a Y-axis of the matrix as defined above. The elements in the matrix may be stimulated more directly by the image processing unit, i.e. without the use of a motor that stimulates the elements.

I n an embodiment, the elements are linear ultrasonic piezo motors adapted to move along a Z-axis upon receiving the one or more signals from the image processing unit.

I n another embodiment, the elements are electromagnets adapted to move along a Z-axis upon receiving the one or more signals from the image processing unit.

I n a further embodiment, the elements are electroactive polymer actuators adapted to move along a Z-axis upon receiving the one or more signals from the image processing unit.

The more direct stimulation improves the speed of translation of the movement of the object into tactile stimulation of the receptor of the user. This also improves flexibi lity of the use of the system because writing in Braille is possible if each element is stimulated individually. The system can thus be used for e.g. teaching.

I n one embodiment, the elements are embedded in a polymer. The polymer may be polyurethane and the like. Embedding of the elements in the polymer makes the receptor stimulating device more robust and less sensitive to damage. This increases the life time of the receptor stimulating device and thus reduces costs for the user.

I n a further embodiment, the image processing unit uses tracking software to register the movement of the light emitting device. The tracking software improves the speed of transport of the signal from the light emitting device into the movement of the elements in the matrix of the receptor stimulating device.

The object of the invention is also achieved by a method for tra nslating visual movements of an object into tactile stimulations of a user, comprising:

- emitting light at a wavelength larger than 700 nm by a light emitting device,

- ca pturing a series of images from the emitted light by a n I R camera, wherein the camera is disposed at a distance between 0.5 and 20 meter from the light emitting device,

- receiving the series of images from the I R camera, determining two dimensional positions of the light emitting device in a defined plane based on the received images and generating one or more signals representing the determined two dimensional positions of the light emitting device by an image processing unit, - receiving the signals from the image processing unit by a receptor stimulating device comprising a two dimensional matrix of a plurality of elements,

- activating an element of the matrix corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals, and

- stimulating at least one tactile receptor of a part of the user by a receptor stimulating device, such that translation of the movement of the object to the elements is realize in real time. In one embodiment of the method, the camera has a resolution of at least 1665 x 1088, and a latency between 1 and 20 milliseconds. The camera and the receptor stimulating device may be defined according to any of the embodiments mentioned above.

A further object of the invention relates to a use of the system as defined above or the method defined above for visualizing the movement of an object for a user having impaired vision.

A further object of the invention relates to a use of the system as defined above or the method defined above for visualizing movements of a baton of a conductor for a user having impaired vision, wherein the light emitting device is attached to the baton. The invention relates to a use of the system and the method by a visually impaired person to play music while following visual instructions of a conductor. In one embodiment, the invention relates to a use of the system as defined above or the method defined above for providing a tactile pattern redition.

The present invention also relates to a teaching system for translating visual movements of an input member into tactile stimulations of a user, comprising:

- a processing unit adapted to receive signals from the input member and to generate one or more signals based on the received signals, and

- at least one receptor stimulating device comprising a two dimensional matrix having a plurality of elements adapted to, upon activation, stimulate at least one tactile receptor of a part of the user, and the receptor stimulating device is adapted to receive the signals from the processing unit, and to activate one or more of the elements based on the received signals, characterized in that the system further comprises

a signaling device designed to be connected to the input member and configured to emit a signal to the processing unit, whereby the signaling device is positioned at a distance between 0.5 and 20 meters from the receptor stimulating device,

the processing unit is adapted to determine two dimensional positions of the signaling device in a defined 2D coordinate system based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device, and

the receptor stimulating device is adapted to activate an element of the matrix corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals, such that translation of the movement of the input member to the elements is realize in real time. The input member may be a stick or pen held by a teacher or a keyboard or a computer mouse. In the systems of the invention, the signaling device can be connected to a plurality of receptor stimulating devices. Thus, a teacher using or holding the input member can teach a plurality of pupils at the same time and in real time. For example, the writing of a teacher on a board ortablet can immediately be translated into Braille forthe pupils. Especially when the receptor stimulating device is positioned under a foot of the pupil, the pupil will be able to use its hands to make notes or respond to the teacher. The systems can thus also be used by people that are deaf and visually impaired. The signal may be a moving signal. The camera and the receptor stimulating device may be defined according to any of the embodiments mentioned above.

The advantage of the teaching system are apparent from the advantageous mentioned above.

One embodiment relates to a use of the teaching system as defined above for visualizing the input of the input member for a user having impaired vision. The input member may be a keyboard used by the teacher for writing, or a data screen used to visualize an object or scheme. The object on the screen can be directly translated to the receptor stimulating device.

Brief description of the drawings

The invention will now be explained more closely by the description of different embodiments of the invention and with reference to the appended figures.

Fig. 1 shows a block diagram of a system for translating visual movements of an object into tactile stimulations of the invention,

F ig. 2 illustrates an example of a system of the invention.

F ig. 3 shows an example of a receptor stimulating device positioned under a foot,

F ig. 4 shows an example of a baton of a conductor provided with a light emitting device.

Fig. 5 shows details of a receptor stimulating device according to one embodiment of the invention.

Figs. ( show two linear motors in a frame of the receptor stimulating device.

Fig. 7 shows an example of a matrix of elements and a position of a writing ball in a matrix.

Fig. 8 illustrates an example of a translation of a movement of a stick into signals of activated elements.

Fig. 9 shows a flow diagram of an example of a method of the invention.

Fig. 10 shows a block diagram of another system for translating visual movements of an object into tactile stimulations of the invention.

Fig. 11 shows a flow diagram of an example of a method of the other system of the invention.

Detailed description of preferred embodiments of the invention Figure 1 shows a block diagram of a system for translating visual movements of an object into tactile stimulations of a user according to an embodiment of the invention. The system comprises a light emitting device 1 designed to be attached to the object, which movements are to be detected, and configured to emit light at a wavelength larger than 700 nm, i.e. in the IR spectrum.

The system further comprises an IR camera 2 adapted to register the IR light emitted from the light emitting device 1 and to capture a series of images of the light emitting device 1.

The IR camera 2 may operate at a wavelength from 700 nm, or between 690 and 14000 nm. The IR camera may operate at a wavelength between 690 and 1050 nm. If the IR camera 2 is a thermal imaging camera, the wavelength may be between 1000 and 5000 nm (short wave camera) or between 5000 and 14000 nm. Examples of thermal imaging camera may be a FLIR E 60 or Testo865.

The IR camera 2 must have a high resolution that prevents latency in recording of images from the light emitting device 1. Preferably, the IR camera has a resolution of at least 640 x 480, or 1280 x 1024, or 1665 x 1088, or 2048 x 2048. The latency may be below 10, 9, 8, 7, 6, 5, 4, 3,

2 milliseconds, or between 1 and 20, or 1 and 10 milliseconds. The pixel size may be between

3 and 7 micrometers, or between 4 and 6 micrometers. The IR camera and/or the image processing unit 4 may use a tracking application that allows tracking of the movement of the light emitting device 5. Preferably, the IR camera is a 2D camera and the images are 2D images.

Any light emitting device 1 may be used in the system that emits light at a wavelength larger than 700 nm, or between 700 and 1050 nm, or between 700 and 1000 nm, or between 800 and 900 nm, or between 825 and 875 nm. If the light emitting device is a temporized element, it may emit light at a wavelength between 1000 and 14000 nm, or between 100 and 5000 nm, or between 5000 and 14000 nm. Suitably, the light emitting device 1 is an IR Light Emitting Diode (LED). The light emitting device 1 may be one or more temporized element. Examples of the temporized element may be a resistor. Any type of resistor may be used that can be attached to a moving object. Examples may be a metal film resistor. The temporized element may also be a peltier element. Peltier elements can be heated or cooled, which increases flexibility of use of the light emitting device. An example of a peltier element may be a Quickcool QC 32-0.6-1.2. The dimensions of the light emitting device is preferably about 10 x 10 x 5 mm, or 8 x 8 x 3 mm or smaller. A plurality of light emitting devices may be organized in a casing attached or connected to the object (not shown).

The IR camera is disposed at a distance from the light emitting device 1. Preferably, the I R camera is disposed at a distance of between 0.5m and 20m or between lm and 5, or 1 and 3m from the light emitting device 1 so that the view of the camera and accordingly the images will contain the light emitting device 1 even though the light emitting device is moved around, without moving the camera. Preferably, the camera 2 has a fixed position in the surroundings of the moving object. The system also comprises an image processing unit 4 connected to the IR camera 2 and adapted to receive the series of images from the IR camera 2, to determine two dimensional positions of the light emitting device in a defined plane based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device based on the received images. The positions of the light emitting device are determined in a 2D coordinate system defined in relation to the view of the camera.

The camera comprises a lens, i.e. wide angle which defines the camera's image capture zone. The 2D coordinate system has two coordinate axes, such X and Y axes. The image processing unit 4 is, for example, configured to generate one signal corresponding to an X coordinate of the determined position and another signal corresponding to a Y coordinate of the determined position. Alternatively, the signal is a data signal comprising data packages including information about the position, such that coordinates of the position. The image processing unit 4 comprises input and output means, processing means, such as a CPU, a FPGA or similar hardware, and memory means, such as ROM, RAM or similar hardware. The image processing unit 4 also comprises software for providing image processing of the images. The image processing unit is, for example, a computer, such as, a PC. Appropriate data processing means and software for carrying out image processing as such are known by the skilled person and will thus not be explained in detail.

The system further comprises a receptor stimulating device 5 comprising a matrix 16 having a plurality of elements 6 adapted to, upon activation, stimulate at least one tactile receptor of a part of the user. For example, the elements are elongated and are moved in a longitudinal direction upon activation. The tactile receptors of the user are stimulated by applying pressure on the receptor. Alternatively, the elements are caused to vibrate upon activation, and the tactile receptors of a part of the user are stimulated by the vibrations. The matrix defines a two dimensional plane corresponding to the defined plane of the 2D positions of the light emitting device. Each element of the matrix corresponds to a position of the light emitting device. The receptor stimulating device 5 is connected to the image processing unit 4 and adapted to receive the signals from the image processing unit 4, and to activate an element of the matrix 16 corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals. The element 6 of the matrix having a position in the matrix corresponding to the received two dimensional position, is active upon receiving the position que. A series of positions is received. Suitably, the receptor stimulating device 5 is adapted to keep the elements 6 activated for a defined time period of between 0.1ms and Is after they have been activated. Thus, several of the elements can be activated simultaneously or sequentially and thereby form a pattern, as shown in figure 8, which pattern can be recognized by the user. The pattern of activated elements corresponds to the movements of the light emitting device and accordingly to the movements of the object. Figure 2 illustrates a practical example of the system. In this example, the light emitting device 1 is attached to a baton moved by a conductor. Movement of the light emitting device is recorded by the IR camera 2. The IR camera captures a series of images from the emitted light device. The series of images are received using a wire 3 or wireless by the image processing unit 4. The image processing unit 4 translates the input from the IR camera into a signal using software. The signal is sent to the receptor stimulating device 5. The receptor stimulating device causes activation of the elements 6 comprised in the receptor stimulating device. In figure 2, the receptor stimulating device is positioned under a sole of a foot 7 of a musician 8 playing a flute, while seated on a chair 9.

Preferably all the parts of the system can be held together and easily transported in a suitcase in order to be used at different locations.

By using adapted software, the images from the IR camera 2 are translated into a signal, which is send as an output signal to the receptor stimulating device 5. Various receptor stimulating devices and various elements may be used. The receptor stimulating device is a device capable of receiving signals from the image processing unit 4 and translating these signals into element activity. Such a device may use fluid or air to activate microvalves in electro-tactile element 6, or the device may use electricity to provide an electrical impulse or a vibration. Different types of elements may be used, such as electronic or electrical elements, vibration elements, heat elements, pressure elements, and the like.

Figure 3 shows a foot 7 of a user positioned on the elements of the receptor stimulating device. The receptor stimulating device 5 comprises a matrix 16 of a plurality of elements 6. The matrix extends along an X-axis and a Y-axis as shown in figure 7, 8. The matrix 16 of elements 6 may cover a portion of the area under the foot or the entire area under the foot. In one embodiment, the receptor stimulating device comprises two linear electromagnetic motors positioned in a frame 18 under the matrix 16. One motor 19a can be moved along an X-axis X and one motor 19b can be moved along a Y-axis Y. The motors may be positioned on top of each other as shown in figures 6a, 6b. The motors receive the output signal from the image processing unit 4, or an Χ,Υ-signal. Depending on that signal, the motors will move along the Χ,Υ-axes of the matrix of the elements 6. Figure 6d shows that the area of the matrix 16 may be less than the area of the receptor stimulating device. The area may be between 20 x 20 cm or 10 x 10 cm 4 x 4 cm, or 10 x 5 cm, and so on. The invention is not limited to any specific area and can be varied depending on the application and user. A smaller area reduces costs.

The motors may be two linear electromagnetic motors 19a, 19b (figures 6a, 6b) or linear ultrasonic piezo motors 19a, 19b as shown in figures 6c.

A reverse piezoelectric effect can be explained as an internal generation of a mechanical strain resulting from an applied electrical field. Different types of materials exhibit piezoelectric properties. Examples of materials may be ceramic, tendon, silk, wood, enamel, dentin, DNA, viral proteins, quartz-analogous crystal, such as langasite (La3Ga5Si014) or gallium orthophosphate (GaP04), lithium niobate (LiNb03), lithium tantalate (LiTa03), synthetic ceramics such as tetragonal unit cell of lead titanate, non-ferroelectric piezoelectric materials, such as AIN and ZnO, tungsten-bronze, barium titanate (BaTi03), lead zirconate titanate (Pb[ZrxTil-x]03 with 0 < x < 1) more commonly known as PZT, potassium niobate (KNb03), Sodium tungstate (Na2W03), Ba2NaNb505, Pb2KNb5015, lead-free piezoceramics, sodium potassium niobate ((K,Na)Nb03) also known as NKN, Bismuth ferrite (BiFe03), sodium niobate NaNb03, Bismuth titanate Bi4Ti3012, sodium bismuth titanate NaBi(Ti03)2, lll-V and ll-VI semiconductors, polymers, polyvinylidene fluoride (PVDF) and organic nanostructures.

Attached to the motors is a pushing member 20. This pushing member is in contact with the elements that are positioned above the pushing member in the direction of a Z axis Z. As shown in figure 5, the pushing member pushes the elements up. The sensors move down by gravity. The pushing member 20 may be a writing ball 20b attached to a holder 20a. The member pushes each individual element up, when the motors 19a, 19b move along the X- and Y-axes.

The elements 6 may be movable pins 17. The extent by which the pins are pushed upwards toward the foot must be such that the tactile receptors in the skin are stimulated by the pressure of the pins. The length of the element can be increased in upward extension. This allows to effectively demark an additional piece of information valuable to a user, such as for musical interpretation. Solenoid magnets can me employed for the purpose.

The elements 6 may also be moved individually upon receiving a signal from the image processing unit. i.e. without the use of a motor.

The elements 6 may be individual linear ultrasonic piezo motors that can move along a Z-axis Z upon receiving the one or more signals from the image processing unit 4. An example of such a piezo motor may be an MPO-05 Piezo, such as the MPO-050015.

The elements 6 may be individual electromagnets that can move along a Z-axis upon receiving the one or more signals from the image processing unit. Solenoid magnets can me employed for the purpose.

The elements 6 may be individual electroactive polymer actuators that can move along a Z- axis upon receiving the one or more signals from the image processing unit.

Electroactive polymers, or EAPs, are polymers that exhibit a change in size or shape when stimulated by an electric field. The most common applications of this type of material are in actuators and sensors. A typical characteristic property of an EAP is that they will undergo a large amount of deformation while sustaining large forces. The majority of historic actuators are made of ceramic piezoelectric materials. In the context of this application, EAPs are defined as electroactive polymer actuators not including piezoelectric materials, which are mentioned separately. Examples of EAPs may be electrostrictive polymers and dielectric elastomers. Other examples may be ferroelectric polymers, electrostrictive graft polymers, liquid crystalline polymers, ionic EAPs, in which actuation is caused by the displacement of ions inside the polymer, electrorheological fluid, ionic polymer-metal composite and stimuli-responsive gels.

EAP materials allow to pack many actuators in a small area such as the matrix 16 without interferences. Individual stimulated elements allow for display of an active, full-page and refreshable Braille dots to a user.

The elements 6 may be supported by a cloth or canvas on a bottom side of the elements. Alternatively, the elements may be embedded in a polymer 13, preferably a biodegradable polymer as shown in figure 6e. Examples of suitable polymers may be polyurethane.

Figure 9 shows a chart illustration of a method for translating visual movements into tactile stimulations. It will be understood that some of the blocks of the flow chart can be implemented by computer program instructions.

The method may comprise the steps outlined below.

Emitting light of a wavelength larger than 700 nm by a light emitting device 1 attached to the object. Any light emitting device as mentioned above can be used for this purpose. The object may be moving.

Capturing a series of images from the emitted light by an IR camera 2. The camera is disposed at a distance between 0.5 and 20 meter from the light emitting device. Any of the cameras mentioned above may be used.

Receiving the series of images from the camera, determining two dimensional positions of the light emitting device in a defined plane based on the received images and generating one or more signals representing the determined two dimensional positions of the light emitting device.

Receiving the signals from the image processing unit 4 by a receptor stimulating device 5 comprising a two dimensional matrix of a plurality of elements 6.

Activating an element of the matrix 16 corresponding to the received two dimensional position of the light emitting device 1 upon receiving the one or more signals.

Stimulating at least one tactile receptor of a part of the user by a receptor stimulating device. Any of the receptor stimulating devices outlined above may be used for this purpose.

The translation of the movement of the object to the elements is realize in real time.

The system or method may be used for visualizing an object for a user being blind or having impaired vision. The system and the method are especially useful for musicians using wind instruments, which do not allow stimulation of tactile receptors in a mouth of the user. The receptor may be located anywhere on the body of the user. The receptor may be located at the sole of the foot of the user. The user may be a musician and the moving object may be a baton 10 of a conductor 11. The elements 6 may be pins 17 as described above. Alternatively, the individual elements 6 can be stimulated in the same manner, such that the same pattern or line is felt by the user.

Figure 7 shows a top side of the matrix 16. As illustrated in the figure, by pushing up a sequence of pins by the pushing member, a cursor 21 is provided on the matrix. By moving the pushing member 20 along the X axis a "line" Ix can be felt on the sole of the foot 7. By moving the pushing member along the Y axis a "line" ly can be felt on the sole of the foot 7. The "line" exists of a sequence of individual upward and downward moving pins 17 along the area of the matrix 16.

Figure 8 symbolizes a translation on the matrix of the movement of the light emitting device, which has been moved up, down and then to the right. These types of movements are commonly used with a baton of a conductor 11 to assign a task to an orchestra. Because the elements 6 react at the same time as the light emitting device is moved, i.e. without delay, the system of the present invention can be used by a blind person playing in an orchestra. Examples of further application can be for educational purpose, such as teaching dance or yoga: where tactile stimulation is the preferred means of receiving instruction. The deaf-blind can be especially appreciative of this form of communication and instruction. The invention also relates to a teaching system for translating visual movements of an input member 12 into tactile stimulations of a user as shown in figure 10 and 11. The input member may be a pin or a keyboard.

This system comprises a processing unit 4 adapted to receive signals from the input member and to generate one or more signals based on the received signals, and a receptor stimulating device 5. The receptor stimulating device 5 comprises a two dimensional matrix 16 having a plurality of elements 6 adapted to, upon activation, stimulate at least one tactile receptor of a part of the user. The receptor stimulating device is adapted to receive the signals from the processing unit, and to activate one or more of the elements 6 based on the received signals. Any of the receptor stimulating device mentioned above can be used for this purpose.

The system further comprises a signaling device la designed to be connected to the input member and configured to emit a signal to the processing unit 4. The signaling device la may be attached to or connected to the input member. When the input member 12 is a keyboard or computer mouse, the signaling device may be a data screen with images or moving images. The signaling device la is positioned at a distance between 0.5 and 20 meters, or 1 and 5 meters, or about 3 meters, from the receptor stimulating device 5.

The processing unit 4 is adapted to determine two dimensional positions of the signaling device in a defined 2D coordinate system based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device, and the receptor stimulating device 5 is adapted to activate an element of the matrix 16 corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals, such that translation of the movement of the input member to the elements is realize in real time.

The systems of the invention are especially useful for communication with visually impaired persons as well as persons that are both deaf and visually impaired.

The present invention is not limited to the embodiments disclosed but may be varied and modified within the scope of the following claims. For example, the elements may be vibrating or heat elements. Different methods may be used to move the pins, or different types of pushing member 20 may be applied.

1 light emitting device or IR diode

2 IR camera

3 wire

4 image processing unit

5 receptor stimulating device

6 element

7 foot

8 musician

9 chair

10 stick

11 conductor

12 input member

13 polymer

16 matrix

17 pin

18 frame

19a,19b motor

20a,20b pushing member holder, writing ball

21 cursor

r recording

ix, ly line along X-or Y-axis

X X axis

Y Y axis

z Z axis