TECHNICAL FIELD

This invention is about an electronic device that integrates the sense of touch and sight while taking in text as input to the brain. The feasibility of this idea is based don the neuroscience hypothesis, which proposes that human senses can be integrated. The invention, which is based on the concept of converting a text into tactile (vibration), helps to acquire information without the need for the senses of sight and hearing, which can be beneficial for both visually impaired and sighted people.

BACKGROUND

At times when new information is growing exponentially, people face difficulties in keeping up with this high growth rate. Although much time and effort were spent throughout human history in developing various devices, tools and techniques to improve the learning process. Only little attention was paid to how the human brain consumes or receives information and what innovations can be made in this regard. Spaced repetition, a method used in concepts and knowledge acquisition, is known to be efficient, although it is an energy time-consuming technique due to its requirement for occupying our most used senses such as eyes and ears. Moreover, visually impaired people are required to have their hands occupied while reading braille using their fingertips.

Sensory integration, the method of passing sensory information through a different sensory pathway, has been documented for decades. (Eagleman, 2020). Typically, this involves translating visual information from a video feed into touch on the skin/tongue, or into soundscape (Bach-y-Rita et. al., 1969; Danilov and Tyler, 2005; Grant et. al., 2016; Amedi et. al., 2007; Striem-Amit et. al., 2012). Sensory integration devices have also been created and tested for various modalities, including balance- to-touch to alleviate balance disorders and touch-to-touch to give somatosensation to someone with nerve damage or prosthetic limb wearers (Tyler et. al., 2003). Sound- to-touch sensory integration devices have also been developed to help people with hearing loss and deafness (Weisenberger and Miller, 1987; Weisenberger and Percy, 1995). Such devices can help people with sensory impairments by giving them access the sensory information they cannot otherwise perceive. Although sensory integration has been experimentally fruitful, it lacked practicality due to size, expense, or inconvenience of wearing (Bach-y-Rita, 1983; Maidenbaum et. al., 2014). In the light of these, our invention aims to provide a practical and useful electronic sensory integration device.

The invention is able to convert text into vibration patterns and it comprises four vibration motors together with an advanced operating system. So, it can help people process newly learned information into long-term memory more easily and in a shorter time through technique of spaced repetition. Because the invention uses the sense of touch as a sensory pathway, users will have an alternative information input to their brain. In addition, neuroplastic changes must occur in the brain for the sensory integration device to be fully utilized. (Perrotta, et. al., 2020). The invention is based on these neuroplastic changes and is conducted under the assumption that the application of the electronic device will be beneficial in the real world.

The braille language is a great alternative for reading for the visually impaired people. However, it requires the hands of the reader to be equipped during the process of reading. So, no device in the background is available for the visually impaired people to have a communication with vibrations. Thus, our invention can also be used in that purpose.

Mobile Apps can be a fantastic addition to the quest to learn new languages successfully. Various language apps and software help millions of language enthusiasts worldwide learning different languages in a more personalized, relaxed, and accessible strategy. Several language apps have gotten popular over the last few years such as Memorize, Quizlet, AnkiDroid Flashcards, Recall, etc. And indeed, there are some distinct advantages of online language learning; they are convenient and easy to use, affordable and can provide much-needed foreign language practice. The pros of language learning apps are apparent. However, there are few aspects of language learning apps. The major disadvantage of language learning apps is that they require lots of time and self-discipline. Freedom and flexibility are substantial reasons to gain language through an app, but it also has a well-known drawback — lack of discipline. Without self-control, proper time-management, and responsibility, even a simple language will become one of the hardest languages to learn. Humans are inherently lazy when there is no stringent timeline to follow. Learners will have ample excuses and justifications to skip their sessions if there is no compulsion. The absence of self-discipline is the number one case of language learning failure. First, we need to build learning with apps into our schedule and consistently stick to using them. There will be no-one to monitor our practice, to remind us to use the apps, to push us. We are our trainer, and we will have to do it all by ourselves. It may sound beneficial to some, but complete independence only works up to a point.

BRIEF DESCRIPTION OF THE INVENTION

Our invention is designed to specifically tackle above mentioned problems by providing access to information through haptic feedback. Whereas the user is only required to provide what he/she is welling to learn only once, and the rest is will be taken care of. Our invention will take the responsibility to provide the pre-given information back to the user repeatedly throughout time when the user could be busy dealing with any of his/her other daily life activities. Thereby, by using the invention as a learning tool, acquiring information into the long-term memory will be easier and less time consuming than ever. For visually impaired people the invention can be used as an alternative for the braille language, where the visually impaired person can upload a text of any size, and have it converted into patterns of vibration in their wrist which will allow them to be hands free during the process of reading.

LIST OF FIGURES

Figure 1 . The Forgetting Curve showing the effect of reviewing

Figure 2. General View of the Invention

Figure 3. The Case

Figure 4. Upper View of the Case

Figure 5. Cross section View of the Case

Figure 6. The Flexible strap

Figure 6A. Upper view of the Flexible strap

Figure 6B. A-A Cross Sectional view of the Flexible strap

Figure 7. The states of vibrations

Figure 8. An example of the text “vib” converted into patterns of vibration

The equivalents of the numbers given in the figures:

1 . The Case 2. The flexible strap

3. The connection between the strap and the housing

4. The power button

5. The USB power connection port

6. The LED indicator

7. The battery

8. The isolation part

9. The rigid PCB

10. The microcontroller

11 . The Bluetooth module

12. The driver for actuators

13. The connection to the flexible PCB

14. The actuator

15. The connection to the housing

16. The support case for the actuators (motors)

17. Flexible PCB

DETAILED DESCRIPTION OF THE INVENTION

In this invention, the background on sensory integration in the field of neuroplasticity was reviewed, which has been the focus of attention of researchers recently, but for which there is not enough study yet, an electronic device has been developed that integrates the sense of touch and sight. The feasibility of this idea is based on the neuroscience hypothesis, which proposes that human senses can be integrated.

Before starting to describe the invention, the meanings of the terms related to field of invention are given below to make it easier for the reader to understand.

Neuroplasticity, brain plasticity, or neural plasticity is the brain’s ability to undergo structural or physiological changes. Recent research has shown that the brain can change in both childhood and adulthood.

Sensory Integration refers to the concept of feeding information into the brain through an atypical sensory pathway. Our brain integrates information that come from different senses to form a perception of the environment. Visual information can be transferred through a sensory integration device into a representation that can be processed through the sense of touch, granting the person the potential to process this information through touch.

Long-term memory is the part where acquired information is stored, after it has been sufficiently repeated in the short-term memory and is part of the Atkinson-Shiffrin model of memory.

The forgetting curve is the assumption that information stored in memory decreases over time. This curve shows how information disappears over time when no attempt is made to retain it in memory (Figure 1 ).

Spaced Repetition is an effective memorization technique in which content is rehearsed repeatedly on a schedule determined by the spaced repetition algorithm, with the aim of improving the retention of information in long-term memory. Spaced repetition, a method used in vocabulary or knowledge acquisition, is known to be efficient, although it is a very time-consuming technique. The invention uses the spaced repetition technique, helps to remember the new acquired information and can save time in the learning process. Besides, the invention can be used to create a new channel in communication. The sense of touch can be used as an alternative input to the brain, especially when the senses of sight and hearing are already occupied.

Braille is a tactile writing system used by people who are visually impaired. It can be read either on embossed paper or by using refreshable braille displays that connect to computers and smartphone devices

The invention can be in any form of wearable devices such as wristband, vest, belt, hat and/or shoes etc. as long as the haptic feedback (vibration) actuators (14) are in a direct contact with the skin. And it is made of two sections, a case (1 ) and a flexible strap (2).

The case (1 ) is a housing for the internal components; a connection between the strap and the housing (3), a power button (4), a USB power connection port (5) and an LED indicator (6) on the outside. In the inside it comprises a battery (7), an isolation part (8), a rigid PCB (Printed Circuit Board) (9), a microcontroller (10), a bluetooth module (11 ), a driver for actuators (12) and a connection to the flexible PCB (13). The strap (2) comprises a connection to the housing (15) and at least one support case for the actuators (16), at least one actuator (14) (preferably at least four) and a flexible PCB (17). The number of the actuators (14) and the support cases (16) can be as much as the strap (2) can take. A mobile application has been developed and it can communicate via bluetooth module (11 ) with the invention so that the invention can be easily controlled by the user. The invention converts the input data in the text format by the mobile application into vibration patterns by the software with an algorithm comprised within the microcontroller (10).

The case (1): The case (1 ) is the housing for the internal components (the rigid PCB (9) and its components, the connection to the flexible PCB (13), the battery (7), the power button (4), the LED indicator (6) and the USB port (5)) as well as the connections to the strap (3).

The connection between the strap and the housing (3): These are the slots to connect the strap (2) to the case (1 ). The form of the slots (3) corresponds the form of the strap (2) connection parts so they fit tight to the slot (3). Unless the user pulls the straps (2) off, they won’t come out because of tight fitting. One side of the housings (3) comprises a connection to the flexible PCB (13) through on of the connections to the strap, for allowing a path for the flexible PCB (17) that connects to the actuators (14) in the strap.

The power button (4): This button allows the user to turn on or turn off the device. The USB power connection port (5): This port allows the user to charge the device. It works as a power input.

The LED indicator (6): The LED indicator (6) provides a color based visual output for indicating the device when turned on, a bluetooth connection being established and low battery level.

The isolation part (8): This is an extension of the case (1 ) on the inside. It provides isolation between the battery (7) and the rigid PCB (9) for preventing a direct contact, thus heat transmission.

The rigid PCB (9): For preventing the possibility of having a weak output signal from the microcontroller (10) and not providing enough voltage to make the actuators (14) to run. The rigid PCB (9) comprises the microcontroller (10), the bluetooth module (11 ), the driver for the actuators (12), the LED indicator (6), the power button (6) and the USB power connection port (5).

The microcontroller (10): The microcontroller (10) controls the circuits and functions of the invention. And it also comprises the software which all the algorithm runs in that makes the invention unique. The software: The main purpose of the software is taking a file in the text format as input and converting each character of the text into vibration patterns then delivering as haptic feedback output by the actuators (14) once and/or repeatedly reproduced as output according to the principle of spaced repetition. Each letter is pre-assigned to a unique vibration pattern by the algorithm that converts the user-input text into patterns of vibration, and the word list is saved in an internal memory of the microcontroller (10) to be reproduced at a predetermined time interval.

To obtain enough patterns of vibration to cover all letters in multiple languages (rather than only high and low), each of the four actuators (14) can take one of three states: null, dit, or dah (Figure 7). The difference between each state is the time duration in which the actuator (14) remains in the high or low states. It takes a total of 150 ms to produce the vibration pattern of each letter to meet the average reading speed (6-9 letters per second) and allow the brain to process a usual rate of information input. Each of the four actuators (14) can take one of three states at a time, each state has a duration of 150 ms and these states are reproduced simultaneously by the actuators (Figure 8). In total, a number of 3 ⁿ (where n is the number of actuators) patterns can be created.

After converting the text into vibration patterns, the software applies the spaced repetition technique. Meaning, the text is taken as an input by software and each character of the text is executed simultaneously by the software as soon as it is typed by a text input hardware/software (e.g., Keyboard or a touch screen). The entered text (or word list) is saved to the internal memory of the microcontroller (10). With this algorithm, the recorded file is produced immediately and/or repeatedly as patterns of vibration, with an interval of time that is pre-set by the user.

The algorithm comprises unique patterns of vibration are pre-defined for each letter/character. The vibration patterns are defined by their frequency and/or time duration and any time duration and/or frequency in the range of 50 - 230 Hz can be used. So, during the process of algorithm each input letter/character is converted into its pre-defined pattern of vibration on the algorithm. To obtain enough patterns of vibration to cover all letters in multiple languages (rather than just on or off), each of the four actuators (14) can take one of three states: null, dit, or dah (Figure 7). Where the algorithm can be set to take any number of states. The state patterns can be added to the software according to the need. The difference between each state can be the time duration in which the actuator (14) remains in the high or low states and/or any frequency in the range of 50 - 230 Hz, that the actuator (14) is running on. It is set to take a total of 150 ms in time duration to produce the vibration pattern of each letter to meet the average reading speed (6-9 letters per second) and allow the brain to process a usual rate of information input rate.

The Bluetooth module (11): Provides wireless communication between the device and the mobile application to transfer the written text at a low power consumption.

The actuators (14): Provides haptic feedback output. In this application, the invention is set to use four actuators (14). However, any number of actuators (14) can be used. The actuators (14) can be at least one of a linear resonant actuator [LRA], electroactive polymer [EAP] actuator, electromechanical polymer [EMP] actuator, etc.), a motor (e.g., brushless motor, brushed motor, direct current (DC) motor, alternating current (AC) motor, eccentric rotating mass (ERM), etc.), a piezoelectric device and/or any other form of vibratory element.

The driver for actuators (12): Provides enough output voltage to run the actuators (14) by preforming as a voltage regulator and it controls the running frequencies of the actuators by taking a PWM signal as input from the microcontroller (10).

The mobile application: The mobile application has a user interface and it can communicate via bluetooth with the invention so that the invention can be easily controlled by the user. The user interface enables user to provide a text input to the microcontroller (10) to be converted into vibration patterns. It also provides options for applying spaced repetition. The user can change the intervals of applying spaced repetition technique. So, it can be once in a day or may be three times in a minute. It all can be changed from the user interface. When the interval is changed from the mobile application, the command of change is sent to the device via bluetooth module (11 ). And the necessary adjustment is done by the microcontroller (10). This applies the change to the device.

The goal of this invention is to facilitate the learning process and save time and energy for both visual impaired and sighted people. For example, the invention could be used for an often-encountered difficulty in learning foreign languages: as an aid in learning and memorizing new vocabulary or concepts. By this Meaning, the newly acquired information can be easily transferred to long-term memory. The invention can also be used to create an alternative communication channel. Sensory integration is especially beneficial for people with visual or hearing impairments, or for people whose vision or hearing are occupied and want to acquire information through a different sensory pathway. In addition, the user’s brain will learn the vibration patterns associated with the letters over time and will acquire the ability to a better distinguishing between the different patterns of vibration. This can take the usage of the invention one step further by establishing direct connection with a mobile device or a computer. Thus, the invention can be used as an information output device for the mobile application, perceptible by the sense of touch, also notifications can be produced in this way. Apart from that, the invention can be used to form new habits such as learning to focus, in case of attention deficits. Owing to the vibrations sent at regular intervals, it can help the user by reminding him/her to return to the subject on which needs to be focused on, thus helping to eliminate the lack of attention.