Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
SMART CONTACT LENS AND MULTIMEDIA SYSTEM INCLUDING THE SMART CONTACT LENS
Document Type and Number:
WIPO Patent Application WO/2018/109570
Kind Code:
A1
Abstract:
A smart contact lens and a multimedia system including the smart contact lenses. The smart contact lens includes: a substrate; and multiple detecting elements in multiple edge areas of the substrate, the detecting elements generating detection signals when contacting eyelids of a user wearing the smart contact lens, to detect lines of sight of the user. By the multiple detecting elements generating detection signals when contacting eyelids of the user in the edge areas of the smart contact lenses, lines of sight of the user may be simply and conveniently detected, which is relatively high in detection precision, small in calculation amount, may obtain a detection result in real- time, and is not limited to a detection range. A function of detecting lines of sight of the user may be simply and conveniently achieved, and expandability and flexibility of application of the function may be improved.

Inventors:
HE FUTAO (CN)
WANG XIAOBO (CN)
TAN JERRY (CN)
Application Number:
PCT/IB2017/052336
Publication Date:
June 21, 2018
Filing Date:
April 24, 2017
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
SONY MOBILE COMMUNICATIONS INC (JP)
SONY MOBILE COMMUNICATIONS USA INC (US)
International Classes:
G06F3/01; A61B3/113; G02B1/04; G02C7/04; G03B17/48
Foreign References:
US20150362749A12015-12-17
US20160097940A12016-04-07
US20160091737A12016-03-31
US9111473B12015-08-18
GB2281838A1995-03-15
Other References:
None
Attorney, Agent or Firm:
STEYER, Grant (US)
Download PDF:
Claims:
CLAIMS

1. A smart contact lens, characterized in that the smart contact lens comprises: a substrate; and

multiple detecting elements provided in multiple edge areas of the substrate, the detecting elements generating detection signals when contacting eyelids of a user wearing the smart contact lens, to detect lines of sight of the user.

2. The smart contact lens according to claim 1, characterized in that,

the multiple detecting elements are provided in all or part of edge areas of the substrate.

3. The smart contact lens according to either one of claims 1 or 2, characterized in that,

the multiple detecting elements are capacitance contact elements, and result in a change of an output voltage and/or current when contacting the eyelids of the user, to generate the detection signals.

4. The smart contact lens according to any one of claims 1-3, characterized in that the smart contact lens further comprises:

a wireless communication unit configured to perform signal transmission with external electronic equipment.

5. The smart contact lens according to any one of claims 1-4, characterized in that the smart contact lens further comprises:

an image capturing unit configured to capture an image according to the detected lines of sight of the user.

6. The smart contact lens according to claim 5, characterized in including two smart contact lenses that are respectively worn on the left eye and the right eye of the user; and the image capturing unit determines a focus position for capturing an image according to three-dimensional coordinates of an intersection of the detected line of sight of the left eye and line of sight of the right eye of the user.

7. The smart contact lens according to any one of claims 1-5, characterized in that the smart contact lens further comprises:

a power supplying unit configured to accumulate electric power by at least one of wireless charging, solar energy charging and biological energy charging, to supply power to the smart contact lens.

8. A multimedia system, characterized in that the multimedia system comprises: at least one of the smart contact lens as claimed in any one of claims 1-7, worn on the left eye and/or the right eye of a user; and

electronic equipment capable of performing signal transmission with the smart contact lens(es);

wherein, the smart contact lens(es) or the electronic equipment comprise(s):

a processing unit configured to process the detection signals generated by the detecting elements of the smart contact lens(es).

9. The multimedia system according to claim 8, characterized in that the processing unit comprises:

a determining unit configured to determine moving directions and moving angles of lines of sight of the user according to the detection signals.

10. The multimedia system according to claim 9, characterized in that the determining unit comprises:

a first determining unit configured to determine a location distribution of detecting elements in the multiple detecting elements contacting the eyelids of the user according to the detection signals generated by the detecting elements; and

a second determining unit configured to determine the moving directions and moving angles of the lines of sight of the user according to a comparison result of the determined location distribution of detecting elements contacting the eyelids of the user and a reference location distribution.

11. The multimedia system according to claim 10, characterized in that, when the comparison result is that the number of detecting elements generating detection signals at a side of a first horizontal direction is increased, the second determining unit is configured to determine that the eyeballs of the user move to the first horizontal direction;

when the comparison result is that the number of detecting elements generating detection signals at a side of a second horizontal direction is increased, the second determining unit is configured to determine that the eyeballs of the user move to the second horizontal direction;

when the comparison result is that the number of detecting elements generating detection signals at a side of a first vertical direction is increased and the number of detecting elements generating detection signals at a side of a second vertical direction is decreased, the second determining unit is configured to determine that the eyeballs of the user move to the first vertical direction;

and when the comparison result is that the number of detecting elements generating detection signals at a side of a second vertical direction is increased, the second determining unit is configured to determine that the eyeballs of the user move to the second vertical direction.

12. The multimedia system according to claim 10 or 11, characterized in that, the second determining unit is configured to determine the moving angles of the lines of sight of the user according to a difference between the determined a location distribution of detecting elements contacting the eyelids of the user and the reference location distribution.

13. The multimedia system according to any one of claims 10-12, characterized in that the determining unit further comprises:

a third determining unit configured to determine the distribution of a reference position according to a location distribution of detecting elements generating detection signals when the user is looking straight ahead.

14. The multimedia system according to any of claims 9-13, characterized in that the processing unit further comprises:

a movement controlling unit configured to control movement of an operating cursor displayed on a display of the electronic equipment according to the moving directions and moving angles of the lines of sight of the user.

15. The multimedia system according to any of claims 9-14, characterized in that the multimedia system comprises:

two smart contact lenses respectively worn on the left eye and right eye of the user; and the determining unit is configured to respectively determine moving directions and moving angles of the line of sight of the left eye and the line of sight of the right eye of the user according to detection signals generated by the detecting elements of the two smart contact lenses respectively worn on the left eye and right eye of the user.

16. The multimedia system according to claim 15, characterized in that the processing unit further comprises:

a first calculating unit configured to calculate the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user according to the moving directions and moving angles of the line of sight of the left eye and the line of sight of the right eye of the user.

17. The multimedia system according to claim 16, characterized in that the processing unit further comprises:

a first display controlling unit configured to control three-dimensional display of the smart contact lenses or the electronic equipment according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user.

18. The multimedia system according to claim 16 or 17, characterized in that the processing unit further comprises:

a second display controlling unit configured to control augmented reality display of the smart contact lenses or the electronic equipment according to the three- dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user.

19. The multimedia system according to any of claims 16-18, characterized in that the processing unit further comprises:

a second calculating unit configured to, according to three-dimensional coordinates of the intersections of the line of sight of the left eye and line of sight of the right eye of the user at a first moment and a second moment respectively calculated by the first calculating unit, calculate a distance between the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment and the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the second moment.

20. The multimedia system according to claim 19, characterized in that, the first calculating unit calculates the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment based on blink of one of the left eye and the right eye of the user at the first moment;

and the first calculating unit calculates the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the second moment based on blink of the other one of the left eye and the right eye of the user at the second moment.

Description:
SMART CONTACT LENS AND MULTIMEDIA SYSTEM INCLUDING THE

SMART CONTACT LENS

Technical Field

This disclosure relates to the field of information technologies, and in particular to a smart contact lens and a multimedia system including the smart contact lenses.

Background

Currently, as there is continuous development of information technologies, application of various smart devices is more and more widespread. For example, more and more attention is paid to smart contact lenses, as one of the smart devices, due to their features of small volumes, and being flexible and convenient to wear etc. Existing smart contact lenses have various forms. For example, a smart contact lens capable of displaying contents of a mobile phone of a user, a smart contact lens capable of detecting a blood glucose level of a user, and a smart contact lens capable of capturing images, have appeared.

Furthermore, as development of multimedia technologies and smart control technologies continues, technologies of controlling equipment by detecting lines of sight of a user have appeared, such as a technology of controlling movement of a mouse cursor on a display by eyeballs of a user, in which the eyeballs of the user are tracked by providing a camera, thereby controlling the mouse cursor on the display.

It should be noted that the above description of the background is merely provided for clear and complete explanation of this disclosure and for easy understanding by those skilled in the art. And it should not be understood that the above technical solution is known to those skilled in the art as it is described in the background of this disclosure.

Summary

The above existing method for detecting the lines of sight of the user needs to provide an extra camera, and to use the camera to determine positions of the eyeballs to track the eyeballs. That method is relatively low in detection precision and relatively large in amount of or number of calculations (sometimes referred to as "calculation amount"), and it leads to that a detection result cannot be obtained in realtime. Furthermore, that technique is limited by a detection range of the camera.

Embodiments of this disclosure provide a smart contact lens and a multimedia system including a smart contact lens, in which by providing multiple detecting elements generating detection signals when contacting eyelids of a user in edge areas of the smart contact lens, lines of sight of the user may be simply and conveniently detected, which is relatively high in detection precision, small in calculation amount, may obtain a detection result in real-time, and is not limited to a detection range. Furthermore, a function of detecting the lines of sight of the user may be simply and conveniently achieved, and expandability and flexibility of application of the function may be improved.

According to a first aspect of the embodiments of this disclosure, there is provided a smart contact lens, including: a substrate; and multiple detecting elements provided in multiple edge areas of the substrate, the detecting elements generating detection signals when contacting eyelids of a user wearing the smart contact lens, to detect lines of sight of the user.

According to a second aspect of the embodiments of this disclosure, the multiple detecting elements are provided in all or part of edge areas of the substrate.

According to a third aspect of the embodiments of this disclosure, the multiple detecting elements are capacitance contact elements, and result in a change of an output voltage and/or current when contacting the eyelids of the user, to generate the detection signals.

According to a fourth aspect of the embodiments of this disclosure, the smart contact lens further includes: a wireless communication unit configured to perform signal transmission with external electronic equipment.

According to a fifth aspect of the embodiments of this disclosure, the smart contact lens further includes: an image capturing unit configured to capture an image per the detected lines of sight of the user.

According to a sixth aspect of the embodiments of this disclosure, the smart contact lenses are respectively worn on the left eye and the right eye of the user; and the image capturing unit determines a focus position for capturing an image according to three- dimensional coordinates of an intersection of the detected line of sight of the left eye and line of sight of the right eye of the user.

According to a seventh aspect of the embodiments of this disclosure, the smart contact lenses further include: a power supplying unit configured to accumulate electric power by at least one of wireless charging, solar energy charging and biological energy charging, so as to supply power to the smart contact lens(es).

According to an eighth aspect of the embodiments of this disclosure, there is provided a multimedia system, the multimedia system includes: at least one of the smart contact lens(es) as described in the first aspect, worn on the left eye and/or the right eye of a user; and electronic equipment capable of performing signal transmission with the smart contact lens(es); the smart contact lens(es) or the electronic equipment include(s): a processing unit configured to process the detection signals generated by the detecting elements of the smart contact lens(es).

According to a ninth aspect of the embodiments of this disclosure, the processing unit includes: a determining unit configured to determine moving directions and moving angles of lines of sight of the user according to the detection signals.

According to a tenth aspect of the embodiments of this disclosure, the determining unit includes: a first determining unit configured to determine a location distribution of detecting elements in the multiple detecting elements contacting the eyelids of the user according to the detection signals generated by the detecting elements; and a second determining unit configured to determine the moving directions and moving angles of the lines of sight of the user according to a comparison result of the determined location distribution of detecting elements contacting the eyelids of the user and a reference location distribution.

According to an eleventh aspect of the embodiments of this disclosure, when the comparison result is that the number of detecting elements generating detection signals at a side of a first horizontal direction is increased, the second determining unit is configured to determine that the eyeballs of the user move to the first horizontal direction; when the comparison result is that the number of detecting elements generating detection signals at a side of a second horizontal direction is increased, the second determining unit is configured to determine that the eyeballs of the user move to the second horizontal direction; when the comparison result is that the number of detecting elements generating detection signals at a side of a first vertical direction is increased and the number of detecting elements generating detection signals at a side of a second vertical direction is decreased, the second determining unit is configured to determine that the eyeballs of the user move to the first vertical direction; and when the comparison result is that the number of detecting elements generating detection signals at a side of a second vertical direction is increased, the second determining unit is configured to determine that the eyeballs of the user move to the second vertical direction.

According to a twelfth aspect of the embodiments of this disclosure, the second determining unit is configured to determine the moving angles of the lines of sight of the user according to a difference between the determined a location distribution of detecting elements contacting the eyelids of the user and the reference location distribution.

According to a thirteenth aspect of the embodiments of this disclosure, the determining unit further includes: a third determining unit configured to determine the distribution of a reference position according to a location distribution of detecting elements generating detection signals when the user is looking straight ahead.

According to a fourteenth aspect of the embodiments of this disclosure, the processing unit further includes: a movement controlling unit configured to control movement of an operating cursor displayed on a display of the electronic equipment according to the moving directions and moving angles of the lines of sight of the user.

According to a fifteenth aspect of the embodiments of this disclosure, the multimedia system includes: two smart contact lenses respectively worn on the left eye and right eye of the user; and the determining unit is configured to respectively determine moving directions and moving angles of the line of sight of the left eye and the line of sight of the right eye of the user according to detection signals generated by the detecting elements of the two smart contact lenses respectively worn on the left eye and right eye of the user.

According to a sixteenth aspect of the embodiments of this disclosure, the processing unit further includes: a first calculating unit configured to calculate the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user according to the moving directions and moving angles of the line of sight of the left eye and the line of sight of the right eye of the user.

According to a seventeenth aspect of the embodiments of this disclosure, the processing unit further includes: a first display controlling unit configured to control three-dimensional display of the smart contact lenses or the electronic equipment according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user.

According to an eighteenth aspect of the embodiments of this disclosure, the processing unit further includes: a second display controlling unit configured to control augmented reality display of the smart contact lenses or the electronic equipment according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user.

According to a nineteenth aspect of the embodiments of this disclosure, the processing unit further includes: a second calculating unit configured to, according to three-dimensional coordinates of the intersections of the line of sight of the left eye and line of sight of the right eye of the user at a first moment and a second moment respectively calculated by the first calculating unit, calculate a distance between the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment and the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the second moment.

According to a twentieth aspect of the embodiments of this disclosure, the first calculating unit calculates the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment based on blink of one of the left eye and the right eye of the user at the first moment; and the first calculating unit calculates the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the second moment based on blink of the other one of the left eye and the right eye of the user at the second moment.

An advantage of the embodiments of this disclosure exists in that by providing multiple detecting elements generating detection signals when contacting eyelids of a user in edge areas of the smart contact lens, lines of sight of the user may be simply and conveniently detected, which is relatively high in detection precision, small in calculation amount, may obtain a detection result in real-time, and is not limited to a detection range. And furthermore, a function of detecting the lines of sight of the user may be simply and conveniently achieved, and expandability and flexibility of application of the function may be improved.

With reference to the following description and drawings, the embodiments of this disclosure are disclosed in detail, and the principles of this disclosure and the manners of use are indicated. It should be understood that the scope of the embodiments of this disclosure is not limited thereto. The embodiments of this disclosure contain many alternations, modifications and equivalents within the scope of the terms of the appended claims.

Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.

It should be emphasized that the term "comprises/comprising/includes/including" when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

Brief Description of the Drawings

Many aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. To facilitate illustrating and describing some parts of the disclosure, corresponding portions of the drawings may be exaggerated or reduced. Elements and features depicted in one drawing or embodiment of the disclosure may be combined with elements and features depicted in one or more additional drawings or embodiments. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views and may be used to designate like or similar parts in more than one embodiment.

In the drawings:

FIG. 1 is a schematic diagram of the smart contact lens of Embodiment 1 of this disclosure;

FIG. 2 is another schematic diagram of the smart contact lens of Embodiment 1 of this disclosure;

FIG. 3 is a schematic diagram of the processing unit 104 of Embodiment 1 of this disclosure;

FIG. 4 is a schematic diagram of the determining unit 301 of Embodiment 1 of this disclosure;

FIG. 5 is a schematic diagram of a case where the user is looking straight ahead of Embodiment 1 of this disclosure;

FIG. 6 is a schematic diagram of a case where the eyeballs of the user are moving in a horizontal direction of Embodiment 1 of this disclosure;

FIG. 7 is a schematic diagram of a case where the eyeballs of the user are moving to a first vertical direction of Embodiment 1 of this disclosure;

FIG. 8 is a schematic diagram of a case where the eyeballs of the user are moving to a second vertical direction of Embodiment 1 of this disclosure;

FIGs. 9-12 are schematic diagrams of controlling movement of an operational cursor of the electronic equipment by the lines of sight of the user of Embodiment 1 of this disclosure;

FIG. 13 is a schematic diagram of an intersection of the line of sight of the left eye and the line of sight of the right eye of Embodiment 1 of this disclosure;

FIG. 14 is a schematic diagram of measuring a size of an object of Embodiment 1 of this disclosure; FIG. 15 is a schematic diagram of the multimedia system of Embodiment 2 of this disclosure; and

FIG. 16 is a block diagram of the electronic equipment of Embodiment 2 of this disclosure.

Description

The interchangeable terms "electronic equipment" and "electronic apparatus", in this disclosure, may relate to any type of appropriate electronic apparatus, and examples of such electronic apparatus include a computer, a mobile phone, a smart mobile phone, a photo camera, a video camera, a tablet PC, a telephone, a media player, and a game device, etc.

The term "unit" may have conventional meaning in the field of electronics, electrical devices and/or electronic devices and may include, for example, electrical and/or electronic circuitry, devices, modules, processors, memories, logic solid state and/or discrete devices, computer programs or instructions for carrying out respective tasks, procedures, computations, outputs, and/or displaying functions, and so on, as such as those that are described herein.

Preferred embodiments of this disclosure shall be described below with reference to the accompanying drawings.

Embodiment 1

An embodiment of this disclosure provides a smart contact lens. FIG. 1 is a schematic diagram of the smart contact lens of Embodiment 1 of this disclosure. As shown in FIG. 1, the smart contact lens 100 includes:

a substrate 101; and

multiple detecting elements 102 provided in multiple edge areas of the substrate 101, the detecting elements 102 generating detection signals when contacting eyelids of a user wearing the smart contact lens, to detect lines of sight of the user. Reference to edge areas may include, for example, at the edge area, near the edge, adjacent the edge, or such other similar location so as to provide for generating detection signals as described herein.

It can be seen from the above embodiment that by providing the multiple detecting elements generating detection signals when contacting the eyelids of the user in the edge areas of the smart contact lens, the lines of sight of the user may be simply and conveniently detected, which is relatively high in detection precision, small in calculation amount, may obtain a detection result in real-time, and is not limited to a detection range. And furthermore, a function of detecting the lines of sight of the user may be simply and conveniently achieved, and expandability and flexibility of application of the function may be improved.

In this embodiment, the substrate 101 may be made from a transparent material, and an existing method may be used for making the substrate. For example, the substrate may be a material that is the same or similar to that of which conventional contact lenses may be made or may be such other material as may be suitable for use as is disclosed herein.

In this embodiment, a shape of the substrate 101 may be designed as actually demanded. The term "as demanded" may also mean "as required," as desired," as preferred," and so on, e.g., as demanded by a user, as demanded by circumstances or desired function, etc. For example, the substrate 101 may be of a circular shape, or an elliptical shape, etc.

In this embodiment, the multiple detecting elements 102 may be provided in multiple edge areas of the substrate 101. For example, the multiple detecting elements 102 may be implanted in the substrate 101 by using an existing method.

In this embodiment, the multiple detecting elements 102 may be provided in all or part of edge areas of the substrate 101. For example, as shown in FIG. 1, the multiple detecting elements 102 are provided continuously in, at, near, adjacent, etc., as mentioned above, the whole edges of the substrate 101, with a density of distribution that may be set according to a detection precision and an actual demand.

In this embodiment, the multiple detecting elements 102 may also be provided in part of the edge areas of the substrate 101. FIG. 2 is another schematic diagram of the smart contact lenses of Embodiment 1 of this disclosure. As shown in FIG. 2, edge areas of the substrate 101 in four directions or locations, e.g., upper, lower, left and right directions, are provided with multiple detecting elements 102, and for brevity and description other elements are not shown.

In this embodiment, the detecting elements 102 may be capacitance contact elements provided on a surface of the substrate 101, and result in changes of an output voltage and/or current when contacting eyelids of the user, thereby generating the detection signals. When the capacitance contact elements are used as the detecting elements 102, their structures are simple, and are easy to manufacture; and by simple changes of the voltage and/or current, the detection signals may be obtained, thus a detection speed is relatively fast. For example, the capacitance contact elements may be of existing or known type structures.

In this embodiment, the smart contact lenses 100 may further include a wireless communication unit 103 configured to perform signal transmission with external electronic equipment. The communication unit 103 is shown in FIG. 1, but also similarly may be included in the smart contact lens 100 of FIG. 2.

In this embodiment, the external electronic equipment may be various types of electronic devices, such as a computer, a mobile phone, a smart mobile phone, a photo camera, an image camera, a tablet PC, a telephone, a media player, and a game device, etc.

In this embodiment, the wireless communication unit 103 may perform signal transmission with the external electronic equipment by using an existing wireless communication method, such as Bluetooth, and WTFI, etc.

In this embodiment, the smart contact lens 100 may further include a processing unit 104 (shown in FIG. 1, but similarly may be included in the contact lens 100 of FIG. 2) configured to process the detection signals generated by the detecting elements 102.

In this embodiment, the processing unit 104 is, for example, an IC (integrated circuit) chip, such as a CPU chip, and, if desired, other electronic component(s), e.g., memory, input/output circuitry and so on; and is implanted in the smart contact lenses.

In this embodiment, the processing unit 104 may be provided in the electronic equipment performing signal transmission with the smart contact lens, rather than provided in the smart contact lenses 100, to process the detection signals.

A structure and function of the processing unit 104 of this embodiment shall be illustrated and described below.

FIG. 3 is a schematic diagram of the processing unit 104 of Embodiment 1 of this disclosure. As shown in FIG. 3, the processing unit 104 includes:

a determining unit 301 configured to determine moving directions and moving angles of lines of sight of the user according to the detection signals;

a movement controlling unit 302 configured to control movement of an operating cursor displayed on a display of the electronic equipment capable of performing signal transmission with the smart contact lenses according to the moving directions and moving angles of the lines of sight of the user;

a first calculating unit 303 configured to calculate the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user according to the moving directions and moving angles of the line of sight of the left eye and the line of sight of the right eye of the user;

a first display controlling unit 304 configured to control three-dimensional display of the smart contact lenses or the electronic equipment according to the three- dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user;

a second display controlling unit 305 configured to control augmented reality display of the smart contact lenses or the electronic equipment according to the three- dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user; and

a second calculating unit 306 configured to, according to three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at a first moment and a second moment respectively calculated by the first calculating unit, calculate a distance between the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment and the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the second moment.

In this embodiment, the movement controlling unit 302, the first calculating unit 303, the first display controlling unit 304, the second display controlling unit 305 and the second calculating unit 306 are optional, and may be provided as demanded by actual application.

In this embodiment, the determining unit 301 determines the moving directions and moving angles of lines of sight of the user according to the detection signals. A structure of the determining unit 301 and a method of determination shall be illustrated below.

FIG. 4 is a schematic diagram of the determining unit 301 of Embodiment 1 of this disclosure. As shown in FIG. 4, the determining unit 301 includes:

a first determining unit 401 configured to determine a location distribution of detecting elements in the multiple detecting elements contacting the eyelids of the user according to the detection signals generated by the detecting elements; and

a second determining unit 402 configured to determine the moving directions and moving angles of the lines of sight of the user according to a comparison result of the determined location distribution of detecting elements contacting the eyelids of the user and a reference location distribution.

And the determining unit 301 may further include:

a third determining unit 403 configured to determine the distribution of a reference position according to a location distribution of detecting elements generating detection signals when the user is looking straight ahead.

In this embodiment, for the convenience of explanation of the directions, the following definitions are given: a direction the user is facing after wearing the smart contact lenses is taken as a reference direction; one of a left direction and a right direction is defined as a first horizontal direction, and the other one of the left direction and the right direction is defined as a second horizontal direction, an upward direction is defined as a first vertical direction, and a downward direction is defined as a second vertical direction.

FIG. 5 is a schematic diagram of a case where the user is looking straight ahead of Embodiment 1 of this disclosure. As shown in FIG. 5, the user is looking straight ahead, and a location distribution of the detecting elements, that is, the detecting elements generating the detection signals, is taken as the reference location distribution. For example, the detecting elements at the upper left half are taken as the detecting elements at a side of the first horizontal direction, the detecting elements at the right half are taken as the detecting elements at a side of the second horizontal direction, the detecting elements at the upper half are taken as the detecting elements at a side of the first vertical direction, and the detecting elements at the lower half are taken as the detecting elements at a side of the second vertical direction.

In this embodiment, for different users, the third determining unit 403 is first needed to determine their distribution of reference positions, so as to be adapted for structures of eyes of different users.

In this embodiment, the second determining unit 402 determines the moving directions and moving angles of lines of sight of the user according to a comparison result. For example,

when the comparison result is that the number of detecting elements generating detection signals at a side of the first horizontal direction is increased, the second determining unit determines that the eyeballs of the user move to the first horizontal direction;

when the comparison result is that the number of detecting elements generating detection signals at a side of the second horizontal direction is increased, the second determining unit determines that the eyeballs of the user move to the second horizontal direction;

when the comparison result is that the number of detecting elements generating detection signals at a side of the first vertical direction is increased and the number of detecting elements generating detection signals at a side of the second vertical direction is decreased, the second determining unit determines that the eyeballs of the user move to the first vertical direction; and

when the comparison result is that the number of detecting elements generating detection signals at a side of the second vertical direction is increased, the second determining unit determines that the eyeballs of the user move to the second vertical direction.

In this embodiment, the moving directions of the eyeballs of the user are in consistence with the moving directions of the lines of sight of the user. FIG. 6 is a schematic diagram of a case where the eyeballs of the user are moving in a horizontal direction of Embodiment 1 of this disclosure. As shown in FIG. 6, when the number of the detecting elements contacting the eyelids of the user at the side of the first horizontal direction (the left side) or the side of the second horizontal direction (the right side) is increased, that is, when the number of the detecting elements generating detection signals at the side of the first horizontal direction or the side of the second horizontal is increased, it is determined that the eyeballs of the user move to the first or the second horizontal direction.

FIG. 7 is a schematic diagram of a case where the eyeballs of the user are moving in the first vertical direction of Embodiment 1 of this disclosure. As shown in FIG. 7, when the number of detecting elements contacting the eyelids of the user at the side of the first vertical direction (the upper side) is increased and the number of detecting elements contacting the eyelids of the user at the side of the second vertical direction (the lower side) is decreased, that is, when the number of detecting elements generating detection signals at the side of the first vertical direction is increased and the number of detecting elements generating detection signals at the side of the second vertical direction is decreased, it is determined that the eyeballs of the user move to the first vertical direction (the upper side).

FIG. 8 is a schematic diagram of a case where the eyeballs of the user are moving in the second vertical direction of Embodiment 1 of this disclosure. As shown in FIG. 8, when the number of detecting elements contacting the eyelids of the user at the side of the second vertical direction (the lower side), that is, the number of the detecting elements generating detection signals at the side of the second vertical direction is increased, as the lower eyelid of the user is substantially kept unmoved, it is determined that the eyeballs of the user move to the second vertical direction (the lower side), even though the number of detecting elements generating detection signals at the side of the first vertical direction (the upper side) is increased.

In this embodiment, the second determining unit 402 determines the moving angles of lines of sight of the user according to a difference between the determined a location distribution of detecting elements contacting the eyelids of the user and the reference location distribution.

For example, in determining the reference location distribution, the third determining unit 403 may obtain a correspondence relationship between the moving angles of lines of sight of the user and the difference between position distribution, and the second determining unit 402 determines the moving angles of lines of sight of the user according to the correspondence relationship.

The structure of the determining unit 301 of the processing unit 104 and a method for determining the moving directions and moving angles of lines of sight of the user are illustrated above.

In this embodiment, the movement controlling unit 302 of the processing unit 104 controls the movement of the operating cursor displayed on the display of the electronic equipment capable of performing signal transmission with the smart contact lenses according to the moving directions and moving angles of the lines of sight of the user. In this way, the movement of the operating cursor of the electronic equipment may be simply and conveniently controlled by wearing the smart contact lenses.

For example, the operating cursor displayed on the display of the electronic equipment is controlled to be in consistence with the moving directions of the lines of sight of the user, and a distance of movement of the operating cursor is proportional to the moving angles of the lines of sight of the user, which may be obtained by calculating according to a predetermined correspondence relationship, such as establishing a correspondence table according to data obtained by user test.

For example, the operating cursor is a mouse cursor, and the processing unit 104 transmits a control signal of the mouse cursor to a mouse control module in the electronic equipment, and performs corresponding control on the mouse cursor based on a mouse protocol.

FIGs. 9-12 are schematic diagrams of controlling the movement of the operating cursor 901 of the exemplary electronic equipment 902 by the lines of sight of the user of Embodiment 1 of this disclosure. As shown in FIG. 9, when the eyeballs of the user are substantially looking straight ahead, the operating cursor is placed at the center of the display of the electronic equipment. As is shown in FIG. 10, when the eyeballs of the user move towards a side in the horizontal direction, the operating cursor is controlled to move to the same horizontal direction. As is shown in FIG. 11, when the eyeballs of the user move upwards, the operating cursor is controlled to move upwards, and as shown in FIG. 12, when the eyeballs of the user move downwards, the operating cursor is controlled to move downwards.

In this embodiment, the user may select to wear a smart contact lens on the left eye or on the right eye, or wear smart contact lenses on both the left eye and the right eye, as actually demanded. For example, when it is needed to control the operating cursor on the display of the electronic equipment, the left eye or the right eye may wear a smart contact lens.

In this embodiment, when both the left eye and the right eye of the user wear the smart contact lenses, the determining unit 301 may respectively determine the moving directions and the moving angles of the line of sight of the left eye and the line of sight of the right eye of the user according to the detection signals generated by the detecting elements of the smart contact lenses worn respectively on the left eye and the right eye of the user.

The first calculating unit 303 calculates the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user according to the moving directions and moving angles of the line of sight of the left eye and the line of sight of the right eye of the user. In this way, more function applications may be expanded by using the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user.

The first calculating unit 303 calculating the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user shall be illustrated below.

FIG. 13 is a schematic diagram of the intersection of the line of sight of the left eye and the line of sight of the right eye of Embodiment 1 of this disclosure. As shown in FIG. 13, A and B respectively denote centers of the eyeballs of the left eye and the right eye of the user, F' denotes an intersection of the line of sight of the left eye and the line of sight of the right eye before movement of the eyeballs of the user (a previous moment), and F denotes an intersection of the line of sight of the left eye and the line of sight of the right eye after movement of the eyeballs of the user (a current moment). A central point of a segment AB is taken as an origin, an axis when the segment AB is located is taken as the X axis, an axis in parallel with a vertical direction of a human body and passing through the origin is taken the Z axis, and Y axis passes through the origin and is perpendicular to the XZ plane. F'(x,y,0) is a projection of a space point F(x,y,z) on the XY plane. ZBAF' and ZABF' may be obtained by detecting the movement of the eyeballs at the previous moment, and |BF' | may be obtained by calculating with reference to a pupil distance |AB| of the user, thereby obtaining two- dimensional coordinates F'(x,y) of F' in the XY plane. Furthermore, a distance |FF' | between F' and F may be obtained with reference to |BF' | by using a result of detecting the movement of the eyeballs at the current moment, i.e. the moving angle FBF' of the right eye. And three-dimensional coordinates F(x,y,z) of F may be obtained with reference to the two-dimensional coordinates F'(x,y) of F' in the XY plane.

In this embodiment, the smart contact lenses or the electronic equipment capable of performing signal transmission with the smart contact lenses have a function of three- dimensional display, and the first display controlling unit 304 controls the three- dimensional display of the smart contact lenses or the electronic equipment according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user. In this way, a visual effect of the three- dimensional display may be effectively improved.

For example, with the control by the first display controlling unit 304, more details may be displayed at the intersection of the line of sight of the left eye and line of sight of the right eye of the user, thereby improving the visual effect of the three-dimensional display.

In this embodiment, the smart contact lenses or the electronic equipment capable of performing signal transmission with the smart contact lenses have a function of augmented reality (AR) display, and the second display controlling unit 305 controls the augmented reality display of the smart contact lenses or the electronic equipment according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user. In this way, a visual effect of the augmented reality display may be effectively improved.

For example, with the control by the second display controlling unit 305, rendering of contrast and details are reinforced at the intersection of the line of sight of the left eye and line of sight of the right eye of the user, consistence of a focus of rendering of a picture or a scenario and a focus of vision may be achieved, and stronger real senses may be brought to the user, while a sense of giddiness brought about by inconsistence of a focus of rendering of a picture or a scenario and a focus of vision may be eliminated.

In this embodiment, according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment and the second moment respectively calculated by the first calculating unit 303, the second calculating unit 306 calculates the distance between the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment and the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the second moment. In this way, a size of an object may simply and accurately be measured.

For example, the user looks towards an end of an object to be measured, and at this moment, by blink of the left eye of the user, such a moment may be locked and the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at this moment may be calculated. Immediately thereafter, the lines of sight of the user move and the user looks towards the other end of the object to be measured, and at this moment, by blink of the right eye of the user, such a moment may be locked and the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user at this moment may be calculated, and a distance between the intersections of the line of sight of the left eye and line of sight of the right eye of the user at the two moments may be calculated, thereby obtaining the size of the object to be measured.

FIG. 14 is a schematic diagram of measuring a size of an object of Embodiment 1 of this disclosure. As shown in FIG. 14, the left eye blinks when the user looks towards an end PI of the object to be measured, and the first calculating unit 303 calculates three-dimensional coordinates Pl(xl,yl,zl) of PI; the right eye blinks when the user looks towards the other end P2 of the object to be measured, and the first calculating unit 303 calculates three-dimensional coordinates P2(x2,y2,z2) of P2; and the second calculating unit 306 calculates a distance between PI and P2 according to formula (1) below:

D = V[(xl - x2) 2 + (yl - y2) 2 + (zl - z2) 2 ] (1); where, D denotes the distance between PI and P2, (xl,yl,zl) denotes the three- dimensional coordinates of PI, and P2(x2,y2,z2) denotes the three-dimensional coordinates of P2.

In this embodiment, the smart contact lens 100 may further include an image capturing unit 105 configured to capture an image according to the detected lines of sight of the user. For example, it determines a focus position for capturing an image according to the three-dimensional coordinates of the intersection of the detected line of sight of the left eye and line of sight of the right eye of the user. In this way, an object focused by the user may be quickly captured without needing to convert a coordinate system of the camera, and a captured image of relatively high quality may be obtained.

In this embodiment, the intersection of the detected line of sight of the left eye and line of sight of the right eye of the user may be obtained through calculation by the first calculating unit 303.

In this embodiment, the smart contact lens 100 may further include a power supplying unit 106 configured to accumulate electric power by at least one of wireless charging, solar energy charging and biological energy charging, to supply power to the smart contact lens. For example, it supplies power to the detecting elements 102, the processing unit 104, and the image capturing unit 105, etc.

For example, wireless charging may be performed by capacitance, charging may be performed by providing a solar cell, or charging may be performed by performing transformation from biological energies to electrical energies by using secretion liquids in the eyes. In this embodiment, the wireless communication unit 103, the processing unit 104, the image capturing unit 105, and the power supplying unit 106, etc., may be implanted in the smart contact lenses by using an existing method, and these units may be integrated in an integrated circuit or a functional module, or may be provided separately. Furthermore, positions of these units on the substrate 101 may be determined according to an actual situation or as demanded or desired.

It can be seen from the above embodiment that by providing the multiple detecting elements generating detection signals when contacting the eyelids of the user in the edge areas of the smart contact lenses, the lines of sight of the user may be simply and conveniently detected, which is relatively high in detection precision, small in calculation amount, may obtain a detection result in real-time, and is not limited to a detection range. And furthermore, a function of detecting the lines of sight of the user may be simply and conveniently achieved, and expandability and flexibility of application of the function may be improved.

Embodiment 2

An embodiment of this disclosure provides a multimedia system. FIG. 15 is a schematic diagram of the multimedia system of Embodiment 2 of this disclosure. As shown in FIG. 15, the multimedia system 1500 includes: at least one smart contact lens 1501, worn on the left eye and/or the right eye of a user; and electronic equipment 1502 capable of performing signal transmission with the smart contact lens(es). The smart contact lens(es) 1501 or the electronic equipment 1502 include(s) a processing unit (e.g., as shown at 104 in FIG. 1, but not shown here) configured to process detection signals generated by detecting elements of the smart contact lens(es).

In this embodiment, a structure and functions of the smart contact lens 1501 are completely the same as those of the smart contact lens 100 in Embodiment 1, and shall not be described herein any further.

In this embodiment, the electronic equipment may be various types of electronic devices, such as a computer, a mobile phone, a smart mobile phone, a photo camera, a video camera, a tablet PC, a telephone, a media player, and a game device, etc.

In this embodiment, the processing unit may be provided in the smart contact lens 1501, and may also be provided in the electronic equipment 1502, and its structure and functions are completely the same as those of the processing unit 104 in Embodiment 1, which shall not be described herein any further.

When the processing unit is provided in the electronic equipment 1502, a structure and functions of the electronic equipment shall be illustrated below.

FIG. 16 is a block diagram of the electronic equipment of Embodiment 2 of this disclosure. As shown in FIG. 16, the electronic equipment 1600 (e.g., in FIG. 15 also designated 1502) may include a central processing unit 1601 and a memory 1602, the memory 1602 being coupled to the central processing unit 1601. It should be noted that this figure is illustrative only, and other types of structures may also be used, so as to supplement or replace this structure and achieve telecommunications function or other functions.

The central processing unit 1601 executes the functions of the processing unit, and may be configured to process the detection signals generated by the detecting elements of the smart contact lenses.

For example, the processing the detection signals generated by the detecting elements of the smart contact lenses may include: determining moving directions and moving angles of lines of sight of the user according to the detection signals; controlling movement of an operating cursor displayed on a display of the electronic equipment according to the moving directions and moving angles of the lines of sight of the user; calculating the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user according to the moving directions and moving angles of the line of sight of the left eye and the line of sight of the right eye of the user; controlling three-dimensional display of the smart contact lenses or the electronic equipment according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user; controlling augmented reality display of the smart contact lenses or the electronic equipment according to the three-dimensional coordinates of the intersection of the line of sight of the left eye and line of sight of the right eye of the user; and according to respectively calculated three-dimensional coordinates of the intersections of the line of sight of the left eye and line of sight of the right eye of the user at a first moment and a second moment, calculating a distance between the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the first moment and the intersection of the line of sight of the left eye and line of sight of the right eye of the user at the second moment.

As shown in FIG. 16, the electronic equipment 1600 may further include a communication module 1603, an input unit 1604, an audio processor 1605, a loudspeaker 1605-1, a microphone 1605-2, a display unit 1606, a power supply 1607, and an antenna 1608. It should be noted that the electronic equipment 1600 does not necessarily include all the parts shown in FIG. 16, and furthermore, the electronic equipment 1600 may include parts not shown in FIG. 16, and the related art may be referred to.

As shown in FIG. 16, the central processing unit 1601 is sometimes referred to as a controller or control, and may include a microprocessor or other processor devices and/or logic devices. The central processing unit 1601 receives input and controls operations of every components of the electronic equipment 1600.

The memory 1602 may be, for example, one or more of a buffer memory, a flash memory, a hard drive, a mobile medium, a volatile memory, a nonvolatile memory, or other suitable devices, which may store predefined or preconfigured information, and may further store a program executing related information. The central processing unit 1601 may execute the program stored in the memory 1602, so as to realize information storage or processing, etc. Functions of other parts are similar to those of the prior art, which shall not be described herein any further. The parts of the electronic equipment 1600 may be realized by specific hardware, firmware, software, or any combination thereof, without departing from the scope of the present disclosure.

It can be seen from the above embodiment that by providing the multiple detecting elements generating detection signals when contacting the eyelids of the user in the edge areas of the smart contact lenses, the lines of sight of the user may be simply and conveniently detected, which is relatively high in detection precision, small in calculation amount, may obtain a detection result in real-time, and is not limited to a detection range. And furthermore, a function of detecting the lines of sight of the user may be simply and conveniently achieved, and expandability and flexibility of application of the function may be improved.

The above apparatuses and methods of the present disclosure may be implemented by hardware, or by hardware in combination with software. The present disclosure relates to such a computer-readable program that when the program is executed by a logic device, the logic device is enabled to carry out the apparatus or components as described above, or to carry out the methods or steps as described above.

The present disclosure also relates to a storage medium for storing the above program, such as a hard disk, a floppy disk, a CD, a DVD, and a flash memory, etc.

The present disclosure is described above with reference to particular embodiments. However, it should be understood by those skilled in the art that such a description is illustrative only, and not intended to limit the protection scope of the present disclosure. Various variants and modifications may be made by those skilled in the art according to the principles of the present disclosure, and such variants and modifications fall within the scope of the present disclosure.