FIELD OF THE INVENTION

The present invention relates generally to an imaging and display system. More specifically, the present invention relates to a smart mirror system and/or device for collecting and visualizing a subject’s physical appearance information using sensors, cameras, and artificial intelligence technologies.

BACKGROUND OF THE INVENTION

The conventional mirror (i.e., reflective surface) is the most common and reliable tool for individuals to use in reviewing their appearance in real-time, with many people looking in mirrors daily to check their appearance. However, the data that users obtain from traditional mirrors immediately disappear after walking away from the mirror.

People sometimes wish to compare gradual changes in their appearance before a mirror but taking a succession of pictures over time can be a cumbersome and inconsistent solution. In another example, people may feel as if they have gained weight but not know exactly how their body has changed if they lack visual data that allow them to compare their current condition with their past conditions. Trying to make mental notes on appearance status that allow comparison between past status and current status can be frustrating, especially when realizing that physical status has changed gradually unnoticed.

Several technologies exist that can deliver information while users look in mirrors of various forms. However, most such information offered by current technologies is not related to appearance, instead relating to weather, video media, time, or temperature — all of which users can obtain via many other devices, such as smartphones or computers. Currently, no system specializes in collecting and visualizing information relating to a user’s appearance.

An image capturing device integrated into mirrors is thus in demand. A few prior attempts to meet this demand have combined a camera with a screen that replaces the conventional mirror. However, digital imitation mirrors have focused on capturing and transferring an image rather than collecting that image and using it to convey meaningful information about the user. Thus, there is a need to provide a system or device that specializes in collecting and visualizing a subject’s appearance information both during and after uses of the mirror.

The present invention aims to solve the aforementioned problems and/or improve on conventional devices by integrating sensors and other related devices into a set of mirrors that form a substantially improved image collecting and display device.

SUMMARY OF THE INVENTION

The present invention offers a smart mirror system that effectively and efficiently collects and visualizes appearance information and data of a user with artificial intelligence (AI) technologies, including, but not limited to, facial recognition, machine learning, deep learning. The smart mirror system acquires the user’s appearance data using various sensors and multiple cameras while the user looks in a mirror of the smart mirror system. The cameras can capture three-dimensional (3D) image/video data of the user in a 360-degree range. The sensors can measure distances of all visible features of the user’s appearance including outfit, pose, etc., measure body temperature, color, etc. of the user. Additionally, the sensors can measure environmental temperature, relative humidity, light and color, etc.

The smart mirror system allows the user to access the collected appearance information even when away from the mirror. The user’s appearance data that are effortlessly collected enables predictive analyses of future conditions based on the acquired appearance data and detects changes over time. The smart mirror system allows the user to obtain many categories of appearance information, including, but not limited to, current appearance, daily outfit, changes in body shape/condition, and medical condition, per the desired use of the acquired data. Additionally, the mirror system acquires video data together with sensor data to present videos including, but not limited to, dressing, clothing fittings, shaving, hair-dressing, face makeup, poses, exercises, etc., to the user. The mirror system can provide simulated images and/or videos in the form of a three-dimensional (3D) avatar to the user using AI technologies, including, but not limited to, deep learning, machine learning, facial recognition, etc. The 3D avatar may include, but is not limited to, a recommended outfit, pose, user’s body measurements and/or simulations, etc. Further, the mirror system may present videos including, but not limited to, an instructor’s exercise video, a standard/recommended pose, etc., to the user along with the current video of the user. The mirror system may provide multiple mirrors to catch a 3D image/video of the user.

The smart mirror system can be made in one piece, or a plurality of mirrors connected in a form of a foldable and movable system that further provides physical flexibility to the user. A computing device of the smart mirror system may include a memory and allow the user appearance data to be saved on the system. The computing device provides secure data transfer to any extraneous storage medium, including, but not limited to, a portable disk drive, a flash drive, etc. Further, the computing device allows the mirror system to communicate to an extraneous device, including, but not limited to, a server, a computer, a cloud device, a mobile device, a personal computing (PC) device, etc., through a wireless network, including, but not limited to, Internet. Thus, the present invention offers the user a versatile, flexible, user-friendly, and effective smart mirror system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front view of a smart mirror system of the present invention.

FIG. 2 is a perspective rear view of one embodiment of the present invention, wherein a computing device attached to the back face of the smart mirror system. FIG. 3 is a perspective rear view of the smart mirror system of the present invention in a folded state.

FIG. 4 is a top view of the smart mirror system of the present invention.

FIG. 5 is an illustration showing one embodiment of the smart mirror system of the present invention, wherein a display is attached to the back face of the mirror.

FIG. 6 is an illustration showing another embodiment of the smart mirror system of the present invention, wherein the display is attached to the mirror surface of the mirror.

FIG. 7 is a system diagram of a computing device of the smart mirror system of the present invention.

FIG. 8 is a flowchart diagram of the computing device of the present invention, wherein the computing device communicates with a server and a personal computing (PC) device through an extraneous network.

FIG. 9 is an illustration showing one embodiment of the smart mirror system of the present invention in use, wherein acquired sensor data and a three-dimensional (3D) avatar of the user is displayed on the display.

FIG. 10 is an illustration showing another embodiment of the smart mirror system of the present invention in use, wherein acquired sensor data, outfit recommendations, and a simulated three-dimensional (3D) avatar of the user are displayed on the display.

FIG. 11 is an illustration showing another embodiment of the smart mirror system of the present invention in use, wherein acquired sensor data, user’s pose, and an instructor’s pose are displayed on the display.

DETAIL DESCRIPTIONS OF THE INVENTION

All illustrations of the drawings are for the purpose of describing selected versions of the present invention and are not intended to limit the scope of the present invention.

The present invention comprises a smart mirror system that effectively and efficiently acquires and visualizes appearance information of a user with artificial intelligence (AI) technologies through the use of multiple cameras and various sensors. As shown in FIG. 1 to FIG. 11, the smart mirror system of the present invention comprises a mirror assembly 10, a display 50, and a computing device 70. More specifically, the mirror assembly 10 comprises a mirror 11, a plurality of cameras 12, and a plurality of distance detectors 13. The mirror 11 comprises a mirror surface 32 and a back face 33. Both the plurality of cameras 12 and the plurality of distance detectors 13 are distributed on the mirror surface 32 of the mirror 11. Additionally, both the plurality of cameras 12 and the plurality of distance detectors 13 are electronically connected to the computing device 70 through a wired or wireless connection. The display 50 is attached to the mirror 11 of the mirror assembly 10 and is electronically connected to the computer device 70 as well. The computing device 70 is mounted to the back face 33 of the mirror 11 of the mirror assembly 10. Additionally, the computing device 70 is adapted to acquire a plurality of appearance data of the user through the plurality of cameras 12 and the plurality of distance detectors 13. Further, the computing device 70 is adapted to display appearance information on the display 50 using the AI technologies, including, but not limited to, machine learning, deep learning, facial recognition, etc.

As can be seen in FIG. 1 to FIG. 4, and FIG. 8, in the preferred embodiment of the present invention, the mirror 11 of the smart mirror assembly 10 may be a foldable mirror. Such a foldable mirror may be one continuous piece of mirror, or comprises a plurality of mirrors 31. The mirror 11 can include a special mirror that allows light through it to display information/data directly on the mirror 11 and the rest of the mirror surface 32 can reflect an object as does a normal mirror. Additionally, the mirror assembly 10 comprises the plurality of cameras 12 and the plurality of distance detectors 13, a plurality of sensors 14, a light 15, a speaker 16, a plurality of handles 17, and a plurality of wheels 18. More specifically, the plurality of cameras 12 may include, but is not limited to, a video camera, a Charge Coupled Device (CCD) camera, a camera and/or camera set-up enabling the capturing of 3D-images, an Intel ^® Realsense D400 series camera, a stereoscopic camera, a camera integrated with infrared (IR) sensor, a camera comprising image sensors, including, but not limited to, CCD, Complementary Metal Oxide Semiconductor (CMOS), X3 ^® PRO 10M™ CMOS image sensor from Foveon ^® , etc. The plurality of distance detectors 13 may include, but is not limited to, an IR sensor, a light detection and ranging (LiDAR) sensor, a short frequency infrared sensor, touchless LiDAR sensor, a LiDAR camera, an Intel ^® Realsense L515, etc. The plurality of sensors 14 is distributed across the mirror surface 32 of the mirror 11 and electronically connected to the computing device 70. The plurality of sensors 14 may include, but is not limited to, a temperature sensor that measures room temperature and/or the user’s body temperature, a relative humidity sensor that measures room relative humidity, a proximity sensor, a touchless proximity sensor, a motion sensor, a color sensor, a touch sensor, a light sensor, a brightness sensor, a light dependent resistors (LDR) sensor, a microphone/sound sensor, IR sensor, etc. The IR sensor identifies the user’s height, shape, and distance. Additionally, the IR sensor allows the user’s image to be distinguished from the background. The color sensor is used to identify color, providing advanced pixel data for the captured image and helping to identify the user from the background. The light sensor can measure the illuminance and automatically adjust the brightness so that the subject can be properly illuminated with appropriate brightness.

The plurality of sensors 14 provides the sensor data to the computing device 70 for real-time analyses of the user’s appearance, body conditions, environmental conditions, etc., and to display actual measured data and analysis results to the mirror system of the present invention. The light 15 is mounted to the mirror surface 32 of the mirror 11 and electronically connected to the computing device 70. The light 15 may include, but is not limited to, a light-emitting diode (LED), an organic light-emitting diode (OLED) light, etc., which can be attached to any place on the mirror 11, including the edges. The lighting can be of different colors to improve the existing lighting. The lighting may set the mood by changing the level and color, and a spotlight can allow users to see themselves brightly reflected in the mirror 11. Similarly, the speaker 16 is mounted to the mirror surface 32 of the mirror 11 and electronically connected to the computing device 70 as well. The speaker 16, together with the microphone/sound sensor of the plurality of sensors 14, provides interactions between the smart mirror system of the present invention and the user. Further, a plurality of handles 17 is mounted to the back face 33 of the mirror 11 to provide flexible handling of the smart mirror system in situations, including, but not limited to, folding, extending, rotating, etc. The plurality of wheels is mounted to the bottom of the mirror system 10 and offers easy moving, rotating, etc., of the smart mirror system.

As can be seen in FIG. 1 to FIG. 11, the smart mirror system of the present invention comprises the display 50. The display 50 may comprise a plurality of displays. Additionally, the computing device 70 may be adapted to project a specific display of the plurality of displays to each of the plurality of mirrors 31 of the mirror 11. As can be seen in FIG. 5, in the preferred embodiment of the present invention, the display 50 comprises a display panel 51, which is attached to the back face 33 of the mirror 11 to display the user appearance information and data on the mirror surface 32. As can be seen in FIG. 6, in an alternative embodiment, the display 50 comprises a transparent display 52, which is positioned to the mirror surface 32 of the mirror 11. The display 50 can show current time, date, temperature, humidity, etc. Additionally, the user may choose to check the user’s appearance data via the display 50, as can be seen in FIG. 9 to FIG. 11. Further, the smart mirror system can be configured to use machine leaming/deep learning algorithms to provide suggestions based on predictive analyses of the user’s future conditions and detected changes over time. The user can access the information/data on a personal computing (PC) device, including, but not limited to, a mobile phone, a smart phone, a tablet, a smart watch, etc., or a website. The user can use natural languages to operate different types of applications powered by the computing device 70.

As can be seen in FIG. 7 to FIG. 8, the computing device 70 of the smart mirror system comprises a central processing unit (CPU) 71, an AI module 73, a graphics processing unit (GPU) 72, a deep learning accelerator 74, and a vision accelerator 75, a memory unit 76. Specifically, the AI module 73 is electronically connected to the CPU 71, which is adapted to process the plurality of appearance data through the AI module 73 before displaying the appearance information of the user on the display 50. The deep learning accelerator 74 is electronically connected to the AI module 73. Additionally, the GPU 72 is electronically connected to the CPU 71, while the vision accelerator 75 is electronically connected to the GPU 72. In the preferred embodiment of the present invention, the CPU 71 is adapted to acquire a plurality of 3D data from the plurality of cameras 12 of the mirror assembly 10 through the GPU 72 and the vision accelerator 75, wherein each of the plurality of cameras 12 is a 3D camera. With the 3D data of the user’s appearance, the CPU 71 is adapted to process the plurality of 3D data through the AI module 73 and the deep learning accelerator 74. Subsequently, the CPU 71 creates a 3D avatar of the user and displays the 3D avatar of the user to the display 50. As can be seen in FIG. 9, in an alternative embodiment, the smart mirror system may use 3D video data acquired by the CPU 71 of the computing device 70 through the plurality of cameras 12 of the mirror assembly 10. Thus, the resulting 3D avatar is a 3D avatar video presentation of the user. Additionally, the 3D avatar and/or 3D avatar video of the user may be displayed in a 360-degree rotating view on the display 50. In another embodiment of the present invention, the CPU 71 may be adapted to provide a plurality of appearance analysis results using the plurality of 3D data and the plurality appearance data of the user through the AI module 73, wherein the plurality of appearance analysis results includes a change of body condition of the user. The CPU 71 then displays the plurality of appearance analysis results to the display 50. As can be seen in FIG. 10, in another embodiment of the present invention, the plurality of appearance analysis results includes an outfit video of the user, and the plurality of appearance analysis results includes an outfit video of the 3D avatar with an outfit recommended by the AI module 73 of the computing device 70. In this process, the AI technologies, including, but not limited to, deep learning, machine learning, facial recognition, etc., are implemented through the AI module 73 of the computing device 70 to conduct the learning and analysis of the appearance input data and to create the outfit video for the user. As can be seen in FIG.

11, in yet another embodiment, the AI module 73 may conduct learning and analysis of a body pose of the user and provides pose image/video of the user as the plurality of appearance analysis results, which is displayed on the display 50. Further, the CPU 71 may simultaneously display a body pose of any instructor to the user to guide the user in any form of task, including, but not limited to, exercises, fitness lessons, physical training sessions, yoga lessons, or any other suitable teaching and learning. In another embodiment of the present invention, the AI module 73 may conduct learning and analysis of exercise videos of the user, thus the CPU 71 can display the actual video of the user with a simultaneous showing of an instructor’s video to provide efficient and effective learning to the user. As can be seen in FIG. 8, the computing device 70 of the smart mirror system comprises the memory unit 76, which is electronically connected to the CPU 71 of the computing device 70. The CPU 71 is adapted to securely save the plurality of appearance data of the user to the memory unit 76. Additionally, the memory unit 76 may comprise a storage device, including, but not limited to hard disk drive (HDD), solid-state drive (SSD), or any other storage device. Additionally, the storage device may include an extraneous storage device, which is electronically connected to the memory unit 76. The connection may be made through a wired connection, including, but not limited to, a universal serial bus (USB), or a protected/secure wireless connection, including, but not limited to, Bluetooth ^® . The CPU 71 is adapted to securely save the plurality of appearance data of the user to the extraneous electronic storage device through the memory unit 76. The storage device is used not only to enhance the storage capacity of the memory unit, but also to provide data security, especially when collecting the user’s body information and data. Thus, the storage device and the memory unit 76 can safely and securely keep the personal image/video and other data within the smart mirror system and utilize the data for machine leaming/deep learning of the AI module 73.

As can be seen in FIG. 8, the smart mirror system of the present invention comprises a server 77, a database 78, an extraneous communication network 79, and a personal computing (PC) device 90. Specifically, the database 78 is stored on the server 77. The server 77 may include, but is not limited to, a server computer, a laptop computer, a desktop computer, a cloud computing device, and any other suitable computing device. Additionally, the server 77 is managed by an administrator, including, but not limited to, a system administrator, a computer professional, a software engineer, a software specialist, an information technology (IT) engineer, and any other suitable person. The PC 90 device is electronically connected to the server 77 through the extraneous communication network 79, and the server 77 is electronically connected to the computing device 70 through the extraneous communication network 79 as well. The extraneous communication network 79 may include a wireless network, including, but not limited to, Internet, the fifth generation (5G) network, etc. Additionally, the server 77 may be adapted to communicate with the computing device 70 to securely transfer all information, data, and analysis results and make safe storage to an external medium per the user’s request and/or by the administrator. The database 78 is used to store all the data captured by the smart mirror system and the computing device 70 can be configured to access the database78 to transfer any and all necessary data through the extraneous communication network 79. Although the invention has been explained in relation to its preferred embodiment, it is to be understood that many other possible modifications and variations can be made without departing from the spirit and scope of the invention as hereinafter claimed.