CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority of United States Patent Application No. 17/086,708 filed November 2, 2020, and the priority of United States Patent Application No. 17/106,376 filed November 30, 2020, the disclosures of which are hereby incorporated by reference in their entirety. Additionally, this application claims priority of PCT Application No. PCT/EP 2020/086015 filed on December 14, 2020, the disclosure of which is hereby incorporated by reference in its entirety as well.

TECHNICAL FIELD

[0002] The present solution relates to a system comprising at least one device, for example a fitness device, wherein the device comprises at least one display for displaying content to user facing a front side of the device.

BACKGROUND

[0003] Fitness devices which are to be used in gyms or at home and which include at least one display for displaying exercise content to a user become more and more popular. The exercise content may be a video retrieved from a local memory of the fitness device or from a memory of a computing device connected to the fitness device via a wired or wireless network. Alternatively, or additionally exercise content may be provided as a stream, for example via the Internet. Based on the exercise content the user is instructed how to perform different fitness exercises. In order to provide a new and motivating experience to the user and providing the user with sufficient details on how to carry out specific exercise properly, the at least one display of the fitness device may be rather large and held by a frame structure of the fitness device having more than 1 meter in length and more than 40 cm in width.

[0004] In this context fitness devices are also known which include a semi-reflective mirror arranged in front of the at least one display. Such a mirror may reflect incident light from a user facing a front surface of the mirror to present a reflected image of the user. Further, the mirror may transmit incident light from a rear surface of the mirror so that exercise content displayed via the at least one the display is superimposed on at least a portion of the reflected image of the user. Thereby, the user may see his/her reflected image in combination with a visual element, like an image of a trainer or avatar in the exercise content, so that the user may easily notice whether a specific exercise is performed properly or not.

[0005] In addition or as an alternative to a mirror a fitness device may be equipped with an image capturing unit, such as a camera, configured to capture at least one image of the user while exercise content is displayed via the at least one display in order to provide electronic feedback on how well the user is following the instructions of the exercise content and performs an exercise displayed to the user. The user may also receive visual and/or acoustic feedback on her/his current training.

[0006] It is an object of the present solution to provide for a device, in particular a fitness device which may provide for an improved user experience, for example, by an enhanced usability and/or an improved user interface for controlling operations at the device.

SUMMARY

[0007] The proposed solution in particular relates to a system including at least one device, for example a fitness device, and at least one computing device included in or coupled to the at least one device. The at least one device of the system comprises a mirror which reflects incident light from the user facing a front surface of the mirror to present a reflected image of the user and which transmits incident light from the rear surface of the mirror. Additionally, the at least one device further comprises at least one display arranged behind the mirror for displaying content superimposed on at least a portion of the reflected image of the user, wherein the at least one display is configured to display at least one visual element. The visual element can be part of the content which is presented at least partially superimposed on at least a portion of the reflected image or the visual element can be presented in addition to such a content. Furthermore, at least one image capturing unit is part of the at least one device for determining a position of the user in front of the device. The at least one computing device comprises at least one processor and a memory including instructions which, when executed by the at least one processor, cause the at least one processor to determine, based on image data from the at least one image capturing unit of the device, the position of the user in front of the device and to define at least one of an appearance, a size and a position of the at least one visual element for display at the at least one display depending on the determined position of the user. [0008] Defining at least one of an appearance, a size, and a position of the at least one visual element in the present context in particular includes that at least one of an initial (after starting presenting content) appearance, an initial size, and an initial position of the at least one visual element at the at least one display is determined depending on a determined position of the user. Defining at least one of an appearance, a size, and a position of the at least one visual element furthermore shall also in particular include that at least one of the appearance, the size, and the position of the at least one visual element is adapted and/or varied depending on the determined position of the user. The proposed system thus takes into account a position of the user which is determined in a contactless manner using at least one image capturing unit for defining at least one of the appearance, the size, and the position of the at least one visual element displayed on the at least one display arranged behind the mirror of the device.

[0009] At least one of an appearance, a size, and a position of the at least one visual element may thus depend on a position of the user, which is determined using at least one image capturing unit of the device, for example at least one camera. The computing device may generate visual element data, based on the determined position of the user, to be transmitted to the at least one display for having the visual element displayed with an appearance (including the presence of a certain holographic effect), size and/or position defined by the computing device. This approach may, for example, result in increasing a holographic effect, augmenting the visual element and/or adapting hue, color and/or brightness of a visual element depending on a determined position of the user. For example, the visual element may be displayed with an increased size and/or increased brightness in case the user is detected farer away from the device.

[0010] In an exemplary embodiment, the proposed system implements a contactless user position determination method configured to determine a position of the user in front of the display in a contactless manner. The contactless user position determination may, for example, allow to generate position data indicating a determined position of the user (for example also including a posture of the user) while the user performs an exercise. The at least one computing device may be configured to generate the above-mentioned feedback additionally based on such position data. Based on the position data the at least one computing device may for example evaluate whether the user correctly performs an exercise the user is instructed to do by the displayed content. Based on the position data the at least one computing device may also be configured to adapt an appearance of at least one visual element displayed by the display. This may include changing at least one of a size, a shading, a color, a hue and a brightness of the at least one visual element displayed at the device and indicating how well the user is performing an exercise in front of the display. One option for supporting determination of a user position in a contactless manner may include additionally using a laser positioning system.

[0011] Generally, a visual element may be just a “passive” element, for example, relating to parts of a video or a stream instructing the user how a fitness exercise is to be performed or indicating to the user parameters to be taken into account when performing a fitness exercise, such as a number of repetitions or a threshold pulse which should not be exceeded. Additionally or alternatively, a displayed visual element may also relate to biometric data of the user. Further, also additionally or alternatively, a visual element may be “active” and may therefore define an interface element of a user interface of the device. In such an embodiment, the at least one visual element may thus define a region at which a user touching a touch-sensitive portion of the front surface of the mirror generates an actuation signal further processed by a computing device coupled to the at least one display. Touching a region of the mirror where visual element is displayed may thus trigger an operation event, e.g., in particular including representing a new or altered visual element.

[0012] For providing the device with a touch-sensitive portion a capacitive field may be applied to at least a portion of the mirror. The at least one touch-sensitive portion may generate of a userinterface of the device at a front side of the device.

[0013] Content and/or a visual element may therefore be displayed in addition to the reflected image of the user - not meaning that a visual element or the content is necessarily displayed completely superimposed on the reflected image. The content and/or the visual element may be displayed side-by-side with the reflected image of the user or just partially superimposed on the reflected image (e.g., depending on the size of the reflected image)

[0014] In an exemplary embodiment, the instructions, when executed by the at least one processor, may cause the at least one processor of the computing device to adjust at least one of the appearance, the size and the position of a visual element based on a change in the determined position of the user. A corresponding adjustment of at least one of the appearance, the size and the precision of the visual element may depend on a change in position exceeding at least one threshold value. Thereby, an adjustment of the corresponding parameters of the displayed visual element may be avoided due to every movement of the user and, for example, due to minor movements of the user. Just a change within a predefined tolerance range may be accepted without adjusting appearance, size and/or position of the visual element.

[0015] In an exemplary embodiment, a position and/or a size of the reflected image of the user on the front surface of the mirror is determined based on image data from the at least one image capturing unit. The at least one computing device may thus be configured to determine a position (e.g., in x and y coordinates of a Cartesian coordinate system) and/or a size of the reflected image of the user. Thereby, a system may know where the optical reflection of the user is and what the size of this reflection is. For determining a position and/or size of the reflected image based on at least one image captured by the at least one image capturing unit of the device the at least one computing device may make use of calibration or setup data, for example, indicating an optical capturing angle of the image capturing unit, an angle of the device with respect to the ground (where the device is located) and measured or calculated dimensions of the user in front of device. Based on the corresponding information it is mathematically possible to conclude on a position and a size of the reflected image of the user in front of the device.

[0016] In an exemplary embodiment, at least one of the appearance, the size, and the position of the at least one visual element may then be adjusted with respect to a determined position and/or a determined size of the reflected image of the user. This in particular includes adapting in realtime at least one of the appearance, the size, and the position of the at least one visual element depending on the determined position and/or determined size of reflected image of the user and to synchronize at least one of the appearance, the size, and the position of the at least one visual element with a position of the user in front of the device. For example, adjusting at least one of the size and the position of the at least one visual element with respect to a determined position and/or a determined size of the reflected image of the user may include adjusting superimposition of the at least one visual element on at least a portion of the reflected image of the user. The system may thus adapt at least one visual element, which is displayed superimposed on the reflected image of the user, in such a way that the super imposition is maintained, even when a change in position of the user occurs.

[0017] In an exemplary embodiment, the computing device comprises a face detection module configured to extract, from at least one image captured of the user in front of the device by the at least one image capturing unit, position information relating to a position of a head and of eyes of the user in the at least one captured image. Based on generally known motion tracking and face detection algorithms the system may thus be configured to generate user-specific position data indicating a position of a head and of eyes of the user in a captured (video) image. Based on the at least one captured image the system may thus not only determine a position and/or a size of the reflected image of the user on the front face of the mirror. Rather, in this embodiment, the system may furthermore be capable of determining a viewing perspective of the user on the front surfaces of the mirror. A viewing perspective of the user on the front surface of the mirror may thus be algorithmically generated (calculated) based on the position information generated by the face detection module. Based on the determined viewing perspective of the user the at least one of the appearance, the size, and the position of the at least one visual element may then be defined for display. This may, for example, be advantageous to ensure that the at least one visual element superimposed on at least a portion of the reflected image of the user continues to be perceived by the user as superimposed.

[0018] The at least one image capturing unit may for example comprise at least one of a monocular camera, a stereoscopic camera, a time-of-flight camera, and an infrared sensor.

[0019] For further increasing the user experience, the at least one display and the mirror may be configured to create a holographic effect for at least one visual element (of or supplemental to the content) by an additional projection of the at least one visual element on the mirror. For generating the additional projection, a front side of the at least one display in which the content is displayed is arranged at predetermined maximum distance to the rear surface of the mirror and the mirror has a predetermined transparency-opacity ratio. The at least one display of the device may be configured to display the at least one visual element on a black toned background, wherein the display and the mirror are further configured to create a holographic effect for the at least one visual element by an additional projection of the at least one visual element on the mirror.

[0020] The instructions may, when executed by the at least one processor, furthermore cause the at least one processor to pre-process content to be displayed by the at least one display of the device. A pre-processing of the content may comprise identifying at least one main visual element in the content for display and blackening out a background of the at least one identified main visual element in the content for display. In a corresponding embodiment, content to be displayed may be computationally analyzed by the at least one computing device for adapting the content in such a way that a desired holographic effect for the visual element is achieved. The identified at least one main visual element in the content for display may then constitute the at least one visual element, which is additionally projected on the mirror as a kind of duplicate for the desired holographic effect. For example, an algorithm implemented by the instructions may identify at least one key area, person or object within a content for display, such as a video or stream, which is mapped and identified as a main visual element (this may also include that several, separate main visual elements are identified). Based on the identified main visual element(s) the rest of an image in a frame of the video or stream may then blacked out in order to provide the main visual element(s) with a holographic effect based on a proposed configuration of the device. For example, a video or stream showing a person, in particular a trainer performing a fitness exercises, may be pre-processed in a way that the person is visible and the rest of the content surrounding the person is covered by a black toned background. In such a case, the person may be visible in a holographic form, which might be of great benefit for augmenting and presenting the person, not only as part of a video or a stream for exercise content but also for videoconferencing or in other situations where a certain visual element is considered to be of (more) importance compared to the rest of an information in the video or stream.

[0021] By arranging the display with its front side at a predetermined minimum distance to the rear surface of the mirror and providing the mirror with a predetermined transparency-opacity ratio a holographic effect may be achieved on the mirror (and thus at a front side of a display surface of the device) so that the display additionally projects the at least one visual elements on the mirror, in particular on the rear surface of the mirror. The minimum distance of the display to the rear surface of the mirror is based on a given transparency-opacity ratio of the mirror, thereby making sure that a duplicate of the visual element is projected on the mirror (directly in front of the representation of the visual element on the display), resulting in a holographic effect for the at least one visual element. The user may thus perceive the visual element with glowing and/or hollow effect improving visibility and visual impression of the displayed visual element. Using a black toned (and this for example a black or gray) background on which the visual element is a displayed combined with an arrangement of the display at a predetermined minimum distance to the rear surface of the mirror and a predetermined transparency-opacity ratio of the mirror may also generate a certain light reflection and refraction phenomena resulting in a favorable holographic effect for displayed visual elements. The proposed device may in particular be configured to display exercise content and therefore be used as a fitness device. The proposed solution shall however not be construed to be limited in this regard. In particular, the proposed solution may also be implemented in a device not related to fitness exercise, e.g., a device for gaming and/or videoconferencing .

[0022] In an exemplary embodiment, the at least one display is configured to display the at least one visual element (including just a portion of it) with a hue in the range of 0° to 345° in the HSL or HSV color space (HSL standing for “hue, saturation, lightness” and HSV standing for “hue, saturation, value” as alternative representations of the RGB color model).

[0023] Alternatively or additionally, the at least one display may be configured to display the black tone background with a black or gray color having a color code, according to or complying with the RGB color code, in which r=g=b. For a black background, r, g and b may of the same value and equal to 0. Exemplary gray backgrounds may be defined by rgb(220,220,220), rgb(211,211, 21 l, rgb(192, 192, 192), rgb(169, 169, 169), rgb(128, 128, 128) or rgb(105,105,105).

[0024] Generally, the at least one display may be coupled to a computing device configured to calculate values for r, b and g (parameters) depending on the transparency-capacity ratio of the mirror. For example, for a given transparency-opacity ratio of the mirror the computing device - which may be a part of the device or just coupled to it - may define suitable values for displaying a corresponding black toned background so as to assure that the visual element displayed results in the desired projection on the mirror. The computing device may comprise at least one processor and a memory including instructions which, when executed by the at least one processor, cause the at least one processor to set the RGB values - or corresponding values in an alternative representation of the RGB color model - depending on a given transparency-opacity ratio of the mirror, namely based on a computationally implemented relationship between the color values and the transparency-opacity ratio.

[0025] In an exemplary embodiment, the predetermined minimum distance lies in the range of 1.5 millimeters to 7 millimeters. It has been found that a corresponding distance between the display and the mirror in certain configurations provides for a beneficial holographic effect agreeable to users of the device.

[0026] The transparency-opacity ratio of the mirror may lie in the range of *4 to %. This means that a transparency of the (semi-reflective) mirror may range from 20% to 75% with the opacity respectively lying between 80% and 25%.

[0027] In an exemplary embodiment, the mirror may include a reflective film power coating for reflecting the incident light from the user facing the front side of the mirror. By using a reflective film powder coating the mirror may include a protective glass cover to which the reflective film powder is applied. Such a configuration might be beneficial with respect to costs and also with respect to facilitating manufacture of the mirror with a desired transparency-opacity ratio. [0028] In an exemplary embodiment, the mirror has a thickness of more than 0.5 mm. The mirror is a thus at least 0.3 millimeters thick, wherein the thickness defines a distance between the front surface and the rear surface of the mirror. A certain minimum thickness of the mirror may be beneficial for achieving a desired holographic effect for visual elements displayed on the device.

[0029] In an exemplary embodiment, the at least one image capturing unit may be configured to capture at least one image of the user while the user performs an exercise in front of the display (for example while an exercise content is displayed which instructs the user how to perform an exercise). The at least one computing device may be configured to generate a feedback to the user resulting from a comparison, by the at least one computing device, of the image (data) with reference data. The reference data may be stored in a memory of the at least one computing device and may relate to a completely correct performance of a fitness exercise displayed. The feedback to the user may be visual and/or audible. The at least one computing device may, for example, be configured to alter, in response to triggering the at least one operation event and based on the temperature data, the feedback to the user. Altering the feedback to the user may, for example, include informing the user visually and/or audibly - taking, for example, also into account temperature data provided by at least one temperature sensing unit of the device as described above - that the user is heating up too much or that the user sufficiently warmed up during warm-up session. The feedback may also include an alarm if a determined temperature exceeds a threshold implying too much physical effort by the user while performing an exercise.

[0030] In an exemplary embodiment, the device may further comprise at least one contactless temperature sensing unit configured to determine at least one temperature of at least one body part of the user in a contactless manner and to generate temperature data indicative of the at least one temperature. The at least one computing device is coupled to the at least one contactless temperature sensing unit, wherein the at least one computing device is configured to electronically trigger, based on the temperature data, at least one operation event. The at least one computing device may thus be configured to trigger, based on the temperature data, at least one operation event at the device or at a remote device communicating with the device. Such an embodiment may thus allow for triggering one or more pre-defined operation events depending on temperature data provided by a contactless temperature sensing unit which temperature data is indicative of temperature of at least one body part of the user. Accordingly, thermal information about a user to which content is displayed by the device may be used to control one or more functions by the device and/or a display of the device, for example, resulting in calling at least one software routine and/or altering at least a part of the content to be displayed to the user based on the temperature data.

[0031] The at least one temperature sensing unit may for example include a pyrometric device. A pyrometric device allows for determining at least one temperature value for an object based on temperature equivalents. A pyrometric device is thus capable of providing temperature data indicative of a temperature of at least a body part of a user in front of the device. Such temperature data may be further processed in embodiments of the proposed solution to control one or more functions by the device and/or a display of the device.

[0032] In an exemplary embodiment, the at least one computing device and the at least one display are configured to display content and at least one (additional) visual element to the user. The at least one computing device may be configured to alter, in response to triggering the at least one operation event and based on the temperature data, at least one of the content and the at least one visual element displayed to the user. This may include that the at least one display is configured to display exercise content and an additional content-related visual element indicating calories burned by the user performing an exercise depicted in the exercise content. The at least one computing device may be configured to alter the visual element indicating calories burned by the user based on the temperature data. In this exemplary embodiment, the temperature data is further processed by the at least one computing device in order to calculate or estimate calories burned by the user when doing a certain kind of exercise. Using temperature data in this context may allow for estimating the calories burned more precisely.

[0033] In one embodiment, the at least one display may alternatively or additionally be configured to display exercise content including a visual element indicating a training schedule for the user and the at least one computing device is configured to alter at least a portion of the training schedule based on the temperature data and to alter the visual element to depict the alteration in the at least one portion of the training schedule. The training schedule (which is indicated by the later altered visual element) may include an exercise which is currently performed by the user. Therefore, the at least one portion of the training schedule which is altered based on the temperature data may relate to at least one subsequent exercise in the training schedule which is thus changed or adapted. The at least one computing device, based on an implemented algorithm, is thus capable automatically changing a subsequent exercise to be displayed to the user dependent on the received temperature data. If, for example, a user heats up too much while doing a certain exercise (thus exceeding a pre-defined temperature threshold) the computing device alters the training schedule so that a less exhausting fitness exercise follows for display which allows the user to cool down at least a bit.

[0034] In one embodiment, altering at least one portion of the training schedule based on the temperature data received for a current user performance may (also) include altering future exercises of the training schedule which are scheduled for at least one subsequent training or workout session. Accordingly, a training schedule may be adapted based on the temperature data in order to suggest a more challenging or less exhausting fitness exercise to the user by the device. Altering a training schedule may also include that the at least one computing device is configured to check for a sufficient warm-up by the user based on the temperature data. This may include comparing, by the at least one computing device, stored temperature reference data with the received temperature data. In case a temperature threshold is reached, for example a predefined threshold level relating to a temperature distribution for one or more body parts of the user, the at least one computing device may be configured to give access to, activate and/or unlock additional content for display by the device in response to the operation event. Merely in case a certain temperature level is reached which minimizes the risk for injury, the user is thus allowed, by getting corresponding additional content displayed by the device, to perform certain fitness exercises belonging to a workout session of the training schedule.

[0035] In an exemplary embodiment, the at least one display is be configured to display exercise content including a visual element depicting a temperature profile and/or a heat map for the user. A heat map may illustrate - by at least one of a size, a shading, color, hue and brightness - different temperature values in body part of a body of the user. Accordingly, a heat map allows the user to (in real-time and afterwards) easily retrieve information on heat distribution in particular on currently trained body parts.

[0036] The at least one computing device may be configured alter, in response to triggering the at least one operation event and based on the temperature data, at least one portion of a depicted heat map. The received temperature data may thus, for example, allow for an update of the heat map, for example simultaneously to a user performing a fitness exercise in front of the device.

[0037] Based on the temperature data the at least one computing device may, for example, alter at least one portion of the heat map by changing at least one of a size, a shading, a color, a hue and a brightness of a region of the heat map.

[0038] The device may comprise a frame structure which defines a continuous frame inside which the mirror is held. The mirror may thus be circumferentially completely encompassed by the frame. In an exemplary embodiment, the mirror may have a length of at least one 1.0 m and a width of at least 40 cm. Thereby, a user can see himself/herself in full height on the front surface of the mirror.

[0039] The device may comprise an interface region including at least one of a connector for power supply, a connector for a wired network connection, a connector for a video or sound interface, like an HDMI plug or the like, and a switch, in particular a power switch. The interface region thus for example allows for connecting the device to a power supply, at least one computing device and/or a network via at least one cable. The interface region may for example be disposed at the back side of the frame structure so that the interface region is easily accessible at the back side.

[0040] The frame structure may define a continuous a frame inside which the mirror is held. In one embodiment, the mirror abuts on a bottom part and a top part of the frame as well as on two opposing side members connecting the bottom part and the top part. A mirror whose front surface extends over substantially the whole front side (except for the part of the frame structure at the circumference of the front side) is thus securely held within a frame of the frame structure which protects the mirror edges from damages and also facilitates assembly of the device. In an embodiment the frame of the frame structure has a thickness of less than 3 mm or even less than 2 mm. At the circumference of the mirror the frame is therefore rather slim and almost not perceivable at the front side.

[0041] In an exemplary embodiment, the front side of the device has a height and a width and the at least one display has a display height which is greater than one half of the height of the front side and has a display width which is greater than one half of the width of the front side. Accordingly, the at least one display of the proposed stand-alone device may be relatively large and may provide, for example, for a screen having a size of more than 34 inches, in particular more than 38 inches or 40 inches.

[0042] In an exemplary embodiment, the at least one computing device and the at least one display are configured to display a visual element in the form of a heat map at least partially superimposed on a reflected image of the user. The heat map, in particular a heat map with a holographic effect, may thus be displayed as an overlay on at least a body part in the reflected image of the user, thereby illustrating at least one determined temperature value for the body part of the user. In addition, the proposed device may also comprise a user position determination system for tracking position of the user (including a posture of the user) for correctly overlaying the heat map on the reflected image of the user in real-time.

[0043] In an exemplary embodiment, the proposed device further comprises at least one communication interface for communicating with at least one supervisor device. The at least one computing device may, for example, be configured to transmit temperature data to the at least one supervisor device via the at least one communication interface. This may, for example, include transmitting the temperature data wirelessly via the communication interface the remote supervisor device. The remote supervisor device may then, for example, be used by a human trainer, mentor or therapist monitoring the performance of the user in front of the device.

[0044] The at least one computing device may be configured to receive, in response to the operation event, at least one instruction signal from the supervisor device and to, in response to the received at least one instruction signal, alter a) at least one of the content and at least one visual element displayed to the user via the display and/or b) give the user access, activate and/or unlock additional content for display by the device. After and/or during a transmittal of temperature data and while user performs an exercise in front of the device, a user of the supervisor device may trigger sending at least one instruction signal, for example a message or command, to the device in order to provide a certain feedback to the user by altering least one of the content and a visual element. Additionally or alternatively, the at least one instruction signal may allow the user to access, activate and/or unlock additional content based on the instructions of the user of the supervisor device. The user of (second) supervisor device monitoring the performance of the user of the (first) device may thus for example check for a correct warm-up or a correct performance of an exercise by the user, in particular based on the contactlessly determined data on the user temperature. This allows for an effective monitoring by a remote human trainer, mentor or therapist using the supervisor device although not being in the same room with the user performing the exercise in front of the display.

[0045] In order to dynamically and individually handle triggered operation events based on temperature data in an effective way, the at least one computing device may be configured to evaluate the temperature data using machine learning. Additionally or alternatively, the at least one computing device may be configured to transmit the temperature data to a remote analysis computing device implementing machine learning algorithms and to receive from the remote analysis computing device an analysis of transmitted temperature data, in response to transmitting the temperature data to the remote analysis computing device. Machine learning at the at least one computing device or the remote analysis computing device, for example one or more (cloud) servers, may include using a neural network. In one exemplary embodiment, a manifold learning process may be used, such as a principal component analysis (PCA). Also a non-negative matrix factorization may be implemented, in particular instead of a PCA. For an exemplary embodiment evaluating the temperature data using machine learning algorithms, e.g., at the at least one computing device or at the at least one remote analysis computing device, and t-distributed stochastic neighbor embedding (t-SNE) might be useful.

[0046] Generally, the proposed device may be a stand-alone fitness device configured to display exercise content to the user.

[0047] According to a further aspect of the claimed solution a method of operating a system comprising at least one device is proposed, which device comprises a mirror which reflects incident light from a user facing a front surface of the mirror to present a reflected image of the user and which transmits incident light from a rear surface of the mirror; at least one display arranged behind the mirror for displaying content superimposed on at least a portion of the reflected image of the user, wherein the at least one display is configured to display at least one visual element; and at least one image capturing unit for determining a position of the user. The method includes determining, based on image data from the at least one image capturing unit, the position of the user in front of the device, and defining (including later adapting and/or varying) at least one of an appearance, a size, and a position of the at least one visual element for display at the at least one display depending on the determined position of the user.

[0048] Embodiments of the proposed method may in particular relate to operating an embodiment of a proposed system. Accordingly, features and advantages mentioned above and below in the context with an embodiment of a proposed system shall therefore also apply to embodiments of the proposed method and vice versa.

BRIEF DESCRIPTION OF THE DRAWINGS

[0049] FIG 1A is a perspective view on an embodiment of a proposed system with a view to a front side of a device of the system.

[0050] FIG IB is a perspective view of the device with a view to a back side. [0051] FIG 2A is a front view of the device.

[0052] FIG 2B is back vow of the device.

[0053] FIG 3 is a side view of the device illustrating dimensions of and angles between components of the device.

[0054] FIG 4 is exploded view of the device.

[0055] FIG 5 is an enlarged detail of the back side of the device showing a mounting region for a stand of the device.

[0056] FIG 6 is an enlarged detail of a bottom part of a frame structure of the device.

[0057] FIG 7 is an enlarged detail of an upper portion of the device showing details of a (first) camera of the device and a power switch accessible at the front surface.

[0058] FIG 8 is a perspective view on the frame structure of the device to which a display is mounted without showing a mirror and a back plate of the device.

[0059] FIG 9A shows the display with its front surface.

[0060] FIG 9B shows the display with its rear surface.

[0061] FIGs 10A, 10B and 10C show different cross-sectional views of the device at an edge section.

[0062] FIG 11A shows the device while displaying exercise content to a user in front of the device.

[0063] FIG 1 IB shows an enlarged detail of the device illustrating a reflected image of the user and superimposed visual elements of the exercise content each displayed with a holographic effect.

[0064] FIG 12 shows the device with exemplary fitness content displayed to another user in front of the device.

[0065] FIG 13 shows details of a user interface of the device including interface elements displayed with a holographic effect

[0066] FIG 14A shows a front view of the device with another type of graphical user interface. [0067] FIG 14B show an interior of an exemplary embodiment of the device in a rear view without a back plate.

[0068] FIG 15A shows a side view of device with a user in front of the device.

[0069] FIG 15B show a graphical user interface of the device of FIG 15A in a calibration phase.

[0070] FIG 16 shows the s device with a graphical user interface in particular depicting a heat map overlaid on a reflected image of the user.

[0071] FIG 17 shows a front view of the device with the graphical user interface depicting different element in another arrangement.

[0072] FIG 18 shows a flow chart illustrating a proposed method implemented in the device for defining at least one of an appearance, a size, and a position of at least one visual element for display depending on a determined position of a user.

DETAILED DESCRIPTION

[0073] FIG 1 illustrates an embodiment of a proposed system with a device in the form of a stand-alone device 1 (sometimes also called a “smart mirror” or an “interactive screen”). The device 1 of the embodiment of FIG 1 may be configured as a stand-alone fitness device to be used in a gym or at home (referred to as “fitness device” in the following), for example used for workouts, physiotherapy, and physical rehabilitation. The fitness device 1 comprises a frame assembly including a frame structure 10, a display 11 and a mirror 12 (the display 11 and the mirror 12 defining a display assembly of the fitness device 1). The frame structure 10 defines a continuous frame surrounding the mirror 12. The mirror 12 is thus held within the frame of the frame structure 10. The display 11 is held behind the mirror 12. The display 11 includes a screen for displaying exercise content to a user standing in front of the fitness device 1. The frame assembly of the fitness device 1 defines a front side 1A and a back side IB. The exercise content displayed via the display

11 may hence be watched by a user facing the front side 1 A.

[0074] The mirror 12 may reflect incident light from the user facing a front surface of the mirror

12 to present a reflected image of the user. The mirror 12 may further transmit incident light from the rear surface of the mirror 12 so that exercise content displayed via the display 11 may be superimposed on at least a portion of the reflected image of the user. By superimposing exercise content on at least a portion of the selected image of the user the fitness device 1 may provide instant feedback on a performance of the user when imitating fitness exercises based on the displayed exercise content.

[0075] Alternatively or additionally, at least one camera may be part of the fitness device to capture images of the user during a fitness exercise in order to electronically provide feedback on the user’s performance whether the user imitates a certain exercise properly and., e.g., to which extent.

[0076] The fitness device 1 as shown in FIG 1A includes the display 11 having a screen with a diameter of at least 34 inches, for example more than 40 inches. Accordingly, the frame structure 10 defines a surface area at the front side of more than 0.5 m ² . For example, a substantially rectangular surface at the front side 1 measures more than 1.5 m in height and more than 0.4 m in width.

[0077] Given the size and weight of the fitness device 1 a secure and stable position of the fitness device 1 with its frame structure 10 is crucial. For this reason, the fitness device 1 of FIG 1A may rest via at least two different components at a floor. A first component is provided by bottom part 100 of the frame structure 10. The bottom part 100 allows the frame structure 10 to rest on the floor and thus provides a support directly below the display 11. In addition, a stand 2 is provided which allows the frame structure 10 to be positioned inclined to the vertical at the floor and to be nevertheless held in a stable position. The stand 2 extends at an angle to the back side IB of the frame structure 10 so that a base portion of the stand 2 (formed by a base member 21) is configured to rest on the floor being the bottom part 100 in a specified distance to the bottom part 100.

[0078] As further illustrated by FIG IB, the stand 2 of this embodiment is designed as a rectangular frame including a base member 21, a crossbar 22 and two parallel lateral bars 20.1, 20.2. The lateral bars 20.1, 20.2 are connected to each other at a first (upper) end via the crossbar 22 and at a second (lower) end via the base member 21. At the crossbar 22 the stand 2 is fixed to a back plate 13 the fitness device 1. The back plate 13 is attached to the frame structure 10 at the back side IB.

[0079] In a display area of the frame structure 10 at which at the display 11 is held, the mounting plate 13 defines a mounting region. At this mounting region, the crossbar 22 is attached to the back plate 13 via an attachment portion of the stand 2. The stand 2 is fixed to the back plate 13 (and for example a component of the frame structure 10 covered by the back plate 13) via several fastening elements, for example in the form of screws, bolts or rivets which at least partially extend through the attachment portion at the crossbar 22.

[0080] The attachment portion of the crossbar 22 is fixed to the back plate 13 at a location above a recess portion or recess 130 of the back plate 13 and below two grille portions 132.1 and 132.2 for loudspeakers of the fitness device 1. The recess 130 includes an interface region 131. At an edge section of the recess 130 the interface region may be accessible for the user and may comprise at least one of a connector for a power supply, a connector for a wired network connection, one or more connectors for a video or sound interface and a main power switch.

[0081] As can be seen from the front view of FIG 2 A, the fitness device 1 may for example in particular include one or more cameras 5.1, 5.2, 5.3 for capturing images of the user facing the front side 1A. For example, a first camera 5.1 may be positioned at the front side 1A in the middle of a portion above the display 11. Additional second and third cameras 5.2 and 5.3 are positioned on lateral sides so that a display recess 101 for a screen of the display 11 is arranged between optics of the second and third cameras 5.2 and 5.3.

[0082] The first, second and third cameras 5.1, 5.2 and 5.3 may be part of a camera system of the fitness device 1 which are controlled by a software component running on at least one computing device 6 of the fitness device 1. The first camera 5.1 may be provided for communication, entertainment and/or displaying visual elements, in particular interface elements, depending on a predetermined position of the user. The second and third cameras 5.2 and 5.3 may provide image data for a motion analysis module of the fitness device (implemented by the computing device 6). The motion analysis module may be provided for analyzing position and posture of a user when performing a fitness exercise in front of the fitness device 1. Accordingly, the different cameras 5.1, 5.2 and 5.3 as well as the display 11 are coupled to the computing device 6, and in particular its motion analysis module.

[0083] In addition, the computing device 6 may receive input(s) i including a video or stream to be presented on the display 11. The input i may be already pre-processed for displaying the respective content with a holographic effect (explained below in greater detail). Alternatively, the content received as input i may be pre-processed by the computing device 6, for example using artificial intelligence. The input i may also include an instruction signal from a supervisor device with which the fitness device 1 communicates while a user performs an exercise in front of the fitness device 1, which will be explained in greater detail in the following.

[0084] In addition, the computing device 6 may include or be connected to a contactless user position determination system. Said contactless user position determination system may comprise a calibration module recognizing proportion differences between images captured by the second and third cameras 5.2 and 5.3. Thereby, the calibration module may also be configured to detect at which distance and thus position from the fitness device 1 a user is located. This may result in calibrating the motion analysis module with x and y coordinates which may then also be used for defining at least one of an appearance, a size, and a position of at least one visual element to be displayed on the display 11. Hence, visual elements display may be dynamically adapted based on image data provided by the camera system including the second and third cameras 5.2, 5.3. For example, depending on the distance of a user to the fitness device 1 a displayed visual element may be augmented and/or displayed with an increased brightness. Additionally or alternatively, the contactless user position determination system of the fitness device may comprise a laser positioning system for determining a position of a user in front of the display 11 with respect to the fitness device 1 in order to dynamically adapt information and thus in particular visual elements displayed by the display 11.

[0085] In addition or in alternative to the second and third cameras 5.2 and 5.3 the contactless user position determination system may comprise a laser positioning system 15 for determining a position of the user in front of the fitness device 1. An intersection of two laser beams of two laser emitting devices of the laser positioning system 15 at a point of intersection KP in front of the fitness device 1 may be used for precisely determining where a user is located (see also FIGs 15A and 16). The laser positioning system 15 may in particular allow for securely determining a distance of a user to the front side 1A.

[0086] In addition, a proximity sensor 16 may be part of fitness device 1 in order to detect a user approaching the fitness device 1. A detected approach may for example result in switching the fitness device 1 from an off-state to an on-state and/or displaying certain interface elements on the display 11 given that the user approaching the fitness device 1 is then in reach for touching the mirror 12.

[0087] The fitness device 1 further comprises a pyrometric device 5.4. The pyrometric device may, for example, include one or more thermal cameras, one or more infrared sensors or one or more other sensors being part of a corresponding contactless temperature sensing unit. The pyrometric device is capable of determining temperature values for body parts of a user in front of the fitness device 1. Temperature values may be measured contactlessly by the pyrometric device 5.4 for generating corresponding temperature data. The temperature data generated by the pyrometric device 5.4 is received by the computing device 6 and for example further processed by a machine learning module 60 of the computing device 6. The machine learning module 60 may apply machine learning algorithms for analyzing the received temperature data. For example, a principal component analysis (PCA) or a t-distributed stochastic neighbor embedding (t-SNE) may be used to generate data to be presented on the display 11 using the temperature data. Based on the analyzed temperature data, for example, a heat map, a temperature profile and/or one or more diagrams may be displayed as visual elements on the display 11.

[0088] Temperature data from the pyrometric device 5.4 may also be transmitted via a communication interface of a connectivity module 61 of the computing devices 6 to a remote analysis computing device, for example one or more cloud servers.

[0089] Based on analysis results provided by the machine learning module 60 or via the remote analysis computing device temperatures and their distribution over the different body parts of the user located in front of the fitness device 1 may be determined and visualized. This may for example include temperature distributions for a head, a neck, shoulders, arms, the upper and lower body, and the legs of the user.

[0090] The computing device 6 furthermore includes a computer vision module 62 and the face detection module 63. The computer vision module 62 may implement a computer vision algorithm configured to extract body characteristics of a user P performing an exercise in front of the fitness device 1. The computer vision module 62 may furthermore be configured to extract skeleton poses based on a skeleton model (such as for example provided by OpenPose) for generating skeleton data based on at least one image captured by at least one of the cameras 5.1-5.3. The face detection module 63 may furthermore implement a face detection algorithm configured to extract a position of a head and of eyes of the user P in the image captured by at least one of the cameras 5.1-5.3. Knowing the position of the head of the user and the eyes of the user may in particular allow for determining a supposed viewing perspective of the user on the front of the mirror 12 as explained in greater detail below.

[0091] The back view of FIG 2B further illustrates the position, dimensions, and the configuration of the stand 2 with respect to the frame structure 10. Here, a height of the stand 2 is greater than one half of a total height h of the frame structure 10 (cf. also FIG 3). Thereby, the attachment portion of the crossbar 22 is fastened to the mounting region at the back side 1A at a height (distance to the bottom part 100) which is greater than one half of the total height h of the frame structure 10 but less than two-thirds of the total height.

[0092] In the shown example, the crossbar 22 is fixed to the back plate 13 via three fastening elements, for example three screws, at three mounting holes 220, 221, and 222. In addition, the first (upper) end of each lateral bar 20.1, 20.2 is separately fixed to the back plate 13 via mounting holes 201, 202 for providing an additional fixation of the stand 2 and its parts to the back plate 13 and thus to the frame assembly of the fitness device 1.

[0093] As can be seen from the side view of FIG 3, the frame of the frame structure 10 has a total height h which is several times bigger than a width w of the frame. Accordingly, the frame of the frame structure 10 is relatively thin or slim which provides for an elegant outer appearance of the fitness device 1. For example, the width w lies in the range of 2 mm to 5 mm, whereas the total height h is the greater of then 1.5 m, for example lies in the range of 1.5 m to 1.8 m.

[0094] The stand 2 extends from its attachment portion at the crossbar 22 to its base member 21 at an angle of about 18° to 25° to the frame structure 10. Thereby, the frame structure 10 (and the display 11) may be held in a stable inclined position. Accordingly, the frame structure 10 does not extend in a completely vertical plane but with a certain inclination to the vertical, namely at an angle oc of less than 90° relative to the floor. For example, the angle oc lies in the range of 75° to 82°.

[0095] The base member 21 of the stand 2 is furthermore resting on the floor at a distance d to the bottom part 100. For example, the distance d lies in the range of 40 cm to 50 cm.

[0096] The exploded view of FIG 4 in particular illustrates an inclined mounting interface of the stand 2 for abutting on the back side 13. Via the inclined mounting interface 22 the stand 2 may abut along a defined plane area against the back plate 13 while the stand 2 being angled away from the back plate 13. FIG 4 furthermore illustrates screws 4 to be used for fastening the stand 2 via the mounting holes 220, 221, 222 and 201, 202 to the back plate 13 and also to a part of the frame structure 10 covered by the back plate 13. [0097] The enlarged view of FIG 5 shows that the mounting holes 220, 221 and 222 of the crossbar 22 as well as the mounting holes 201 and 202 of the two lateral bars 20.1 and 20.2 in greater detail.

[0098] The enlarged view of FIG 6 furthermore illustrates that the bottom part 100 of the frame structure 10 may include at least two height adjusters 3 A, 3B. Via this height adjusters 3 A, 3B of the bottom part 100 the frame structure 10 may rest on the floor. By turning the height adjusters 3 A, 3B in a threaded hole of the frame structure 10 irregularities of the floor may be compensated for and an inclination of the frame structure may be adjusted.

[0099] FIG 7 shows an enlarged view of an upper portion of the fitness device 1. Therefore, FIG 7 in particular illustrates the arrangement of the first camera 5.1 above the display recess 101. Additionally, FIG 7 also shows a power switch B on a comer portion at the front side 1A. By pressing the power switch B the fitness device 1 may be switched on or off.

[00100] FIG 8 shows the rectangular frame structure 10 of the fitness device 1 with the display

11 mounted to it. In FIG 8 a rear surface 11B of the display 11 is visible. The display 11 is fixed within the frame structure 10 at a predefined position and orientation so as to be held in a particular relative position to the mirror.

[00101] FIGs 9A and 9B show the display.i l with a front surface 11A and its rear surface 11B, respectively, without the further components of the fitness device 1.

[00102] The cross-sectional views of FIGs 10A, 10B and 10C further illustrate the arrangement of the mirror 12 and the display 11 at the fitness device 1. In the illustrated embodiment, the mirror

12 has a thickness of more than 0.3 millimeters and is provided with a reflective film powder coating for reflecting the incident light from a user. The mirror 12 is held in the frame structure 10 at a mirror enclosure 102 so that the display 11 has it to be fixed at the frame structure 10 behind the mirror 12 at a predefined minimum distance d _m of at least 1.5 millimeters. In addition, the mirror 12 is provided with a specific transparency-opacity ratio of at least *4 (up to %). Accordingly, the mirror 12 may have a transparency between 20% and 75%. In combination with the display 11 arranged at the predetermined minimum distance d _m behind the mirror 12 parts of content displayed on the display 11 on a black toned background may be presented with a certain holographic effect. A visual element displayed on correspondingly black tone background having a hue in the range of 0° to 345° in the HSL or HSV color code domain results in an additional projection of the visual element on the rear surface of the mirror 12. Thereby, a duplicate of the visual element is projected and a kind of glow and hollow effect for the displayed visual element is achieved. In this case, by using a black tone background and the reflective surface of the mirror 12 with its predefined transparency-opacity ratio, a light reflection refraction phenomena may be generated so that a user may perceive a displayed (non-black and, compared to the background, brighter) visual element as a kind of floating hologram at the front side 1A of the fitness device 1.

[00103] A corresponding holographic effect, which is further illustrated by exemplary embodiments of FIGs 11A, 11B, 12 and 13, may be altered on the basis of a ratio between the black tone of the background and the transparency of the mirror 12. As a rule of thumb, it may be considered that the lighter the black toned background, the smaller the transparency of the mirror 12 should be in order to provide the user with an agreeable holographic effect. For this reason, at least one processor of the computing device 6, in an exemplary embodiment, may be configured to pre-process content to be displayed, depending on a known/preset transparency-opacity ratio of the mirror 12, in order to black out a background in the content to be displayed and to present at least one visual element on the background (which was identified, by the at least one processor, as a main visual element in the content) so that this visual element appears with a certain hue and brightness on the black tone background.

[00104] FIG 11A shows an exemplary embodiment of the fitness device 1 showing exercise content. The exercise content includes a visual element 7.1 in the form of a visual representation of a trainer showing the exercise to be performed. The visual element 7.1 is displayed as part of a user interface 7 by the display 11. The user interface 7 is displayed on a black toned background, whereas the visual element 7.1 - positioned in the center of the display 11 - is displayed with lighter and brighter colors having a hue in the range of 0° to 345°. In addition to the virtual trainer of the visual element 7.1 additional visual elements are displayed, like are visual element 7.2 in the left upper corner of the user interface 7 indicating a number of repetitions to be performed. The different visual elements 7.1, 7.2 are displayed such that perception of the visual elements 7.1, 7.2 is facilitated for a user P in front of the fitness device 1.

[00105] As can be seen from the enlarged view of the user interface 7 in FIG 11B, the visual elements 7.1 and 7.2 are superimposed on a reflection PR of the user P on the mirror 12. Based on the previously described configuration of the mirror 12 and the display 11 in combination with the black toned background of the user interface 7 and the lighter colors of the displayed visual elements 7.1 and 7.2, a holographic effect is achieved for the displayed visual elements 7.1 and 7.2. In particular, a duplicate image is projected in each case on the rear surface of the mirror 12 so that each visual element 7.1, 7.2 may be perceived by the user P as a floating hologram on the front side 1A. This may include that the visual elements 7.1 and 7.2 are generated with a kind of glow and hollow effect on the black toned background of the user interface 7.

[00106] The corresponding holographic effect is used, for example, to augment a virtual or live streamed trainer or avatar as a holographic image while a user P is doing exercises or any other comparable movement for which guidance by the fitness device 1 by a particular visual element might improve the user experience.

[00107] As further illustrated by FIG 12, the fitness device 1 may dynamically vary at least one of an appearance, a size, and a position of at least one of the visual elements 7.1, 7.2 on the user interface 7 depending on the detected decision of the user P in front of the fitness device 1.

[00108] Additionally, visual elements with a holographic effect may be presented on the fitness device 1 as interface elements 7.31, 7.32, 7.33 for providing commands to the computing device 6. For example, different interface elements 7.31, 7.32 and 7.33 may allow a user, by touching a corresponding touch- sensitive portion of the mirror 12 at which the corresponding interface elements 7.31, 7.32 and 7.33 is displayed, to trigger a certain operation event and thus command. Such a command might, for example, cause selecting a certain program or exercise content to be displayed. In the exemplary embodiment of FIG 13, each of the interface elements 7.31, 7.32 and 7.33 are displayed with the holographic effect to be perceived as a kind of floating hologram on the reflective display surface of the fitness device 1. In addition, visual elements 7.41, 7.42 comprising letters for, for example, describing subprograms, titles or menu items may be presented with duplicate letters on the mirror 12 so as to also achieve a glow and hollow effect and the letters also appearing as a floating hologram.

[00109] The visual elements 7.41 and 7.42 and the interface elements 7.31, 7.32 and 7.33 of FIG 13 are hence presented with a duplicated holographic image 7.41h, 7.42h or 7.3 Ih, 7.32h, 7.33h, respectively. This duplicate is visible for the user P together with the respective visual element displayed on the display 11 behind the mirror 12. A touch input on the displayed interface elements 7.31, 7.32 and 7.33 may for example allow a navigation through different menus. A user P may also be able to navigate across different holographic views. The holographic view may, for example, differ in a ratio between the black tone of the background and a hue of at least a portion of a visual/interface element 7.1, 7.2, 7.31, 7.32, 7.33, 7.41, 7.42.

[00110] The computing device 6 of the fitness device 1 may receive information from different network input sources as part of the input i, for example, from at least one other smart device located in a house or gym. Such information may also be used to augment holographic content, in particular, if such information allows for determining a position of the user with respect to the fitness device 1 and/or a potential approach of the user to the fitness device 1.

[00111] The fitness device 1 may for example also receive information from a connected car or any comparable vehicle connected to the Internet. The fitness device 1 may also equipped with at least one speaker and/or at least one microphone in order to allow for using the fitness device 1 for videoconferencing. Also, in this context, the described holographic effects for displayed visual elements may be beneficial. For such a functionality it may also be possible to vary at least one of an appearance, a size, and a position of at least one visual element depending on a position of the user P in front of the fitness device 1. This may, for example, include augmenting a representation of a remote communication partner in a videoconference depending on the position of the user in front of the fitness device 1.

[00112] From the foregoing description it should be clear that the fitness device 1 may in particular include at least one of a microphone for capturing sound inputs, a video camera, a speaker sound system to accompany the visuals presented and a connectivity module 61 (see FIG 2A) including at least one communication interface, e.g., for a Wi-Fi and/or Bluetooth connection. The connectivity module may, e.g., allow exchange of data and multimedia content via the Internet and/or other local devices. Based on the connectivity module the fitness device 1 may in particular be capable and configured to transmit to and/or receive signals from at least one other device, for example from at least one other local or remote fitness device also equipped with a connectivity module, a mobile phone and/or a computing device.

[00113] FIG 14A shows the front side 1A of the fitness device 1 with the display 11 showing schematically a different arrangement of interface elements 7.31, 7.32 and 7.33.

[00114] The schematic view of an interior of the fitness device 1 viewed from the rear shown in FIG 14B illustrates in particular the arrangement of the different optical, thermal and laser sensor devices/units in the fitness device 1. In addition to the first camera 5.1 and the pyrometric device 5.4 additional sensors 5.5, 5.6 and 5.7, for example optical sensors for determining a position including a posture of the user P in front of the fitness device 1, are illustrated. On one or more printed circuit boards 14A, 14B different driver modules 140, 141 and 142 are arranged. As shown in FIG 14B additional loudspeakers LI and L2 may be also included below the display 11. [00115] FIG 15A in particular illustrates how a position of the user P (having a height H smaller than the height of the fitness device 1) in front of the fitness device 1 is determined and corresponding position data generated and how temperatures for body parts of the user P are determined and corresponding temperature data is generated by the pyrometric device 5.4. In the side the view of FIG 15A the user P is located in front of the fitness device 1 at a certain distance s. This distance s of the user P with respect to the front side 1A is determined by using the point of intersection KP of the laser positioning system 15. Via the pyrometric device 5.4 measuring area M at the front of the user P is sensed in a contactless manner in order to generate temperature data indicative of (surface) temperature values of the different body parts of the user. Based on the generated data several visual elements visualizing certain parts of the data are displayed together with the reflected image PR of the user P on the user interface 7 at the front side 1A of the fitness device 1, as shown in FIG 15B.

[00116] On the user interface 7 the reflected image PR is, for example, visible together with additional visual elements in the form of a histogram or bar chart 7.5 and a representation of training schedule 7.6. The user interface 7 of FIG 15B may be representative for a calibration phase of the fitness device 1 before starting a workout and a preceding warm-up.

[00117] After the calibration phase the fitness device 1 may display fitness content instructing the user P to perform a sequence of exercises as part of the training schedule. While performing the exercises the performance of the user P may be tracked and mapped against reference data thereby automatically providing feedback to the user P via the display 11 on how well the instructed exercise is carried out.

[00118] As shown for the user interface 7 of FIG 16, such feedback may include displaying a heat map 7.8* superimposed on the reflected image PR of the user P. The heat map 7.8* is displayed as an overlay on the reflected image PR, in one embodiment with a holographic effect as explained above. The heat map / overlay 7.8* comprises several portions 7.8a to 7.8h associated with different, identified body parts of the user, for example an arm, the upper body, the head, the lower body, and a leg. The corresponding portions or zones 7.8a to 7.8h of the heat map 7.8* overlaying an associated body part visually indicate by at least one of a color, a shading, hue and a brightness certain temperature or temperature distribution in the respectively associated body part.

[00119] In addition, a visual element 7.7 may visualize a heartbeat rate of the user P. Furthermore, based on the temperature data provided by the parametric device 5.4 an additional heat map 7.8 is displayed for a representation of the user P. By different temperature indicators 7.81, 7.82, 7.83 and 7.84 of the heat map 7.8 determined temperatures of different body regions of the user P are visualized.

[00120] The fitness device 1 of FIG 16 also includes an identification device ID at the front side 1A. Such an identification device ID may for example include a fingerprint sensor or RFID sensor for authenticating the user P.

[00121] In an exemplary embodiment, the fitness device 1 detects a human body and thus user P in front of the fitness device 1, for example using the proximity sensor 16 and/or the second and third cameras 5.2 and 5.3. Based on a position of the user P detected with the aid of the laser position system 15 and based on temperature data generated by the pyrometric device 5.4 a heat map 7,8*, 7.8 may be overlaid or superimposed on the reflected image PR at the front side 1A of the fitness device 1. Machine learning algorithms may be used to map a temperature distribution due to certain body parts / regions based on the temperature data generated by the parametric device 5.4. With the holographic effect described above the different portions or zones 7.8a to 7.8h; 7.81 to 7.84 of the heat map 7.8*, 7.8 may be monitored and presented on the reflected image PR of the user P. The holographic effect may augment and emphasize temperature differences and a temperature distribution on the body of the user P while performing an exercise, while playing a videogame or while performing any kind of physical activity.

[00122] The temperature data may, for example also together with other biometrical data of the user P, streamed and distributed across one or more additional fitness devices in front of which people simultaneously perform an identical physical activity. The fitness device 1 or a remote computing device connected to the fitness device 1 may then provide a ranking and thus a kind of competition functionality based on the temperature data (and, for example, additionally based on the other biometric information). The user P of the fitness device 1 may see its results in relation to the results of the other users augmented on his/her associated fitness device 1.

[00123] Based on the temperature data provided by the pyrometric device 5.4 the computing device 6 may also be able to provide a dynamic feedback to the user P taking into account how user P is heating up while performing exercise and/or how certain body parts are affected by an exercise performed. This may for example include to restrict access to further exercise content and thus certain exercises of a training schedule if generated temperature data does not indicate that the user P has sufficiently warmed-up for minimizing the risk of an injury when performing the subsequence fitness exercises. Based on reaching a certain warmup temperature an operation event may be triggered in the software via which the computing device 6 processes the temperature data. In response to this triggered operation event the computing device 6 may then give access to, activate and/or unlock additional content for display to the user P so that the user P is allowed to start a workout session according to a personal training schedule for the user P.

[00124] Based on the temperature data provided by the pyrometric device 5.4 the computing device 6 may also capable of electronically triggering one or more other operating events, for example not only resulting in allowing access to a workout session after having correctly performed a warm-up but for example also resulting in at least one of a) altering a visual appearance of portions/zones of a heat map 7.8*, 7.8, b) altering a visual indicator of calories burned in visual element 7.5, c) re-arranging or adapting current or future elements of a training schedule displayed in visual elements 7.6, d) visually and/or audibly adapting a feedback to the user P on how good the instructed exercise is performed and e) providing an visual and/or audible alarm to the user in the case. The temperature data may in particular be evaluated in real-time in order to trigger one or more operating events at the fitness device 1 while the user P performs an exercise in front of the fitness device 1.

[00125] FIG 17 shows the user interface 7 with the reflected image PR of the user P below a representation of virtual trainer visualized by visual elements 7.1 and accompanied by another arrangement of previously mentioned visual elements 7.5, 7.6 and 7.8 combined with a training histogram in a visual elements 7.9. Additionally centrally display interface elements 7.3a and 7.3b allow for controlling and selecting displayed functions. FIG 17 thus illustrates a different arrangement of (passive) visual elements and interface elements superimposed on a reflected image PR of the user P.

[00126] The fitness device 1 may transmit temperature data to a remote supervisor device via which another user, such as a human trainer, mentor, or therapist, monitors the user P while performing an exercise. Based on the temperature data provided to the remote supervisor device the other user of this remote supervisor device may trigger generation and sending of an instruction signal, for example including a message or command, to the fitness device 1. In response to the receipt of such an instruction signal to the computing device 6 of fitness device, the user P may be visually and/or audibly informed of an instruction by the other user. Such an instruction may for example result in a personalized feedback on the current performance of the user, a change in the displayed exercise content and/or in displaying a waming/advisory notice. [00127] At the end of a fitness exercise or a completed training schedule a summary of the measured data and its analysis may be provided to the user P via the display of the fitness device 1 or may be sent to another device, like a mobile device of the user P or the remote supervisor device.

[00128] The machine learning module 61 of the computing device 6 and/or machine learning algorithms of the remote analysis computing device may furthermore generate feedback and/or adapt feedback to the user P on the user’s performance while exercise content is displayed at the display 11 of the fitness device 1. The corresponding machine learning algorithms may then also take into account the temperature data generated by the pyrometric device 5.4.

[00129] As illustrated by the flowchart of Figure 18 the fitness device 1 and a system comprising such a fitness device 1 may - in particular, irrespective of providing dynamic feedback to the user P based on data provided by a pyrometic device 5.4 - improve a user experience based on userspecific visualization of visual elements, such as the visual elements 7.1, 7.2, 7.3, 7.31, 7.32, 7.33, 7.41, 7.42, 7.6, 7.7, 7.8, 7.9 and 7.8* as previously discussed. In this context, the computing device 6 of the fitness device 1 may in particular take into account the distance s of the user P to the fitness device 1, an angle a of the front of the fitness device 1 to the ground where the fitness device 1 is located and an optical capturing angle P of a camera 5.1 capturing at least one image of the user P, as, for example, indicated in Figure 15 A. The computing device 6 may in particular be configured to, based on image data provided from at least one of the cameras 5.1-5.3, the position and/or the size of the reflected image PR of the user P on the front face of the mirror 12.

[00130] At least one of the appearance, the size, and the position of at least one visual element with respect to determined position and/or determined size of the reflected image PR may then be (automatically) adjusted. This in particular includes adjusting a superimposition of at least visual element on at least a portion of the reflected image PR of the user P. Based on corresponding configuration or setup data, the computing device 1 may determine a position in x, y coordinates of the reflected image PR of the user P and also determine what size the reflected image PR has. For storing corresponding configuration or setup data in a memory of the computing device 6 a calibration may have taken place. When the user P later uses the fitness device 1 the content to be displayed to the user P may thus be specifically adapted to the user’s position in front of the fitness device 1. This may particularly include initialization and continuing synchronization of a representation of visual elements with respect to the reflected image PR of the user P at the front of the fitness device 1 in real- time. In this context at least one of the cameras 5.1-5.3 may be a monocular camera, a stereoscopic camera, or a time-of-flight camera. Optionally, at least one infrared sensor (or any other type of sensor capable of measuring a distance and a volume of a user P located in front of the fitness device 1) may be used.

[00131] As illustrated in Figure 18, at least one of the cameras 5.1-5.3 may capture video images of the user P standing in front of the fitness device 1. The computer vision module 62 of the computing device 6 may then extract body characteristics of the user P and skeleton poses in realtime to generate skeleton data. Afterwards, this generated skeleton data may be further processed, taking into account a device configuration to generate user-specific position data. The device configuration in this context may include the aforementioned parameters such as the angles a and P, but also the distance d between the mirror 12 and the display 11 arranged behind the display 12 for creating a holographic effect. Based on the measured or calculated height H of the user P the generated position data allows for determining the position and the size of the user reflection and thus of the reflected image PR on the front face of the mirror 12. The computing device 6 may thereby continuously adjust and synchronize a position and a size of displayed visual elements so that their superimposition on the reflected image PR and/or their relative positions with respect to the reflected image PR are maintained.

[00132] The face detection module 63 of the computing device 6 furthermore allows for extracting position information from a captured image relating to a position of a head and of a face of the user P including the position of the eyes of the user P. Based on this position information a viewing perspective of the user P towards/on the front surface of the mirror 12 may be calculated which thus also takes into account for defining at least one of an appearance, a size and a position of at least one of the visual elements 7.1, 7.2, 7.3, 7.31, 7.32, 7.33, 7.41, 7.42, 7.6, 7.7, 7.8, 7.9 and 7.8* to be displayed. In other words, by knowing the position of the user P in front of the fitness device 1 and the position of the head F and the eyes of the user P, the dimensions and the volume of the user P, the computing device 6 may determine a position and a size of the reflected image PR and may combine this information with the calculated viewing perspective of the user P in order to present visual elements so that they visually overlap with at least certain portions of the reflected image PR or are presented in a specific relative position to the reflected image PR SO that the user P may perceive the visual elements in an improved way. The proposed system thus allows for an improved user experience and improved content presentation via the display 11 and the mirror 12 in real-time.

[00133] By using a stereoscopic camera, that is capable of calculating a distance between a location of the stereoscopic camera and an object in the captured image, neither the height H of the user P nor the distance s of the user P to the fitness device 1 needs to be known in advance or calculated for synchronizing display of visual elements with the position of the user P. The same advantages may be achieved by using at least one infrared sensor or any sensor that is capable of measuring the distance s and a volume of the user P located in front of the fitness device 1.

[00134] The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be implemented in another embodiment, even if not specifically shown or described. The same elements may also be varied in one or more ways. Such variations are not to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.