Virtual and Augmented Reality Boxing Activity or Game, Systems, Methods, and Devices

Related Applications

[0001] This application claims the benefit of U.S. provisional patent application number 63/271,613, filed October 25, 2021, the entire contents of which are hereby fully incorporated herein by reference for all purposes.

Field of the Invention

[0002] This invention relates generally to Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR), hereinafter collectively referred to as “XR,” and, more particularly, to methods, systems, and devices supporting gaming, entertainment, exercise, and training in an XR environment.

Background

[0003] Virtual and augmented reality devices allow users to view and interact with virtual environments. Users may effectively immerse themselves in a digitally-enhanced environment and interact with that environment. For example, a user may interact (e.g., play a game) in a virtual environment, where the user's real-world movements are translated to movements and actions in the virtual world. Thus, e.g., a user may simulate a game of boxing, tennis or fencing, or the like in a virtual environment by their real-world movements.

[0004] A user may see a view of their virtual environment with a wearable XR device, such as a virtual reality (VR) headset or augmented reality (AR) glasses, or the like (generally referred to as a head-mounted display (HMD)).

[0005] People should get regular exercise, and many attend group or individual exercise programs at gyms, schools, or the like. At these programs, users are instructed and guided through exercise routines, usually by a person (e.g., a coach) who can monitor, instruct, and encourage participants. A good coach or instructor may customize a routine for a user and may modify the routine based on that user's performance. However, in some situations (e.g., during quarantine for a pandemic), users may not be able or willing to attend such programs outside the safety of their homes.

[0006] It is desirable, and an object of this invention, to provide instruction and guided personalized exercise routines and/or training.

[0007] A further object of this invention is to provide various features to the experience that provide new challenges and interest to the player during activity. [0008] Yet another object of this invention is to provide a boxing-based exercise routine within an XR environment using XR equipment.

[0009] The present invention is specified in the claims as well as in the below description. Preferred embodiments are particularly specified in the dependent claims and the description of various embodiments.

Summary

[0010] The present invention is specified in the claims as well as in the below description. Preferred embodiments are particularly specified in the dependent claims and the description of various embodiments.

[0011] A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that, in operation, causes the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

[0012] One general aspect includes a computer-implemented method for simulating a dodging move in boxing by a person within a virtual environment. The person uses a device in a real-world environment. The device comprises a virtual reality (VR) headset being worn by the person. The VR headset is capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world. The method may include: (a) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles, including at least one dodge bar, wherein the dodge bar is elongated, and defines a right side and a left side and may be presented to the person within the virtual world at a predetermined angle with respect to horizontal. The method may also include (b) presenting, based on the determining, in (a), a first at least one dodge bar to the person in the virtual world and at a predetermined angle with respect to the horizontal. The method may also include: (c) allowing the person to virtually interact with the presented at least one dodge bar in the virtual world by positioning their body in the real world so that the location of the virtual representation of the VR headset is positioned below the virtual location of the at least one dodge bar. The method may also include: (d) comparing the location of the virtual representation of the VR headset with respect to the virtual location of at least one dodge bar. The method may include: (e) recording the results of the comparing step. [0013] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0014] Implementations may include one or more of the following features, alone and/or in combination(s):

• The method where the at least one dodge bar is presented to the person horizontally.

• The method where the predetermined angle of the at least one dodge bar is such that the right side is lower than the left side, with respect to the horizontal.

• The method where the predetermined angle of the at least one dodge bar is such that the left side is lower than the right side, with respect to the horizontal.

• The method where the at least one dodge bar includes a series of repeating dodge bars, each with a predetermined angle, with respect to the horizontal, and each one presented to the person in the virtual world.

• The method where the VR headset further includes an angle of tilt value.

• The method where the comparing step further comprises comparing the angle of tilt value with the predetermined angle of the at least one dodge bar.

[0015] Another general aspect includes a computer-implemented method for measuring breathing characteristics of a person punching in a boxing game within a virtual environment. The person uses a device in a real-world environment. The device comprises a virtual reality (VR) headset worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world. The VR headset includes a microphone. The method may include (a) audio-recording, at a first time prior to boxing, the sounds of the person exhaling to establish a first audio breathing signature. The method may also include (b) audio-recording, at a second time, while boxing, the sounds of the person exhaling to establish a second audio breathing signature. The method may also include: (c) comparing the first and second audio breathing signatures to determine a match. The method also may include: (d) recording the results of the comparing step.

[0016] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0017] Implementations may include one or more of the following features, alone and/or in combination(s):

• The method where the last recording step includes providing information regarding breathing advice to the person.

• The method where the last recording step includes providing information regarding breathing advice to the person, in real time.

• The method where the second audio-recording step takes place as the person executes a punch in the boxing game.

• The method where the first and second audio-recording steps include the use of the VR headset microphone.

• The method where the first and second audio-recording steps include the use of a microphone, independent of the VR headset.

• The method where the last recording step includes providing information regarding breathing advice to the person, when not boxing.

• The method where the last recording step includes providing information regarding breathing advice to the person, in response to the comparing step failing to match.

[0018] Another general aspect includes a computer-implemented method for simulating a blocking move by a person playing a boxing game within a virtual environment. The person uses a device in a real-world environment, wherein the device comprises a virtual reality (VR) headset and handheld controllers being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world. Each handheld controller allows the person to control the movement of virtual boxing implements in the virtual world. Each of the virtual boxing implements includes a different color, the person having a front, and at least one side. The method may include (a) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles, including at least one block object, wherein the block object includes a first color. The method may also include: (b) presenting, to the person in the virtual world, based on the determining, in (a), a block object traveling along a prescribed trajectory to the front of the person. The method may also include (c) allowing the person to virtually interact with the presented block object in the virtual world. The method may also include: (d) hitting, by the person, the block object having the first color with the at least one virtual boxing implement having a second color. The method may also include (e) comparing, in the blocking step (d), the first and second colors. The method may also include: (f) recording the results of the comparing step.

[0019] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. [0020] Implementations may include one or more of the following features, alone and/or in combination(s):

• The method where the block object is successfully hit when the comparing step shows a color match.

• The method where the block object includes two colors, one for each virtual boxing implement, so the person must use both virtual boxing implements in the hitting step.

• The method where the block object travels along a prescribed trajectory to the side of the person.

• The method where the block object travels along a random trajectory towards the person in the virtual world.

• The method where the colors of the block object are visible to the person only after the block object passes a predetermined point along the trajectory towards the person.

• The method where the predetermined point along the trajectory is the halfway point.

[0021] Another general aspect includes a computer-implemented method for counting and displaying a number of consecutive successful responses to object interactions by a person within a virtual environment. The person uses a device in a real-world environment, wherein the device comprises a virtual reality (VR) headset and handheld controllers being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world. Each handheld controller allows the person to control the movement of virtual implements in the virtual world. The method may include (a) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles. The method may also include: (b) presenting, to the person in the virtual world, based on the determining, in (a), at least one projectile. The method may also include: (c) allowing the person to virtually interact with the presented projectile in the virtual world. The method may also include: (d) determining if the interaction of the allowing step was successful or unsuccessful. The method may also include: (e) counting, in response to the determining step, consecutive successful interactions, the counting having a first value. The method may also include: (f) displaying a second value, in response to the first value reaching the second value. The method may also include: (g) replacing the displayed second value of the displaying step with the first value in response to the determining step determining an unsuccessful interaction for a predetermined period of time. The method may also include: (h) setting the counter of the counting step to zero after the predetermined period of time of the replacing step ends. The method may also include: (i) repeating steps (b) through (i).

[0022] Another general aspect includes a computer-implemented method for simulating a boxing punch by a person within a virtual environment. The person uses a device in a real- world environment, wherein the device comprises a virtual reality (VR) headset and handheld controllers being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world. Each handheld controller allowing the person to control the movement of virtual boxing implements in the virtual world. Each of the virtual boxing implements includes a different color, the person having a front, and at least one side. The method may include: (a) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles, including at least one virtual punch target, the at least one punch target includes a body shaped to define a pointed section and an opposing flat section, the pointed section displaying the direction of the punch required by the player for a successful punching interaction. The method may also include: (b) presenting, to the person in the virtual world, based on the determining, in (a), at least one punch target, defining a first punch direction, and traveling along a prescribed trajectory to the person in the virtual world. The method may include: (c) allowing the person to virtually interact with the presented at least one punch target, based on the displayed first punch direction. The method may also include: (d) hitting, by the person, the at least one punch target in a second punch direction. The method may also include: (e) comparing, in the blocking step (d), the first and second punch directions. The method may include: (f) recording the results of the comparing step.

[0023] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0024] Implementations may include one or more of the following features, alone and/or in combination(s):

• The method where the at least one punch target has a first color.

• The method where the hitting step further requires that the player punch the at least the punch target with a virtual boxing implement having a color that matches the first color of the at least one punch target.

• The method where the first punch direction of the at least one punch target is up.

• The method where the first punch direction of the at least one punch target is to the left side.

• The method where the first punch direction of the at least one punch target is to the right side.

• The method where the hitting step requires that the person hit the opposing flat section of the at least one punch target, in the direction of the first punch direction.

[0025] Below is a list of process (method) embodiments. Those will be indicated with the letter “P.” Whenever such embodiments are referred to, this will be done by referring to “P" embodiments.

Pl. A computer-implemented method for simulating a dodging move in boxing to a person within a virtual environment, the person using a device in a real-world environment, wherein the device comprises a virtual reality (VR) headset being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world, the method comprising:

(A) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles, including at least one dodge bar, wherein the dodge bar is elongated, and defines a right side and a left side and may be presented to the person within the virtual world at a predetermined angle with respect to horizontal;

(B) presenting, based on the determining, in (A), a first at least one dodge bar to the person in the virtual world and at a predetermined angle with respect to the horizontal;

(C) allowing the person to virtually interact with the presented at least one dodge bar in the virtual world by positioning their body in the real world so that the location of the virtual representation of the VR headset is positioned below the virtual location of the at least one dodge bar;

(D) comparing the location of the virtual representation of the VR headset with respect to the virtual location of the at least one dodge bar; and

(E) recording the results of the comparing step.

P2. The computer-implemented method of any of the method embodiments, wherein said at least one dodge bar is presented to the person horizontally.

P3. The computer-implemented method of any of the method embodiments, wherein said predetermined angle of the at least one dodge bar is such that the right side is lower than the left side, with respect to the horizontal. P4. The computer-implemented method of any of the method embodiments, wherein said predetermined angle of the at least one dodge bar is such that the left side is lower than the right side, with respect to the horizontal.

P5. The computer-implemented method of any of the method embodiments, wherein the at least one dodge bar includes a series of repeating dodge bars, each one with a predetermined angle, with respect to the horizontal, and each one presented to the person in the virtual world.

P6. The computer-implemented method of any of the method embodiments, wherein said VR headset further includes an angle of tilt value.

P7. The computer-implemented method of method embodiment s) P6, wherein said comparing step further comprises comparing said angle of tilt value with said predetermined angle of the at least one dodge bar.

P8. A computer-implemented method for measuring breathing characteristics of a person punching in a boxing game within a virtual environment, the person using a device in a real-world environment, wherein the device comprises a virtual reality (VR) headset being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world, the VR headset includes a microphone, the method comprising:

(A) audio- recording, at a first time prior to boxing, the sounds of the person exhaling to establish a first audio breathing signature;

(B) audio-recording, at a second time, while boxing, the sounds of the person exhaling to establish a second audio breathing signature;

(C) comparing the first and second audio breathing signatures to determine a match; and

(D) recording the results of the comparing step.

P9. A computer-implemented method of any of method embodiment s) P8, wherein the last recording step includes providing information regarding breathing advice to the person.

PIO. A computer-implemented method of any of method embodiment(s) P8-P9, wherein the last recording step includes providing information regarding breathing advice to the person, in real time.

Pll. A computer-implemented method of any of method embodiment(s) P8-P10, wherein the second audio-recording step takes place as the person executes a punch in the boxing game. P12. A computer-implemented method of any of method embodiment(s) P8-P11, wherein the first and second audio-recording steps include the use of the VR headset microphone.

P13. A computer-implemented method of any of method embodiment(s) P8-P12, wherein the first and second audio-recording steps include the use of a microphone, independent of the VR headset.

P14. A computer-implemented method of any of method embodiment(s) P8-P13, wherein the last recording step includes providing information regarding breathing advice to the person, when not boxing.

P15. A computer-implemented method of any of method embodiment(s) P8-P14, wherein the last recording step includes providing information regarding breathing advice to the person, in response to the comparing step failing to match.

P16. A computer-implemented method for simulating a blocking move by a person playing a boxing game within a virtual environment, the person using a device in a real-world environment, wherein the device comprises a virtual reality (VR) headset and handheld controllers being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world, each handheld controller allowing the person to control the movement of virtual boxing implements in the virtual world, each of the virtual boxing implements includes a different color, the person having a front, and at least one side, the method comprising:

(A) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles, including at least one block object, wherein the block object includes a first color;

(B) presenting, to the person in the virtual world, based on the determining, in (A), a block object traveling along a prescribed trajectory to the front of the person;

(C) allowing the person to virtually interact with the presented block object in the virtual world;

(D) hitting, by the person, the block object having the first color with the at least one virtual boxing implement having a second color;

(E) comparing, in the blocking step (D), the first and second colors; and

(F) recording the results of the comparing step.

P17. The computer-implemented method of embodiment(s) P16, wherein the block object is successfully hit when the comparing step shows a color match. P18. The computer-implemented method of any of embodiment(s) P16-P17, wherein the block object includes two colors, one for each virtual boxing implements so that the person must use both virtual boxing implements in the hitting step.

P19. A computer-implemented method of any of embodiment! s) P16-P18, wherein the block object travels along a prescribed trajectory to the side of the person.

P20. A computer-implemented method of any of embodiment! s) P16-P19, wherein the block object travels along a random trajectory towards the person in the virtual world.

P21. A computer-implemented method of any of embodiment! s) P16-P20, wherein the colors of the block object are visible to the person only after the block object passes a predetermined point along the trajectory towards the person.

P22. A computer-implemented method of embodiment! s) P21, wherein the predetermined point along the trajectory is the halfway point.

P23. A computer-implemented method for counting and displaying a number of consecutive successful responses to object interactions by a person within a virtual environment, the person using a device in a real-world environment, wherein the device comprises a virtual reality (VR) headset and handheld controllers being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world, each handheld controller allowing the person to control the movement of virtual implements in the virtual world, the method comprising:

(A) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles;

(B) presenting, to the person in the virtual world, based on the determining, in (A), at least one projectile;

(C) allowing the person to virtually interact with the presented projectile in the virtual world;

(D) determining if the interaction of the allowing step was successful or unsuccessful;

(E) counting, in response to the determining step, consecutive successful interactions, the counting having a first value;

(F) displaying a second value, in response to the first value reaching the second value; (G) replacing the displayed second value of the displaying step with the first value in response to the determining step determining an unsuccessful interaction for a predetermined period of time;

(H) resetting the counter of the counting step to zero after the predetermined period of time of the replacing step ends; and

(I) repeating steps (B) through (I).

P24. A computer-implemented method for simulating a boxing punch by a person within a virtual environment, the person using a device in a real-world environment, wherein the device comprises a virtual reality (VR) headset and handheld controllers being worn by the person, the VR headset being capable of providing images of scenes and objects to the person through the VR headset to generate a visual representation of a virtual world, each handheld controller allowing the person to control the movement of virtual boxing implements in the virtual world, each of the virtual boxing implements includes a different color, the person having a front, and at least one side, the method comprising:

(A) determining a visual routine for the person to view in the virtual world using the VR headset, the visual routine comprising a series of images representing projectiles, including at least one virtual punch target, the at least one punch target includes a body shaped to define a pointed section and an opposing flat section, the pointed section displaying the direction of the punch required by the player for a successful punching interaction;

(B) presenting, to the person in the virtual world, based on the determining, in (A), at least one punch target, defining a first punch direction, and traveling along a prescribed trajectory to the person in the virtual world;

(C) allowing the person to virtually interact with the presented at least one punch target, based on the displayed first punch direction;

(D) hitting, by the person, the at least one punch target in a second punch direction;

(E) comparing, in the blocking step (D), the first and second punch directions; and

(F) recording the results of the comparing step.

P25. A computer-implemented method of embodiment s) P24, wherein the at least one punch target has a first color.

P26. A computer-implemented method of any of embodiment s) P24-P25, wherein the hitting step further requires that the player punch the at least one punch target with a virtual boxing implement having a color that matches the first color of the at least one punch target. P27. A computer-implemented method of any of embodiment! s) P24-P26, wherein the first punch direction of the at least one punch target is up.

P28. A computer-implemented method of any of embodiment! s) P24-P27, wherein the first punch direction of the at least one punch target is to the left side.

P29. A computer-implemented method of any of embodiment! s) P24-P28, wherein the first punch direction of the at least one punch target is to the right side.

P30. A computer-implemented method of any of embodiment! s) P24-P29, wherein the hitting step requires that the person hit the opposing flat section of the at least one punch target, in the direction of the first punch direction.

[0026] Below are device embodiments, indicated with the letter “D.”

D31. A device, comprising:

(a) hardware including memory and at least one processor, and

(b) a service running on the hardware, wherein the service is configured to perform the method of any of the method embodiments P1-P30.

[0027] Below is an article of manufacture embodiment, indicated with the letter “M.”

M32. An article of manufacture comprising non-transitory computer-readable media having computer-readable instructions stored thereon, the computer readable instructions including instructions for implementing a computer-implemented method, the method operable on a device comprising hardware including memory and at least one processor and running a service on the hardware, the method comprising the method of any of the preceding method embodiments P1-P30.

[0028] Below is computer-readable recording medium embodiment, indicated with the letter “R ”

R33. A non-transitory computer-readable recording medium storing one or more programs, which, when executed, cause one or more processors to, at least: perform the method of any of the preceding method embodiments P1-P30.

[0029] The above features, along with additional details of the invention, are described further in the examples herein, which are intended to further illustrate the invention but are not intended to limit its scope in any way.

Brief Description of the Drawings:

[0030] Objects, features, and characteristics of the present invention as well as the methods of operation and functions of the related elements of structure, and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification.

[0031] FIG. 1 depicts aspects of a virtual reality personalized and customized exercise and training system, according to exemplary embodiments hereof;

[0032] FIG. 2 depicts aspects of an exercise and training system, according to exemplary embodiments hereof;

[0033] FIG. 3 depicts aspects of a mapping and transforming telemetry data, according to exemplary embodiments hereof;

[0034] FIG. 4 depicts aspects of exemplary data structures for a boxing system, according to exemplary embodiments hereof;

[0035] FIG. 5a depicts an exemplary straight punch target for a boxing system, according to exemplary embodiments hereof;

[0036] FIG. 5b depicts a screenshot or image of aspects of an implementation, according to exemplary embodiments hereof;

[0037] FIG. 6a depicts an exemplary left hook punch target for a boxing system, according to exemplary embodiments hereof;

[0038] FIG. 6b depicts a screenshot or image of aspects of an implementation, according to exemplary embodiments hereof;

[0039] FIG. 7 depicts an exemplary right hook punch target for a boxing system, according to exemplary embodiments hereof;

[0040] FIG. 8a depicts an exemplary uppercut punch target for a boxing system, according to exemplary embodiments hereof;

[0041] FIG. 8b depicts a screenshot or image of aspects of an implementation, according to exemplary embodiments hereof;

[0042] FIG. 9a depicts an exemplary block object for a boxing system, according to exemplary embodiments hereof;

[0043] FIG. 9b depicts a screenshot or image of aspects of an implementation, according to exemplary embodiments hereof;

[0044] FIG. 10 depicts a screenshot or image of aspects of an implementation, according to exemplary embodiments hereof;

[0045] FIG. 11 depicts a screenshot or image of aspects of an implementation of a single duck down bar, according to exemplary embodiments hereof; [0046] FIG. 12a depicts a screenshot or image of aspects of an implementation of a series of different dodge bars, according to exemplary embodiments hereof;

[0047] FIG. 12b depicts a screenshot or image of aspects of an implementation of a series of different dodge bars, according to exemplary embodiments hereof;

[0048] FIG. 13 depicts an exemplary virtual platform for a boxing system, according to exemplary embodiments hereof;

[0049] FIG. 14 depicts a screenshot or image of aspects of an implementation of a virtual boxing session, according to exemplary embodiments hereof;

[0050] FIG. 15a depicts a screenshot or image of aspects of an implementation of a virtual boxing session, showing a counter, according to exemplary embodiments hereof;

[0051] FIG. 15b is a logical block diagram depicting aspects of an implementation of a counting system, according to exemplary embodiments hereof;

[0052] FIG. 16 is a logical block diagram depicting aspects of a computer system; and

[0053] FIG. 17 depicts an exemplary mat for a boxing system, according to exemplary embodiments hereof.

Detailed Description of the Preferred Exemplary Embodiments: a) Glossary and Abbreviations:

[0054] As used herein, unless used otherwise, the following terms or abbreviations have the following meanings:

[0055] Augmented Reality (AR) refers to or means an interactive experience of a real- world environment where select objects that reside in the real world are enhanced by computer-generated perceptual information, often across multiple sensory modalities, such as visual, auditory, and haptic.

[0056] Virtual Reality (VR) refers to or means an interactive experience wherein a person interacts within a computer-generated, three-dimensional environment, a “virtual world,” using electronic devices, such as hand-held controllers.

[0057] Mixed Reality (MR) refers to or means an interactive system that uses both virtual reality and augmented reality technologies to create an environment where physical and virtual objects can exist and interact in real-time.

[0058] XR is a term used herein which refers to the three current forms of altered reality, Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR). [0059] The Guard position is a boxing term in which the boxer is in a protective and ready position, with their hands held high to protect the head and chin and their forearms are tucked against the torso to impede body shots. b) Description:

[0060] In the following, exemplary embodiments of the invention will be described, referring to the figures. These examples are provided to further understand the invention, without limiting its scope. The exemplary embodiments described herein are described as being applied to virtual reality, including the use of a VR headset being worn by a VR player, which displays a VR game having a VR environment within a VR display. It is to be understood that the present technology, as described, may equally be applied to all XR devices, without departing from the gist of the invention and that the application towards VR devices is just exemplary.

[0061] In the following description, a series of features and/or steps are described. The skilled person will appreciate that unless required by the context, the order of features and steps is not critical for the resulting configuration and its effect. Further, it will be apparent to the skilled person that irrespective of the order of features and steps, the presence or absence of time delay between steps, can be present between some or all of the described steps.

[0062] It will be appreciated that variations to the foregoing embodiments of the invention can be made while still falling within the scope of the invention. Alternative features serving the same, equivalent, or similar purpose can replace features disclosed in the specification, unless stated otherwise. Thus, unless stated otherwise, each feature disclosed represents one example of a generic series of equivalent or similar features.

[0063] Although the terms “game” and “gaming” are used throughout this description, the present technology can be applied to a variety of electronic devices and immersive experiences, including, but not limited to, games, educational interactions, such as online teaching of math or languages, fitness-related activities using electronic devices, such as the below-described fitness program, called “Supernatural,” for use with an XR headset, and even business activities. For reasons of simplicity and clarity, all the applications of the present technology are generally referred to as “games” and “gameplay” herein.

[0064] With the development of virtual reality (VR) and with recent improvements of accurate inertial sensors, high-resolution displays, and specifically-developed 3-D software, playing a VR game can become a truly immersive and often emotional experience. The hardware requirements to achieve a truly immersive, interactive and realistic gaming experience lends itself perfectly to providing valuable biometric and accurate body movement information, in real time, without adding additional sensors.

[0065] For example, a popular VR system called the Quest 2, designed and manufactured by Oculus, a brand of Facebook Technologies, Inc., located in Menlo Park, California, includes a VR headset and two handheld wireless controllers. This VR headset includes forward-facing cameras, a gyroscope and accelerometer (also called an IMU or Inertial Measurement Unit), and an array of infrared LEDs and an IMU located within each hand controller. These sensors are very accurate and provide precise orientation and position information of both the headset and each controller, in 3-D space, essentially in real time (updated 1,000 times per second) during gameplay. Game developers of VR games use this information to effectively establish the location and orientation of the player’s head, their hands and fingers in real time during a game. Software developers create virtual handheld objects, such as tennis rackets, batons, and boxing gloves which are designed to appear in the VR world, i.e., as seen by a player through at least the player’s VR headset (and also through remotely connected displays, if any are connected). The virtual handheld objects are directly controlled by the player’s hand movements in the real world. Therefore, the location, orientation, speed and direction of movement of the virtual handheld objects, in real time, are known by the controlling system, as well.

[0066] The use of a VR system allows a player’s senses to become truly isolated from the surrounding real-world environment. By wearing a typical VR headset, the player can only view the images that are presented on the VR display, similar to the focus an audience gains when watching a movie in a dark movie theater. The VR headset effectively provides a 3- Dimensional movie theater experience. The VR system also provides sound input for the player’s ears, thereby further enhancing the sense that the experience is real. By controlling both visual and auditory inputs to a player, and effectively separating the player from real- world sensory inputs (i.e., real world distractions), VR games offer a player a greater chance to focus, improve gaming performance, and, depending on the type of VR software being played, even provide an effective workout.

[0067] To help explain this inventive technology, a representative fitness experience is illustrated in the accompanying figures. It should be noted that the present technology is meant to be applied to a specific type of 3-D virtual reality game that uses projectiles and geometric shapes, projected at a player, to encourage the user to move various muscle groups for the purpose of both entertainment and exercise. One exemplary such game of which the presently described embodiments could be applied is a virtual reality fitness game called “Supernatural.” It was developed by, and is currently available from, a company called Within Unlimited, Inc., located in Los Angeles, California.

[0068] In this particular exemplary VR software game, a player dons a suitable virtual reality headset and hand controllers, such as the above-identified Quest 2, by Facebook’s Oculus brand. Once the Supernatural game begins, a three-dimensional natural environment image, such as a mountain setting, is automatically generated and displayed within the player’s headset, virtually placing the player at a center point within this computer-generated virtual environment, as is well known by those of ordinary skill in the art of VR technology. The player will experience this virtual environment as a realistic three-dimensional image, one that can be viewed in all directions, as if the player were actually standing in the same environment in the real world. As the player moves their head left and right, up and down, the abovedescribed sensors located within the VR headset will detect this head movement in extremely fine resolution. The running software program (e.g., Supernatural) will collect and analyze this sensor data to adjust the displayed environment image in real time (effectively immediately) to match the exact minute increments of head movement, and also the direction and speed of the player’s head to accurately create an illusion of presence, within the environment. The illusion is sufficient to convince the player that they are truly part of the virtual world being displayed, literally right in front of the player’s eyes.

[0069] Continuing with this example, in the fitness game called Supernatural, the player is meant to remain at a substantially fixed location in the real world during gameplay so that their VR presence (i.e., their avatar) remains at a central point within the VR environment.

[0070] A system supporting a real-time virtual reality (VR) environment 100 for a virtual and augmented reality fitness training system is described now with reference to FIG. 1, in which a person, (VR user, player, person boxer, etc.) 102 in a real-world environment or space 112 uses a VR device or headset 104 to view and interact with and within a virtual environment. The VR headset 104 may be connected (wired and/or wirelessly) to a training system 106, e.g., via an access point 108 (e.g., a Wi-Fi access point or the like). Since the user’s activity may include a lot of movement, the VR headset 104 is preferably wirelessly connected to the access point 108. In some cases, the VR headset 104 may connect to the training system 106 via a user device or computer system (not shown). While shown as a separate component, in some embodiments, the access point 108 may be incorporated into the VR headset 104.

[0071] Sensors (not shown in the drawings) in the VR headset 104 and/or other sensors 110 in the user’s environment may track the VR user's actual movements (e.g., head movements, etc.) and other information. The VR headset 104 preferably provides user tracking without external sensors. In a presently preferred implementation, the VR headset 104 is an Oculus Quest headset made by Facebook Technologies, LLC.

[0072] Tracking or telemetry data from the VR headset 104 may be provided in real time (as all or part of data 118) to the training system 106.

[0073] Similarly, data from the sensor(s) 110 may also be provided to the training system 106 (e.g., via the access point 108).

[0074] The user 102 preferably has one or two handheld devices 114-1, 114-2 (collectively handheld device(s) and/or controller(s) 114) (e.g., Oculus Touch Controllers). Hand movement information and/or control information from the handheld controlled s) 114 may be provided with the data 118 to the training system 106 (e.g., via the access point 108).

[0075] In some embodiments, hand movement information and/or control information from the handheld controller(s) 114 may be provided to the VR headset 104 or to another computing device which may then provide that information to the training system 106. In such cases, the handheld controlled s) 114 may communicate wirelessly with the VR headset 104. [0076] In some embodiments, at least some of a user's hand movement information may be determined by tracking one or both of the user's hands (e.g., if the user does not have a handheld controller 114 on/in one or both of their hands, then the controller- free hand(s) may be tracked directly, e.g., using 3D tracking).

[0077] Although described here as using one or two handheld controllers 114, those of skill in the art will understand, upon reading this description, that a user may have no handheld controllers or may have only one. Furthermore, even when a user has a handheld controller in/on their hand, that hand may also (or instead) be tracked directly.

[0078] The VR headset 104 presents the VR user 102 with a view 124 corresponding to that VR user's virtual or augmented environment.

[0079] Preferably, the view 124 of the VR user's virtual environment is shown as if seen from the location, perspective, and orientation of the VR user 102. The VR user's view 124 may be provided as a VR view or as an augmented view (e.g., an AR view).

[0080] In some embodiments, the user 102 may perform an activity such as an exercise routine or a game or the like in the VR user's virtual environment. The training system 106 may provide exercise routine information to the VR headset 104. In presently preferred embodiments, the activity system 126 may provide so-called beat map and /or other information 128 to the headset (e.g., via the network 119 and the access point 108). [0081] As the user progresses through an activity such as an exercise routine, the VR headset 104 may store information about the position and orientation of VR headset 104 and of the controllers 114 for the user’s left and right hands.

[0082] In a present implementation, the user’s activity (and a beat-map) is divided into sections (e.g., 20-second sections), and the information is collected and stored at a high frequency (e.g., 72 Hz) within a section. The VR headset 104 may also store information about the location of targets, portals and all objects that are temporally variant, where they are in the 3-D space, whether any have been hit, etc. at the same or similar frequency. This collected information allows the fitness system to evaluate and/or recreate a scene at any moment in time in the space of that section.

[0083] Collected information may then be sent to the training system 106, preferably in real-time, as all or part of data 118, as the user’s activity/workout continues, and several of these sections may be sent to the training system 106 over the course of an activity/workout. The data 118 that are provided to the training system 106 preferably include beat-map information.

[0084] The training system 106 may be part of backend / cloud framework 120.

The Training System:

[0085] As explained in greater detail below, in some implementations / embodiments, the fitness training system provides a user with an individualized customized VR training routine, tracks the user as they carry out the routine (in VR), modifies the routine if needed, and provides guidance to the user. The routine may involve the user interacting (virtually) with various objects, and the system may monitor and evaluate the user's interactions and movements in order to determine possible modifications to the routine. The system may also use physiological data (e.g., heart rate data) to evaluate a user during a routine.

[0086] With reference to FIG. 2, the training system 106 is a computer system (as discussed below), e.g., one or more servers, with processor(s) 202, memory 204, communication mechanisms 206, etc. One or more training programs 210 run on the training system 106. The training system 106 may store data in and retrieve data from one or more data structures 224 in memory 204 and/or from one or more databases (not shown). The databases may include a user database to store and maintain information about users of the system.

[0087] Although only one user 102 is shown in FIG. 1, it should be appreciated that the training system 106 may interact with multiple users at the same time. It should also be appreciated that the following description of the operation of the training system 106 with one user extends to multiple users. [0088] The training programs 210 of the training system 106 may include data collection mechanism(s) 212, movement/tracking mechanism(s) 214, mapping and transformation mechanism(s) 216, calibration mechanism(s) 218, routine generation mechanism(s) 220, and routine evaluation mechanism(s) 222.

[0089] The data structures 224 may include a routine data structure 226 and a user data structure 228.

[0090] In operation, the data collection mechanism(s) 212 obtains data 118 (FIG. 1) from a user (e.g., user 102 in FIG. 1). The data 118 may include at least some of user movement / telemetry data, information about the location of targets, portals and objects that are temporally variant, where they are in space, whether any have been hit, where and how hard they were hit, etc.

[0091] The movement/tracking mechanism(s) 214 determines or approximates, from that data, the user’s actual movements in the user’s real -world space 112. The user’s movements may be given relative to a 3-D coordinate system 116 the user’s real-world space 112. If the data 118 includes data from the user’s handheld controlled s) 114, the movement/tracking mechanism(s) 214 may also determine movement of one or both of the user’s hands in the user’s real -world space 112. In some cases, the user’s headset 104 may provide the user’s actual 3-D coordinates in the real -world space 112.

[0092] The movement/tracking mechanism(s) 214 may determine or extrapolate aspects of the user’s movement based on machine learning (ML) or other models of user movement. For example, a machine learning mechanism may be trained to recognize certain movements and/or types of movements and may then be used to recognize those movements based on the data 118 provided by the user 102.

[0093] With reference to FIGS. 2 and 3, the mapping and transformation mechanism(s) 216 of FIG. 2 may take the movement/tracking data (as determined by the movement/tracking mechanism(s) 214) and transform those data from the real world coordinate system 116 in the user’s real -world space 112 to corresponding 3-D coordinates in a virtual -world coordinate system 314 in a virtual world 312.

[0094] Those of skill in the art will understand, upon reading this description, that the mapping and transformation mechanism(s) 216 may operate prior to or in conjunction with the movement/tracking mechanism(s) 214. As with all mechanisms described herein, the logical boundaries are used to aid the description and are not intended to limit the scope hereof. [0095] For the sake of this description, the user’s movement data in the real-world space 112 are referred to as the user’s real -world movement data, and the user’s movement data in the virtual -world space 312 are referred to as the user’s virtual movement data.

[0096] In some embodiments, the training system 106 may also receive or have other user data (e.g., physiological data or the like) and may use some of the physiological data (e.g., heart rate, temperature, sweat level, breathing rate, etc.) to determine or evaluate the user’s movements and actions in the virtual space. Such physiological data may be obtained by one or more sensors 121 (FIG. 1) worn by and/or monitoring the user. The sensors 121 may be incorporated into another device, such as, e.g. a watch or the like worn by the user. For example, the sensors 121 may include a heart rate monitor included in an Apple Watch worn by the user.

[0097] The training system 106 may be co-located with the user (e.g., in the same room), or it may be fully or wholly located elsewhere. For example, the training system 106 may be located at a location distinct from the user, in which case the user’s data 118 may be sent to the training system 106 via a network 119 (e.g., the Internet). Although in preferred cases the user’s data 118 are provided to the training system 106 as the data are generated (i.e., in real time), in some cases, the user’s data 118 may be collected and stored at the user’s location, and then sent to the training system 106. When located apart from the user and accessed via a network, the training system 106 may be considered a cloud-based system.

[0098] As noted above, the fitness training system may provide a user with an individualized, customized VR training routine. A user's routine may be stored in a routine data structure 226 in the memory 204 of the training system 106.

Boxing:

[0099] Boxing is an ideal exercise for releasing stress, increasing muscle strength, and improving cardiovascular health. Owing to the rigorous and generally fast pace of the movements of boxing, the player will also typically bum quite a few calories. Of course, boxing in the real world typically holds the risk of impact injury from the opponent, but a person may enjoy the many benefits of boxing in the relative safety of a suitable virtual (VR) environment without the risk by performing the sport in the safety of a suitable virtual reality (VR) environment, while donning a VR headset.

[0100] Regardless, if the boxing activity is being performed in the real world, against a living opponent, or alone, in a VR environment, there are a few basic boxing guidelines which should be followed to ensure that the boxer maintains proper form, proper balance and is able to efficiently and effectively translate hand, arm, and body movement into an impactful force. [0101] An important aspect of the exemplary embodiments shown and described herein, is the application of virtual reality technology to the sport of boxing for the purpose of entertainment, training, and improving fitness and overall health. Briefly, a player dons the above-described VR hardware and “enters” into a virtual environment and effectively becomes a boxer. The boxer holds a VR controller in each hand, which appear as boxing gloves, bare fists, or other implements which appear virtually at the end of the forearm or clenched in virtual fists in the VR world. The boxer may rotate their head to view the extent of the VR environment, including the virtual platform (described below) onto which the virtual boxer will reside during a boxing session.

[0102] At the start of a boxing session, the VR computer system will present the boxer with a boxing routine which includes the formation of a portal, and the generation of a predetermined series of events. Each event interacts with the boxer and challenges the boxer to move in a particular way, depending on the type of event, to properly overcome the event. The boxer continues to respond to each event until the routine is over. This application describes particulars regarding the different types of events, the proper responses required of the boxer to overcome each event, and other aspects of the boxing environment and the boxing game experience.

The Stance:

[0103] Arguably, the most important aspect of boxing is the boxer’s stance and the position of their feet during a workout, or during a boxing match. A proper stance during a boxing match will offer the boxer the best chance to effectively and efficiently deliver the most powerful punch to the opponent (or target), while maintaining good balance, and allowing for a quick response. A proper stance will also provide the boxer with the longest reach possible while punching an opponent or target, again while maintaining good balance. The boxer must maintain good balance during a “fight” or exercise session so that they are ready to throw a solid punch, block any incoming punches, including quickly retreating, and quickly ducking and dodging in place. If the boxer is not standing properly, they will not be able to perform the necessary movements quickly and efficiently. The proper stance provides the boxer with reliable stability and quick mobility.

[0104] There are two main stances used in boxing, “orthodox” and “southpaw.” The most commonly used and well known stance is the orthodox stance. This stance is typically most suitable for right-handed boxers, wherein the strongest arm is the boxer’s right arm. The orthodox stance requires that the boxer’s left foot and left hand be positioned in front of their body, while their right foot and right hand are at their rear. The southpaw stance is opposite to the orthodox stance. Here, the boxer’s right hand and foot are positioned in front of their body and their left hand and left foot are at their rear. Essentially all boxers who benefit from the southpaw stance are left handed. Where the boxer places their feet on the ground determines their advantages and disadvantages in offense, defense, and mobility movements. It is critical that the boxer learns proper foot positioning before any other aspects (hand placement) of boxing stance. Of course, some boxers may switch between orthodox and southpaw stances, either as a strategy to confuse their opponent, or they are ambidextrous. Most boxers hold to one stance, based on their stronger and dominant arm.

[0105] According to exemplary embodiments, it is recognized that a boxer’s hand position will always follow their foot position, so that if a boxer’s feet are in an orthodox foot stance, with their left foot forward, then their left hand will follow and will also be forward. The hands are biologically forced into position, following the foot position, since positioning one foot in front of the other causes the boxer’s torso to rotate, which automatically positions the hands accordingly (the lead foot will always force the same side hand to lead as well). For the boxer to do otherwise, would require them to awkwardly rotate their torso against the set position of their hips. This biological conflict would be awkward and imbalanced, thereby supporting a position that this action would be unlikely. The present computer system may use the relative location of the hand controllers with respect to the boxer’s headset orientation to determine which stance the boxer is in, as they box.

[0106] The present computer system can also measure the speed and acceleration of each hand controller to determine which of the boxer’s hands is dominant and stronger.

[0107] Also, according to exemplary embodiments, and as shown in FIG. 17, a mat 1700 may be provided, for use in the real world, to help the boxer manage their foot positions. The mat is preferably made from a suitable material, such as plastic, rubber, or a textile structure, such as a rug. Regardless of the format, the mat 1700 would include an upper surface 1702 on which the boxer may stand, as they box in the VR world. According to these present embodiments, the mat 1700 includes a discernible left section 1704 and right section 1706 , equally positioned so that the demarcation line therebetween defines a centerline 1708 of the mat. The two sections are preferably identifiable by the boxer by both color and texture, such as integrally-molded ribs 1710 so that the boxer may either feel the delineation between the two halves using their feet, or may look down and understand the location of the centerline by the contrasting colors. The correct foot position of the orthodox stance and the southpaw stance may be printed 1712 onto the upper surface 1702 (note that only the orthodox stance is shown in FIG. 17). [0108] According to embodiments, the present computer system allows the boxer to automatically activate the “passthrough” feature of the Oculus VR headset, whereby the boxer may view the real world through the headset display without removing the headset. According to the embodiments herein, the boxer simply has to look straight down while donning the VR headset to activate the passthrough feature. The computer system may detect when the VR headset is orientated facing straight down (or close to it) and response by turning on the passthrough feature. When this happens, the boxer may see their own feet, and foot position with respect to the mat 1700, including the contrasting colors without having to remove the headset. This would allow the boxer to quickly and easily inspect and adjust their foot position. [0109] According to exemplary embodiments, the internal structure of mat 1700 may further include load-cells (not shown) positioned strategically below the correct positions of the boxer’s feet, such as a load cell below the front and rear feet of the orthodox printed feet 1712, shown in FIG. 17. Load cells convert pressure into a resistance that may be measured and calibrated using a well-known electronic circuit so that the effective pressure (and therefore weight) below each foot of the boxer may be measured in real time. The values of the pressure measurements from the different load cells (not shown) would be transmitted, such as using Bluetooth, or WIFI systems to the present computer system. The computer system can then keep track of how the boxer is positioning their weight over their front and rear feet. It is preferred for a boxer to retain 55-60% of their body weight over their front foot. The present computer system may display the percentage values of weight for the boxer’s front and rear feet in the boxer’s headset display, in real time, all the time, or only when the relative weight shifts outside the desired range. For example, the computer system ideally measures the pressure under each foot of the boxer, when the boxer is in the guard position, and not during actual punching since movement of the boxer would make accurate readings difficult. If, during this time, the readings show that the boxer is effectively leaning back too much, in their stance, wherein the load cells in the mat 1700 measure 70% weight on the rear foot, then the readings or an alert may be displayed to the boxer.

The Platform:

[0110] Referring to FIG. 13, and according to exemplary embodiments, platform 1302 is provided within the virtual environment and includes an open ring 1304 and a rectangular mat 1306. The mat 1306 is positioned within the ring and is where the boxer will stand, as they participate in a boxing session. Of course, the platform 1302, ring 1304 and mat 1306 may be any shape. Platform 1302 may consist of only a solid surface onto which the boxer is positioned. According to exemplary embodiments, the mat 1306 includes a graphic centerline 1308, or other appropriate delineation, which extends from the center point 1310 of the front edge 1312 to the rear edge 1314 of the mat 1306. The centerline 1308 aligns with the portal 404, as shown in FIG. 13. The purpose of the centerline 1308 is to effectively divide the mat 1306 into two equal halves, a right half and a left half. The centerline 1308 may also be defined as the boundary between two contrasting colors of each half of the matte. Regardless, the centerline 1308, however it is formed, provides a reference which the boxer may use to properly align their feet, as they box during a boxing session or match. During a boxing session, as described below, the boxer may be instructed to change between orthodox and southpaw stances and the centerline 1308 will offer them a reference line to help them maintain the proper footing required of the stance.

[0111] For the above described orthodox stance, the boxer’s left foot is on the left side of the centerline 1308 and positioned forward with the left-foot toes generally facing towards the opponent (or portal 404). The toes of the left foot are very close to the centerline 1308. The boxer’s rear right foot is located to the rear of the boxer and forms a large angle with respect to the centerline 1308, and is positioned on the right side of the centerline. By positioning the rear foot at an angle, with respect to the centerline 1308, as shown in FIG. 13, the boxer becomes more stable and will be less likely to fall while boxing. This particular foot position also forces the boxer to stand with the left side of their body facing forward and this presents a smaller frontal area to the opponent, or incoming targets.

[0112] In contrast, a southpaw stance positions the boxer’s feet opposite to that of the above-described orthodox stance. The boxer’s left foot is now located to the rear of the boxer and angled outwardly and on the left side with respect to the centerline 1308. The boxer’s right foot is positioned forward, with the right-foot toes immediately adjacent to the centerline 1308, and on the right side of the mat 1306.

[0113] According to exemplary embodiments, as shown in FIG. 13, an instructional foot placement guide 1320 may appear at different times during a boxing session. The guide 1320 may be any shape, but is shown here as a circle. Within the circular guide are graphic representations of a right foot and a left foot. The guide is visible by the boxer within the virtual environment during a boxing session. The guide 1320 is preferably presented for a prescribed period of time when it is necessary to change the boxing stance during a boxing session, as dictated by the computer gaming system following a computer program for that session. The guide 1320 will appear and the boxer will understand how to reposition their feet in the real world to match the foot pattern shown in the guide 1320. The guide may be displayed for any length of time during the boxing session, but preferably only for a short period of time, such as 10 seconds.

[0114] Regardless of which stance is used, slightly more weight of the boxer should reside over the forward foot. This allows the boxer to be more agile and be able to quickly move back and away from a punch, or a target, or an object, as needed. Once the proper stance is realized by the boxer, the match may begin.

Targets and Objects

[0115] With reference to FIG. 4, a routine 400 may comprise a time-ordered series of events 402. An event 402 may comprise a portal location 404 through which any of a predetermined number of targets 406 and objects 408 are individually formed and directed towards the player (e.g., the boxer), following a prescribed sequence.

[0116] According to exemplary embodiments, and as described in greater detail below, a target 406 is used to elicit a particular punching response by the boxer and uses four different shapes to instruct the boxer which one punch of four types of punches would constitute a proper response to the particular target. The targets are used to instruct the boxer to punch on the offensive, against an unseen opponent, and in doing so, carry out an exercise and also train the different muscle groups used in these punches, as would a real boxer. The targets may be shaped to convey one of four different types of punches, commonly used in boxing.

Punches: o Jab + Cross:

[0117] These are your “straight punches.” The Jab is thrown with the boxer’s front hand (Left hand if using the orthodox stance, and right hand if using the southpaw stance). Both the jab and the cross punches use a similar form. Starting in the guard position, the boxer will take a small step in place, as they extend their arm. The boxer rotates their hand as they extend their arm so that their palm is facing down at the point of impact. o Hooks

[0118] Hook punches are also known as “round” punches because they arc out to the side. Most of the power in these punches comes from using the legs and hips to pivot. The boxer rotates their hips towards the punch, as they lift their arm, keeping their elbow at approximately a 90 degree angle. The boxer strikes the target from either the left or right side. At the point of impact, the boxer may have their hand rotated one of two ways: either with the thumb facing the sky, or with the thumb facing the boxer. o Uppercuts [0119] An uppercut punch gets its power from explosive movement of your legs and hip rotation. From the guard position, the boxer rotates their hand so that their palm faces you. The boxer will then bend and contract their hips slightly as they explode through the legs to strike the target from the bottom. o “Flurries”

[0120] Used in workouts, “flurries” are a quick series of events that use only one type of punch (for example, a series of very fast, back-to-back jabs and crosses). These flurries of speedwork are intended to help the boxer practice speed and agility, with the added challenge to get their hands back into the guard position in between every punch.

[0121] Referring to FIG. 5a, a straight punch target (which includes a jab and a cross punch), depending on the color of the target) 410, is shown. When the virtual boxer sees this punch target advancing on them, they understand by the shape and color to perform a straight punch, using either the left hand or the right hand, whichever colored glove matches the color of the target. This is illustrated in FIG. 5b.

[0122] Once the boxer hits the target with the correct gloved hand, the target may graphically explode and disappear, just disappear, or bounce away from view.

[0123] FIG. 6a shows a left hook target 412, which is arrowhead shaped and includes a pointed section 414 and an opposing rear section 416, having a slight indentation. The rear section of the left hook target 412 is meant to resemble a conventional boxing training pad of the type that is held by a trainer and includes a slightly curved front section to receive punches from a boxer in training. The rear surface is shaped to help guide the boxer to punch the target correctly, in the correct direction, correct wrist orientation, and within the proper plane.

[0124] The pointed section 414, in this case, is pointing to the right, which indicates that this particular type of target is a left hook target, meaning that the boxer must use their left gloved hand to perform a left hook punch. The boxer’s punch moves across the front of their body in the right direction to properly hit this target on the flat section 416. This is illustrated in FIG. 6b, wherein the right hook target is moving towards the boxer in the direction of the arrow 418, and the boxer is using his left hand to perform a hook punch at the target, just as the target reaches them. The illustration shows the boxer just before making contact with the target.

[0125] Once this left hook target 412 is hit, again, the target will graphically explode and disappear, just disappear, or bounce away from view.

[0126] Similarly, as shown in FIG. 7, a right hook target 420 is shown, including a pointed section 422 and an opposing flat section 424. [0127] The pointed section 422, in this case, is pointing to the left, which indicates that this particular type of target is a right hook target, meaning that the boxer must use their right gloved hand to perform a right hook punch. The boxer’s punch would be identical to the one shown in FIG. 6b, except that the boxer’s punch for the right hook would require that the boxer’s right hand moves across the front of their body in the left direction to make contact with the flat section 424 of the target 420. Once this right hook target 420 is hit, again, the target will graphically explode and disappear, just disappear, or bounce away from view.

[0128] According to exemplary embodiments, the fourth type of target is uppercut punch target 426, which is shown in FIG. 8a.

[0129] Similar to the above described left and right hook targets 412, 420, the uppercut target 426 is also arrowhead shaped and also includes a pointed section 428, and an opposing flat section 430. The pointed section, in this case, is pointing up, which indicates that this particular type of target is an uppercut target, meaning that the boxer must use either their right or left gloved hand to perform an uppercut punch. Which glove the boxer should use depends on the color of the target. The color of the target must match the color of the glove used to perform the punch for the boxer to earn a correct hit. In this case, the boxer’s punch would require the correctly colored glove to move upwardly so that the flat section 430 is hit.

[0130] This is illustrated in FIG. 8b, wherein the uppercut target is moving towards the boxer in the direction of the arrow 432, and the boxer is using their right hand to perform an uppercut punch to hit the flat underside surface 430 of the target, just as the target reaches them. The illustration shows the boxer just before making contact with the target.

[0131] Once this uppercut target 426 is hit, the target will graphically explode and disappear, just disappear, or bounce away from view.

[0132] According to exemplary embodiments, and as described in greater detail below, an object 408 may comprise either a block object 440, or a dodge bar 442.

[0133] The purpose of the block object is to simulate an incoming punch by the boxer’s opponent, who, in this exemplary embodiment does not exist in avatar form, but instead is effectively represented by the various targets and objects described herein. The block object represents the opponent’s punch while the above-described targets represent the opponent’s face and body, depending on the location of the projected targets. To this end, the block object may follow a variety of trajectories between the portal, where it is formed, and the virtual boxer. The block object may have a straight trajectory that brings the block object directly in front of the boxer’s face, or to either side of the boxer. [0134] In real boxing, the opponent can throw different types of punches, typically, a jab, a straight punch, a cross punch, a left hook, and a right hook. To simulate these different types of opponent punches, in the present VR boxing simulation, the block objects 440 may be one of three different types, a straight block object 444, which mimics a straight punch, a cross punch, or a jab, a left block object 446, which mimics a left hook punch, and a right block object 448, mimics a right hook punch.

Defense: o Blocking

[0135] The best offense is a good defense! Block targets remind the boxer to get their hands up in the guard position, next to the cheekbones so that the face is protected. The boxer’s gloves should be positioned so that the boxer can see the tops of the gloves, with elbows positioned slightly away from the rib cage. Shoulders relaxed. o Slipping or Dodging:

[0136] Angled “dodge bars” are used to train the boxer to “slip” or dodge left or right. The slip is a defensive move where the boxer rotates their torso to position a side of the shoulder forward, to brace for impact. The boxer contracts their abdominal muscles and rotates slightly to the correct side, all while keeping their eyes forward to see what is coming next. o Ducking

[0137] A horizontal “duck-down” bar instructs the boxer to quickly squat or duck straight down and straight back up into their stance. The boxer maintains their feet in the boxing stance during this move. The boxer’s back remains straight, with just the knees and hips bending sufficiently to avoid the bar, quickly returning to the “ready” stance. o Bob and weave

[0138] A series of shifting dodge bars and duck-down bars instructs the boxer to “bob + weave” under the passing bars. This defensive motion has the boxer moving their head both beneath a bar and simultaneously, laterally to one side or the other. Again, the boxer lowers by bending their knees, not the spine, then shifts left or right with their torso and legs.

[0139] According to exemplary embodiments, a straight block object 444 is shown in FIG. 9a, and includes a first section 450 of a first color, and a second section 452, of a second color. The first and second colors match the right and left glove colors, respectively. The first and second sections of the straight block object 444 may be any shape, but are preferably similar in size, so that both colors are equally represented. The exemplary straight block object 444 is shown in FIG. 9a as having a Yin-Yang” design, wherein the “Yin” part of the design is first section 450, and the “Yang” part is second section 452. The design is symmetrical so that both colors are equally represented.

[0140] The proper response of a boxer encountering a straight block object 444 is to block the incoming block object 444 using both gloves hands, with their arms held out in front of their face and in a crossed-arm (X) arrangement. This is shown in FIG. 9b.

[0141] The boxer may also hold both gloved hands straight up in front of their face, instead of crossing their arms, as long as both gloved hands make contact with the incoming straight block object 444. If the boxer performs this action in response to a straight block object attack, then the response will be considered correct and recorded as such by the system. If the boxer tries to punch the block object 444 with a single gloved hand, or only holds up one gloved hand, for example, the system will not recognize a proper response.

[0142] As with the above-described targets, once a block object is correctly hit, regardless of the type of block object, the object will graphically explode and disappear, just disappear, or bounce away from view.

[0143] Block object 440 may also be a left block object 446, which mimics a right hook attacking punch by the boxer’s opponent. Since this type of incoming attack is on the left side of the boxer, the left block object 446 may be colored only a single color,

[0144] which matches the color of the boxer’s left-hand glove. The proper response of the boxer to this left-side attack is to simply position the left arm to block the incoming left block object 446 FIG. 10 shows a top plan view of an exemplary boxing session, illustrating a left block object 446 being formed at a portal 454. The newly formed left block object 446 will follow a pre-planned trajectory 456 to the boxer’s left side. The trajectory may define a relatively straight path to the boxer, or define a more complicated curvy path. The latter path would provide a greater challenge to the boxer owing to the inherent unpredictability of the incoming “punch.” As described above, a proper and correct response to an incoming left block object 446 by the boxer is to raise or otherwise move their left gloved hand up to the left side of their body and head to make contact with the inbound left block object 446. In this case, since only one color is displayed by the block object, only one gloved hand (having the correct matching color) is required to respond to it. Once a block object is correctly hit, regardless of the type of block object, the object will graphically explode and disappear, just disappear, or bounce away from view. The illustration of FIG. 10 represents the trajectory of an exemplary left block object between the portal 454 and the boxer. Of course, other trajectories may be used, including ones suitable for the below described right block object 448 (not shown). [0145] According to exemplary embodiments and to add to the boxing challenge, the left and right hook block objects, as well as the above-mentioned targets 406 may hide their “glove” color (the color indicating which glove the boxer must use to punch), until reaching a predetermined point along their trajectory to the virtual boxer. With this arrangement, the boxer is forced to be alert and responsive. The point along the trajectory at which the color for each target 406 and object 408 is revealed may vary throughout the boxing session, in response to the boxer’s detected skill level during the session, or in response to the number of consecutive correct hits the boxer has made, the measured speed of the boxer’s punches, or other such indicators.

[0146] Similar to the above-described left block object 446, block object 440 may be a right block object 448, which mimics a left hook attacking punch by the boxer’s opponent. Since this type of incoming attack is on the right side, the right block object 448 may be colored only a single color, which matches the color of the boxer’s right hand glove. The proper response of the boxer to this right-side attack is to simply position the right arm to block the incoming right block object 448.

[0147] Once a block object is correctly hit, regardless of the type of block object, the object will graphically explode and disappear, just disappear, or bounce away from view. [0148] According to exemplary embodiments, the block objects 440, regardless of the type (straight, left and right) may be virtually represented using any unique shape, within the VR environment, including a sphere, which is shown in FIG. 9a. The appearance of the block object 440 should not be similar to the appearance of any of the above mentioned targets 406 or objects 408, to the point of confusion. It is important that the boxer working out in a boxing session within the VR environment is able to quickly and easily understand and identify each type of target and each type of object advancing towards them.

[0149] According to other embodiments of this technology, a tactile impulse is generated (to be felt by the boxer) in response to the boxer not properly hitting and destroying either a block object 408, or a target 406. The tactile impulse can be generated by the hand controllers that are part of the VR system, or can be generated using a separate vibrating generator, or another motion-generating device which is directly, or indirectly attached to the real boxer in the real world. These vibration generators, or motion generating devices may be secured to, or formed integral with, the boxer’s headset, or attached to the boxer’s body as a separate element (e.g., a headband, a wristband, a vest, etc.), with vibrations occurring in one of multiple available positions on the user to match their digital experience [0150] According to exemplary embodiments, and as introduced above, an object 408 may also comprise a dodge bar 442, which is used to encourage and teach the virtual boxer (and the real one) to quickly move their head side to side, and down, in varying combinations, to simulate avoiding an incoming punch by an unseen opponent. Similar to the other objects 408, and the targets 406, the dodge bar 442 is formed at the portal location 404 and may be a dodgeleft bar 460, a dodge-right bar 462, or a duck down bar 464. In each case, the size and shape of the bar is the same, as shown in FIG. 11, but how the bar tilts defines which type it is. If the bar is angled to the left, it is a dodge-right bar 462 since the boxer would have to move their head to the right to avoid hitting it. Similarly, if the bar is angled to the right, it is a dodge-left bar 460 since the boxer would have to move their head to the left to avoid hitting it. If the bar is horizontal, it is a duck-down bar 464, and the boxer would have to squat down to avoid hitting it.

[0151] Regardless of the type of dodge bar 442, each bar may be presented to the boxer either individually, as a single dodge bar, as shown in FIG. 11, or as a consecutive series 470 of dodge bars 442, one right after the other. In such instance, the series 470 of dodge bars 442 may consist of the same type of bar, such as a series 470 of ten (as an example) horizontally disposed duck-down bars 464, one after the other. The boxer, confronting this particular series 470 of duck-down bars 464, during a boxing session, would have to squat down low enough to avoid hitting the bars, as they passed by overhead. The series 470 of duck-down bars 464 would prolong the time the boxer would be required to squat down, and thereby increase the muscle workout to the boxer’s leg muscles. However, during this time, the boxer would be able to rest their upper body. By switching between targets 406 and block objects 408, and dodge bars 442, the gaming computer may effectively control and balance the workout applied to the different muscle groups of the boxer, a period of upper body workout, followed by a period of lower body workout. The number of bars in the series 470 may be selected based on the skill level of the boxer (player), or the rated difficulty of the particular boxing session, or another parameter.

[0152] Referring to FIG. 12a and FIG. 12b, a series 470 of dodge bars 442 are being generated at a portal location 404 and thereafter presented to a boxer. This particular series 470 of dodge bars 442 comprise a mix of dodge bar types. The first dodge bar to reach the boxer is a duck-down bar 464, followed by a slight dodge-right bar 462, angled at 10 degrees to the left. The next bar in the series is another dodge right bar 462, but this one is angled 20 degrees to the left. The bars in this series 470 continue to be dodge-right bars, but each disposed at a different angle with respect to the horizontal. This arrangement of different angled dodge bars being presented to the boxer allows the gaming computer to control the squat movement, body “slip” and head movement of the boxer as the series 470 progresses. To negotiate this particular series 470 shown in FIG. 12a, the boxer must first squat down to avoid the first duck-down bar 464, and then angle their body and tilt their head both to the right to avoid hitting the following dodge-right bars 462. Eventually, near the end of the series 470, shown in FIG. 12a, the dodge-right bars 462 being presented to the boxer change back to being a duckdown bar 464, and then being angled down into a dodge-left bar 460. The presentation of various types of dodge-bars 442 help provide the boxer with a workout towards their core and neck muscles, as well as train agility and speed. FIG. 12b illustrates the series 470 of dodge bars 442, from the boxer’s perspective.

Streak Counter:

[0153] According to exemplary embodiments, and as shown in FIG. 15a and FIG. 15b, a streak counting system 1500, having a display 1502, which is provided within the virtual environment, in the field of view of the boxer. The display 1502 may be positioned within, or adjacent to, a portal 1504 so that it may be quickly and easily viewed by the boxer during gameplay. The display 1502 is used to display successful consecutive punches and / or dodges completed by the boxer during any period of gameplay. The streak counting system automatically resets when the boxer punches a target incorrectly, or misses a target, or is hit by a dodge bar, or does not properly block a block object. The display 1502 may display single numbers for every successful interaction made, showing the numbers as they count up one by one, or may only display set numerical intervals, such as, e.g., displaying 25, when at least 25 successful consecutive interactions are made, then 50, when at least 50 is reached, then 75, etc. [0154] Of course, the higher the number displayed, the more skilled the boxer would appear to be, and perhaps the more motivated to play well.

[0155] According to exemplary embodiments, and as shown in FIG. 15b, a streak counting process is shown as a flow diagram. After starting at Block 1510, two variables are set to prescribed values, at Block 1512. A value “N” is set to zero and a value “T” is set to one. The value “N” represents the current number of consecutive successful hits, blocks, or dodges the boxer makes, without fail. The value “T” is a multiplier for the 25 count intervals, as explained below.

[0156] At Block 1514, the computer system sends a boxer (player) a target to hit, or an object to either block or dodge. At Block 1516, if the boxer fails to respond correctly to the target, block object, or dodge bar correctly, (e.g., by using the wrong colored glove to hit a target), the process returns to Block 1512, and the variables N and T are reset to zero and one, respectively. At Block 1516, if the boxer does respond correctly to the target, block object or dodge bar, then the value of N is increased by one, in Block 1518. The process continues to Block 1520. Here, if N is NOT greater than or equal to the product of T and “25,” then the process returns to Block 1514 and the computer system sends a boxer (player) another target to hit, or an object to either block or dodge. For example, in the first pass, the value of T is one, so the product of T and 25 is 25. If the value of N is one, then clearly, one is fewer than 25, and therefore, the process will return to Block 1514. If and when N IS greater than the product of T and 25, then the product value will be displayed in display 1502 (e.g., if T=l, then the product of T and 25 is 25, and “25” will be displayed in display 1502). The number “25” will be displayed by display 1502, either continuously or for a predetermined period of time. Also, the value of T will increase by one in Block 1522 before the process returns to Block 1514 to repeat the cycle. This process ensures that if the boxer reaches at least 25 consecutive successful interactions, but fewer than 50, then the value “25” will be displayed. When at least 50 consecutive successful interactions are reached, then the value “50” will be displayed, and so on. Of course, the interval number to be displayed can be any value, including displaying the value of N all the time. Also, according to another exemplary embodiment, when the boxer fails to respond to an interaction correctly at Block 1516, an indication is made to the boxer that the current “run” of successful interactions has ended. This can be done, for example, by causing the display 1502 to show the current value of N (which would be the highest value reached for that particular consecutive run) that was reached steady for a few seconds or by blinking a few times.

Breathing Detection

[0157] The two most important things to remember about breathing when boxing is to breathe slowly and always breathe in through your nose but exhale through your mouth when punching. When a boxer throws a punch, either in the real world or in the virtual one, it is best that the boxer does not hold their breath but exhales through their mouth quickly. The purpose of this sharp exhale is to engage the core and connect the punch to your body. This breathing technique helps with both timing and power. Breathing through a wide open mouth is generally bad practice during boxing since an open mouth leaves the boxer’s jaw vulnerable to injury from a punch. Exhaling quickly through a closed, clenched mouth protects the boxer’s jaw and also generates a distinctive hissing sound which may be picked up by the microphone that is integral with the VR headset.

[0158] The present computer system may calibrate the exhalation sound during a setup period by asking the boxer to breathe out quickly several times, while recording the sample sounds. The computer system can then compare the captured sound signature as the boxer boxes, matching the timing of the boxer’s outburst of exhalation with inertia and accelerometer data from the boxer’s hand controller. The comparison will indicate if the boxer is breathing correctly and is exhaling quickly during a punch. Other breathing-related information may be obtained from recording the boxer’s breathing sounds, including breath rate, duration of inhalation and/or exhalation, and consistency of breathing. The computer system may convey the results to the boxer after the boxing session, or in real time, as the boxer punches in a session, and instruct the boxer accordingly, and if necessary. If a boxer’s breathing can be corrected, the boxer’s stamina may improve, allowing them to box within the VR environment longer. It is preferred to use the integral microphone of the VR headset, but a separate device including a microphone, power, and necessary controlling and transmitting circuitry may be provided as an attachment to either the VR headset, controllers, or to the boxer/player themselves.

Real Time

[0159] Those of ordinary skill in the art will realize and understand, upon reading this description, that, as used herein, the term “real time” means near real time or sufficiently real time. It should be appreciated that there are inherent delays in electronic components and in network-based communication (e.g., based on network traffic and distances), and these delays may cause delays in data reaching various components. Inherent delays in the system do not change the real time nature of the data. In some cases, the term “real time data” may refer to data obtained in sufficient time to make the data useful for its intended purpose.

[0160] Although the term “real time” may be used here, it should be appreciated that the system is not limited by this term or by how much time is actually taken. In some cases, realtime computation may refer to online computation, i.e., a computation that produces its answer(s) as data arrives, and generally keeps up with continuously arriving data. The term “online” computation is compared to an “offline” or “batch” computation.

[0161] In some cases, in the context of a Virtual Reality (VR), Mixed Reality (MR), or Augmented Reality (AR) system, the term "real-time" may mean sufficient time to allow a user's interactions and/or movements with the system to be reflected in the system in a manner that appears or is perceived to be immediate and without perceptible lag.

Computing

[0162] The applications, services, mechanisms, operations, and acts shown and described above are implemented, at least in part, by software running on one or more computers. [0163] Programs that implement such methods (as well as other types of data) may be stored and transmitted using a variety of media (e.g., computer-readable media) in a number of manners. Hard-wired circuitry or custom hardware may be used in place of, or in combination with, some or all of the software instructions that can implement the processes of various embodiments. Thus, various combinations of hardware and software may be used instead of software only.

[0164] One of ordinary skill in the art will readily appreciate and understand, upon reading this description, that the various processes described herein may be implemented by, e.g., appropriately programmed general-purpose computers, special-purpose computers and [0165] computing devices. One or more such computers or computing devices may be referred to as a computer system.

[0166] FIG. 16 is a schematic diagram of a computer system 1600 upon which embodiments of the present disclosure may be implemented and carried out.

[0167] According to the present example, the computer system 1600 includes a bus 1602 (/.<?., interconnect), one or more processors 1604, a main memory 1606, read-only memory 1608, removable storage media 1610, mass storage 1612, and one or more communications ports 1614. Communication port(s) 1614 may be connected to one or more networks (not shown) by way of which the computer system 1600 may receive and/or transmit data.

[0168] As used herein, a “processor” means one or more microprocessors, central processing units (CPUs), computing devices, microcontrollers, digital signal processors, or like devices or any combination thereof, regardless of their architecture. An apparatus that performs a process can include, e.g., a processor and those devices such as input devices and output devices that are appropriate to perform the process.

[0169] Processor(s) 1604 can be any known processor, such as, but not limited to, an Intel® Itanium® or Itanium 2® processor(s), AMD® Opteron® or Athlon MP® processor(s), or Motorola® lines of processors, and the like. Communications port(s) 1614 can be any of an Ethernet port, a Gigabit port using copper or fiber, or a USB port, and the like.

Communications port(s) 1614 may be chosen depending on a network such as a Local [0170] Area Network (LAN), a Wide Area Network (WAN), or any network to which the computer system 1600 connects. The computer system 1600 may be in communication with peripheral devices (e.g., display screen 1616, input device(s) 1618) via Input / Output (I/O) port 1620.

[0171] Main memory 1606 can be Random Access Memory (RAM), or any other dynamic storage device(s) commonly known in the art. Read-only memory (ROM) 1608 can be any static storage device(s), such as Programmable Read-Only Memory (PROM) chips for storing static information, such as instructions for processor(s) 1604. Mass storage 1612 can be used to store information and instructions. For example, hard disk drives, an optical disc, an array of disks such as Redundant Array of Independent Disks (RAID), or any other mass storage devices may be used.

[0172] Bus 1602 communicatively couples processor(s) 1604 with the other memory, storage, and communications blocks. Bus 1602 can be a PCI / PCI-X, SCSI, a Universal Serial Bus (USB) based system bus (or other) depending on the storage devices used, and the like. Removable storage media 1610 can be any kind of external storage, including hard-drives, floppy drives, USB drives, Compact Disc - Read Only Memory (CD-ROM), Compact Disc - Re-Writable (CD-RW), Digital Versatile Disk - Read Only Memory (DVD-ROM), etc.

[0173] Embodiments herein may be provided as one or more computer program products, which may include a machine-readable medium having stored thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process. As used herein, the term “machine-readable medium” refers to any medium, a plurality of the same, or a combination of different media, which participate in providing data (e.g., instructions, data structures) that may be read by a computer, a processor or a like device. Such a medium may take many forms, including but not limited to nonvolatile media, volatile media, and transmission media. Non-volatile media include, for example, optical or magnetic disks and other persistent memory. Volatile media include dynamic random-access memory, which typically constitutes the main memory of the computer. Transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise a system bus coupled to the processor. Transmission media may include or convey acoustic waves, light waves, and electromagnetic emissions, such as those generated during radio frequency (RF) and infrared (IR) data communications.

[0174] The machine-readable medium may include, but is not limited to, floppy diskettes, optical discs, CD-ROMs, magneto-optical disks, ROMs, RAMs, erasable programmable readonly memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or other type of media/machine- readable medium suitable for storing electronic instructions. Moreover, embodiments herein may also be downloaded as a computer program product, wherein the program may be transferred from a remote computer to a requesting computer by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., modem or network connection). [0175] Various forms of computer-readable media may be involved in carrying data (e.g., sequences of instructions) to a processor. For example, data may be (i) delivered from RAM to a processor; (ii) carried over a wireless transmission medium; (iii) formatted and/or transmitted according to numerous formats, standards, or protocols; and/or (iv) encrypted in any of a variety of ways well known in the art.

[0176] A computer-readable medium can store (in any appropriate format) those program elements which are appropriate to perform the methods.

[0177] As shown, main memory 1606 is encoded with application(s) 1622 that support(s) the functionality as discussed herein (the application(s) 1622 may be an application(s) that provides some or all of the functionality of the services / mechanisms described herein, e.g., training program(s) 210, FIG. 2). Application(s) 1622 (and/or other resources as described herein) can be embodied as software code such as data and/or logic instructions (e.g., code stored in the memory or on another computer-readable medium such as a disk) that supports processing functionality according to different embodiments described herein.

[0178] During operation of one embodiment, processor(s) 1604 accesses main memory 1606 via the use of bus 1602 in order to launch, run, execute, interpret, or otherwise perform the logic instructions of the application(s) 1622. Execution of application(s) 1622 produces processing functionality of the service related to the application(s). In other words, the process(es) 1624 represent one or more portions of the application(s) 1622 performing within or upon the processor(s) 1604 in the computer system 1600.

[0179] For example, process(es) 1624 may include an AR application process corresponding to VR sharing application 230.

[0180] It should be noted that in addition to the process(es) 1624 that carries(carry) out operations as discussed herein, other embodiments herein include the application(s)

[0181] 1622 itself (i.e., the un-executed or non-performing logic instructions and/or data).

The application(s) 1622 may be stored on a computer-readable medium (e.g., a repository) such as a disk or in an optical medium. According to other embodiments, the application(s) 1622 can also be stored in a memory type system such as in firmware, read only memory (ROM), or, as in this example, as executable code within the main memory 1606 (e.g., within Random Access Memory or RAM). For example, application(s) 1622 may also be stored in removable storage media 1610, read-only memory 1608, and/or mass storage device 1612.

[0182] Those skilled in the art will understand that the computer system 1600 can include other processes and/or software and hardware components, such as an operating system that controls allocation and use of hardware resources. [0183] As discussed herein, embodiments of the present invention include various steps or acts or operations. A variety of these steps may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general- purpose or special-purpose processor programmed with the instructions to perform the operations. Alternatively, the steps may be performed by a combination of hardware, software, and/or firmware. The term “module” refers to a self-contained functional component, which can include hardware, software, firmware, or any combination thereof.

[0184] One of ordinary skill in the art will readily appreciate and understand, upon reading this description, that embodiments of an apparatus may include a computer/computing device operable to perform some (but not necessarily all) of the described process.

[0185] Embodiments of a computer-readable medium storing a program or data structure include a computer-readable medium storing a program that, when executed, can cause a processor to perform some (but not necessarily all) of the described process.

[0186] Where a process is described herein, those of ordinary skill in the art will appreciate that the process may operate without any user intervention. In another embodiment, the process includes some human intervention (e.g., a step is performed by or with the assistance of a human).

[0187] Although embodiments hereof are described using an integrated device (e.g., a smartphone), those of ordinary skill in the art will appreciate and understand, upon reading this description, that the approaches described herein may be used on any computing device that includes a display and at least one camera that can capture a real-time video image of a user. For example, the system may be integrated into a heads-up display of a car or the like. In such cases, the rear camera may be omitted.

Conclusion

[0188] As used herein, including in the claims, the phrase “at least some” means “one or more” and includes the case of only one. Thus, e.g., the phrase “at least some ABCs” means “one or more ABCs” and includes the case of only one ABC.

[0189] The term “at least one” should be understood as meaning “one or more,” and therefore includes both embodiments that include one or multiple components. Furthermore, dependent claims that refer to independent claims that describe features with “at least one” have the same meaning, both when the feature is referred to as “the” and “the at least one.” [0190] As used in this description, the term “portion” means some or all. So, for example, “A portion of X” may include some of “X” or all of “X.” In the context of a conversation, the term “portion” means some or all of the conversation.

[0191] As used herein, including in the claims, the phrase “based on” means “based in part on” or “based, at least in part, on” and is not exclusive. Thus, e.g., the phrase “based on factor X” means “based in part on factor X” or “based, at least in part, on factor X.” Unless specifically stated by use of the word “only,” the phrase “based on X” does not mean “based only on X.”

[0192] As used herein, including in the claims, the phrase “using” means “using at least” and is not exclusive. Thus, e.g., the phrase “using X” means “using at least X.” Unless specifically stated by the use of the word “only,” the phrase “using X” does not mean “using only X.”

[0193] As used herein, including in the claims, the phrase “corresponds to” means “corresponds in part to” or “corresponds, at least in part, to,” and is not exclusive. Thus, e.g., the phrase “corresponds to factor X” means “corresponds in part to factor X” or “corresponds, at least in part, to factor X.” Unless specifically stated by use of the word “only,” the phrase “corresponds to X” does not mean “corresponds only to X.”

[0194] In general, as used herein, including in the claims, unless the word “only” is specifically used in a phrase, it should not be read into that phrase.

[0195] As used herein, including in the claims, the phrase “distinct” means “at least partially distinct.” Unless specifically stated, distinct does not mean fully distinct. Thus, e.g., the phrase, “X is distinct from Y” means that “X is at least partially distinct from Y,” and does [0196] not mean that “X is fully distinct from Y .” Thus, as used herein, including in the claims, the phrase “X is distinct from Y” means that X differs from Y in at least some way. [0197] It should be appreciated that the words “first” and “second” in the description and claims are used to distinguish or identify and not to show a serial or numerical limitation. Similarly, the use of letter or numerical labels (such as “(a),” “(b),” and the like) are used to help distinguish and / or identify and not to show any serial or numerical limitation or ordering. [0198] No ordering is implied by any of the labeled boxes in any of the flow diagrams unless specifically shown and stated. When disconnected boxes are shown in a diagram the activities associated with those boxes may be performed in any order, including fully or partially in parallel.

[0199] As used herein, including in the claims, singular forms of terms are to be construed as also including the plural form and vice versa, unless the context indicates otherwise. Thus, it should be noted that as used herein, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.

[0200] Throughout the description and claims, the terms “comprise,” “including,” “having,” and “contain” and their variations should be understood as meaning “including but not limited to” and are not intended to exclude other components.

[0201] The present invention also covers the exact terms, features, values, and ranges, etc., in case these terms, features, values and ranges etc. are used in conjunction with terms such as about, around, generally, substantially, essentially, at least etc. (i.e., "about 3" shall also cover exactly 3 or "substantially constant" shall also cover exactly constant).

[0202] Use of exemplary language, such as “for instance,” “such as,” “for example” and the like, is merely intended to better illustrate the invention and does not indicate a limitation on the scope of the invention unless so claimed. Any steps described in the specification may be performed in any order or simultaneously unless the context clearly indicates otherwise. [0203] All of the features and/or steps disclosed in the specification can be combined in any combination, except for combinations where at least some of the features and/or steps are mutually exclusive. In particular, preferred features of the invention are applicable to all aspects of the invention and may be used in any combination.

[0204] Reference numerals have just been referred to for reasons of quicker understanding and are not intended to limit the scope of the present invention in any manner.

[0205] While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Incorporation by Reference

[0206] The entire content of each of the following patent applications/publications are hereby fully incorporated herein by reference for all purposes:

1. U.S. Patent Application No. 17/236,543, filed April 21, 2021, published as US 20210362029 Al, November 25, 2021, titled “Virtual And Augmented Reality Personalized and Customized Fitness Training Activity or Game, Methods, Devices, and Systems.”

2. PCT/IB2021/053307, filed April 21, 2021, published as WO/2021/214695 on October 28, 2021.

3. PCT/US2022/046132, filed October 8, 2022, titled "System to Determine a Real-Time User-Engagement State During Immersive Electronic Experiences."

4. PCT/US2022/046894, filed October 17, 2022, titled “Virtual And Augmented Reality Fitness Training Activity Or Games, Systems, Methods, And Devices.”