IN AN ACTIVITY

FIELD OF THE INVENTION

[0001] The present invention relates to methods and systems for capturing motion of a user performing an activity, and more particularly it relates to providing feedback to enhance performance and/or execution of the activity.

DESCRIPTION OF THE RELATED ART

[0002] Motion capture is the process of recording the movement of objects or individuals. It is used in military, entertainment, sports, medical applications, and for validation of computer vision and robotics. In filmmaking and video game development, it refers to recording actions of human actors, and using that information to animate digital character models in 2D or 3D computer animation. When used to capture the face, and fingers or captures subtle expressions, it is often referred to as performance capture.

[0003] Motion capture technology is well known in the art, back in the l970s researchers started using potentiometers to track human motion, by attaching these electronic devices to the human body and as the person moved the potentiometers would“record” the angle of the joints. Based on these measurements, the acceleration and velocity could be derived from those angles ultimately resulting in a rather primitive construction of a 3D model that mimicked the actions of the human body.

[0004] Existing systems for capturing motion include various components such as: fixed multiple motion capture cameras; multiple sensors; body suits for tracking motion; a state-of-the art computer to acquire and process all the data from the cameras; 2D video recording system; display screen for feedback; and an operator. Predictably, such setups are expensive and require experienced personnel and can only be found at specialized motion capture labs, which makes this technology inaccessible to most people who require it for everyday use. In addition, most of the components in these existing systems are bulky and are fixed in place in the motion capture labs, and therefore are not portable and do not lend themselves to everyday use in places such as home environments; work environments; hospital settings; rehabilitation environments; or gymnasiums. [0005] Surprisingly, not a lot of technologies have been developed to prevent injury and improve the safety of sports. Nowadays a fair amount of safety in sports is done by following the right protocols, instructions etc. For example, coaches endeavor to make sure their athletes are doing the right exercises by being there and observing them. There is a lack of technologies that can prevent an injury or give information to a coach on whether the player is playing dangerously or safely, during training. In the past concussions were a huge problem for teams and players, as players would end up with a concussion and not even know because there was no suitable technology available, or procedures in place to identify the occurrence of a concussion. While protection gear in sports usually protects the player from getting physical injuries, other injuries are due to the user’s own personal movements of his body or anatomical parts, such as incorrect form for a particular activity, improper execution of an activity.

[0006] Learning a new sport or a new exercise can be difficult and dangerous. As mentioned above, there are many research studies that state that a large number of people that get injured by sports and exercising. However, there are other problems involved with learning and performing an exercise and/or sport. The responsibility of a coach or personal trainer is to coach and train other so that they can learn and understand how to perform an activity or sport, and ultimately achieve better performance results. Research studies have shown that personal training and coaching do make a difference when comparing people that received professional help and people who did not. Unfortunately, personal training or coaching is not always available to everyone, and can be expensive, intimidating and might not fit an individual’s current life style and routine. Accordingly, the portable motion capture apparatus can act as a personal trainer and has the added benefit of providing very precise information regarding how the participant’s body moves and feedback to improve the participant’s movements and motion in whatever activity the participant wishes to improve. Given the portability of the apparatus, it can be used at the gym, at home, or while travelling; and does not require body suits to enable the tracking of motion.

[0007] Very similar to sports, work safety is dependent on following the right regulations and using the right safety wear and gear. Many physically-intensive jobs required employers to follow sets of rules and guidelines that were developed to protect the workers while performing physical jobs with high risk of injury. [0008] It is an object of the present invention to mitigate or obviate at least one of the above- mentioned disadvantages.

SUMMARY OF THE INVENTION

[0009] In one of its aspects, there is provided a motion capture apparatus comprising:

an imager for capturing motion of a user and/or at least one anatomical part of the user within a specified field of vision, the motion comprising a plurality of frames represented as captured motion data, wherein the motion is associated with a contemporaneous user activity; a depth sensor adapted to sense a location of the user and/or the at least one anatomical part of the user;

at least one memory operable to store computer-executable instructions; and

at least one processor configured to access the at least one memory and execute the computer-executable instructions to:

analyze the captured motion data and recognize captured motion of the user and/or the at least one anatomical part of the user;

compare the captured motion to a reference motion pertaining to a predetermined user activity;

determine whether the captured motion of the user and/or the at least one anatomical part of the user matches the reference motion; and

provide feedback for correcting the motion of the at least one anatomical part of the user such that the motion of the at least one anatomical part of the user is substantially similar to the reference motion;

thereby enhancing performance in the predetermined user activity.

[0010] In another of its aspects, there is provided a rehabilitation training method for a user based on a predetermined protocol, the method comprising:

with an imager, capturing motion of the user and/or at least one anatomical part of the user within a specified field of vision, the motion comprising a plurality of frames represented as captured motion data, wherein the motion is associated with a contemporaneous user activity; with a depth sensor, sensing a location of the user and/or at least one anatomical part of the user; with at least one processor, executing computer-executable instructions stored in at least one memory operable to at least:

analyze the captured motion data and recognize captured motion of the user and/or the at least one anatomical part of the user;

compare the captured motion to a reference motion pertaining to a predetermined user activity;

determine whether the captured motion of the user and/or the at least one anatomical part of the user matches the reference motion;

provide feedback for correcting the motion of the at least one anatomical part of the user such that the motion of the at least one anatomical part of the user is substantially similar to the reference motion; and

when the captured motion of the at least one anatomical part of the user is not substantially similar to the reference motion, instruct the user to initiate the user activity in accordance with the predetermined protocol.

[0011] In another of its aspects, there is provided a method for capturing motion of a user performing an activity, and providing feedback to enhance performance and/or execution of the activity, the method comprising:

with an imager, capturing motion of the user and/or at least one anatomical part of the user within a specified field of vision, the motion comprising a plurality of frames represented as captured motion data, wherein the motion is associated with a contemporaneous user activity; with a depth sensor, sensing a location of the user and/or at least one anatomical part of the user;

with at least one processor, executing computer-executable instructions stored in at least one memory operable to at least:

analyze the captured motion data and recognize captured motion of the user and/or the at least one anatomical part of the user;

compare the captured motion to a reference motion pertaining to a predetermined user activity; determine whether the captured motion of the user and/or the at least one anatomical part of the user matches the reference motion; and

provide feedback for correcting the motion of the at least one anatomical part of the user such that the motion of the at least one anatomical part of the user is substantially similar to the reference motion;

thereby enhancing performance in the particular user activity.

[0012] Advantageously, these methods and systems are useful in motion tracking of the human body; and more specifically gait tracking of the human body. These methods and systems allow users to not only track their movement but also learn about their bodies, while performing an activity or exercise, and provide feedback to augment execution of the activity or exercise in order to enhance performance.

[0013] The portable motion capture apparatus may be used in a variety of applications, such as: sports, fitness training or fitness instruction, work safety; health care monitoring and rehabilitation.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Several exemplary embodiments of the present invention will now be described, by way of example only, with reference to the appended drawings in which:

[0015] Figure la shows an exemplary image capture apparatus;

[0016] Figure lb shows a transparent view of the exemplary image capture apparatus;

[0017] Figure lc shows an exploded view of the exemplary image capture apparatus;

[0018] Figure 2 shows a top-level component architecture diagram of the exemplary image capture apparatus;

[0019] Figure 3 shows an exemplary operating environment, in which one or more technologies may be implemented;

[0020] Figures 4a - 4c show various exemplary user interfaces associated with profile creation and calibration;

[0021] Figure 5 shows a high-level data flow diagram between various components within the operating environment; [0022] Figures 6a and 6b show various target points for tracking during an exemplary activity for capture and analysis;

[0023] Figures 7a and 7b show tracked motion of the various target points during the exemplary activity;

[0024] Figures 8a - 8c show various exemplary user interfaces associated with capturing motion; and

[0025] Figure 9 shows an exemplary flow diagram of an algorithm associated with an exemplary artificial intelligence (AI) engine.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

[0026] The detailed description of exemplary embodiments of the invention herein makes reference to the accompanying block diagrams and schematic diagrams, which show the exemplary embodiment by way of illustration and its best mode. While these exemplary embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, it should be understood that other embodiments may be realized and that logical and mechanical changes may be made without departing from the spirit and scope of the invention. Thus, the detailed description herein is presented for purposes of illustration only and not of limitation. For example, the steps recited in any of the method or process descriptions may be executed in any order and are not limited to the order presented.

[0027] Moreover, it should be appreciated that the particular implementations shown and described herein are illustrative of the invention and are not intended to otherwise limit the scope of the present invention in any way. Indeed, for the sake of brevity, certain sub-components of the individual operating components, conventional data networking, application development and other functional aspects of the systems may not be described in detail herein. Furthermore, the connecting lines shown in the various figures contained herein are intended to represent exemplary functional relationships and/or physical couplings between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in a practical system.

[0028] Figures la to lc show a motion capture apparatus, generally identified by numeral 10, in an exemplary embodiment. Motion capture apparatus 10 captures movements of an target, such as the human body, performed in its field of vision, tracks and analyzes the motion of the human body, and outputs analytical reports, outcomes, and information and conclusions based on the motion observed, and provide feedback to a user or a third party.

[0029] As can be seen in Figures la to lc, motion capture apparatus 10 comprises housing 12 which accommodates printed circuit board l4a, b and c with electronic circuitry 16 comprising analog circuits, digital circuits and mixed-signal or hybrid circuits. Housing 12 comprises front housing portion l3a and back housing portion l3b. Apparatus 10 is portable and compact, and may include various form factors.

[0030] Figure 2 shows a top-level component architecture diagram of electronic circuitry 16 which includes computing system 18 comprising a processing unit, such as microprocessor 20, system memory 22, storage device 23, gesture module 24, I/O controller 25, and power supply module 26 for supplying power to the various components of electronic circuitry 16. A number of input/output (I/O) devices may be coupled to I/O controller 25, such as, touch screen and/or display screen 28, and image capture device or camera 29, and one or more depth sensors 30 which are collectively known as camera system 31. I/O devices may include a keyboard, a mouse, a trackball, a microphone, a printing device, and a speaker, etc. Gesture module 24 comprises executable instructions to receive and interpret human gestures that are associated with commands for interacting with apparatus 10.

[0031] Communications interface device 32, also coupled to I/O controller 25 provides networking capabilities via a wired or wireless connection, such as, Ethernet, WiFi™, and/or other suitable network format, to enable connection to one or more networked computers, or other networked devices, via a communications network 33. Accordingly, camera 29 may be configured to record and/or buffer video, and send video to user devices 70, 71 and/or server or cloud- computing system 72 via the communication network 33, such as the Internet.

[0032] The components of computing system 18 may be coupled by interconnection mechanism 34, which may include one or more buses (e.g., between components that are integrated within a same machine) and/or a network (e.g., between components that reside on separate discrete machines). Interconnection mechanism 34 enables communications (e.g., signals, data, instructions) to be exchanged between components of electronic circuitry 16. Heat sink 35a transfers the heat generated by electronic circuitry 16 on printed circuit boards l4a, b and c from the interior of housing 12 in to the surrounding air via vent 35b in back housing portion l3b.

[0033] Power-supply module 26 may include a power connection and/or a local battery 27a and power-supply circuitry for conditioning and regulating the power to the various components of electronic circuitry 16, and charging port 27b. For example, the power connection may connect the apparatus to a power source such as a line voltage source. In some instances, an AC power source can be used to repeatedly charge a (e.g., rechargeable) local battery 27a, such that the battery may be used later to supply power to apparatus 10 when the AC power source is not available. Power module 26 comprises switching device 36 and LED indicator light 37 for indicating the operating status of apparatus 10, i.e. on/off, charging or fully charged. Both switching device 36 and LED indicator light 37 may be located on one of the faces of housing 12. Electronic circuitry 16 also includes other peripheral components, such as, with EO ports, for example, USB hub 38 with USB ports 39 to perform updates, verifications, calibration and other more advanced features to camera system 31 and computing system 18.

[0034] Referring once again to Figure lc, housing 12 includes one face 40 with two orifices 42, 44, which accommodate camera lens 46 of camera 29 and depth sensor 30, respectively. In another implementation, camera lens 46 may be used in conjunction with different types of filters, or substituted with filters, to provide varying motion capture results. In another implementation, housing 12 is water proof, dust proof and shock-resistant, as apparatus 10 may be submitted to various non-ideal operating environments.

[0035] Depth sensor 30 evaluates the distance between camera lens 46 and the target in front of camera 29 that is being recorded, and the captured video includes distance data corresponding to the distances between lens 46 of the camera 29 and predefined points of the target or individual. Accordingly, the captured image data is received by computing system 18 from camera 29, and microprocessor 20 and/or video processor executes program instructions on the captured image data obtained from camera system 31 to identify the primary human body joints, such as head, elbow, shoulders, hips etc., and calculate the displacement and distances between “pre programmed” joints of the human body, or user-defined joints. Microprocessor 20 and/or video processor outputs a video with x, y and z coordinates of all the pre-programmed or user-defined human body joints. This output data and information regarding the human motion recorded by camera system 31 is displayed on display screen 28 and is also transmitted, via communications network 33, to computing device 72, associated with databases 73, 74, as shown in exemplary operating environment 75 of Figure 3. One or more databases 73, 74, may be any type of data repository or combination of data repositories, which store records or other representations of data comprising user profile data and movement data.

[0036] Mobile device 70 and computing device 71 comprise computing system 76, similar to computing system 18, and comprises similar components, such as, microprocessor 78, memory 80, power source 82, I/O interface 84 with input means and output means coupled thereto, such as user interface 86 comprising a keyboard, touch screen and/or display screen 88, speaker, and communication interface adapter 89, among others.

[0037] Turning now to the programmed instructions, computing system 18 comprises a first set of program instructions and a second set of instructions stored in memory 22. The first set of program instructions comprises algorithms, such as a motion capture detection algorithm or logic, with a plurality of lines of code executable by microprocessor 20 to at least: interpret the recorded video as a 3D joint tracking capture; perform joint angle, velocity and position calculations; and output the results of the calculations to display screen 28. A detailed description of the functionality of the first set of instructions will be described later.

[0038] The second set of instructions comprises a plurality of lines of code executable by microprocessor 20 to at least: allow input of personal user data, display the captured video data on display screen 28 and to provide visual and auditory feedback to the user, and other data, such as statistical data. Generally, the second set of instructions comprises an application which provides a means for the user to set up a profile with information include name, gender, height, weight, and other types of descriptive information via touch screen and/or display screen 28, voice commands or gestures. When the user is in proximity of apparatus 10 then touch screen and/or display screen 28 may be used, otherwise the user can interact with apparatus 10 via voice commands or gestures.

[0039] The user profile may also comprise data such as goals, activity regimens, and routines that will allow the platform 72 to adapt the algorithms based on the user generated data and/or user dimensions, as will be described later. [0040] Figure 4a shows exemplary user interface 100 with“Sign in” button 102 for a registered user who has already created an account, and“Create new account” button 104 for an un-registered user, and image or video 106 on display screen 28 of apparatus 10. In Figure 4b, the user is presented with exemplary user interface 106 with a plurality of input fields l08a-e, radio buttons l lOa-d for creating a profile by inputting information such as, age, gender, and name 112, including their body proportions, e.g. measurements, weight, height, etc. The user profile may also store other information provided by the user, such as, images or videos 114, which allows the algorithms to be optimized for the user. The profiles are stored in account database 73 associated with account management module 90 of remote computing device 72 or a separate account management server. Preferably, the user can revise the profile periodically to take into account in changes in previously inputted data, such as changes in body weight, girth, height, etc., or add/remove certain information. In Figure 4c, there is shown exemplary user interface 116 which shows a calibration process to automatically compensate for every user's anatomical differences, including gait. Accordingly, before providing the feedback in accordance with the operation of the methods and systems of the invention, the user is implored to perform certain motions and exercises (which vary depending on the chosen activity), and the application program is configured to display instructions on display screen 28 to guide a user through the calibration process. As an example, exemplary user interface 112 comprises GUI portion 118 with title 120 and calibration steps 122 instructing the user to perform certain motions or exercises, GUI portion 123 with images or video 124 pertaining to the certain motions or exercises, as a guide to the user, and GUI portion 126 includes“NEXT” button 128 actuable by the user to progress to the next calibration screen, and calibration progress bar 130.

[0041] Using the video data captured by camera 29 during the calibration procedure, algorithms and trajectory models in computing system 18 are changed according to the user’s own body and movements, thereby permitting further customization and therefore more accurate motion capture of future instances of activities performed in the field of view of the camera 29 by the same user, as will be discussed in more detail below.

[0042] Now turning to computing system 18, which is tasked to perform calculations on the video recorded received from camera system 31, and provide various outputs and feedback on apparatus display screen 28 and assist the user in executing particular activities. Generally, computing system 18 executes application programs with development packages for motion capture. These application programs include algorithms which generate data points from the video images associated with a target object, and those data points are then used to extract the position data and motion data, and interpret motion of the target object, and generate a movement profile.

[0043] In one exemplary implementation, upon enablement of the application on apparatus 10, and user performs an activity in front of camera system 31 and the captured the user movements are sent to remote computing device 72. Using matching algorithms, remote computing device 72 compares the received individual results to results of other users, or with predefined standards based on aggregated data, to identify the user activity. Accordingly, the matching algorithms match the movements picked up by camera system 31 to pre-recorded movements stored in database 74, and the movements that closest resemble the actions performed are selected (152). Once identified, remote computing device 72 determines whether the captured movements are within an acceptable range or correspond within predefined parameters. When the user’s movements correspond to the reference movements of the identified activity, then the user continues to perform the activity according to the activity regimen, otherwise the requisite feedback is provided to the user, and the user is then once again prompted to perform the user activity.

[0044] In another exemplary implementation, as illustrated in dataflow diagram of Figure 5, upon enablement of the application on apparatus 10, an applications page comprising a plurality of activities for selection by the user are displayed on display screen 28. As an example, via dialogs on display screen 28 or the speaker, the user is prompted to choose an activity (140), such as a fitness activity. Next, an exemplary instructional video of the activity being performed is presented on display screen 28 (142) and is prompted to perform the movements corresponding to the instructional video in front of camera 29 (144). As the user performs the activity, camera system 31 captures the user movements (146), including the depth data and distance data, and that information is sent to computing system 18 which processes the information (148), and generates a movement profile. An application program on apparatus 10 is executed by microprocessor 20 to communicatively couple to remote computing device 72, and send the results, that is the depth data, distance data and movement profile, to remote computing device 72. Using matching algorithms, remote computing device 72 compares the received individual results to results of other users, or with predefined standards based on aggregated data, to identify the user activity (150). Accordingly, the matching algorithms match the movements picked up by camera system 31 to pre-recorded movements stored in database 74, and the movements that closest resemble the actions performed are selected (152), and this process can be optimized by a profile operator algorithm. In more detail, matching algorithms are responsible to match one set of data to another. Once identified, remote computing device 72 determines whether the results are within an acceptable range or correspond within predefined parameters (154). When the user’s movements correspond to the reference movements of the identified activity, then the user continues to perform the activity according to the activity regimen (156), otherwise the requisite feedback is provided to the user (158), and the user is then once again prompted to perform the user activity (144).

[0045] Further to the steps outlined in Figure 5, when the user performs an activity in front of camera 29, the captured user movements are converted into digital data, and that digital data is used to create a movement profile. The movement profile is sent to computing device 72 or cloud computing platform as a query in motion database 74, where a search is performed in order to match the observed movement profile with the stored movement profiles in motion database 74. Once a match is found, motion database 74 outputs the matching movement profile and/or results back to apparatus 10 and the user is able to view and/or hear the feedback. Several types of feedback are possible, such as, motion matching, error matching, variances and others. If the user activity is performed incorrectly, an error message is outputted informing the user of the incorrect movements and advice is provided to the user on how to correct the movements or form.

[0046] In one example, the user chooses to perform a biceps curl, as an activity. As shown in Figures 6a and 6b, for a biceps curl movement there are several points of focus, such as:

reference point (r); shoulder joint (1); elbow joint (2); and wrist joint (3), including the various angles defined between these points of focus. From the starting position in Figure 6a, an angle is formed between segments (e.g. between segment 1 and segment 2 and reference point r). In another example, the point of focus may be any joint, such as ball and socket joint (shoulder joint, hip joint), pivot joint (elbow joint), hinge joint (elbow joint, knee joint, ankle joint), condyloid joint (finger joints, toe joints), and spine joints; and associated angles. [0047] To form a movement profile, a motion (such as the biceps curl) comprises a starting position and an ending position as well as a movement envelope and other more advanced parameters. The starting position is represented by the data in Table 1 and the ending position is represented by the data in Table 2. Joint couple refers to a pair comprised of one segment and the reference point or a pair of two segments.

[0048] Table 1 Table 2

[0049] The movement envelope is another set of parameters that ensure the movement performed by the user is within a certain predetermined range of motion (ROM). Figures 7a and 7b show a figurative representation of the movement envelope for the biceps curl. The ellipse represents a 2D space in which the right arm can move for the movement to still be considered a biceps curl. If the arm moves outside of the envelope defined by the ellipse, an algorithm executed by microprocessor 20 analyzes the captured images in a particular manner to detect errors, and provides feedback advising the user of the proper manner to perform the movement or activity. It should be noted that the ROM of the user will change as he or she performs the exercise, therefore the movement envelope will also change as the motion is performed.

[0050] The combination of the datasets associated with the focus points forms a movement profile. Accordingly, the matching algorithm searches the database for different movements until the closest one to the profile observed by camera 29 is found. An inordinate number of data points (e.g. greater than 10 ⁶ ) is analyzed to identify key features of each movement and eliminate the movements that do not comprise those key features. As an example, if an arm exercise is to be performed it is understood that the arms will move, therefore if camera 29 picks up leg activity only then all the upper body movements are eliminated and the search is performed on lower body profiles only. By classifying the upper body and lower body as different“classes”, this distinction substantially reduces search times and optimizes the searches. As remote computing device 72 matches observed movements to other movements already stored in the motion database 74, the accuracy of the system is therefore dependent on the quantity of datasets available for analysis. As such, the more data there is to analyze and the more accurate the system becomes.

[0051] As an example, the movement profiles are stored in movement database 74, Table 3 shows an exemplary movement profile table.

[0052] Table 3

[0053] The matching algorithms use the data points received by camera 29 and create a profile similar to the one depicted in Table 3 in real time, and the profile is modified every time the data points change. When the user finishes a full cycle of the movement or motion, cloud computing platform 72 searches motion database 74 for a movement profile that matches the observed movement profile.

[0054] The entire process of movement matching is highly dependent on what motion is observed by camera 29. For example, if the user wants to correct his or her biceps curl movements, he or she should only perform the biceps curl in front of camera system 31, since if other movements, different from the biceps curls, are performed then cloud computing platform 72 would not be able to accurately determine how to correct the motion, and/or which motion to search on motion database 74.

[0055] In order to make the system more user friendly, the recorded video of the user performing an activity may be edited on the fly by the user using apparatus 10 in order to extract only the desired portion of the video comprising the activity for analysis and correction. In one exemplary scenario, a user looks to correct the movement of his biceps curl launches the application on apparatus 10 and turns on camera 29. Now suppose that just before the user is ready to perform the biceps curl exercise, he realizes that he dropped something, looks down to the floor, and subsequently kneels down to get the dropped item, with camera 29 all the while capturing all of the user’s movements. Accordingly, as the system is unable to determine at this point if the motion of kneeling down is part of the movement to be observed for analysis and correction, or whether the user motion is in preparation to start the selected activity, therefore the system will try to match and correct all of the observed movements. To prevent the system from trying to analyze motions that are not intended to be analyzed, the application on apparatus 10 comprises an editing feature. After the motion is performed, the user actuates“Stop Capture” button 306 on display screen 28 of application on apparatus 10, and a video editing screen is launched, such as a pop-up. The video editing screen will allow the user to select the beginning and the end of the motion he or she wants to be analyzed. This can be accomplished by selecting a beginning frame and ending frame using the video timeline.

[0056] Figure 8a shows an exemplary user interface 300 of the application on display screen 28 of apparatus 10 just before the user starts the video capture. The user actuates“Start Capture” button 302 and camera 29 starts capturing motion, a dialog on the display screen 28 will show that the system has been successfully initiated and GUI portion 304 of display screen 28 starts displaying, in real time, the user motions being recorded by camera 29. In Figure 8b, upon completion of the desired user activity,“Stop Capture” button 306 on exemplary user interface 308 is actuated to terminate the recording by camera 29. Next, as shown in exemplary user interface 310 of Figure 8c, two buttons“Select start” button 312 and“Select end” button 314 appear on display screen 28, along with the recorded video, and timeline 316 associated with the recorded video 318 extending from the beginning of user activity (ti) to the end of user activity (t ₂ ). Accordingly, the user can view recorded video 318, and cue it back and forth in order to determine the beginning of the biceps curl exercise (marker 320), and the end of the biceps curl exercise (marker 322). After selecting the desired beginning and end of the intended user activity, any unintended user activity from the recorded video 318 is automatically edited out by apparatus 10, and only the edited video comprising the intended user activity is analyzed to match the movement and identify corrections using the proprietary algorithms and obtain the results of the search. The results are then outputted to apparatus 10, where the user can review the feedback, errors, including areas for improvement, and improvement tips. The feedback may include a video playback of the recorded motion along with corrected movements overlaying the original recorded video, and/or accompanying audio. For examples, the corrected movements comprises a semi transparent representation of the user performing the activity in the desired way or form.

[0057] Memory 22 of computing system 18 includes movement profile modification module 330 (not shown) with instructions executable by microprocessor 20 to modify a movement profile pre-saved in memory 23 to properly be matched to a motion performed by the user. Movement profile modification module 330 compensates for the difference in body dimensions and shape between different users and the“model template” used for all calculations. For example, for the motion of the biceps curl, a personal trainer or bodybuilder performs the exercises for recordal by camera system 31, and after the motion is recorded, the results are filtered and uploaded to motion database 74 of movement profiles, as reference data. Accordingly, if the personal trainer or bodybuilder is 5’ 5” in height, then his movements will be completely different in angle and speed when compared to an individual who is 4’ 11” or 6 3 in height. To compensate for the difference in height and in body shape, a profile operator is assigned to every single user (based on height, weight, shape etc.), and the moment the user sets up the mobile app. If there are any changes in the user’s body shape and height, for example, if the user loses weight, gains weight, or grows taller, further calibrations can be performed, in order to order to improve accuracy and relevancy of the feedback.

[0058] Tables 4, 5 and 6 show the movement profile of the biceps curl being modified by an operator. Table 6 shows the modified results that will be used by the system for that user.

[0059] Table 4

[0060] Table 5

Table 6

[0062] As an example, the profile operator in this case reduces the joint angle by 1 degree per inch height difference between the user and the model used to generate the standard biceps curl movement profile. The system adjusts the angles accordingly by reducing each value by 4 degrees, because the user is 4 inches shorter than the person used for the saved movement profile. In this case, for the sake of simplicity, the height of the individual was the only consideration.

[0063] On remote computing device 72, received data from motion capture apparatus 10 is saved, analyzed by an artificial intelligence (AI) engine 340 before it can be displayed on apparatus 10 or user mobile device 70 or other coupled devices. The AI engine 340 performs different types of logic on the different types of information. Table 7 shows a list of data types and logic types.

[0064] Table 7

[0065] The relationship between the data types and logic types can be understood by analyzing an exemplary logic algorithm associated with exemplary artificial AI engine 340, as shown in Figure 9. AI engine 340 is trained through a machine-learning process to recognize captured motion data that corresponds to a predetermined user activity. Looking at the top left comer of the schematic, the users input data into the system which is stored in database Dl. The data is analyzed by a classifier which uses a machine-learning decision tree or a support vector machine (SVM). For example, SVM may be configured to generate data points from the captured motion associated with the target object, such as the user and/or an anatomical part of the user. The data points are then used to extract the position data and motion data, and interpret motion of the target object to generate capture motion products. Next, the captured motion products are compared to a threshold or thresholds determined through machine learning such that the SVM classifies a value above the threshold as representative of an a user movement wherein the captured user movements corresponding to a predetermined act activity pertaining to the user and/or an anatomical part of the user; and classify a value below the threshold as representative of an a user movement wherein the captured user movements not corresponding to a predetermined act activity pertaining to the user and/or an anatomical part of the user..

Accordingly, data in Dl can go through Ll first or through L2 / L3 directly. In Ll the data will go through operations which are basic mathematical calculations that are independent of other users or parameters. This result will be stored back into Dl. Data from Dl eventually will flow through L2 and/or L3 where it will be used for basic and advanced user only data calculations, which depend on other users’ data and other data. For example, calculating the number of hours each day the average user exercises using the system. Depending on how complex the

calculation is the system will use L2 for more basic or L3 for more advanced operations. The data from L2 and L3 will be stored in D2 which is the database for user data that has been calculated using dependent logics. The platform owner or operator also acts as a user in Figure 9 and can upload data to the system, all data uploaded by the platform owner or operator is stored into D3. D2 and D3 databases flow through L4 and L5 before reaching D4. Accordingly, L4 performs system wide calculations that will not require an artificial intelligence (AI) type system. For example, calculating how close the movements of the user are to those of a famous basketball player. L5 performs advanced AI calculations that in order to provide the requisite feedback to the user with respect to exactly he or she needs to do in the future in order to achieve a certain fitness goal or a desired performance level. These calculations are system wide because data provided by the platform operator or owner is used in D3; and L4 and L5 use D2 and D3, with the final data stored into D4, which is reserved only for results and final calculation data. Results in D4 that affect the user are sent to the apparatus 10, that is connected to camera 28 and linked to the user account.

[0066] In another implementation, apparatus 10 can thus be used for professions in which incorrect and repeated motions during execution of a task or activity over prolonged periods of time can lead to injury. For example, office jobs, such as call centers, require employees to sit in front of the computer for many hours at the time. Accordingly, apparatus 10 may be used to promote good posture, by informing employees of the general status of their posture, or whether it can be improved. For employees performing manufacturing or manual labour jobs, apparatus 10 may be used to enhance protocols and procedures to ensure good health and safety of employees and track compliance of the protocols and procedures. Oftentimes, a number of companies rely on reports from managers and employees to inform them about their general health.

[0067] In another implementation, apparatus 10 can be used for health care monitoring. In a hospital setting, there is a variety of technologies ranging from surveillance cameras to wearable devices which are typically available to patients, however, when the patient leaves the hospital, and goes home to continue rehabilitating or to recover, the patient no longer has access to most of the monitoring devices that are available in the hospital setting. Therefore, the patient usually relies on periodic check-ups or rehabilitation session with a nurse or hospital staff back at the hospital. However, once patients finish their rehabilitation session in the hospital and return home, the lack of orientation and information prevents them from continuing

exercising/rehabilitating and tracking their goals e.g. such as frequency of activity, the number of repetitions, and milestones. The methods of apparatus 10 may be used to track patients at home and chart their respective progress and review goals, as they recover during the rehabilitation process. For example, patients that sustain spinal cord injuries for example, are advised to stand for a least one hour every day, as a preventive measure against pressure sores and other complications that arise just from sitting in a wheel chair for too long. The physician or nurse responsible for the rehabilitation of a patient with spinal cord injury will only know if the exercises and protocols are being followed if the patients inform them or if they are present during the time of the rehabilitation. Exercises perfomed at home or without the supervision of a nurse or clinical staff can only be reported back to the clinic or hospital by means of reports, mail, telephone calls and/or other types of communication. The methods of apparatus 10 allows for tracking progress at home or a non-medical facility environment by enabling the portable motion capture apparatus to sending the captured video information to computing device 74, and receiving feedback with detailed graphics and joint movement for correcting movements, activity statistics, and protocol compliance assessment and reports. In addition, the afore-mentioned feedback may be transmitted to a medical professional, such as nurse or physician, or a third party, such as a parent, guardian, employer or insurance company to track patient compliance. Accordingly, the methods of apparatus 10 allows for the determination of insurance

reimbursements or disability income support, based on the feedback.

[0068] In another implementation, a large number of patients can be tracked a single hospital staff member via the platform.

[0069] In another implementation, the output data and information regarding the human motion recorded by camera system 31 may also be directly transmitted to a user device, such as mobile device 70 or computing device 71, or indirectly via communications network 33, for review by the user.

[0070] Remote computing device 72 may be a server which executes a web server application, examples of which may include but are not limited to: Microsoft IIS, Novell Webserver™, or Apache® Webserver, that allows for HTTP (i.e., HyperText Transfer Protocol) access to server computer 72 via network 33 (Webserver is a trademark of Novell Corporation in the United States, other countries, or both; and Apache is a registered trademark of Apache Software Foundation in the United States, other countries, or both).

[0071] Communications network 33 can include a series of network nodes (e.g., the clients and servers) that can be interconnected by network devices and wired and/or wireless communication lines (such as, public carrier lines, private lines, satellite lines, etc.) that enable the network nodes to communicate. The transfer of data between network nodes can be facilitated by network devices, such as routers, switches, multiplexers, bridges, gateways, etc., that can manipulate and/or route data from an originating node to a server node regardless of dissimilarities in the network topology (such as, bus, star, token ring, mesh, or hybrids thereof), spatial distance (such as, LAN, MAN, WAN, Internet), transmission technology (such as, TCP/IP, Systems Network Architecture), data type (such as, data, voice, video, multimedia), nature of connection (such as, switched, non-switched, dial-up, dedicated, or virtual), and/or physical link (such as, optical fiber, coaxial cable, twisted pair, wireless, etc.) between the correspondents within the network. Network 33 may be connected to one or more secondary networks, examples of which may include but are not limited to: a local area network; a wide area network; or an intranet, for example.

[0072] Databases 73, 74 may be, include or interface to, for example, the Oracle™ relational database sold commercially by Oracle Corp. Other databases, such as Informix™, DB2 (Database 2), Sybase or other data storage or query formats, platforms or resources such as OLAP (On Line Analytical Processing), SQL (Standard Query Language), a storage area network (SAN), Microsoft Access™ or others may also be used, incorporated or accessed in the invention. Alternatively, database 20 is communicatively coupled to remote computing device 72.

[0073] User device 70, computing device 71 and remote computing device 72 may communicate with each other using network-enabled code. Network enabled code may be, include or interface to, for example, HyperText Markup Language (HTML), Dynamic HTML, Extensible Markup Language (XML), Extensible Stylesheet Language (XSL), Document Style Semantics and Specification Language (DSSSL), Cascading Style Sheets (CSS), Synchronized Multimedia Integration Language (SMWL), Wireless Markup Language (WML), ava™, ava™ Beans, Enterprise ava™ Beans, ini™, C, C++, Perl, UNIX Shell, Visual Basic or Visual Basic Script, Virtual Reality Markup Language (VRML), Python, or other compilers, assemblers, interpreters or other computer languages or platforms.

[0074] The microprocessor and operating system together define a computer platform for which application programs in high-level programming languages are written. It should be understood that the invention is not limited to a particular computer system platform, microprocessor, operating system, or network. Also, it should be apparent to those skilled in the art that the present invention is not limited to a specific programming language or computer system. Further, it should be appreciated that other appropriate programming languages and other appropriate computer systems could also be used.

[0075] User device 70 or computing device 71 include any device, such as, a personal computer, laptop, tablet, computer server, personal digital assistant (PDA), cellular phone, smartphone, or smart watch. For example, various aspects of the invention may be distributed among one or more computer systems (e.g., servers) configured to provide a service to one or more client computers, or to perform an overall task as part of a distributed system. For example, various aspects of the invention may be performed on a client-server, hybrid client-server, or multi-tier system that includes components distributed among one or more server systems that perform various functions according to various embodiments of the invention. These components may be executable, intermediate (e.g., IL) or interpreted (e.g., Java) code which communicate over a communication network (e.g., the Internet) using a communication protocol (e.g., TCP/IP).

[0076] The logic of Figure 9 can be implemented in software and/or hardware, but in the preferred embodiment, the logic is implemented as software that is stored in memory and executed by a microprocessor. The logic (as well as the other software and software logic described in this document), which comprises an ordered listing of executable instructions for implementing logical functions, can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a“non-transitory computer-readable medium” can be any means that can contain or store the program for use by or in connection with the instruction execution system, apparatus, or device. The non-transitory computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the non-transitory computer-readable medium would include the following: a portable computer diskette (magnetic), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) or DVD (optical). The software and software logic described above comprises an ordered listing of executable instructions for implementing logical functions, can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

[0077] In other implementations, various aspects of the invention may be distributed among one or more computer systems (e.g., servers) configured to provide a service to one or more client computers, or to perform an overall task as part of a distributed system. For example, various aspects of the invention may be performed on a client-server, hybrid client-server, or multi-tier system that includes components distributed among one or more server systems that perform various functions according to various embodiments of the invention. These components may be executable, intermediate (e.g., IL) or interpreted (e.g., Java) code which communicate over a communication network (e.g., the Internet) using a communication protocol (e.g., TCP/IP).

[0078] In other implementations, motion capture apparatus 10 may be used for validation of computer vision and in the field of robotics.

[0079] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or elements of any or all the claims. As used herein, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, no element described herein is required for the practice of the invention unless expressly described as "essential" or "critical." [0080] The preceding detailed description of exemplary embodiments of the invention makes reference to the accompanying drawings, which show the exemplary embodiment by way of illustration. While these exemplary embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, it should be understood that other embodiments may be realized and that logical and mechanical changes may be made without departing from the spirit and scope of the invention. For example, the steps recited in any of the method or process claims may be executed in any order and are not limited to the order presented. Further, the present invention may be practiced using one or more servers, as necessary. Thus, the preceding detailed description is presented for purposes of illustration only and not of limitation, and the scope of the invention is defined by the preceding description, and with respect to the attached claims.