The example and non-limiting embodiments of the present invention relate to an image processing arrangement for analysis of physical exercise in context of physiotherapy.

BACKGROUND

Physical exercise carried out a framework of physiotherapy constitutes a commonly applied approach for restoration of mobility for persons recovering from an injury or from an illness as well as for maintaining mobility of elderly people or persons with a chronic mobility-affecting condition.

In many scenarios at least part of such physical exercise can be taken without explicit guidance from medical personnel e.g. at home. While such self- directed exercise is convenient in terms of the subject being free in selecting time and place of the exercise, a downside of such self-directed exercise is lack of monitoring and feedback from the medical personnel.

SUMMARY

Therefore, an object of the present invention is to provide a tool that enables conducting, analyzing and monitoring physical exercise in the framework of physiotherapy in an efficient and predictable manner. According to an example embodiment, a method for facilitating physical exercise for physiotherapy is provided, the method comprising obtaining exercise definition data for an exercise session, where the exercise definition data defines at least one movement to be performed by a patient in the course of the exercise session; receiving, in the course of the exercise session, a video stream depicting a patient performing said at least one movement defined for the exercise session; deriving, via image analysis carried out based on images of the video stream, movement performance data that is descriptive of observed motion of the patient depicted in said images of the video stream in comparison to the exercise definition data; and storing the exercise performance data for subsequent analysis by a member of medical staff.

According to another example embodiment, an apparatus for facilitating physical exercise for physiotherapy is provided, the apparatus comprising: means for obtaining exercise definition data for an exercise session, where the exercise definition data defines at least one movement to be performed by a patient in the course of the exercise session; means for receiving, in the course of the exercise session, a video stream depicting a patient performing said at least one movement defined for the exercise session; means for deriving, via image analysis carried out based on images of the video stream, movement performance data that is descriptive of observed motion of the patient depicted in said images of the video stream in comparison to the exercise definition data; and means for storing the exercise performance data for subsequent analysis by a member of medical staff.

According to another example embodiment, an apparatus for facilitating physical exercise for physiotherapy is provided, wherein the apparatus comprises at least one processor; and at least one memory including computer program code, which, when executed by the at least one processor, causes the apparatus to: obtain exercise definition data for an exercise session, where the exercise definition data defines at least one movement to be performed by a patient in the course of the exercise session; receive, in the course of the exercise session, a video stream depicting a patient performing said at least one movement defined for the exercise session; derive, via image analysis carried out based on images of the video stream, movement performance data that is descriptive of observed motion of the patient depicted in said images of the video stream in comparison to the exercise definition data; and store the exercise performance data for subsequent analysis by a member of medical staff.

According to another example embodiment, a computer program for facilitating physical exercise for physiotherapy is provided, the computer program comprising computer readable program code configured to cause performing at least a method according to the example embodiment described in the foregoing when said program code is executed on a computing apparatus.

The computer program according to an example embodiment may be embodied on a volatile or a non-volatile computer-readable record medium, for example as a computer program product comprising at least one computer readable non-transitory medium having program code stored thereon, the program which when executed by an apparatus cause the apparatus at least to perform the operations described hereinbefore for the computer program according to an example embodiment of the invention. The exemplifying embodiments of the invention presented in this patent application are not to be interpreted to pose limitations to the applicability of the appended claims. The verb "to comprise" and its derivatives are used in this patent application as an open limitation that does not exclude the existence of also unrecited features. The features described hereinafter are mutually freely combinable unless explicitly stated otherwise.

Some features of the invention are set forth in the appended claims. Aspects of the invention, however, both as to its construction and its method of operation, together with additional objects and advantages thereof, will be best understood from the following description of some example embodiments when read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF FIGURES

The embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, where

Figure 1 illustrates a block diagram of some logical components and/or entities of an exercise management system according to an example;

Figure 2 schematically illustrates some elements of a graphical user interface (GUI) according to an example;

Figure 3 illustrates a flowchart depicting a method according to an example; Figure 4A schematically illustrates exercise definition data according to an example;

Figure 4B schematically illustrates exercise performance data according to an example; and Figure 5 illustrates a block diagram of some elements of an apparatus according to an example.

DESCRIPTION OF SOME EMBODIMENTS

Figure 1 illustrates a block diagram of some logical components and/or entities of an exercise management system 100 according to an example. The exercise management system 100 comprises a client entity 110, an administrator entity 120 and a server entity 130. Each of the client entity 110 and the administrator entity 120 is communicatively coupled to the server entity 130, for example, via a communication network such as the Internet.

The client entity 110 may be provided by a software application running on a computer device, such as a laptop computer, a desktop computer, a tablet computer, a mobile phone (a smartphone), etc. As an example, the software application serving to provide the client entity 110 may be provided as a software module or software program installed in the computer device and executed in the framework of an operating system of the computer device. As another example, the software application may be provided using program code that is executable within a web browser application operating in the framework of an operating system of the computer device. The computer device serving to provide the client entity 110 may comprise any computing device equipped with a camera for recording images of a user, a user interface (Ul) for receiving user input and for displaying images to the user and a communication interface for communicating with other computer devices, e.g. one or more computer devices arranged to provide the server entity 130, via the communication network. Like the client entity 110, also the administrator entity 120 may be provided by a software application running on a computer device, e.g. as a software module or software program installed in the computer device and executed in the framework of an operating system of the computer device or as program code that is executable within a web browser application operating in the framework of an operating system of the computer device.

In an example, respective dedicated software applications may be provided to provide client entity 110 and the administrator entity 120. In another example, the same software application may be arranged to selectively serve as one of the client entity 110 and the administrator entity 120. As an example in this regard, the selection between operation as the client entity 110 or the administrator entity 120 may be provided via different user accounts: each user of the exercise management system 100 may be provided with an user account and the usage of the software application requires the user to sign in to his/her user account using a username and a password assigned to his/her user account. Each user account is designated either as a client or an administrator and hence the user signing in to user account automatically adapts the software application to serve the client entity 110 or as the administrator entity 120 in accordance with the designation of the user account.

The server entity 130 may be provided by one or more computer devices arranged to provide a server functionality. The one or more computer devices may be arranged to provide a cloud computing service. The server entity 130 has access to a memory for storing data to be transferred to the client entity 110 and/or to the administrator entity 120 (e.g. via downloading by the respective one of the client entity 110 and the administrator entity 120) and for storing data received from the client entity 110 and/or from the administrator entity 120 (e.g. via uploading by the respective one of the client entity 110 and the administrator entity 120) and a to communication interface for communication with other computer devices, e.g. the computer device arranged to provide the client entity 110 and/or the administrator entity 120. A patient may access the exercise management system 100 via the client entity 110. In this regard, the client entity 110 may enable the patient to access the exercise management system 100 via a graphical user interface (GUI) operable via the Ul of the computer device serving to implement the client entity 110. In this regard, the Ul of the computer device may be provided using standard components known in the art, such as a (traditional) display together with a pointing device such as a mouse or a touch-sensitive display (e.g. touchscreen). Figure 2 schematically illustrates some elements of a GUI 150:

- a control view 151 including Ul elements that enable the patient to provide user input to the client entity 110;

- a patient video view 152 for displaying a video stream illustrating the patient in the course of an exercise session, obtained from the camera entity of a computer device included in or coupled to the computer device implementing the client entity 110;

- an instruction view 153 for displaying textual or visual instructions to the patient before or during the exercise session; and

- a feedback view 154 for displaying information that is characteristics of the observed motion of the patient in the course of an exercise session.

As pointed out, the illustration of Figure 2 is schematic only and serves to provide a non-limiting example of elements and/or information that may be displayed in the GUI 150, whereas different arrangement of the control view 151 , patient video view 152, the instruction view 153 and the feedback view 154 may be applied, one or more of the control view 151 , the patient video view 152, the instruction view 153 and the feedback view 154 may be omitted from the GUI 150 and/or the GUI 150 may include additional elements in addition to and/or instead of the control view 151 , the patient video view 152, the instruction view 153 and the feedback view 154.

As described in the foregoing, the computer device that implements the client entity 110 is provided with or is communicatively coupled to a camera entity that is capable of providing a sequence of digital images that constitute a video stream, whereas the client entity 110 is arranged to receive the sequence of digital images from the camera entity. The client entity 1 10 facilitates a patient conducting an exercise session within the field of view of the camera entity while analyzing one or more aspects of the patient performing movements defined for the exercise session while conducting the exercise session, where the analysis is based at least in part on digital images received from camera entity in the course of the exercise session. According to an example, the video stream obtained from the camera entity may be displayed to the patient via the patient video view 152 of the GUI 150 whereas at least some information derived via analysis of the of the patient’s movement may be displayed to the user via the feedback view 154 and/or the patient video view 152 of the GUI 150.

According to an example, an outline of a patient operating the client entity 1 10 to conduct an exercise session and derive exercise performance data may comprise the following aspects:

- the patient operates the client entity 1 10 to transfer exercise definition data for an exercise session from the server entity 130 e.g. via a respective Ul element provided as part of the control view 151 of the GUI 150;

- the patient initiates the exercise session e.g. via a respective Ul element provided as part of the control view 151 of the GUI 150;

- the client entity 1 10 receives, from the camera entity, a video stream depicting the patient in the course of the exercise session, which may be displayed e.g. in the patient video view 152 of the GUI 150,

- the client entity 1 10 carries out a movement analysis pertaining to the patient based at least in part on images of the video stream;

- the client entity 1 10 displays feedback to the patient via the GUI in the course of the exercise session e.g. via the feedback view 154 and/or via the patient video view 152 of the GUI 150;

- the client entity 1 10 composes exercise performance data based on the movement analysis carried out in the course of the exercise session and/or after the exercise session; and - the client entity 110 transfers the exercise performance data to the server entity 130.

It should be noted, however, that the above outline serves as a high level description of a non-limiting example pertaining to the client entity 110 conducting an exercise session, which may be varied or complemented in a number of ways without departing from the scope of the exercise management system 100 according to the present disclosure.

Moreover, the above outline pertains to operation of the client entity 110 for carrying out an exercise session for a patient. The aspect of the user client entity 110 conducting the exercise session may be preceded and/or followed related operations carried out via the administrator entity 120. As particular examples in this regard, before the patient conducting an exercise session via the client entity 110 e.g. a member of medical staff, such as a physiotherapist, may apply the administrator entity 120 to define or select an exercise session for a given patient and store it in the server entity 130 for subsequent downloading by the client entity 110 under control of the given patient. Likewise, after the given patient has conducted the exercise session using the client entity 110 (thereby resulting in transferring the exercise performance data to the server entity 130), the member of medical staff may acquire the exercise performance data from the server entity 130 for subsequent analysis and monitoring purposes.

In the following, various aspects pertaining a user operating the client entity 110 to conduct an exercise session to derive the exercise performance data for subsequent inspection by a member of medical staff are described via non- limiting examples. In this regard, the exercise session may be carried out in the framework of a method 200 illustrated by a flowchart depicted in Figure 3. The method 200 may be carried out by the computer device applied to execute the software application arranged to implement the client entity 110. The operations described with references to blocks 202 to 210 of the method 200 may be varied or complemented in a number of ways without departing from the scope of the exercise management system 100 according to the present disclosure, for example in accordance with the examples described in the foregoing and in the following.

The method 200 commences from obtaining exercise definition data for an exercise session, where the exercise definition data defines at least one movement for the exercise session, as indicated in block 202. In this regard, the exercise definition data for the exercise session defines one or more movements for the exercise session, thereby defining a sequence of movements to be performed by a patient in front of (e.g. within the field of view) the camera entity included in or coupled to the computer device implementing the client entity 110 in the course of the exercise session. The exercise definition data may be received (e.g. downloaded) from the server entity 130.

Figure 4A schematically illustrates exercise definition data 300 according to a non-limiting example, where the exercise definition data includes a sequence of one or more movement definitions 301 -k, represented in the schematic illustration of Figure 4A by the movement definitions from 301 -1 to 310-K. In this regard, Figure 4A serves to illustrate the logical arrangement of the exercise definition data 300 rather than its physical arrangement while transferred between one of the client entity 110 and the administrator entity 120 and the server entity 130 and/or while stored in a memory of a computer device. The exercise definition data 300 may optionally further define (durations of) pauses between the movements defined therein. Each movement definition 301 -k includes an identification of a respective movement and it may include an indication of the number of repetitions for the respective movement and/or an indication of time allocated for the respective movement in the exercise session. Moreover, the movement definition 301 -k may include one or more reference parameters that define reference motion associated with the movement identified in the movement definition 301 -k. The movement defined in the movement definition 301 -k comprises a movement of a limb of the patient and the reference motion parameters for the movement definition 301 -k serve to characterize intended motion of the limb of the patient in the course of the respective movement of the exercise session. Non-limiting examples of defining the movements in the exercise definition data are provided in the following.

The sequence of movements defined via the exercise definition data 300 serve as physical exercise defined or selected by a member of medical staff, e.g. by a medical practitioner such as a doctor or by a physiotherapist, for a given patient, typically with the aim of facilitating restoration of mobility of a patient who is recovering from an injury or from an illness or facilitating continued mobility of an elderly person or a person with a chronic mobility-affecting condition. Consequently, the sequence of movements aims at serving a well- defined medical or medically-motivated purpose, whereas an ability to monitor and analyze the manner at which the patient is conducting the physical exercise defined by the exercise session provides valuable information that enables both ensuring that physical exercise is executed in a manner that contributes towards a desired effect and enables monitoring mobility of the patient and development thereof based on objective information.

The movement definition 301 -k may further comprise a respective textual description for the movement identified therein, which textual description may be displayed to the patient via the instruction view 153 of the GUI 150 in the course of the exercise session, e.g. before initiating the first movement of the exercise session, before initiating the respective movement of the exercise session and/or in the course of the respective movement of the exercise session. Alternatively or additionally, the movement definition 301 -k may further comprise a respective audio description for the movement identified therein, which audio description may be played back e.g. via a speaker arrangement included in or coupled to the computer device implementing the client entity 110 in the course of the exercise session, e.g. before initiating the first movement of the exercise session, before initiating the respective movement of the exercise session and/or in the course of the respective movement of the exercise session. Alternatively or additionally, the movement definition 301 -k may further comprise a respective visual description for the movement identified therein, provided e.g. as a respective video stream depicting a person performing the movement identified therein or an animation that is descriptive of the movement identified therein, which visual description may be rendered to the patient via the instruction view 153 of the GUI 150 in the course of the exercise session, e.g. before initiating the first movement of the exercise session, before initiating the respective movement of the exercise session and/or in the course of the respective movement of the exercise session. In a variation of the above example, the visual description may be provided as a set of still images that are descriptive of the movement identified in the movement definition 301 -k. The method 200 further comprises receiving, in the course of the exercise session, a video stream depicting the patient performing the at least one movement defined for the exercise session in the course of the exercise session, as indicated in block 204. In this regard, the video stream is received from the camera entity included in or coupled to the computer device implementing the client entity 110 in the course of the exercise session. The received video stream may be displayed to the user via the patient video view 152 of the GUI 150.

The method 200 further comprises deriving, via image analysis carried out based on images of the video stream, exercise performance data that is descriptive of observed motion of the patient depicted in said images of the video stream in comparison to the exercise definition data 300, as indicated in block 206. In this regard, the image analysis may comprise extracting, for each movement defined in the exercise definition data 300, respective one or more movement parameters that are descriptive of observed motion of the patient performing the respective movement based on images of the video stream illustrating the patient performing the respective motion. The image analysis may be followed by deriving one or more comparison values that are descriptive of observed motion of the patient in comparison to the definition of said at least one movement in the exercise definition data 300 based on differences between the one or more extracted movement parameters and respective ones of the one or more reference movement parameters. Hence, the one or more movement parameters and/or the one or more comparison values may be provided as (part of) the exercise performance data pertaining to the respective movement defined in the exercise definition data 300. Herein, the one or more movement parameters that serve to characterize the observed motion of the patient are preferably descriptive of the same characteristics of motion (of a limb of the patient) that are defined via the one or more reference parameters of the corresponding motion definition 301 -k, thereby allowing for direct comparison between the observed motion and the reference motion.

Figure 4B schematically illustrates exercise performance data 400 according to a non-limiting example, where the exercise performance data includes a sequence of one or more movement observations 401 -k, represented in the schematic illustration of Figure 4B by the movement observations from 401 -1 to 410-K. Each movement observation 401 -k is associated with (or corresponds to) a respective movement definition 301 -k of the exercise definition data 300. Along the lines of the illustration of Figure 4A, also Figure 4B serves to illustrate the logical arrangement of the exercise performance data 400 rather than its physical arrangement while transferred between the server entity 130 and the client entity 1 10 and/or while stored in a memory of a computer device.

Each movement observation 401 -k includes an identification of a respective movement and it may include an indication of the observed number of repetitions for the respective movement. Moreover, the movement observation 401 -k further includes the one or more movement parameters extracted via the image analysis for the movement identified in the movement observation 401 -k and/or the one or more comparison values that are descriptive of the observed motion of the patient in comparison to the corresponding reference motion defined in the movement definition 301 -k. The movement observation 401 -k may comprise a respective set of one or more movement parameters and/or the set of one or more comparison values for each observed repetition of the movement identified in the movement observation 401 -k, thereby characterizing each repetition of the movement by the patient separately. Alternatively or additionally, the movement observation 401 -k may comprise a (further) set of one or more movement parameters that serve to characterize the patient motion in performing the movements defined via the movement definition 301 -k in general, e.g. respective averages of the one or more movement parameters over all observed repetitions of the movement identified in the movement observation 401 -k and/or respective averages of the one or more comparison values over all observed repetitions of the movement identified in the movement observation 401 -k.

As an example, the one or more comparison values for the movement observation 401 -k may comprise respective one or more comparison values that are descriptive of the manner in which the patient has performed the respective movement. In this regard, the one or more comparison values may be indicative of the difference between the observed motion for a respective movement and the reference motion defined for the respective movement in the exercise definition 301 -k. In an example, the one or more comparison values may comprise an indication whether the one or more movement parameters extracted for the respective movement indicate motion that matches that defined by the corresponding one or more reference movement parameters defined for the respective movement, thereby providing a binary indication that serves as a qualitative indication concerning whether the observed movement (or one repetition thereof) is successfully performed by the patient. In another example, the one or more comparison values may comprise an indication of the (extent of) difference between the one or movement parameters extracted for the respective movement and the corresponding one or more reference parameters, thereby providing a quantitative and/or qualitative indication of the match between the observed motion and the reference motion.

The method 200 may, optionally, further comprise displaying at least part of the exercise performance data to the patient in the course of the exercise session, as indicated in block 208. In this regard, displaying the exercise performance data 400 (or part thereof) may comprise, for example, displaying the exercise performance data via the feedback view 154 of the GUI and/or via the patient video view 152 of the GUI. In the latter example, the data may be displayed in a predefined spatial portion of the patient video view 152, thereby overlaying it on the video stream illustrating the patient in the course of the exercise session.

The method 200 further comprises storing the exercise performance data 400 for subsequent analysis by a member of medical staff, as indicated in block 210. In this regard, the storing may comprise storing the exercise performance data 400 in a memory of the computer device applied to execute the software application arranged to implement the client entity 110 via operation of the method 200 and/or transferring (e.g. uploading) the exercise performance data 400 to the server entity 130 for storage in the memory therein.

Referring back to the exercise definition data 300 described e.g. in context of operations pertaining to blocks 202 and 206, each movement defined therein typically involves a respective motion of a limb of the patient, for example motion of an arm or an upper arm, motion of a forearm, motion of a hand, motion of a thigh, motion of a leg, motion of a foot, etc. The defined motion of a limb implicitly concerns rotational motion about a joint that joins the limb to the torso or about a joint that joins two parts of a limb to each other. Examples of such joints hence include a shoulder joint, an elbow joint, a wrist joint, a hip joint, a knee joint, an ankle joint, etc. Hence, definition of the motion defines the limb and the joint under consideration together with at least one characteristic of rotational motion of said limb about said joint. In this regard, the definition of motion may be provided via respective definitions of a reference direction (or a reference point) and a first target position of the limb under consideration with respect to the applied reference direction (or reference point).

The applied reference direction may depend on the limb under consideration. As an example, the reference direction may comprise the direction (or position) of another limb or part thereof, direction (or position) of a torso of the patient, direction (or position) of some other part of the body of the patient, a horizontal direction, a vertical direction, etc. Non-limiting examples in this regard include the following: movement of an upper arm with respect to the torso or with respect to a horizontal direction, movement of a forearm with respect to the upper arm, movement of a hand with respect to the forearm, movement of a thigh with respect to the torso, with respect to the pelvis, with respect to the other thigh or with respect to a horizontal direction, movement of a leg with respect to the thigh, movement of a foot with respect to the leg.

Moreover, since the movement of the kind described above implicitly concerns (repeated) rotational motion of the limb that goes back and forth between a first rotational position and a second rotational position, the definition of motion may be further provided via a second target position with respect to the applied reference direction (or reference point). The second target position, together with the first target position, may be applied to define a reference extent of rotational motion of the limb under consideration. According to an example, characteristics of such rotational motion may be defined, for example, by a first target angle between the first target position and the applied reference direction, possibly together with a second target angle between the second target position and the applied reference direction. Concrete non-limiting examples of applicable movement definitions along via the first and/or the second target angle include an angle between the (upper) arm and the torso, an angle between the (upper) arm and the horizontal direction (or the horizontal plane), an angle between an upper arm and a forearm (of the same arm), an angle between the thigh and the horizontal direction (or the horizontal plane), an angle between the thigh and the leg, etc.

As described in the foregoing, the movement definition 301 -k defines the movement identified therein via the one or more reference parameters that are descriptive of the intended motion (of the limb) of the patient. As an example in this regard, the one or more reference parameters may identify the limb and the joint under consideration, together with a definition of at least one characteristic of the rotational motion of the identified limb. In this regard, the one or more reference parameters may define a respective reference angle, possibly together with a reference direction. Hence, the one or more reference parameters may include one or more of the following :

- an identification of the limb under consideration,

- an identification of the joint under consideration, i.e. the joint about which the motion of the identified limb is to be analyzed,

- an identification of a reference direction,

- a first target angle between the identified limb and the identified reference direction, and

- a second target angle between the identified limb and the identified reference direction.

Referring back to the image analysis described in context of operations pertaining to block 206, the client entity 1 10 may carry out image analysis on basis of a sub-sequence of images of the video stream to identify and track respective image plane positions illustrating the limb under consideration and the image plane positions illustrating the applied reference direction in the same image, and deriving the one or more movement parameters based on the identified image plane positions illustrating the limb under consideration with respect to the image plane positions of the applied reference direction over the sub-sequence of images. In this regard, the client entity 1 10 may compute, based on the respective identified image plane positions of the limb under consideration and the applied reference direction in the sub-sequence of images, a respective observed angle between the limb under consideration and the applied reference direction for each image of the sub-sequence. Consequently, the resulting (time) series of observed angles may be applied to derive the one or more comparison values that are indicative whether the observed angle represents motion defined by the first target angle (e.g. the observed angle reaching the first target angle), indicative whether the change of the observed angle over time represents rotational motion defined by the difference between the first and second target angles (e.g. the observed variation of angle over time reaching the difference between the first and second target angles), indicative of the (extent of) difference between the observed angle and the first target angle, and/or indicative of the (extent of) difference between the observed the change in the observed angle over time and the difference between the first and second angles.

The image analysis applied for identifying and tracking the image plane positions illustrating the limb under consideration and the applied reference direction may comprise identifying and tracking respective image plane positions illustrating one or more joints of the person depicted in the images over a sub-sequence of images. In an example, the image analysis may be applied to identify and track respective image plane positions of a predefined set of joints, e.g. one or more of a shoulder joint, an elbow joint, a wrist joint, a hip joint, a knee joint, an ankle joint. In another example, the image analysis may be applied to identify and track respective image plane position of one or more joints that are relevant for observing motion of the limb (or part thereof) under consideration in view of the one or more reference movement parameters defined for the corresponding motion defined in the exercise definition data. In this regard, the identification and tracking may involve identifying and tracking respective image plane position of the joint under consideration and at least one adjacent joint. In this regard, joint(s) adjacent to one under consideration are readily definable via human anatomy. Non-limiting examples in this regard are provided in the following: joints adjacent to an elbow joint under consideration include the wrist joint and the shoulder joint of the same arm;

joints adjacent to a shoulder joint under consideration include the elbow joint of the same arm, and the shoulder joint of the other arm and the hip joint in the same side of the torso as the shoulder joint under consideration;

joints adjacent to a hip joint under consideration include the knee joint in the same leg, and the hip joint of the other leg and the shoulder joint in the same side of the torso as the hip joint under consideration;

joints adjacent to a knee joint under consideration include the ankle joint and the hip joint of the same leg. As concrete non-limiting examples in this regard, motion of an upper arm with respect to a horizontal plane may be observed via identifying and tracking respective image plane positions of the shoulder joint and the elbow joint of the limb under consideration, motion of a forearm with respect to the upper arm may be observed via identifying and tracking respective image plane positions of the shoulder joint, the elbow joint and the wrist joint of the limb under consideration, whereas motion of an upper arm with respect to the torso may be observed via identifying and tracking respective image plane positions of the shoulder joint and the elbow joint of the limb under consideration and the position of the hip joint on the side of the limb under consideration. Herein, a first image plane position in a given image may indicate the image plane position of the shoulder joint, a second image plane position in the given image may indicate the image plane position of the elbow joint, a third image plane position in the given image may indicate the image plane position of the wrist joint and/or a fourth image plane position in the given image may indicate the image plane position of the hip joint. With these position indications, one or more of the following may be derived:

- The angle between the upper arm and the horizontal plane in the given image may be derived as an angle between a first (imaginary) line that connects the first and second image plane positions and a second (imaginary) line that is aligned with the horizontal direction in the image plane;

- The angle between the upper arm and the forearm in the given image may be derived as an angle between the first (imaginary) line that connects the first and second image plane positions and a third (imaginary) line that connects the second and third image plane positions;

- The angle between the upper arm and the torso in the given image may be derived as an angle between the first (imaginary) line that connects the first and second image plane positions and a fourth (imaginary) line that connects the first and fourth image plane positions According to a non-limiting example, identification and tracking of respective image plane position of the one or more joints in images of the video stream may rely on an artificial neural network (ANN). The ANN applied for identifying respective image plane positions of one or more joints under consideration may be trained, for example, using a suitable supervised learning technique known in the art. The supervised learning may be based on a large dataset of training images representing a plurality of person in various positions where respective image plane positions of one or more joints of interest are indicated for each training image. Consequently, the training yields an ANN that takes an image as an input and provides respective image plane positions of joints under consideration as an output.

According to another non-limiting example, identification and tracking of respective image plane position of the one or more joints in images of the video stream may be based on image processing techniques known in the art, for example such that respective motion vectors (known e.g. in the field of video compression) that are descriptive of motion between an image of the sub sequence and a reference image are derived for images of the sub-sequence and the motion vectors are analyzed to find respective pivot points of rotating movement. The identified pivot points represent image plane positions of respective joints of the human body, whereas each pivot point may be mapped to a respective joint of the human body in view of the knowledge of the human anatomy.

Identification and tracking of the image plane position of a given joint may result in a time series of image plane coordinates that indicate the image plane positions of the given joint in respective images of the video stream and hence at respective points in time. In this regard, image plane coordinates that indicate the image plane position of the given joint may be provided with a timestamp that indicates (relative) time associated with the respective image plane coordinates. Subsequent detection of observed motion may be carried on based on respective time series of image plane coordinates derived for the joints considered in detecting the motion of the limb under consideration. As an example in this regard, the time series of image plane coordinates derived for the joints considered in detecting the motion of the limb under consideration may be applied to derive the time series of observed angles between the limb under consideration and the applied reference direction for each image of the sub-sequence and, consequently, deriving the one or more comparison values that are descriptive of observed motion of the patient in comparison to the definition of said at least one movement in the exercise definition data 300.

Referring back to carrying out the exercise session according to the method 200 described in the foregoing, the flow of operation in conducting the exercise session may be at least in part dependent on user input received via the GUI 150 and/or via definitions received in the exercise definition data 300. Non limiting example in this regard are provided in the following.

According to an example, the control view 151 of the GUI 150 may comprise a Ul element that enables the patient to select one of plurality of exercise sessions made available to him/her by operations carried out via the administrator entity 120 by of a member of medical staff, e.g. a physiotherapist and to download the respective exercise definition data 300 from the server entity 130 to the client entity 1 10. In another example, the client entity 1 10 may be arranged to automatically download, from the server entity 130, the exercise definition 300 made available to him/her upon activating the software application arranged to implement the client entity 1 10. In another example, alternatively or additionally, the control view 151 may comprise a Ul element that enables the patient to initiate an exercise session in accordance with the exercise definition data 300 obtained from the server entity 130. In another example, the client entity 1 10 may be arranged to automatically initiate the exercise session in accordance with the exercise definition data 300 obtained from the server entity 130 in response to meeting one or more predefined conditions with respect to visibility of the patient in images of the video stream and/or the client entity 1 10 may be arranged to continue the exercise session until the predefined one or more conditions are no longer met (or until the end of the exercise session). Such predefined conditions may comprise, for example, one or more of the following: a requirement for at least a predefined portion of the patient being illustrated in images of the video stream, a requirement for the joint under consideration and at least one adjacent joint being illustrated in images of the video stream, a requirement for the patient being distinguishable from the background in images of the video stream at a sufficient reliability, etc.

In a further example, alternatively or additionally, the control view 151 may comprise respective Ul elements that enable the patient to pause and/or terminate an ongoing exercise session. In a further example, alternatively or additionally, the control view 151 may further include a Ul element that enables the patient to terminate the movement he/she is currently performing and to move to the next movement defined in the sequence of movements defined in the exercise definition data 300. Instead of or in addition to the Ul elements of the control view 151 of the GUI 150, one or more of the downloading the exercise definition data 300, initiating, pausing and terminating the exercise session or moving to the next movement of the sequence of movements defined in the exercise definition data 300 may be carried out in response to a respective predefined voice command by the patient and/or in response to a respective gesture by the patient. In this regard, the voice interface may be provided using a suitable speech recognition technique known in the art in the computer device applied to execute the software application arranged to implement the client entity 1 10.

According to an example, assuming that the movement definition 301 -k includes an indication of the number of repetitions for the movement identified therein, the client entity 1 10 may count the successfully completed repetitions of the movement defined by the movement definition 310-k by the patient and display an indication in this regard via the feedback view 154 of the GUI 150 and/or (as information overlaid on) the patient video view 152. The displayed indication may be, for example, a numerical indication or a graphical indication. Alternatively or additionally, the client entity 1 10 may provide an audible indication regarding the successfully completed repetitions of the movement defined by the movement definition 301 -k. Consequently, the exercise session may automatically proceed to the next movement in the sequence defined in the exercise definition data 300 in response to the patient having successfully carried out the indicated number of repetitions of the movement defined by the movement definition 310-k. Identification of the successfully complemented repetitions may rely on the one or more comparison values described in the foregoing.

According to an example, assuming that the movement definition 301 -k includes an indication of the time allocated for the respective movement of the exercise session, the client entity 1 10 may automatically move to the next movement in the sequence defined in the exercise definition data 300 in response to the time allocated for the respective movement having elapsed.

According to an example, in the course of the exercise session e.g. the control view 151 or the feedback view 154 of the GU1 150 may comprise a visual (e.g. textual or graphical) identification of the progress of the exercise, for example an indication that is descriptive of the position of the movement currently being performed in relation to other movements of the exercise session.

Figure 5 illustrates a block diagram of some components of an exemplifying apparatus 900. The apparatus 900 may comprise further components, elements or portions that are not depicted in Figure 5. The apparatus 900 may be employed e.g. in implementing one or more components described in the foregoing in context of the client entity 1 10.

The apparatus 900 comprises a processor 916 and a memory 915 for storing data and computer program code 917. The memory 915 and a portion of the computer program code 917 stored therein may be further arranged to, with the processor 916, to implement at least some of the operations, procedures and/or functions described in the foregoing in context of the client entity 1 10.

The apparatus 900 comprises a communication portion 912 for communication with other devices. The communication portion 912 comprises at least one communication apparatus that enables wired or wireless communication with other apparatuses. A communication apparatus of the communication portion 912 may also be referred to as a respective communication means.

The apparatus 900 may further comprise user I/O (input/output) components 918 that may be arranged, possibly together with the processor 916 and a portion of the computer program code 917, to provide a user interface for receiving input from a user of the apparatus 900 and/or providing output to the user of the apparatus 900 to control at least some aspects of operation of the client entity 1 10 implemented by the apparatus 900. The user I/O components 918 may comprise hardware components such as a display, a touchscreen, a touchpad, a mouse, a keyboard, and/or an arrangement of one or more keys or buttons, etc. The user I/O components 918 may be also referred to as peripherals. The processor 916 may be arranged to control operation of the apparatus 900 e.g. in accordance with a portion of the computer program code 917 and possibly further in accordance with the user input received via the user I/O components 918 and/or in accordance with information received via the communication portion 912.

Although the processor 916 is depicted as a single component, it may be implemented as one or more separate processing components. Similarly, although the memory 915 is depicted as a single component, it may be implemented as one or more separate components, some or all of which may be integrated/removable and/or may provide permanent / semi-permanent/ dynamic/cached storage.

The computer program code 917 stored in the memory 915, may comprise computer-executable instructions that control one or more aspects of operation of the apparatus 900 when loaded into the processor 916. As an example, the computer-executable instructions may be provided as one or more sequences of one or more instructions. The processor 916 is able to load and execute the computer program code 917 by reading the one or more sequences of one or more instructions included therein from the memory 915. The one or more sequences of one or more instructions may be configured to, when executed by the processor 916, cause the apparatus 900 to carry out at least some of the operations, procedures and/or functions described in the foregoing in context of the client entity 110.

Hence, the apparatus 900 may comprise at least one processor 916 and at least one memory 915 including the computer program code 917 for one or more programs, the at least one memory 915 and the computer program code 917 configured to, with the at least one processor 916, cause the apparatus 900 to perform at least some of the operations, procedures and/or functions described in the foregoing in context of the client entity 110.

The computer programs stored in the memory 915 may be provided e.g. as a respective computer program product comprising at least one computer- readable non-transitory medium having the computer program code 917 stored thereon, the computer program code, when executed by the apparatus 900, causes the apparatus 900 at least to perform at least some of the operations, procedures and/or functions described in the foregoing in context of the client entity 110. The computer-readable non-transitory medium may comprise a memory device or a record medium such as a CD-ROM, a DVD, a Blu-ray disc or another article of manufacture that tangibly embodies the computer program. As another example, the computer program may be provided as a signal configured to reliably transfer the computer program. Reference(s) to a processor should not be understood to encompass only programmable processors, but also dedicated circuits such as field- programmable gate arrays (FPGA), application specific circuits (ASIC), signal processors, etc. Features described in the preceding description may be used in combinations other than the combinations explicitly described. Although functions have been described with reference to certain features, those functions may be performable by other features whether described or not. Although features have been described with reference to certain embodiments, those features may also be present in other embodiments whether described or not.