Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
PTBOT: BIOMECHANICS-AWARE PHYSICAL THERAPY ROBOT
Document Type and Number:
WIPO Patent Application WO/2023/282747
Kind Code:
A1
Abstract:
The present invention is in the field of a physical therapy apparatus, in particular an apparatus for passive or active exercising. The present invention relates to a physical therapy robot for a mammal, a method of compiling at least one tissue function map of joint muscles of said mammal, a method of providing a trajectory of a body part of said mammal, and a physical therapy robot computer program comprising instructions for operating the physical therapy robot according to the invention.

Inventors:
SETH AJAY (NL)
PETERNEL LUKA (NL)
PRENDERGAST JOSEPH MICAH (NL)
BALVERT STEPHAN (NL)
DRIESSEN TOM (NL)
Application Number:
PCT/NL2022/050392
Publication Date:
January 12, 2023
Filing Date:
July 07, 2022
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
UNIV DELFT TECH (NL)
International Classes:
G16H20/30; A61H1/02; G16H50/50
Domestic Patent References:
WO2015041618A22015-03-26
WO2016008109A12016-01-21
Foreign References:
US9892655B22018-02-13
US20200015713A12020-01-16
Other References:
PRENDERGAST JOSEPH MICAH ET AL: "Biomechanics Aware Collaborative Robot System for Delivery of Safe Physical Therapy in Shoulder Rehabilitation", IEEE ROBOTICS AND AUTOMATION LETTERS, IEEE, vol. 6, no. 4, 16 July 2021 (2021-07-16), pages 7177 - 7184, XP011869808, DOI: 10.1109/LRA.2021.3097375
SETH AJAY ET AL: "OpenSim: Simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement", vol. 14, no. 7, 26 July 2018 (2018-07-26), pages e1006223, XP055899579, Retrieved from the Internet DOI: 10.1371/journal.pcbi.1006223
JAMWAL PRASHANT K ET AL: "Musculoskeletal Model for Path Generation and Modification of an Ankle Rehabilitation Robot", IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, IEEE, PISCATAWAY, NJ, USA, vol. 50, no. 5, 12 May 2020 (2020-05-12), pages 373 - 383, XP011809020, ISSN: 2168-2291, [retrieved on 20200915], DOI: 10.1109/THMS.2020.2989688
PEI Y ET AL: "Trajectory planning of a robot for lower limb rehabilitation", ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY,EMBC, 2011 ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE, IEEE, 30 August 2011 (2011-08-30), pages 1259 - 1263, XP032318895, ISBN: 978-1-4244-4121-1, DOI: 10.1109/IEMBS.2011.6090296
PEI Y ET AL: "Robot-aided rehabilitation task design for inner shoulder muscles", ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC), 2013 34TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE, IEEE, 28 August 2012 (2012-08-28), pages 3922 - 3925, XP032463797, ISSN: 1557-170X, DOI: 10.1109/EMBC.2012.6346824
Attorney, Agent or Firm:
VOGELS, Leonard Johan Paul (NL)
Download PDF:
Claims:
CLAIMS

1. Method for compiling at least two tissue function maps of at least part of a musculo skeletal system for a mammal, comprising the steps of:

- providing at least part of a biomechanical model of a Musculoskeletal structure of the mammal comprising at least one joint;

- defining at least one tissue in the biomechanical model, wherein the at least one tis sue is selected from a muscle effecting the at least one joint, a ligament providing stability to the at least one joint, a tendon connecting a muscle effecting the at least one joint, a bone providing one side of the joint, and a combination thereof;

- for a plurality of states of the joint and the Musculoskeletal structure and based on said biomechanical model, deriving for each state the exerted load on the at least one tissue; and

- aggregating the derived exerted load on the at least one tissue in a tissue function map.

2. Method according to the preceding claim, wherein said biomechanical model further comprises a cost function, in said cost function:

- calculating for each state the exerted load on at least one first tissue and on at least one second tissue; and

- combining the first tissue function map and the second tissue function map to obtain a combined tissue function map, such as using a cost function, in particular said cost func tion comprising (i) a sum of loads of the at least one tissue of a specific state of said tissues, such as a weighted sum, (ii) a product of the mean of said loads with an Euclidian distance, (iii) the maximum or minimum load across at least one tissue, or (iv) a combination thereof.

3. Method according to the preceding claim, further comprising providing a or the cost function, in said cost function calculating at a particular state the derived exerted load based on adding, such as weighted adding, the strain on said at least one tissue derived from said tissue function map.

4. Method according to claim 2, wherein the step of combining comprises the step of se lecting at a particular state for at least one tissue a load, and comparing said load with a threshold load.

5. Method according to any of the preceding claims, wherein the step of deriving com prises the step of calculating the exerted load based on at least one of a characteristic or con stitutive curve that relate force to displacement (strain), force to area (stress, pressure), and velocity (damping, viscosity), such as wherein the step of deriving comprises the step of cal culating the exerted load based on at least one of an active force-length curve, a passive force-length curve, tendon force-length curve, and a normalized tendon force-length curve.

6. Method according to any of the preceding claims, wherein the at least one tissue is a muscle; and wherein the step of deriving is based on muscle activation.

7. Method according to any of the preceding claims, wherein the at least one joint comprises a first part and a second part arranged for forming the at least one joint; wherein the state of the at least one joint comprises the position of the first part relative to the second part, preferably also comprising the velocity of the first part relative to the sec ond part, more preferably also comprising the acceleration of the first part relative to the sec ond part, most preferably also comprising the jerk of the first part relative to the second part; and wherein the step of deriving is based on the state of the at least one joint.

8. Method according to any of the preceding claims, wherein the at least one tissue is a tendon; and wherein the step of deriving comprises the step of calculating the exerted load based on a normalized tendon force-length curve.

9. Method according to any of the preceding claims, comprising the step of introducing in the biomechanical model an injury in the at least one tissue for making the biomechanical model mammal individual-specific.

10. Method according to any of the preceding claims, comprising the step of calculating an exercise path for the at least one joint to traverse, in particular an injured at least one joint, wherein the exercise path prevents injury to the at least one joint and/or the at least one tis sue while at the same time stimulating the at least one tissue based on the tissue function map, in particular wherein when said biomechanical model comprises an injury in at least one of the at least one tissue, the exercise path limiting a load to said at least one of the at least one tissue while more in particular at the same time stimulation further tissue of the at least one tissue.

11. Method according to the preceding claim, wherein the step of calculating comprises the step of setting a load threshold, wherein the exercise path remains below the load thresh old, in particular wherein said at least one load threshold defines an injury in at least one tis sue in said biomechanical model.

12. Method according to any of the preceding claims 10-11, wherein the at least one joint has a range of motion; and wherein the step of calculating comprises the steps of:

- selecting waypoints such that use of the range of motion is optimized; and

- calculating a path between waypoints such that the cost function does not exceed a threshold level or load threshold.

13. Method according to the preceding claim, wherein the step of selecting waypoints also comprises selecting a velocity and direction of the velocity at the waypoint, preferably also selecting an acceleration and direction of the acceleration at the waypoint, more preferably also selecting a jerk and direction of the jerk at the waypoint, more in particular a combina tion thereof.

14. Method according to any of the preceding claims, wherein the tissue function map is a precomputed tissue function map.

15. Method according to any of the preceding claims, wherein the mammal is a human, or a pet, such as a dog or a cat, or an Ungulate, such as a horse.

16. Method according to any of the preceding claims, wherein the at least one joint is a monoarticular joint, an oligoarticular joint, or a polyarticular joint, a simple joint, a shoulder joint, a hip joint, a compound joint, a radiocarpal joint, a complex joint, or a knee joint.

17. Method according to any of the preceding claims, wherein the biomechanical model models a hand, an elbow joint, a wrist joint, an axillary joint, a sternoclavicular joint, a verte bral articulation, a temporomandibular joint, a sacroiliac joint, a hip joint, a knee joint, a jaw joint, glenohumeral joint, or an articulation of a foot.

18. Physical therapy robot system for a mammal comprising:

- a collaborative motion delivery device comprising a holder for holding a part, in par ticular a limb, of the mammal for moving the holder relative to the mammal, in particular a collaborative robot; and

- a controller configured for:

- obtaining a tissue function map based on at least part of a biomechanical model of a Musculoskeletal structure of the mammal comprising at least one joint and exerted load for a plurality of states of the at least one joint and the Musculo skeleton, and at least one tis sue selected from a muscle effecting the joint, a ligament providing stability to the joint, a tendon connecting a muscle effecting the joint, a bone providing one side of the joint, and a combination thereof;

- selecting an exercise path through the tissue function map of the at least one tis sue preventing injury to the joint and the at least one tissue while at the same time exercising the at least one tissue based on the tissue function map; and

- instructing the collaborative motion delivery device to move such that the joint traverses the exercise path.

19. Physical therapy robot system according to the preceding claim, wherein obtaining the tissue function map comprises obtaining a tissue function map according to any of the claims 1-17, preferably according to or depending on claim 14.

20. Physical therapy robot system according to any of claims 18-19, wherein the controller is further configured for calculating the load exerted on the at least one tissue based on the tissue function map when traversing the exercise path.

21. Physical therapy robot system according to the preceding claim, comprising a load cell arranged for measuring load exerted between the holder and the held part of the mammal, wherein the load exerted on the at least one tissue based on the tissue function map is further based on measured load.

22. Physical therapy robot system according to any of claims 18-21, wherein the controller is further configured for:

- receiving measurements from a load or force sensor, preferably a strain sensor, ar ranged for measuring the exerted load on the at least one tissue;

- adapting the tissue function map based on the received measurements; and - changing the exercise path based on the adapted tissue function map.

23. Physical therapy robot system according to any of claims 18-22, wherein the motion delivery device comprises a robot arm, preferably having multiple joints providing at least 3 degrees of freedom to the holder, more preferably providing 6 de grees of freedom to the holder; and wherein the robot arm preferably is a collaborative robot arm.

24. Physical therapy robot system according to the any of claims 18-23, wherein the con troller is further configured for calibrating and/or simulating before instructing, wherein cali brating and/or simulating comprises:

- receiving a signal indicating that the part of the mammal is arranged in the holder and the joint is set in a predefined position; and

- receiving a current spatial orientation of the joint relative to the motion delivery de vice

25. Data-processing apparatus comprising a processor configured for carrying out the method of any of the claims 1-18, or for the controller in any of the claims 19-24.

26. Computer program product comprising instructions which, when the program is exe cuted by a computer, cause the computer to carry out the method of any of the claims 1-18, or for the controller in any of the claims 19-24.

27. Computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of the claims 1-18, or for the controller in any of the claims 19-24.

28. Computer-readable storage medium comprising data representing a tissue function map obtained according to any of the claims 1-18, or obtained by the controller in any of the claims 19-24.

29. The method according to any of claims 1-17 and/or the physical therapy robot system ac cording to any of claims 18-24 and/or the data processing apparatus according to claim 25 and/or the computer program product according to any of claims 26-28, further comprising one or more elements according to the description, in particular according to the examples, more in particular at least one of using an L-shaped brace, using a robot end-effector, a robot encoder configured to measure an end-effector pose, one or more frames, a transformer link ing a joint to the position and/or orientation of one or more of the frames, limiting a number of poses, retrieving an equivalent tissue function map, clustering unsafe poses typically into clusters, using robot impedance control, using incremental steps in time and/or space, using stiffness and/or damping, using a reference pose and/or reference velocity, using tele-physio- therapy, using a haptic device, providing a robot feedback force, using a robot commanded pose, using a robot actual pose, using a movement drag force, projecting movement of an in dividual on at least one tissue strain map, visualizing data, using a control loop at a higher frequency than that of visualized data, providing interaction between an individual and the robot, and tuning positional and/or rotational stiffness of the robot.

Description:
PTbot: Biomechanics-Aware Physical Therapy Robot

FIELD OF THE INVENTION

The present invention is in the field of a physical therapy apparatus, in particular an apparatus for passive or active exercising. The present invention relates to a physical therapy robot for a mammal, a method of compiling at least one tissue function map of joint muscles of said mammal, a method of providing a trajectory of a body part of said mammal, and a physical therapy robot computer program comprising instructions for operating the physical therapy robot according to the invention.

BACKGROUND OF THE INVENTION

The present invention is in the field of a physical therapy robot. The robot may be used for strengthening muscles, and for recovery of patients, in particular for recovery of muscles around a joint.

A joint relates to a connection between bones in a body of a vertebrate. The joints and muscles make a skeletal system, having individual elements, to function as a whole. Joints are formed such that the allow for different degrees and types of movement, depending clearly on the function required or permitted. Joints are typically able to withstand compres sion and maintain loads while still being capable of moving. The present invention is con cerned with joints that allow movement, that is having a biomechanical function.

From a biomechanical point of view joints can be subdivided into various groups. A simple joint having two articulation surfaces, a compound joint, having three or more articu lation surfaces, such as a radiocarpal joint, and a complex joint, having two or more articula tion surfaces and an articular disc or meniscus, such as a knee joint. Examples of joints of the human body are a joint of hand, a shoulder joint, an elbow joint, a wrist joint, an axillary joint, a sternoclavicular joint, a vertebral articulation, a temporomandibular joint, a sacroiliac joint, a hip joint, a knee joint, a jaw joint, and an articulation of a foot.

Joints may be damaged, unfortunately, such as by a trauma to the joint, or trauma to tissues around a joint, such as to muscles, tendons, ligaments, articular cartilage, and bone. Such trauma can at least partly be treated by physical therapy. A physical therapist treats a joint or the like typically based on experience, insight in the musculoskeletal system in gen eral, and information on the trauma. Unfortunately it has been found that, at least in some cases, treatment may be insufficient, or even leading to subsequent trauma, as information on the actual state of a joint and muscles is incomplete.

Many musculoskeletal injuries, like rotator-cuff tears and ligament strains and related surgeries, require physical therapy as an integral part of recovery. It is noted that the preva lence of rotator-cuff injuries alone is about 22% in the general population and over 50% for individuals over 60, hence widely occurring in large numbers. It is expected that a significant portion thereof requires therapy to return to work or to retrieve function. The movements provided by the physical therapist induce joint circulation and mobility which help improve joint health. A problem with such a therapy is that patients are not getting enough physical therapy to achieve the benefits on an optimal timeline. Physiotherapy of rotator-cuff muscles after injury and/or surgery is typically prescribed and conducted by human physiotherapists. In limited cases some (mostly passive) mechanical devices have been employed, but these devices do not provide any additional insight of the internal states of the muscles and joint tissues.

The present invention relates to a physical therapy robot. A robot is an apparatus which is capable of carrying out actions automatically. The actions may be repetitive, may relate to a series of actions, and may be complex or simple of nature. Typically a robot comprises a controller, which may be programmable. The controller can be an external control device, or the control may be embedded within the robot. Nowadays robots can be autonomous or semi-autonomous. Some examples of robots are industrial robots, medical operating robots, and patient assist robots. The present invention in particular relates to a collaborative robot.

A collaborative robot can, most of all, safely and effectively interact with a human, while performing simple tasks or being -mostly- idle. It may relate to an industrial robot designed to safely interact with humans, and being programmable for performing simple tasks. For in stance, a collaborative robot is designed to stop if it detects an object or a subject, such as a human, in its way or in its way of movement, or in its way of movement of part of the robot. Collaborative robots typically have impedance control. The impedance control enables the robot to be soft in certain directions, while being stiff in the other directions, allowing the ro bot to act as a guide for the human. Additionally, the impedance control enables easy incor poration of gravity compensation terms for the human arm. Nevertheless, impedance control on its own, usually cannot guarantee the safety of the human with which the robot is interact ing. In particular these robots do not guarantee that loads applied to human will not cause musculoskeletal injuries. This is particularly true for injury sensitive populations, like pa tients suffering from or treated for a musculoskeletal injury. As mentioned, these robots oth erwise are generally safe to interact with.

Several upper-extremity rehabilitation devices have been developed, but they are highly specialized and bulky mechanical systems that are expensive and hard to move be tween patients, which severely limits clinical access to therapy. In addition, these existing devices do not target the physical therapy requirements of a recovering rotator-cuff tear. On the other hand, industrial collaborative robots are certified to safely interact with humans, are less expensive, and more generally applicable compared to highly specialized rehabilitation devices. Some prior art documents relate to robot systems or the like to be used in physical therapy, however none of these evaluate or generate therapeutic and/or diagnostic movement for musculoskeletal injuries, nor provide robotic control systems that follow from these gen erated exercises/motions. For instance WO 2015/041618 A3, US 9,892,655 B2, US 2020/0015713 Al, and W02016/008109 A1 may be mentioned. WO 2015/041618 A3 re cites an upper limb rehabilitation robot that includes force sensing and electromyography sensing (EMG) for feedback. A hybrid impedance controller is used (in various modes of op eration) to control the position of the patient's arm and the force applied to the arm. Only limited biometric data is used by this system, such as a patient’s arm length. Estimation of injury risk of targeted muscles is not at all addressed. US 9,892,655 B2 recites exercise pose feedback system for enabling patients to more accurately perform exercises with or without the help of an accompanying physical therapist. Correct motions, typically performed with the help of a therapist, are performed and recorded. The patient is later able to perform these motions on their own with visual feedback on the accuracy of their reproduced motions pro vided from the feedback system., US 2020/0015713 A1 recites a posture detection and feed back system. It uses one or more three axis accelerometers to estimate the posture of the wearer and visual and/or auditory feedback is provided to alert the user to correct their pos ture. The system is only for back posture. WO 2016/008109 A1 recites a robotic system for interacting and positioning a patient's upper limb. The robotic system interfaces with a com puter for data collection and for generating motor commands. This system does not generate the prescribed motor commands for moving the patient. Further documents which may be relied on are an article of Seth et ah, an article of Jamwal et ah, a first article of Pei et ah, and a second article of Pei et ah. Seth (DOI: 10.1371/joumal.pcbi.1006223) recites background of the present invention. Therein, OpenSim is used as a tool for extracting biomechanical varia bles. It is therefore background art. Jamwal et al. (DOI: 10.1109/THMS.2020.2989688) The study relates to creating a single optimal path that minimizes a physiological criterion (min tissue load). Pei et al. (DOI: 10.1109/IEMBS.2011.6090296) relates to a study that presents a method to form a single traversal trajectory that minimizes a physiological criterion (knee contact force), and Pei et al. (DOI: 10.1109/EMBC.2012.6346824)) is in line therewith.

It is an object of the present invention to overcome one or more disadvantages of the robots of the prior art and to provide alternatives to current robots, without jeopardizing functionality and advantages.

SUMMARY OF THE INVENTION

The present robotic system, the physical therapy robot (PTbot), is found capable to de liver safer and more effective physical therapy to a patient via a collaborative robot, which improves or at least maintains joint health, in particular a collaborative robot arm. The con trol of the collaborative robot and combination with an awareness of the participant's risk of injury provided by a detailed biomechanical model of the joints and muscles involved in the injury, provides such safer and more effective operation of the robot. For instance, using the robot and external sensors, the position, velocity and forces applied to the patient are meas ured, which provide an actual status of the subject, and can be applied to the biomechanical model during movement. From this actual state of the model, the forces in injured tissues can be estimated and the risk of injury assessed during the movement. For example, PTbot ap plied to the rehabilitation of shoulder rotator-cuff muscle injuries can estimate the strain in the rotator-cuff muscles during therapy, and thus can avoid positions and movements that would put injured muscles at risk. The shoulder is used as an exemplary embodiment to demonstrate the feasibility of PTbot to handle the most complex and commonly injured joint in the human musculoskeletal system. With PTbot inventors have also provided the concept of "tissue function maps " that identify e.g. how much muscle strain is induced in any given state of the patient, such as during physical therapy. As a result, the precomputed strain maps plus the current state of the patient can inform the robot about how much strain is being put on the healing muscles in any given configuration of the shoulder including interaction forces applied by the robot. Combining the detailed biomechanical information with for in stance a state-of-the-art trajectory planner and impedance controller ensures that the joint of interest (e.g. shoulder) is safely moved and exercised, providing the patient with greater spa tial and temporal volume of physical therapy. An advantage of impedance control is that the subject does not have to be moved by the robot. For instance, the strain maps can be “felt” by the subject in that movement in a desired direction (towards low strain) are easy (compli ant) whereas moving towards high strain is stiff. Therefore the direction and steepness of the map, the gradient, is also considered key and innovative. The robotic delivery frees the phys iotherapist to manage the therapy of multiple patients in parallel. Advantages of the present invention are therefore that the present robot and optional additional sensors make physical therapy far more measurable and quantitative than the state-of-the-art, which is human phys iotherapist mediated therapy; an objective physics- and anatomy -based model is used to di rectly and quantitatively assess injury risk and guide movement in real-time in contrast to qualitative protocols and years of training; and robot delivery is found more sensitive to mo tion and forces, and thus safer, and can be completely automated with minimal direction from the therapist or primary clinician. Prendergast et al. published after the priority date, (DOI: 10.1109/LRA.2021.3097375), provides details of the present invention. It is not con sidered as prior art. Inventors presented this further embodiment after filing of the priority application, to demonstrate how the map could be employed in the case of physical therapy.

The present invention relates amongst others to creation, storage, navigation or use of physiological (tissue function) maps of neuromusculoskeletal information space: e.g.: tissue force, strains, stresses, EMG level, and collagen growth/alignment. Furthermore, the map is directly related to the subject's joint position, velocity and/or applied muscle, implanted and external forces, where there is a transfer function (or algorithm) between the physiological map and the physical (motion) space of the patient. The present tissue function map differen tiates from maps of the physical space which may be common place. The present tissue func tion map defines what areas are safe or beneficial, and those that are unsafe or unfavourable that is directly linked to the physical space. They may be used to define barriers in physio logical space for where not to go (safety) in physical space or define target regions for spe cific training/therapy benefit. While musculoskeletal modelling helps to create these maps, that is only one way of generating a tissue function map, such as a physiological map.

PTbot is also a solution to a growing shortage of physiotherapists and therapy time. PTbot offers to deliver physical therapy sooner (after injury or post-surgery) and in sufficient volume and variety to improve outcomes and reduce the recovery period. A key advantage of this system compared to the human physiotherapist (or existing devices) is that PTbot har nesses musculoskeletal modelling to gain insights into the inner workings of the patient's joints and muscles and is able to modify and adjust therapy in real-time. PTbot gathers quan titative data, such as limb position, motion, and muscle and joint forces during therapy, which as considered to be valuable data, such as to track a patient’s progress and to improve treatment strategies. This is in contrast with physiotherapists, whom must document their qualitative assessments based on their trained perception. The models underlying PTbot can already improve the quality and specificity of physical therapy by estimating the injury risks of patients so that therapy is safer and more effective. PTbot increases the efficacy of physi cal therapy by: increasing access to therapy, guiding patients to move through a larger range of motion and experience targeted resistance, without the manual manipulation of a physio therapist.

The present invention provides several significant improvements over both conven tional physical therapy and the latest attempts at robotic physical therapy, for instance an in creased access to physical therapy since a human therapist is no longer required to deliver therapy, a greater dosage of physical therapy in terms of volume (time, range-of-motion) and specificity, an increased access for a patient since the present PTbot can operate at any time, an increased therapeutic variety (greater range-of-motion, different speeds and loads), target ing to individual tissues, such as muscles/tendons, a reduced recovery time since physical therapy can be delivered more frequently over a shorter recovery period, a real-time feed back about the safety/injury -risk of specific muscles according to the position, movement and loading of the patient's shoulder, and a quantitative monitoring to track progress and re sponsiveness to therapy over time. It is noted that a conventional physical therapy relies on practitioner experience, a patient’s verbal and tactile feedback and general (population level) guidelines to ensure safe movements during therapy. While a therapist can be effective with sufficient doses of physical therapy, s/he is inherently overly conservative because general guidelines attempt to avoid the possibility of reinjury for all patients and injury types. Posi tive outcomes typically depend on very good patient-therapist communication and signifi cant therapist experience, which is in short supply with limited therapists available to offer therapy sessions. With the present PTbot, therapists can receive real-time feedback on the safety of motions allowing for less conservative, larger range of motion activities that can lead to faster recovery. This provides new opportunities to even automate physical therapy enabling a physical therapist to meet the needs of more patients more frequently, leading to overall better outcomes for more patients.

The present invention relates to obtaining and implementing tissue function maps, such as muscle strain maps, such as in a PT-robot, which can induce lower strains and safer move ments, use of the tissue function maps , optionally with an impedance controller, trajectory planning within the safer regions of the strain map, and a physical prototype of the PTbot demonstrating safety and execution of the modelling and control algorithms. The present PTbot is considered to be a transformative technology, changing physical therapy from a scarce, time-consuming, qualitatively assessed practice of experts, with trained physical ther- apists and their accreditation and oversight bodies, to an automated, specific, and quantita tive service that is widely available to the public at much lower costs.

The present invention is also subject of a scientific paper entitled “Biomechanics Aware Collaborative Robot System for Delivery of Safe Physical Therapy in Shoulder Reha bilitation” by Prendergast et al, which reference and contents thereof are incorporated by ref erence.

Physiotherapy is aimed at recovery of the patient while preventing reinjury. Current physiotherapy involves predominantly manual labour of the physiotherapist. Furthermore, current physiotherapy involves a lot of guess work based on experience for what exercise the patient can safely handle without reinjuring himself. This results often in one-dimensional exercises with conservative limits limiting the movement of the recovering patient. This overcaution may hamper the recovery of the patient.

The strain maps allow the physiotherapist to provide multi-dimensional exercises with a larger range of motion while minimizing the risk of reinjury. The larger range of motion of the exercise causes improved blood flow and mobility of the joint typically attributing to an improvement of the recovery time of the injury. The larger range of motion of the exercise causes nerves in the tissue to be more excited or receiving a stimulus resulting in neural rein forcement in time providing improved mobility to the joint. The current invention provides insight to the state and capabilities of the patient to the physiotherapist. The effect is that the strain maps safely allow for planning of an exercise over a larger range of motion. Further more, the strain maps allow the physiotherapist to quantify the strain or load for the patient, where previously the physiotherapist was only able to evaluate in qualitative terms the loads and strains of the patient. An application of the strain maps may be to use the strain maps to control a collaborative robot such that the collaborative robot causes e.g. a limb of a patient to execute the exercise.

The loads and strains on the muscles, tendons, bones and ligaments can typically not be measured with a simple sensor or from external observation. The loads and strains on the muscles, tendons, bones and ligaments may be measured with the help of a sensor, such as an ultrasound sensor at least for the muscle strain. A sensor for measuring loads may provide real-time feedback for updating the strain maps for thereafter adapting the exercise path.

For rotator cuff injuries, a collaborative robot may incorporate a patient-specific bio mechanical model to inform robotic trajectory planning, patient state estimation and imped ance control. This integration of musculoskeletal modelling within the system allows it to plan therapy trajectories to reduce the strain of targeted muscles and tendons, while enabling increased mobility of the human arm in terms of range of motion. For the tests on the rotator cuff detailed below, only the glenohumeral joint is modelled and only the movement or state of the humerus (upper arm) relative to scapula (shoulder blade) is taken into account. This model allows safe predictions regarding load or strain for the different bones, tendons, liga ments and tendons. During tests limits where set for the movement of the shoulder. The lim- its where, shoulder elevation range -44 to 144 degrees; horizontal abduction, plane of eleva tion, -85 to 180 degrees; and internal and external axial rotation -90 to 90 degrees. To limit the amount of data of the strain maps, the strain maps were calculated with 4 degree incre ments for each position of the rotator cuff. These strain maps were pre-computed for each muscle, tendon, ligament and bone. Interpolation may be used to provide trajectory planning with a higher degree of accuracy.

The tissue function map may be a biomechanical impact map, or a load impact map. Where the term “load” is used, depending on the tissue, and in so far as applicable, also the terms “strain”, “stress”, “force”, and “torque”, are included, or the term can be replaced by said further terms. The tissue function map may be a load map or a strain map. The Typi cally for each tissue a map is composed. Multiple maps are typically combined to form one overall or combined strain map. The tissue function map typically maps the exerted load, such as exerted stress, strain and/or torque. The exerted load is typically a resultant of the combination of the state of the joint, load exerted on the joint and activation of a muscle that span the joint. The load exerted on the joint may also include the weight (due to gravity) of the limb attached to the joint. The state of the joint is at least the position of the joint. The state of the joint may also comprise the speed of the joint at that specific position. The state of the joint may also comprise the acceleration of the joint at that specific position. The state of the joint may also comprise the jerk of that joint at that specific position. The state may comprise combinations of at least the position with one or more of speed, acceleration and jerk.

In determining the strain of each of the rotator-cuff muscle tendons, the strains of those parts were compared for each muscle at each position and the highest strain value was taken to be representative of the strain of the entire muscle tendon. This way, the strain space in cludes the highest possible strain the tendon will undergo at any given position. In an alter native calculation method, the contribution of all the tissue under load or strain is taken into account, such as adding all the values per position or weighted adding all the values per posi tion.

Typically, the tissue function map of the injured tissue is supplemented with infor mation at particular states of the joint representing the injury at a specific state of the joint. This information propagates to the combined tissue function map such that when e.g. an ex ercise is planned this exercise prevents reinjury avoiding these specific states of the joint. So an injury state of the model can be made to better represent the risk to an individual subject by adapting properties of the model and/or evaluation thresholds that are specific to the sub ject and her injury.

A cost function may be a summation of all the values associated with a particular strain or load of the tissue at a particular state of the joint. The cost function may alternatively be a weighted summation of all the values associated with a particular strain or load of the tissue at a particular state of the joint. The cost function may alternatively be the maximum strain or load of the tissue during traversing the exercise path. The biomechanical model of a Musculoskeletal structure of the mammal comprises at least one joint. The joint may comprise a first bone and a second bone both shaped to limit the relative motion between the bones. Forces, loads or strains on the bones associated with a joint may be of interest for Osteoarthritis and osteoporosis. A load may be a force or a torque acting upon a joint of the first or second bone of the joint. A muscle may be modelled as a spring-damper. Inventors can compute forces between two rigid bodies (bones), as long as they are connected, and even for those that can be welded together with 0 degrees of free dom. This can be interesting to look at loads through joints that have fused.

In a first aspect the present invention relates to a method for compiling a tissue func tion map of at least part of a musculoskeletal system for a mammal, such as a strain map, comprising the steps of providing at least part of a biomechanical model of a musculoskele tal structure of the mammal comprising at least one joint; defining at least two tissues in the biomechanical model, typically at least three tissues, wherein the at least one tissue is se lected from a muscle effecting the at least one joint, a ligament providing stability to the at least one joint, a tendon connecting a muscle effecting the at least one joint and a bone providing one side of the joint; for a plurality of states of the joint and the Musculoskeletal structure and based on said biomechanical model, deriving for each state the exerted load on the at least one tissue; wherein the plurality typically relates to at least 50% of states possible within boundary conditions of joint, wherein separate states are spaced apart sufficiently to allow resolution of individual states and yet also providing spatially dense enough distrib uted states, for instance where states are taken 0.01mm-5 mm spaced apart, or wherein states are taken 0.01-5 degrees spaced apart, or both, and aggregating the derived exerted load on the at least one tissue in a tissue function map.

In a second aspect the present invention relates to a physical therapy robot system for a mammal comprising a collaborative motion delivery device preferably comprising a holder for holding a part of the mammal for moving the holder relative to the mammal, in particular a collaborative robot; and a controller configured for obtaining a tissue function map based on at least part of a biomechanical model of a the musculoskeletal structure of the mammal comprising at least one joint and exerted load for a plurality of states of the at least one joint and the musculoskeletal structure, and at least one tissue selected from a muscle effecting the joint, a ligament providing stability to the joint, a tendon connecting a muscle effecting the joint, and a bone providing one side of the joint; selecting an exercise path through the tissue function map of the at least one tissue preventing injury to the joint and the at least one tissue while at the same time exercising the at least one tissue based on the tissue function map, wherein the exercise path may be continuous, semi-continuous, or discrete, such with steps having a similar or the same resolution as the tissue function map; and instructing the holder to move such that the joint traverses the exercise path. It is noted that the present tissue func tion map is considered to implicitly provide a gradient of the map as well. This gradient is typically used by the controller. The controller is typically configured to use the at least one tissue function map and its partial derivatives. In a third aspect the present invention relates to a data-processing apparatus compris ing a processor configured for carrying out the steps of the method of the invention, or the steps of the controller of the invention.

In a fourth aspect the present invention relates to a computer program product compris ing instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the present method, or the steps of the present controller.

In a fifth aspect the present invention relates to a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the present method, or the steps of the present controller.

In a sixth aspect the present invention relates to a computer-readable storage medium comprising data representing a tissue function map according the invention, or obtained by the controller of the invention. Now that inventors know their value, they can use the present robot to reverse the process and to build a map based on thousands of specific measurements in a subject's physical workspace.

Further the invention relates to a method according to the invention and/or the physical therapy robot system according to the invention and/or the data processing apparatus accord ing to the invention and/or the computer program product according to the invention, further comprising one or more elements according to the description, in particular according to the examples, more in particular at least one of using an L-shaped brace, using a robot end-ef fector, a robot encoder configured to measure an end-effector pose, one or more frames, a transformer linking a joint to the position and/or orientation of one or more of the frames, limiting a number of poses, retrieving an equivalent tissue function map, clustering unsafe poses typically into clusters, using robot impedance control, using incremental steps in time and/or space, using stiffness and/or damping, using a reference pose and/or reference veloc ity, using tele-physiotherapy, using a haptic device, providing a robot feedback force, using a robot commanded pose, using a robot actual pose, using a movement drag force, projecting movement of an individual on at least one tissue strain map, visualizing data, using a control loop at a higher frequency than that of visualized data, providing interaction between an in dividual and the robot, and tuning positional and/or rotational stiffness of the robot.

DETAILED DESCRIPTION OF THE INVENTION

In an exemplary embodiment of the present method said biomechanical model further comprises a cost function, in said cost function:

- calculating for each state the exerted load on at least one first tissue and on at least one second tissue; and

- combining the first tissue function map and the second tissue function map to obtain a combined tissue function map, such as using a cost function, in particular a cost function comprising (i) a sum of loads of the at least one tissue of a specific state of said tissues, such as a weighted sum, (ii) a product of the mean of said loads with an Euclidian distance, (iii) the maximum or minimum load across at least one tissue, or (iv) a combination thereof.

In an exemplary embodiment the present method further comprises providing a or the cost function, in said cost calculating at a particular state the derived exerted load based on adding, such as weighted adding, the stain on said at least one tissue derived from said tissue function map.

In an exemplary embodiment of the present method the step of combining comprises the step of selecting at a particular state for at least one tissue a load, and comparing said load with a threshold load.

In an exemplary embodiment of the present method the step of deriving comprises the step of calculating the exerted load based on at least one of a characteristic or constitutive curve that relate force to displacement (strain), force to area (stress, pressure), and velocity (damping, viscosity).

In an exemplary embodiment of the present method the step of deriving comprises the step of calculating the exerted load based on an active force-length curve, a passive force- length curve, tendon force-length curve, and/or a normalized tendon force-length curve.

In an exemplary embodiment of the present method the at least one tissue is a muscle; and the step of deriving is based on muscle activation.

In an exemplary embodiment of the present method the at least one joint comprises a first part and a second part arranged for forming the at least one joint; wherein the state of the at least one joint comprises the position of the first part relative to the second part, preferably also the velocity of the first part relative to the second part, more preferably also the acceleration of the first part relative to the second part, most prefer ably also the jerk of the first part relative to the second part; and wherein the step of deriving is based on the state of the at least one joint.

In an exemplary embodiment of the present method the at least one tissue is a tendon; and wherein the step of deriving comprises the step of calculating the exerted load based on a normalized tendon force-length curve.

In an exemplary embodiment the present method comprises the step of introducing in the biomechanical model an injury in the at least one tissue for making the biomechanical model mammal individual-specific.

In an exemplary embodiment the present method comprises the step of calculating an exercise path for the at least one joint to traverse, in particular an injured at least one joint, wherein the exercise path prevents injury to the at least one joint, or at least avoid a high risk of injury, and the at least one tissue while at the same time stimulating the at least one tissue based on the tissue function map, the stimulation being the benefit of simply moving and be cause unsafe areas wherein the patient can move are avoided.

In an exemplary embodiment of the present method the step of calculating comprises the step of setting a load threshold, wherein the exercise path remains below the load thresh old.

In an exemplary embodiment of the present method the at least one joint has a range of motion; and wherein the step of calculating comprises the steps of:

- selecting waypoints such that use of the range of motion is optimized; and

- calculating a path between waypoints such that the cost function does not exceed a threshold level.

In an exemplary embodiment of the present method the step of selecting waypoints also comprises selecting a velocity and direction of the velocity at the waypoint, preferably also selecting an acceleration and direction of the acceleration at the waypoint, more prefera bly also selecting a jerk and direction of the jerk at the waypoint.

In an exemplary embodiment of the present method the tissue function map is a pre computed tissue function map.

In an exemplary embodiment of the present method the mammal is a human, or a pet, such as a dog or a cat, or an Ungulate, such as a horse.

In an exemplary embodiment of the present method the at least one joint is a monoar ticular joint, an oligoarticular joint, or a polyarticular joint, a simple joint, a shoulder joint, a hip joint, a compound joint, a radiocarpal joint, a complex joint, or a knee joint.

In an exemplary embodiment of the present method the biomechanical model models a hand, an elbow joint, a wrist joint, an axillary joint, a sternoclavicular joint, a vertebral artic ulation, a temporomandibular joint, a sacroiliac joint, a hip joint, a knee joint, a jaw joint, glenohumeral joint, or an articulation of a foot.

In an exemplary embodiment of the present physical therapy robot system obtaining the tissue function map comprises obtaining a tissue function map according to the inven tion, preferably wherein the tissue function map is a precomputed tissue function map.

In an exemplary embodiment of the present physical therapy robot system the control ler is also configured for calculating the load exerted on the at least one tissue based on the tissue function map when traversing the exercise path.

In an exemplary embodiment the present physical therapy robot system comprises a load cell arranged for measuring load exerted between the holder and the held part of the mammal, wherein the load exerted on the at least one tissue based on the tissue function map is further based on measured load.

In an exemplary embodiment of the present physical therapy robot system the control ler is also configured for:

- receiving measurements from a load or force sensor, preferably a strain sensor, ar ranged for measuring the exerted load on the at least one tissue;

- adapting the tissue function map based on the received measurements; and

- changing the exercise path based on the adapted tissue function map.

In an exemplary embodiment of the present physical therapy robot system the motion delivery device comprises a robot arm, preferably having multiple joints providing at least 3 degrees of freedom to the holder, more preferably providing 6 degrees of freedom to the holder; and wherein the robot arm is preferably a collaborative robot arm. In an exemplary embodiment of the present physical therapy robot system the control ler is further configured for calibrating and/or simulating before instructing, wherein cali brating and/or simulating comprises:

- receiving a signal indicating that the part of the mammal is arranged in the holder and the joint is set in a predefined position; and

- receiving a current spatial orientation of the joint relative to the motion delivery de vice

FIGURES

Fig. 1 shows schematics of the rotator cuff and a robot applied thereto.

Fig. 2 shows a biomechanical model of the shoulder.

Fig. 3 shows a workflow of the present robot.

Fig. 4. Tendon strain maps.

Fig. 5. Varying shoulder trajectories.

Fig. 6 shows an estimation of strains in the rotator-cuff

Figs. 7a-d, 8, 9a-d, lOa-e, lla-e, and 12 shows details of a further experiment.

DETAILED DESCITPTION OF THE FIGURES

Fig. 1. Physical system and biomechanical simulation setup a) A Kuka LBR iiwa ro botic manipulator delivers motion to the shoulder of a test subject through an elbow brace and load cell attachment b) An OpenSim biomechanical model of the shoulder matches the movement of the subject and strains in the rotator-cuff muscles (highlighted) are estimated c) The estimated strains of the individual rotator-cuff muscles presented as 2D heatmaps over the shoulder range of motion (axes).

Fig. 2. Biomechanical model of the shoulder with only the rotator-cuff muscles visible for clarity (left). Shoulder joint coordinate system (at glenohumeral joint centre) and the de- grees-of-freedom (DoF): plane of elevation, shoulder elevation and axial rotation of the hu merus are shown (right).

Fig. 3. Workflow of the biomechanics aware robotic system for delivering physical therapy. The biomechanical model is used to generate maps of muscle strains. A simple point selection and planning interface enables the user to select start, end and waypoints in joint space. A low strain path is computed over the strain map and transformed into an end point reference frame that the robotic manipulator can follow at a user-selected speed. Robot position and force measures are fed back to the biomechanical model to update its current state. Data is collected from the robot for further evaluation.

Fig. 4. Tendon strain maps represented by a heat spectrum. Four individual rotator-cuff muscle tendon strains (upper graphs) and total combined (lower graph) were used to plan motions that reduce the potential strain on one or multiple tendons. Path indicates different motion paths for identical starting pose and intermediate points, when different muscle ten dons are prioritized. Fig. 5. Varying shoulder trajectories maintaining low-strain for four different condi tions: (1) two point trajectory with non-compliance, (2) 0 degree axial rotation starting pose, (3) -40 degree axial rotation starting pose and (4) large range of motion are each projected on their respective strain map corresponding to their starting axial-rotation pose.

Fig. 6: (a) OpenSim shoulder model used to estimate strains in specific rotator-cuff muscle tendons (b) 3D surface plot showing the max strain experienced by any tendon of the rotator-cuff muscles at every pose (specified by glenohumeral joint angles) in the reacha ble workspace. The weight of the arm is assumed to be supported in such a way that the shoulder is moved passively (with minimal muscle activation related to gravity compensa tion). In white is an example trajectory (movement) that traverses a large range-of-motion at low rotator-cuff tendon strain.

Fig. 7 shows an overview of the present high strain avoidance system (a) OpenSim shoulder model and rotator-cuff (RC) strain map are overlaid (b) One RC strain map is shown with unsafe (high strain) zones labelled (c) Impedance control to push subjects out of unsafe zones is shown (d) The sigma 7 haptic device and the strain map visualizer can be used for safely teleoperating the system (e) The Kuka LBR iiwa robotic arm is used to control the subject and so that they cannot reach poses that would result in unsafe RC tendon strains.

Fig. 8 shows a workflow of the biomechanics aware robotic system for delivering physical therapy. The biomechanical model is used to generate maps of muscle strains. Clus tering based segmentation is performed to determine unsafe pose regions and ellipse fitting is done to simplify these ’’unsafe zones”. Additional interpolation is done to allow for smooth map transitions during real-time changes to axial -rotation. These Maps and corre sponding safe zones are then used either direction by the robotic control system during pa tient led activities or by the haptic feedback system for guiding and restricting the physio therapist as they lead the patient. Robot position and force measures are fed back to the bio mechanical model to update its current state. Data is collected from the robot for further evaluation.

Fig. 9 shows identification and segmentation of unsafe zones (a) A raw strain map at each axial rotation pose is first generated, (b) DBSCAN is used to cluster hight strain points together within the map, (c) ellipses are fit to these segments to serve as simplified bounda ries which must be used in real-time by the robot to ensure safe motion of the patient (d) multiple axial rotation maps are combined by interpolating between adjacent ellipses so as to ensure smooth force changes during axial rotation.

Fig. 10 shows an example experiment where Axial Rotation of the joint has been fixed (a) The trajectory is shown with time stamps in seconds indicated by the enumerated blue circles (b) shows the starting pose of the subject (c) pose error used by the impedance con troller is shown for those times when the subject has entered the unsafe zone (d) Joint tor ques on the subject’s shoulder as calculated from the end-effector force of the robot are shown. € estimated tendon strain based on the strain map. Fig. 11 shows an example of a full three degree of freedom motion experiment (a) the corresponding strain maps representing the change of axial rotation are shown here as par tially transparent so that the trajectory can be seen. Initially the subject axially rotates 28 de grees which forces them into an unsafe zone. The subject rotates back slowly while also moving the other two degrees of freedom until they eventually return to the original axial ro tation position (b) the full trajectory is plotted on a just the original axial rotation map for ease of visualization (c) the full shoulder pose of the subject is shown (d) the error used by the impedance controller is shown for the time in which the subject is in an unsafe zone (e) calculated joint torques as calculated from the end-effector force.

Fig. 12 shows am example teleoperation experiment along with the force and pose data. The top image shows the strain map and trajectory travelled with the time indicated by the numbers along the subject and PT’s trajectories (a) Initially the subject follows the PT and overall forces are low. (b) as the PT pushes into the unsafe zone they feel the force feed back from the zone boundary, note because it requires some force for the PT to push into this zone there is some oscillation by the PT that also results in some increased forces for the subject (c) Both the subject and PT have exited the unsafe zone (d) The subject has been asked to actively resist at this point while the PT attempts to pull them along (e) the subject relaxes and once again begins to follow the PT

EXAMPLE

Inventors provide a the present physiotherapy system for rotator cuff injuries, based on a collaborative robot that incorporates a patient-specific biomechanical model to inform ro botic trajectory planning, patient state estimation and impedance control. This integration of musculoskeletal modelling within the system allows it to plan therapy trajectories to reduce the strain of targeted muscles and tendons, while enabling increased mobility of the human arm in terms of range of motion. An important component of this system is the strain map of which Fig. lc is an example, which is developed to provide the muscle strains of each of the rotator-cuff muscles at any pose of the subject’s arm. These strain maps provide an integration of muscle strain estimates from the biomechanical model (see Fig. 2) into the present robotic path plan ning and control system and enable quantitative feedback to the system about safety/injury in any patient pose. It is noted that if strains in the muscles are exceeded, a healing muscle/ten don might be re-injured. While a human physiotherapist has only qualitative sensing and no direct access to the muscle strains, they are limited to more conservative movements to avoid re-injuring the healing muscles. By using these strain maps, the present robotic system gains quantitative insight into a subject’s muscle strains, allowing it to safely manoeuvre a subject through less conservative, larger range of motion exercises. In addition, the biomechanical model may be comple mented by external measurements to augment the estimates of the model. This allows for real-time model updates, and more accurate perception and control of the complete robotic system, for safe delivery of robotic physiotherapy. Inventors present in an example shoulder tissue function maps according to which the optimization algorithm plans the trajectories. The sys tem is demonstrated and evaluated with proof of concept experiments on a Kuka LBR iiwa collaborative robot.

A. Strain map computation

To accurately model the internal strains of a subject’s shoulder, the open-source com putational musculoskeletal modelling tool OpenSim is used. A primary goal is to inform the present robot of the tendon strain on each of the four rotator-cuff muscle tendons (strain map) throughout the range of motion of the subject (see Fig. 4). This strain space is obtained using OpenSim and the Thoracoscapular Shoulder Model. To generate the tissue function map, and in particular the muscle strain map, the proposed approach requires position and velocity of the shoulder model (joints) and the applied loads and their location.

As the rotator-cuff muscles span the glenohumeral joint inventors consider only the three degrees of freedom that comprise the motion of the humerus (upper arm) relative to the scapula (shoulder blade). These include shoulder elevation, with a range of motion from -44 to 144 degrees; the plane in which the arm is moved (referred to as plane of elevation throughout), also known as horizontal abduction, with a range from -85 to 180 degrees; in ternal and external (axial) rotation, with a range from -90 to 90 degrees (see Fig. 2). A mus cle and tendon are allowed to reach equilibrium in the model using active force-length curve, passive force-length curve and tendon force-length curve respectively, a force-velocity curve, muscle activation, pennation angle, muscle fibre length, muscle velocity, and tendon length, as well as maximum isometric force. The muscle fibre length at equilibrium is then used to determine a normalized muscle fibre force which is then used to obtain the tendon length from the normalized tendon force-length curve. The tendon length and tendon slack length are then used to compute percent tendon strain. Inventors may precompute activated strain maps in anticipation of the next phase of therapy involving low levels of activity such that the measured input forces between the subject and the robot end-effector can be used as feedback and for map/trajectory updates. Muscles may be “divided”, such as the infraspina tus muscle is divided into an inferior and a superior part, the supraspinatus muscle into an anterior and a posterior part and the subscapularis muscle into an inferior, medial and supe rior part, while the teres minor muscle is not divided.

From a practical point of view the present strain map can be limited, such as including only feasible poses (-20-160 degrees for plane of elevation and 0-144 degrees for shoulder elevation).

The maps are the used for planning and control operations during therapy.

B. Safe Path Planning

With strain space maps computed for each of the four rotator-cuff muscles, these maps are used to plan trajectories that avoid large strains in one or more muscle tendons. The pro cess is automated using a weighted A* approach to path generation. A strain threshold (2 percent strain for the results presented here) is first set, which will denote the barrier within the 3D map. This threshold is easily changed to accommodate more/less restrictive strain limits when that is desired. Regions above this threshold cannot be selected as starting points or waypoints, and the path planner will not cross these barriers. With the barrier map deter mined, it is possible to run A* to find the shortest path between any two points within the strain space, however because the goal is to avoid straining the tendons while increasing range of motion, A* is modified to plan trajectories that result in reduced accumulated strain throughout the entire trajectory. This results in generally longer paths with reduced accumu lated strain. To accomplish this, a distance between each adjacent node within the map is de fined by the strain of that node. In addition, to allow for A* to continue traversing the strain map towards the end goal, a strain distance heuristic is defined. The accumulated strain of each node g(n) in the path along with this strain distance estimate from the end goal h(n)) al lows to assert a cost function associated with each new node explored in the path. The cost function applied is a sum of all strains and a multiplication of a mean strain with a Euclidian distance. The weighted A* approach provides an automated path planning procedure. The planning procedure is adaptable, e.g., such that a complete, larger range of motion trajectory is provided.

C. Trajectory Implementation and Control

Once a path has been chosen in the strain map, this path is transformed to the physical space of the robotic manipulator to allow for implementation of the planned trajectory. To accomplish this, some prior information is about the starting pose of the human relative to the robot base frame, as well as the arm length of the human is provided. Also, subject specific data (arm length, torso height) may be entered along with the starting pose of the human in the robot’s X,Y frame and the starting rotation of the human body. Once all subject-specific data has been entered and a path has been generated, the path is then transformed into the ro bot’s base frame coordinates. A planned path trajectory is then generated to manoeuvre the patient’s arm through the prescribed path generated by the weighted A* algorithm. A linear interpolation may be done between each node in the path prior to the path being transformed to the robot’s (Cartesian) task space. The robot itself operates with a set update rate of 200 Hz. The speed of the path is typically dependent on the number of reference transforms pro vided between each node. This speed is set prior to the initial path planning and is fixed at 5 degrees/second for the tests demonstrated here.

The generated trajectory was controlled by a Cartesian impedance controller with an interaction force vector acting from the robot on the environment, stiffness and damping ma trices in Cartesian space, respectively, and actual and the reference pose vectors of the robot endpoint, respectively. The desired interaction force is controlled in Cartesian space by the joint torques with vector of robot joint torques, a vector of robot joint angles, the gravity vec tor, and the robot Jacobian matrix, mass matrix, and Coriolis and centrifugal matrix, respec tively. A force vector is used to compensate the gravity acting on the human arm and is de pendent on the current configuration of the human shoulder. The joint torques induced into the human shoulder by the robot was used as an input for the musculoskeletal model. A quasi-static approach was used. The human shoulder configuration was calculated through the kinematic model of the shoulder based. on the measured robot endpoint pose, which coincides with the human elbow pose.

In the actual experiment a 6-axis load cell (FTS-Delta SI-330-30, Schunk GmbH &

Co. KG, Germany), and a soft moulded thermoplastic for orthopaedic splinting was used. Two 3D printed mounts for interfacing between the thermoplastic brace and the load cell, and between the load cell and the robotic endpoint were used. Each test trajectory shown was done with and without a subject to demonstrate the impact of the subject’s arm on the ability of the robot to track the reference trajectory. Conducting tests without a subject also allows for confirming that the robot would be capable of completing the reference trajectory. Once this initial run has been conducted, the subject was positioned at the previously set starting pose. Various trajectories have been tested (see fig. 5). Max strains expected during these trajectories and the total expected accumulated strain are shown in Table I. These expected strains using the weighted A* approach are compared to trajectories planned by the conven tional A* algorithm and generally show lower total path strains throughout. Weighted A* and conventional A* approaches are denoted as W-A* and C-A* respectively in I.

TABLE I

WEIGHTED A* VS. CONVENTIONAL A*: MAX AND ACCUMULATED STRAINS

Test Max C-A* Max W-A* Total C-A* Total W-A*

1 0.64 0.45 11.79 10.47

2 0.93 0.93 23.10 22.29

3 0.82 0.82 24.38 22.66

4 0.98 0.98 22.38 19.38

As shown in Table II deviations were not significant enough to substantially increase the strain experienced by the subject during the trajectory and did not result in the subject under going any large peak strains at any point during the exercise. No large reaction forces were measured during any tests. In general, these forces stayed at or below the 40 N force ex pected from the relaxed weight (approximately 4kg) of the subject’s arm.

TABLE II

ACTUAL MAX AND ACCUMULATED STRAINS

Test Max Weighted Actual Total Weighted Actual

1 0.42 9.00

2 0.92 18.73

3 0.85 21.94

4 0.74 13.28

Further example

This further experiment follows at least partly the previously mentioned experiment. Framework To enable the robot to physically interact with the patients and prevent them to acci dentally re-injure themselves, a custom arm-brace is mounted on the robot end-effector and worn by the patient, see Figure 7. The brace, shaped like an “L”, limits the movement of the patients and allows to know the position in space of their elbow, since this is just translated by a known quantity with respect to the end-effector pose x ee GR 6 , measured by the robot’s encoders. Inventors conveniently define the following frames:

• shoulder frame: centered on the humeral head of the patient, its x axis lies in the frontal plane of the patient body and points towards the left shoulder, while the z axis is parallel to the sagittal plane and pointing upwards;

• arm frame: centered on the elbow joint, the x axis points towards the wrist and the z axis towards the shoulder;

• arm brace frame: aligned with the arm frame. The arm is mounted so that the z axis of the robot end effector and the z axis of the arm brace frame coincide.

During the experiments, participants are instructed not to move their torso, so that in ventors can consider the shoulder frame to remain fixed over time. In this way, it is possible to establish a constant transformation that links the state of the glenohumeral joint to the po sition and orientation of the arm frame. The system is therefore always aware of the biome chanical condition of the patients, who are either free to explore their shoulder’s range of motion (RoM) autonomously or are guided in the rehabilitation exercises by the physiothera pist through teleoperation.

Biomechanical Model and Safety

In order to suitably represent the patient with a biomechanical model, inventors lever aged the shoulder model, as it describes the right shoulder joint with a high degree of fidel ity, including all the four rotator cuff muscles that inventors are interested in: subscapularis, teres minor, supraspinatus and infraspinatus. By means of the open-source musculoskeletal modelling tool OpenSim inventors simulated the full RoM of the human shoulder and com pute the strains of the rotator cuff muscles for various human poses. In particular, inventors focused on the value of the induced tendon strain level e, that is indicative of the risk of tear ing or re-injury for the muscle, and is defined as a dimensionless number comprising the ten don length and slack length (i.e., the length at which the tendon starts to generate a resistive force to stretch) respectively.

As previously mentioned, inventors restrict their analysis to the influence that the 3 DoF of the human glenohumeral joint have on the resulting strain level, thus effectively de fining the shoulder state vector as a = [AR PE SE]

(AR is the axial rotation degree of freedom (dof) of the (glenohumeral) joint, PE is the plane of elevation angle (dof) and SE is (shoulder) elevation angle (dof) for the same (gleno humeral) (3 dof) joint) where the values are limited to be - 90° < AR < 90°, - 20° < P E < 160° and 0° < SE < 144° to include only feasible poses. The humerus of the model was posed in all combinations of the 3 shoulder state variables, using 4° increments, and all strain maps were precomputed to allow to adjust the strain maps in real-time based on the position of the patient. For a given value of a, inventors group the strain levels into a single metric consisting of the maximum strain experienced among the rotator cuff muscles. By selecting an appropriate value of strain as a threshold, inventors can classify as “safe” or “unsafe” each pose of the human arm, thus minimizing the risk for re-injury during the rehabilitation exercises.

For the strain maps to be integrated in the robot control, inventors break down the 3- dimensional space in which a is defined into 2-dimensional maps (where AR is fixed): these are referred to as strain maps in the current work. When AR changes, a new map is consid ered. For each one of them, inventors define unsafe clusters (or zones) where the strain level is above the safety threshold, see Figure 9. First, an equivalent map is retrieved by retaining just the points {PE,SE} whose strain is greater than the threshold. As a second step, the un safe points are clustered using the Density -Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm, which is chosen because of its ability to cope with an unknown (but bounded) number of clusters, its capability of identifying non-spherical shapes and its low computational cost. These characteristics are desirable since they ease the pre-computa tion of the clusters among the strain maps without need of supervision. When the unsafe zones are found, they are approximated as minimum-volume enclosing ellipses (MVEEs). This representation of the unsafe zones is chosen since it allows for a more efficient compu tation of the control references (see II-C), at the acceptable cost of constraining the allowed RoM to be possibly more conservative, but still safe.

Strain-Map-Based Impedance Control

To integrate the strain-map-based safety into the robot control in real-time, at every timestep t inventors find the current shoulder state vector at; leverage the strain maps to per form biomechanical safety check; and send commands to the robot.

A Cartesian impedance controller is used to control the interaction force exerted by the robot on the patient: where Fi mp G R 6 is the interaction force vector acting from the robot to the patient, K , D e R 6x6 are the desired stiffness and damping matrices in Cartesian space, x ee , x ' ee GR 6 are the reference pose and velocity for the end-effector. While K is defined to prescribe the desired Cartesian and rotational stiffness, D is obtained using the double diagonalization design technique. The references for (1) are adjusted according to a safety check based on at, where inventors assess whether the shoulder state lies in an unsafe zone in the current strain map (where strain values depending on PE and SE are collected, assuming to fix AR). If this is the case, the closest safe point lying on the elliptic contour of the zone is estimated in real time with Brent’s approach and set as a reference point. To ensure that the patient is readily accompanied to the new reference position, the impedance control stiffness matrix is set to high stiffness (KM), and the damping matrix is adjusted accordingly. This results in a control force Fi mp that pushes the patient perpendicularly towards the zone boundary, ensuring mini mal permanence in the zone itself. Otherwise, if at does not correspond to an unsafe strain, the reference point is set to the current coordinates (PE t , SE t } and a lower stiffness matrix is used (K ). Similarly, a second safety check is concerned with the problem that, even if at is safe in the current strain map, slight changes in the AR value could make it unsafe in an adjacent one (by changing the value of AR also the unsafe zones can potentially shift). This check only affects the AR angle, and spans the third dimension of the shoulder state vector space. In the case in which AR is approaching a value that would entail excessive strains, the impedance control parameters K and D are set to high values like in the previous case. This results in a torque on the z-axis of the robot end-effector that guides the subject towards the safe strain map. With the control methodology formulated above, the patients can freely ex plore the RoM of their glenohumeral joint, while the robot implements quantitative knowledge of their internal safety, expressed in terms of rotator cuff muscle strains. It is noted that AR is a quantity that may vary continuously, while the strain maps are precom puted with a constant discretization of it. This could mean that the patient will likely not be exactly on one strain map, but between two adjacent ones. Therefore, smooth transitioning between them is guaranteed by calculating the closest safe points for both maps and retriev ing the correct reference point by linear interpolation as a function of AR (see Figure 9 (d)).

Tele-Physiotherapy through Haptic Device

In order to demonstrate the usefulness of their setup, inventors implemented a second modality in which a patient can receive therapy from a professional physiotherapist (PT) by leveraging the metric of biomechanical safety introduced in II-B in terms of strains. In this case, inventors allow for a shared-authority control of the patient movement, which is facilitated by the use of a haptic device with which the PT could interact with the patient.

The interface that inventors present makes it possible for the PT to manually move the pa tient’ s elbow through the Sigma7 gripper, while feeling a feedback force that informs the PT about whether the patient is moving in a safe or unsafe manner based on a. In fact, the move ment of the haptic device is transferred to the Kuka end-effector directly, but the haptic de vice RoM is limited on the basis of the value of a, so the PT experiences a repulsive force field in case an unsafe zone is being entered. The feedback force that the PT experiences can be formalized as follows: where the first term implements biomechanical safety as explained above. The second term introduces a force proportional to the error between the commanded pose for the robot and its actual pose is reflected in a feedback force for the PT, so that eventual movements or resistances on the patient side are also felt by the PT. The last term in represents a drag force depending on the velocity of the commanded movement that is generated, to prevent the PT to unwillingly generate excessively fast movements during the therapy. RESULTS

In this section inventors present their further results. One healthy individual acts as subject/patient for the following experiments. Strain maps are generated and divided into safe and unsafe zones by selecting the strain threshold to be 2.4%. A first simpler case is demonstrated, where the subject interacts autonomously with the robot, but the axial rotation of the arm is locked. This case is extended by leaving the patient-robot interaction uncon strained. In both cases the subject is sitting on a normal chair, and the robot is moved to its initial position so the custom arm brace described above can be worn comfortably. The movements that are performed are projected on the strain map that better represents the cur rent state of the subject’s shoulder, and a screen is used to give visual information allowing safe exploration of the shoulder range of motion. Visualization data are updated at 30 Hz, while the control loop runs at 200 Hz. Inventors then integrate the Sigma7 in the experi mental setup to test their teleoperation system. First, inventors consider the case in which the subject movement is constrained to lie on one strain map only. By locking the rotation of the end-effector about its z axis, the resulting trajectory is visualized on a single strain map, since only PE and SE can change. In such a way, interpretation and discussion of the results become possible in a single 2D plot, and the reader can familiarize with how to follow more complex cases as well. The references for the impedance controller formulation are com puted in real time, depending on the interaction with unsafe zones. A selected trajectory of 16 seconds is visualized, during which the patient interacts with one unsafe zone during its exploration of the rehabilitation space: when the current shoulder state at becomes unsafe, the reference is shifted on the closest point on the border of the ellipse, producing robot tor ques that facilitate the user to navigate again in a safe region. The positional and rotational stiffness for the controller was tuned.

Free interaction results

A patient-led robotic-assisted rehabilitation is given. No constraints are prescribed on the motion of the shoulder by the subject, so that a can freely vary inside their feasible bio mechanical range of motion. In the reported results, the movement happens through 8 differ ent strain maps: the trajectory travelled on each one and the strain map themselves.

Teleoperation results

The test reported here for the use of strain-map-based safety proves the effectiveness of the teleoperation approach. One of the authors impersonate the PT operator, and drives the movements of the subject’s elbow. To demonstrate different scenarios, the movement starts in the safe-strain region, and then the patient is driven towards an unsafe zone. As the initial motion is slow and safe, the operator does not feel significant feedback force. When the un safe zone is hit, the operator experiences a repulsive force pointing towards the safe region. For illustration purposes, the operator navigates briefly inside the unsafe region, and then drives the subject away from it. At this point, the subject was instructed to resist the end-ef fector force, so the reference trajectory and the actual one diverges, and this generates feed back force for the operator. Summary

The results inventors showed represent a step forward in the biomechanical-aware safe interaction between robots and humans. It is now possible to deliver sophisticate physi otherapy by employing collaborative robots that, even if are not conceived primarily for this aim, can be turned into reliable physiotherapy tools when controlled correctly. Moreover, they natively grant safety features for human-robot as they respect rigorous standards, which is a great benefit in comparison with custom-built rehabilitation robotic platforms or exo skeletons. Inventors showed that the strain maps that inventors elaborated are suited for a pa tient-led therapy that could be useful to assist during e.g. the shoulder rehabilitation process. Strain tissue maps allow also to receive more expert treatment in the case in which a profes sional physiotherapist interacts with the robot. The approach inventors presented enables them to visualize in real-time the quantitative strain that the therapy is generating in the pa tient’s muscles, possibly leading to a more efficient therapy overall. The way in which the PT interacts with the patient relieves them from physical fatigue, since it is the robot that mainly supports the weight of the patient arm.