CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to US provisional patent application no. 62/572, 167, filed on October 13, 2017, incorporated herein by reference in its entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR

DEVELOPMENT

This invention was made with government support under Grant No. EBO 19473, awarded by the National Institutes of Health. The Government has certain rights in the invention.

BACKGROUND OF THE INVENTION

The advent of minimally invasive surgery (MIS) led to significant benefits for patients at a cost of increase technical difficulty for surgeons. Robotic minimally invasive surgery (RMIS) was introduced to help eliminate some of the outstanding challenges by introducing

improvements such as enhanced 3D vision and additional degrees of freedom. Unfortunately, RMIS resulted in a complete loss of haptic feedback, a problem that has persisted even after more than a decade of technology development.

The loss of haptic feedback has been one of the most persistent issues plaguing robotic surgical systems. Despite the many improvements in robotic surgery, a complete and effective haptic feedback system remains out of reach. This is in large caused by technical challenges involving force sensing and actuation. Since robotic controls which the surgeon interacts with have no mechanical connections to the robotic instruments that are in contact with the tissue, any effective feedback system must not only be able to sense all relevant forces on the instruments, but also convey that information to the surgeon through a simulated touch.

Despite the lack of haptic feedback in laparoscopic and robotic surgery, these surgical techniques have been in use for many decades. Therefore, it is critical to understand the clinical impact and limitations that lack of haptic feedback has placed on laparoscopic and robotic surgeries. During open surgery, the availability of tactile sensation allows surgeons to manipulate soft tissue without applying excessive forces. In laparoscopic and robotic surgery, metal instruments are constantly grasping and manipulating soft tissue without the surgeon being able to easily perceive the extent of forces being applied. Research has shown that these forces can often be excessive and to lead to instances of tissue crush injury, particularly in less experienced surgeons (Wottawa CR, Investigation into the Benefits of Tactile Feedback for Laparoscopic, Robotic, and Remote Surgery. University of California, Los Angeles, 2013). In fact, excessive grip force is not the only performance related impact. Research has also shown that other aspects of the operation such as number of errors and total time to completion are impacted as well, leading to higher overall costs (Enayati N et al., IEEE reviews in biomedical engineering, 2016, 9:49-65; Okamura AM et al., Robotics research, 2010, 361-372; Westebring-Van Der Putten EP et al., Minimally Invasive Therapy & Allied Technologies, 2008, 17(1):3-16).

Application of excessive force is not always just limited to grip forces. Excessive shear forces resulting from unnecessarily large and/or rapid displacements of the robotic arms can also lead to injuries such as tissue tearing and even suture breakage during knot tying tasks (Reiley CE et al., The Journal of thoracic and cardiovascular surgery, 2008, 135(1): 196-202).

Another limitation that arises from the loss of haptic feedback is the loss of palpation as a means of locating various structures such as nerves, vessels and even tumors (Pacchierotti C et al., The International Journal of Robotics Research, 2015, 34(14): 1773-1787). Normally, palpation allows the surgeon to easily identify changes in tissue softness/hardness and therefore find various structures in soft tissue even if they are not visually identifiable. Palpation in robotic and even laparoscopic surgery can prove quite challenging since the surgeon is forced to rely purely on visual cues for identification of tissue characteristics.

Despite the growing popularity of robotic surgery, many critics have long referenced the rate of conversion of robotic procedures to laparotomy, with conversion rates ranging from 1% - 30% depending on patient comorbidities, procedure type and operator skill. Some of these conversions which can result from tissue injuries, suture breakage and operator error can be completely avoided if sense of touch could be restored through implementation of a haptic feedback system.

Thus, there is a need in the art for improved haptic feedback in robotic systems. The present invention addresses this need. SUMMARY OF THE INVENTION

In one aspect, a grasper feedback system comprises a chamber capable of holding a pressure, the chamber positionable in a hinged controller, a pressure regulator having a first port and a second port, the first port fiuidly connected to a pressure source, and a dual-input solenoid valve having a first input, a second input, and an output, the dual-input solenoid valve having a first configuration that fiuidly isolates the first input and fluidly connects the second input to the output and a second configuration that fluidly isolates the second input and fluidly connects the first input to the output, wherein the first input is fluidly connected to the second port of the pressure regulator, and the second input and the output are fluidly connected to the chamber. In one embodiment, the first configuration holds a first pressure in the chamber and enables the pressure regulator to prepare a second pressure in the fluid connection between the second port and the first input. In one embodiment, the second configuration transfers the second pressure to the chamber. In one embodiment, the hinged controller opens and closes to remotely open and close a robotic grasper. In one embodiment, the pressure in the chamber is adjustable to vary the resistance of closing the hinged controller, such that the resistance simulates a robot grasper gripping pressure around an object. In one embodiment, the pressure source comprises a pressurized gas or liquid. In one embodiment, the system further comprises at least one tactile feedback system comprising a gripping surface positioned on the hinged controller, the gripping surface having at least one aperture, a membrane capable of expanding under pressure, the membrane positioned below the aperture, and a pressure source positioned below the membrane, wherein the membrane is expandable through the aperture and beyond the gripping surface while under pressure from the pressure source.

In one embodiment, the tactile feedback system has a zero pressure setting, wherein the membrane is unexpanded. In one embodiment, the tactile feedback system has a first pressure setting, wherein the membrane is expanded through the aperture to match the level of the gripping surface. In one embodiment, the expansion of the membrane through the aperture beyond the gripping surface is adjusted to simulate gripping pressure of a robot grasper remotely controlled by the hinged controller. In one embodiment, the pressure source comprises a pressurized gas or liquid. In one embodiment, the system further comprises at least one vibrating motor. In another aspect, a shear sensor comprises an outer shell having a perimeter enclosing a central space, a movable member positioned within the central space, and at least two flexible piezoresistive sensors positioned opposite from each other on the perimeter facing the central space, wherein the movable member touches the at least two flexible piezoresistive sensors at rest. In one embodiment, the at least two flexible piezoresistive sensors each comprise at least one curved region. In one embodiment, the at least one curved region of the at least two flexible piezoresistive sensors each curves around an adjacent corner of the movable member.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing purposes and features, as well as other purposes and features, will become apparent with reference to the description and accompanying figures below, which are included to provide an understanding of the invention and constitute a part of the specification, in which like numerals represent like elements, and in which:

Figure 1 is a diagram of two forms of haptic feedback;

Figure 2 is an exemplary robotic surgical forceps and trocar;

Figure 3 is an exemplary shear force sensor;

Figure 4 is a diagram of a shear force sensor according to one aspect of the present invention;

Figure 5 is an exemplary tactile feedback actuator;

Figure 6 is a diagram of a pneumatic feedback system according to one aspect of the present invention;

Figure 7 is a partial diagram of a pneumatic feedback system according to one aspect of the present invention; Figure 8 is a graph of pressure over time from an electro-pneumatic valve;

Figure 9 is a flow diagram of a pneumatic feedback system according to one aspect of the present invention;

Figure 10 is an illustration of an experiment according to the present invention;

Figure 11 is a set of force graphs showing results of an experiment related to the present invention; and

Figure 12 is a graph of faults resulting from an experiment related to the present invention.

DETAILED DESCRIPTION

It is to be understood that the figures and descriptions of the present invention have been simplified to illustrate elements that are relevant for a clear understanding of the present invention, while eliminating, for the purpose of clarity, many other elements found in related systems and methods. Those of ordinary skill in the art may recognize that other elements and/or steps are desirable and/or required in implementing the present invention. However, because such elements and steps are well known in the art, and because they do not facilitate a better understanding of the present invention, a discussion of such elements and steps is not provided herein. The disclosure herein is directed to all such variations and modifications to such elements and methods known to those skilled in the art.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, exemplary methods and materials are described.

As used herein, each of the following terms has the meaning associated with it in this section.

The articles "a" and "an" are used herein to refer to one or to more than one (i.e., to at least one) of the grammatical object of the article. By way of example, "an element" means one element or more than one element.

"About" as used herein when referring to a measurable value such as an amount, a temporal duration, and the like, is meant to encompass variations of ±20%, ±10%, ±5%, ±1%, and ±0.1%) from the specified value, as such variations are appropriate.

Throughout this disclosure, various aspects of the invention can be presented in a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the invention. Accordingly, the description of a range should be considered to have specifically disclosed all the possible subranges as well as individual numerical values within that range. For example, description of a range such as from 1 to 6 should be considered to have specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within that range, for example, 1, 2, 2.7, 3, 4, 5, 5.3, 6 and any whole and partial increments therebetween. This applies regardless of the breadth of the range.

In some aspects of the present invention, software executing the instructions provided herein may be stored on a non-transitory computer-readable medium, wherein the software performs some or all of the steps of the present invention when executed on a processor. Aspects of the invention relate to algorithms executed in computer software. Though certain embodiments may be described as written in particular programming languages, or executed on particular operating systems or computing platforms, it is understood that the system and method of the present invention is not limited to any particular computing language, platform, or combination thereof. Software executing the algorithms described herein may be written in any programming language known in the art, compiled or interpreted, including but not limited to C, C++, C#, Objective-C, Java, JavaScript, Python, PHP, Perl, Ruby, or Visual Basic. It is further understood that elements of the present invention may be executed on any acceptable computing platform, including but not limited to a server, a cloud instance, a workstation, a thin client, a mobile device, an embedded microcontroller, a television, or any other suitable computing device known in the art.

Parts of this invention are described as software running on a computing device. Though software described herein may be disclosed as operating on one particular computing device (e.g. a dedicated server or a workstation), it is understood in the art that software is intrinsically portable and that most software running on a dedicated server may also be run, for the purposes of the present invention, on any of a wide range of devices including desktop or mobile devices, laptops, tablets, smartphones, watches, wearable electronics or other wireless digital/cellular phones, televisions, cloud instances, embedded microcontrollers, thin client devices, or any other suitable computing device known in the art.

Similarly, parts of this invention are described as communicating over a variety of wireless or wired computer networks. For the purposes of this invention, the words "network", "networked", and "networking" are understood to encompass wired Ethernet, fiber optic connections, wireless connections including any of the various 802.11 standards, cellular WAN infrastructures such as 3 G or 4G/LTE networks, Bluetooth®, Bluetooth® Low Energy (BLE) or Zigbee® communication links, or any other method by which one electronic device is capable of communicating with another. In some embodiments, elements of the networked portion of the invention may be implemented over a Virtual Private Network (VPN).

Embodiments of the present invention relate to various feedback modalities for conveying tactile information to the user of a surgical robot. In their most abstract form, feedback technologies in surgical robotics can be split into two categories: sensory substitution, and sensory augmentation through haptic feedback.

Sensory substitution is one of the most common methods for providing feedback.

Sensory substitution is the process of mapping or converting data originally aimed for one sensory modality to a different sensory modality. An example of sensory substitution is sensing normal forces applied to tissue from robotic graspers and displaying it to the user as visual stimuli, for example using a force vs. time plot. Feedback through sensory substitution can involve auditory, visual, electrical and even thermal stimuli. Extensive research has been done with regards to the effectiveness of sensory substitution and its ability to convey tactile information. In nearly all cases, particularly with regards to application in surgical environments involving complex tasks, it has been found that sensory substitution can lead to additional mental load and is less ideal that haptic feedback.

Haptics is focused on providing information to the user through a sense of touch. Its goal is to convey information to the user via a simulated touch, the same way that tactile information would normally be processed in the human neurosensory pathways. An example of this type of feedback is detecting normal forces applied to tissue from robotic graspers and providing normal force feedback on an operator's fingertips.

Haptics itself consists of two sub-categories of feedback, tactile and kinesthetic. The reason for this breakdown is that the sense of touch involves not only the mechanoreceptors in the skin and muscles, but also the proprioceptive and attention centers of the brain. Haptic feedback in devices of the present invention may therefore induce a haptic response either as a tactile feedback, i.e. as an induced mechanical force on the skin of a user, or alternatively as a kinesthetic response conveyed to the muscles, joints, or tendons of a user.

Figure 1 shows the primary mechanoreceptors involved in conveying the sense of touch. The sense of touch involves two major groups of mechanoreceptors, those in the skin and those in the muscles. In its most basic form, tactile feedback results from activation of

mechanoreceptors in the skin, while kinesthetic force feedback, which provides a sense of resistance, utilizes mechanoreceptors in the muscles. The data from these sensory receptors, when combined with the proprioceptive and attention centers of the brain, help control and fine tune muscular response, and drive the user's attention to critical events.

Development of methods for sensing forces applied to the tissue at the end effector of a robotic surgical system has been one of the most challenging problems for haptic feedback system, particularly those focused on applications in surgical robotics. This is largely due to size and biocompatibility constraints. Because of the small size of incisions in MIS, robotic and laparoscopic instruments are generally designed to move through trocars with diameters less than 12mm. This means that any sensor that is designed to be installed on either the end effector or the shaft of the tool must be significantly smaller than 12mm wide. In addition to size constraints, sensors positioned inside the body must be biocompatible and able to function within a wet environment without any impact on resolution and dynamic range. Finally, the range of forces applied by different instruments can vary significantly, from 0 - 5N for instruments such as the Cadiere forceps to more than 20N, for example with the da Vinci Prograsp forceps. Although exemplary embodiments are described herein in conjunction with particular surgical instruments, it is understood that devices and methods of the present invention are suitable for use with any surgical instruments, including but not limited to bipolar forceps.

With reference now to Figure 2, an exemplary robotic surgical forceps 201 is shown, in this example the forceps is a da Vinci Cadiere grasper, measuring 5mm x 14 mm. The same forcep is shown in a different view 202 with a pressure sensor 203 installed between the graspers. The size constraints are dictated at least in part by the need to fit through a trocar, for example the 12 mm trocar 204. In alternative embodiments, instruments of the present invention may be used with different sized trocars.

Improved Shear Sensor

One aspect of a device of the present invention relates to a capacitive sensor with a novel method of sensing shear force. In addition to normal force feedback, shear force feedback is very important in certain surgical operations, including but not limited to suturing, grasping, inspecting, and manipulating tissue. Without good shear force feedback in suturing for example, a user has to rely on visual cues to know when a suture was pulled tight. An example of a currently-available shear force sensor is shown in Figure 3. The shear sensor includes a movable plate 301 which has a rough top surface, which is slidably positioned in an enclosure having two normal force sensors 302 and 303. As the plate slides left or right, the normal force sensors are depressed or released, providing a single axis measurement of shear force.

In real world surgical conditions, uni-axial shear sensors are not sufficient to detect all the forces applied to a suture. This is because a suture is rarely ever pulled only in a single axis in which the sensor can detect shear. Therefore, higher-order shear sensors are needed for real- world clinical conditions. Another aspect of a device of the present invention includes a bi-axial shear sensor. This bi-axial sensor design relies on two peizoresi stive sensors, with a novel inner chamber of the sensor's outer shell and a differential technique to identify shear in two axes. An overview of the device is shown in Figure 4.

The sensor includes a slidable plate 401 with a rough top surface, and two curved piezo- resistive sensors 402 and 403. The depicted sensor is configured to measure shear force in both the X axis and the Y axis, using the two curved sensors. As shown in the exemplary graphs of sensor output, the shear force vector's X-axis component is measured based on the difference between the measurements recorded by the two piezo-resistive sensors. For example in 404, the reading from 402 is high, while the reading from 403 is low, indicating a high shear force in the negative X direction. Similarly, in 406, the reading from 403 is higher than the reading from 402, indicating a high shear force in the positive X direction.

The shear force vector Y-axis component is calculated based on the sum of the measurements from 402 and 403. Because the sensors 402 and 403 are curved, the total force sensed increases as the plate 401 moves in the negative Y direction. A higher sum of

measurements therefore results in a negative Y component of the shear force vector, while a lower sum of measurements results in a positive Y component of the shear force vector. In some embodiments, the plate 401 is in mechanical contact with both of the sensors 402 and 403 when at rest. In one embodiment, the plate 401 has a center resting position dictated by one or more springs holding it in place.

Pneumatic Tactile Feedback

Another factor that could potentially contribute to variation among subjects is sensory desensitization. Some subjects in previous studies reported that inflations of pneumatic balloons were difficult to notice, particularly after long periods of using them. This effect may be caused for example by desensitization of mechanoreceptors in the skin. Under normal circumstances, the sense of touch involves simultaneous retrieval of light touch and deep pressure sensors. This is because the skin is not normally in contact with any object and low pressure (i.e. light touch) contact is made only when some grasping action is initiated. This contact is then followed quickly by an increase in pressure, which in turn activates deep pressure sensors in the skin.

While normal force, tactile feedback actuators attempt to recreate this effect, in reality, these actuators only target slow adapting pressure sensors in the skin. This is because the flat design of the PDMS actuators causes light touch to be present at all times. With reference to Figure 5, an exemplary PDMS actuator is shown having a touch surface 501 and a pneumatic airway 502 through which fluid is flowed into or out of an internal chamber, fluidly connected to the membrane on the touch surface 501. When fluid flows into the chamber, the pressure in the chamber increases and bumps 503 on the surface of the membrane inflate, causing a tactile sensation. Conversely, when fluid flows out of the chamber, the bumps deflate, removing the tactile sensation. Depending on how tightly the subject is holding on to the robotic controls, even deep pressure sensors may remain activated. This effect can lead to adaptation in the mechanoreceptors of the skin and in turn, dampen the effect of the pneumatic actuators, particularly at lower pressures. Of course, this dampening effect varies among subjects due to different subject skin types, how often the subject lifts his/her hand off of the controls, and even the length and complexity of the task. Considering that real life surgical procedures can span many hours, haptic feedback systems (HFS) may therefore be ineffective in clinical settings, where prolonged use can lead to degraded

performance.

A pneumatic tactile feedback system of the present invention includes a multiplexed, 5- level solenoid valve array. The first pressure level was selected such that a subject would just barely feel the presence of the membrane once it was inflated. The remaining pressure levels were then chosen such that the difference between the different levels was noticeable. The last pressure level was configured such that it didn't cause discomfort due to high deformation of the skin. One exemplary set of tactile pressure levels is listed below in Table 1.

Table 1

Although certain pressure values are presented in Table 1 above, it is understood that pressure levels may vary based on pneumatic actuator size, material, user, procedure, or a variety of other factors. In some embodiments, one or more pressure levels may be adaptive, and may be set differently for different users, or may alternatively change based on an individual user's behavior during a procedure. Another aspect of the present invention relates to a pneumatic kinesthetic force feedback system, for providing a more natural sense of kinesthesia. In some embodiments, the system comprises a flexible pneumatic tube placed between the graspers of the control console, as shown in Figure 6. The graspers on the control console have a tube positioned between them, such that when the graspers are open 601, the tube 602 does not exert any force on the graspers, while when the graspers close 603, the tube 604 is compressed and is configurable to exert a variable opposing force based on the pressure in the tube. A high pressure (for example 19 PSI) in the tube will cause it to be more rigid, thus exerting an increased opposing force on the graspers as they close. Similarly, decreasing the pressure (for example 0 PSI) in the tube will cause the tube to become more pliable, thus decreasing the opposing force (kinesthetic feedback) on the graspers as they close.

Advantageously, pneumatic kinesthetic feedback systems of the present invention do not require major modifications to the surgical console. Alternative kinesthetic feedback systems operate by fixedly attaching one or more vibration motors on the console. The disclosed system by contrast is an add-on solution, using 3D printed pneumatic actuators with a pneumatic tube positioned in between the graspers in order to simulate the presence of an object in between the graspers. One exemplary embodiment of a pneumatic actuator of the present invention is shown in Figure 7.

A conventional pressure regulation system for such a feedback system, wherein quantized levels of pressure are needed, requires a multiplexed solenoid valve array with each actuator needing as many solenoid valves as the levels of pressure desired. Emulating a continuous pressure regulation system with such an architecture would require far too many solenoid valves, making it prohibitively costly and cumbersome. A more compact pressure regulation system with a high number of pressure levels (i.e. simulating continuous pressure regulation), is advantageous for two reasons: (1) large pressure changes can reduce user performance, particularly in a kinesthetic feedback where sudden changes in feedback can result in the user releasing the grasper and (2) a user-based, adaptive feedback requires pressure levels to be variable and changeable electronically.

Existing systems include electro-pneumatic pressure regulators; however, these systems generally have slow response times and produce significant vibration during pressure changes. To resolve these issues, one embodiment of the present invention includes a dual-valve continuous pressure regulation system to provide continuous pressure regulation with low latency. In one embodiment, a dual-valve pressure regulation system comprises one or more electro-pneumatic valves, for example a Asco Sentronic D valve, placed in series with an SMC solenoid valve. The electro-pneumatic valve may use a direct acting proportional coil to control output pressure. The benefits of this type of electro-pneumatic valve is the stable pressure output that it can provide. The stable response, however, comes at the expense of system response times. System response time can be reduced, but reducing the response time comes at the expense of large pressure overshoots when pressure levels are being changed. The electro- pneumatic valve output pressure may be controlled for example by an analog voltage level. In one embodiment, the output pressure is controlled by providing a voltage of 0 - 10V to an analog control pin. One disadvantage of using an electro-pneumatic valve is that underdamped control systems can result in large pressure overshoots. An example of this phenomenon is shown in Figure 8, where a large peak 801 shows where the internal pressure momentarily, but significantly, exceeds the set point 802. One embodiment of a system of the present invention includes a dual-input solenoid valve in series with an electro-pneumatic valve, configured to absorb the pressure peaks while still delivering fast response times in a compact package. A diagram of an exemplary system is shown in Figure 9. Electro-pneumatic valve 901 has input port 902 and output port 903. Input port 902 may be connected to a fixed pressure source 904, for example a pressure source configured to deliver at least the maximum pressure desired in the system. Output port 903 is connected to one input of two-input solenoid valve 905, which has first input 906, second input 907, and output 908. In the depicted embodiment, second input 907 and output 908 are fluidly connected to one another and to pneumatic tube 909, used for kinesthetic feedback. The two- input valve is switchable in the depicted embodiment such that in a first configuration, the first input port 906 and the output port 908 are fluidly connected, with the second input port 907 isolated, while in a second configuration the second input port 907 and the output port 908 are fluidly connected and the first input port 906 is isolated.

The valves and/or pressure source may be controlled, for example by a microcontroller, embedded computer, or other computing device electrically or communicatively connected to the valves. In some embodiments, the system comprises one or more pressure sensors also connected to the controller to provide feedback to the system. In some embodiments, the two-input valve and the electro-pneumatic valve each have their own controllers and/or pressure sensors, while in other embodiments they are connected to the same controller. In some embodiments, multiple two-valve systems may be controlled by a single controller.

In one configuration, the device of Figure 9 typically maintains the solenoid valve 905 in the first configuration, wherein the first input 906 is fluidly connected to the output 908, and second input 907 is isolated. This has the effect of feeding the regulated pressure from the electro-pneumatic valve 901 to the pneumatic tube 909. When the system needs to change the pressure of in the pneumatic tube 909, the solenoid valve 905 first switches to the second configuration, isolating the first input 906 and fluidly connecting the second input 907 and the output 908. This has the effect of isolating the pressure in the pneumatic tube from the rest of the system. The electro-pneumatic valve 901 is then actuated to change the pressure to the desired level, but any pressure spike is isolated from the pneumatic tube 909 by solenoid valve 905. Once the system detects that the pressure is stabilized at the first input port 906 (for example using a pressure sensor, or alternatively after a fixed or variable amount of time has passed), the solenoid valve 905 switches back to the first configuration, fluidly connecting the output port 908 (and also the pneumatic tube 909) to the first input port 906, now at the updated pressure.

In certain configurations, the system of Figure 9 is capable of stable pressure output with minimizing response time. In one embodiment, the system response time is less than 70 ms. In another embodiment, the average system response time is about 65 ms. In another embodiment, the system response time is less than 50 ms.

EXPERIMENTAL EXAMPLES

The invention is further described in detail by reference to the following experimental examples. These examples are provided for purposes of illustration only, and are not intended to be limiting unless otherwise specified. Thus, the invention should in no way be construed as being limited to the following examples, but rather, should be construed to encompass any and all variations which become evident as a result of the teaching provided herein.

Without further description, it is believed that one of ordinary skill in the art can, using the preceding description and the following illustrative examples, make and utilize the system and method of the present invention. The following working examples therefore, specifically point out the exemplary embodiments of the present invention, and are not to be construed as limiting in any way the remainder of the disclosure. Pneumatic Kinesthetic-Tactile Hybrid Feedback System

The investigation into the pneumatic kinesthetic-tactile feedback system was based on the assertion that this bi-modal feedback solution provided a better method for simulating the sense of touch. The development of the pneumatic kinesthetic force feedback solution is described in greater detail above. The goal of this investigation was to evaluate the pneumatic kinesthetic- tactile feedback system in reducing grip forces during robotic surgical tasks involving tissue grasping.

The evaluation of this hybrid feedback system was conducted as part of two independent studies, each designed to provide a better understanding of the impact that this multi-modal HFS would have on RMIS tasks involving tissue manipulation.

The goal of the experiment was to evaluate the kinesthetic-tactile HFS alongside an existing kinesthetic-tactile feedback system. More specifically, this experiment aimed to not only evaluate the impact of the multi-modal HFS on grip forces, but also the kinesthetic force feedback as a single-modality solution.

The control system for the pneumatic kinesthetic-tactile hybrid HFS was implemented by configuring the logic engine of the Haptics Manager software to utilize the dual-value continuous pressure regulators as a means of changing the pressure in the pneumatic tube responsible for providing kinesthetic force feedback. A total of 15 novice subjects with little to no experience with robotic surgery were recruited to perform single handed peg-transfer tasks using a da Vinci IS 1200 surgical system. With reference to Figure 10, the subjects were asked to use robotic forceps 1003 to pick up a peg 1002 from a post 1001 and drop it on another post at the opposite end of the field. Subjects were given a 2-minute training period prior to the start of the study to familiarize themselves with the robotic system. This training period was sufficient in most cases since the use of clutch, camera and most other complex da Vinci operations was not allowed. In order to eliminate any bias toward either hand, nearly half (7) of the subjects performed the feedback on the right hand while the remaining received feedback on the left hand (8). All subjects were right handed. Each subject was asked to perform four peg transfers during each trial. To eliminate any bias toward the position which a subject may drop the peg, if the subject dropped the peg, the proctor would quickly reset the peg to its original position.

For this study, a FlexiForce B201 sensor 1004 was installed on da Vinci Fenestrated Bipolar forceps. Each subject performed the peg transfer tasks, four separate times as part of four trials, each performed under different feedback conditions: (1) No Feedback (2) Normal Force Tactile Feedback (3) Pneumatic Kinesthetic Force Feedback (4) Hybrid Kinesthetic-Tactile Feedback.

During the trial, the number of faults (i.e. number of times the subject dropped the peg), time-to completion, and the grip force were recorded. Statistical analysis for average grip force was performed using Repeated Measures ANOVA after a Log2 transform which was used to meet the normality assumption. Repeated Measures ANOVA was also used for analysis of peak grip force. Statistical analysis for the number of faults was conducted using Ordinal Repeated Measures ANOVA. Results

The results (see Figure 11) show that the average grip force is significantly lower compared to the no-feedback condition when tactile feedback (p = 0.017), kinesthetic feedback (p = 1.66E-6) or hybrid feedback (p < 1.0E-16) are provided. The bi-modal kinesthetic-tactile HFS also performs better than both tactile-only (p = 5.64E-8) and kinesthetic-only (p = 0.0027) feedback conditions. With regard to the peak grip force, no significant improvement can be seen between the tactile feedback and the no feedback conditions. However, both kinesthetic (p = 0.0008) and hybrid HFS (p = 0.0001) conditions display a significant reduction in peak grip force.

The number of faults (Figure 12) appears to be significantly higher with the hybrid HFS activated, compared to the tactile feedback group (p = 0.012). No significant difference exists between the other conditions.

The results from the peg transfer study show the clear benefits of providing haptic feedback in reducing grip force. All feedback modalities performed significantly better than the no feedback condition. Kinesthetic feedback alone showed significant benefits in reducing the average grip force. Most importantly, there is also a clear indication that the multi-modal kinesthetic-tactile feedback system is significantly better than both single-modality feedback solutions, benefiting from the synergistic relationship from the activation of mechanoreceptors in the skin and muscle. This leads to a more natural sense of touch, allowing the subjects utilizing the system with multi-modal HFS to apply forces nearly 50% less than when no feedback is present.

An interesting observation from the study that is worth discussing is the larger number of faults (i.e. peg drops) in the bi-modal HFS compared to the tactile-only condition (p = 0.012). Based on the large standard error mean, it is clear that there is also significant variation among subjects. The reason for this variation is actually the way the pneumatic kinesthetic feedback functions. The high pressures used for higher feedback levels makes compressing the grasper quite challenging due to increased resistance. When this resistance is coupled with a high tactile feedback level, it appears to result in the subject suddenly relaxing the hold on the graspers and often dropping the peg. This variation is more of a learned behavior which is most likely caused by the lack of experience with the feedback system. Even though some training with the haptic feedback system can eventually eliminate this behavior (as we observed in a series of follow up bench tests), the correct way to ultimately deal with this issue is to develop an adaptable feedback system. Such a feedback system can learn from the user's behavior and automatically lower the pressure levels to help reduce resistance and hence the number of peg drops.

The disclosures of each and every patent, patent application, and publication cited herein are hereby incorporated herein by reference in their entirety. While this invention has been disclosed with reference to specific embodiments, it is apparent that other embodiments and variations of this invention may be devised by others skilled in the art without departing from the true spirit and scope of the invention. The appended claims are intended to be construed to include all such embodiments and equivalent variations.