Technical field of invention

This invention relates to a control system for an endoscopic device, a control method for an endoscopic system and an endoscopic system comprising the control system and an endoscopic device. More specifically the invention relates to an endoscopy control system that enables one or more control signals to be acquired without use of the hands.

Prior art

Today's technologies for surgery have reached a very high level of precision and accuracy, bringing considerable improvements in terms of both results and comfort for the operators using them.

In particular, in surgery based on endoscopic visualisation techniques, both flexible and rigid or semi-rigid endoscopes are in common use. As the name suggests, the two types differ in mechanical properties, which do or do not allow the endoscope to adapt to some extent to the conformation of the part of the body under examination. Flexible endoscopes are often characterised by openings that enable the operator to introduce other instruments and perform surgical procedures; while rigid endoscopes, although generally of high precision, are normally limited to image acquisition.

In general, semi-rigid endoscopic devices are fitted with an articulated operating end (an operating segment at the head of the endoscope) which can perform movements in different directions when controlled by appropriate actuators connected to the other end of the endoscope. The operating end is also equipped with image acquisition means such as a pair of cameras.

The present invention is particularly suitable for the control of endoscopic devices having an operational image acquisition end whose movement can be controlled in (about) at least three axes of rotation, conventionally defined as the X, Y and Z axes. Conventionally, in the technical sector in question, rotation about the X axis identifies a so- called "pan" or "pitch” movement, rotation about the Y axis identifies a so-called "tilt" or "yaw" movement and rotation about the Z axis identifies a so-called "roll" movement. In the jargon these three axes of movement are therefore generally called the pitch, yaw and roll axes respectively.

One example of such endoscopy devices is the semi-rigid Endoeye Flex 3D endoscope from Olympus. When using this endoscope the operator has to operate movements of the flexible segment at the head of the endoscope using appropriate mechanisms at the opposite end while looking at a monitor. This means that the operator deciding how to direct the endoscope inside the patient will always have at least one hand engaged.

Viewers that reproduce a virtual reality in response to movements of the person wearing the viewer are also known. Tracking systems that capture movements of the viewer and accordingly modify the images that can be seen inside it are also known. The effect of this technology is to immerse the individual in a three-dimensional reality that he or she can only see when wearing the viewer.

Italian Patent No. 102015000055827 describes an endoscopy device including a wearable viewer, a processor and an endoscope. The wearable viewer is equipped with a movement tracking system and is connected to a processor that receives position information in relation to a given reference or horizon. The tracking system is based on the transmission/reception of a three-axis spatial location system (accelerometer, gyroscope and magnetometer).

The viewer transfers position information to a processor via cable or wireless communication and the processor transforms the position information into a motion control signal at the operating end of the endoscope, resulting in hands-free control.

The viewer is also equipped with a limb recognition system, also known as Leap Motion, which can be used to switch the display of the images reproduced by the viewer between images captured by the endoscope and those captured by cameras located on the outer surface of the viewer, thus enabling the operator to switch between an endoscopic image display and a display of the surrounding reahty without removing the viewer.

This solution, although functional, has many problems, including the fact that it is still very dependent on use of the hands to control the display, the manner in which the endoscope is controlled, and the fact that the control system is not very safe and reliable, so inadvertent movements of the head and/or hands can result in unwanted control signals for the endoscope and/or the viewer.

It is however desirable, particularly for image acquisition endoscopes for surgical use, that the operator has hands that are as free as possible to operate other surgical instruments; it is also desirable the safety of known control systems for endoscopic devices should be improved.

This raises the technical problem of creating a control system for endoscopic devices that solves or at least attenuates one or more of the aforementioned problems of the known art.

In the context of this problem it is a particular object according to the present invention to provide a control system that is as reliable as possible, can be used and adapted to surgical applications, in which the surgical operator may need to operate other devices or operate directly in the area framed by the endoscope.

It is also desirable that the control system should make it possible to operate an endoscope with improved safety.

In the context of these problems it is also desirable that the control system should be versatile, easy and economical to implement, install and use, and/or be adaptable to the different types of endoscopic devices that are commercially available.

Summary of the invention

According to this invention these results are achieved through a control system for an endoscopy device according to claim 1.

Such a control system for an endoscopy device (10) therefore has first hands-free means for acquiring a control signal on the basis of which movement of the operating segment of the endoscope is controlled on the basis of tracking the movements of the operator's head, and at least second hands-free means for acquiring at least one further control signal that does not require the use of hands, and are capable of acquiring one or more of: a control signal to control blocking or enabling of control of movement of the operating endoscope based on the tracking of head movements, and/or a signal for switching the display of images on the viewer between images from the environmental image acquisition means and images from the endoscopy device, and/or a zoom control signal for an image acquired from the acquisition means of the endoscopy device or an environmental image. This solution therefore makes it possible to release the hands from control of one or more important functions, resulting in improved safety and reliability, as well as versatility for use in surgical applications.

The second hands-free acquisition means may include at least one or more pedals and/or a voice command acquisition system. The use of pedals is particularly advantageous because it allows the surgical operator's voice to be freed from control of the endoscope and image display, but use in combination with a voice acquisition system may be advantageous for maximum versatility, for example when switching between images from the endoscopic device and environmental images.

In particular, it is preferable to provide at least one or preferably at least two pedals with graduated actuation to acquire a corresponding control signal which is proportional to a pressure force exerted on the pedal. These pedals may advantageously be used to adjust the amount of zoom for images from the endoscope acquisition means and/or environmental image acquisition means. This solution, which has never before been envisaged, is optimal for applications of a surgical nature.

One or more pedals can also be provided for the acquisition of a switching signal and, preferably, an operable part of each switching pedal is shaped with an external surface that differs in shape from that of one or more other control signal acquisition pedals. The operator can therefore easily recognise the different pedals by means of his own foot, so safety is further increased.

The switching pedals may preferably acquire a switching signal for the display of images on the viewer between images from the environmental image acquisition means and images from the endoscopy device and/or a control signal to command the disabling or enabling of control of movement based on the tracking of head movements. According to a particularly preferred embodiment of the control system, the processing unit (110) is configured to automatically control disabling or enabling (s-lk) movement control of the operating end of the endoscopy device based on tracking the movements of the operator's head when it receives and acts on a signal for the display of images on the viewer switching from/to the images from the endoscopy device to/from the images from the environmental image acquisition means.

This solution results in maximum system safety and reliability for surgical use, in particular if used in combination with pedals for graduated control of the amount of zoom, as it allows the surgical operator to change from images of the operating site (possibly at maximum magnification) acquired from the endoscope to environmental images (or vice versa), with maximum safety, reliability and a single control signal, the operator thus being able to act promptly and in the most suitable surgical manner if, for example, in the course of surgery it is necessary to change the manner of operation or the viewing point for the site being operated on.

To further improve the safety and reliability of the control system, the processing unit is preferably configured to exclude a control signal coming from one of the second hands-free means for acquiring a control signal when another of said second hands-free acquisition means is acquiring a corresponding control signal and/or until an action controlled by a corresponding control signal acquired from another of said second hands-free acquisition means has been completed.

A further object of this invention is an endoscopy system according to claim 13, a method for controlling an endoscopy system according to claim 14 and a computer program according to claim 15.

Brief description of the figures

Further features and advantages of the invention will be more apparent in the light of the detailed description of a preferred but not exclusive embodiment of the endoscopy device illustrated by way of a non-limiting example with the help of the following plates of drawings in which:

Figure 1 shows a block diagram of an endoscopy system according to the present invention;

Figure 2a is a front perspective view of a wearable viewer as part of the control system according to the present invention;

Figure 2b is a side view of the wearable viewer in Figure 2a;

Figure 3 Figure 2a is a front perspective view of one example of a pedal unit with pedals for the hands-free acquisition of control signals, forming part of a control system according to the present invention;

Figure 4 is a view in vertical cross-section of one of the pedals in the pedal unit of the endoscopy system in Figure 3 during use;

Figure 5 is a block diagram of an example of control logic that implements a method of control according to the present invention.

Detailed description of the invention

With reference to Figure 1, one embodiment of an endoscopy system according to the present invention comprises an endoscopy device 10 and a control system 100.

Endoscopy device 10 comprises an endoscope 11 with a head operating segment that carries means 11a for acquiring images Ill. Endoscope 11 may be a semi-rigid endoscope of the type currently known to those skilled in the art, which is substantially straight, of length varying from 30 to 50 cm, and a cross-section of variable shape averaging 10 mm.

The operating segment at the top of endoscope 11 is flexible, equipped with means 11a for the acquisition of images, preferably of the stereoscopic acquisition type, for example comprising at least one camera, preferably two cameras, positioned at the tip. If desired, one or more illumination devices, for example of the LED type, may be placed on the operating segment to illuminate the area framed by the acquisition means.

Movement of the operating segment can be controlled in at least three axes by means of a drive unit 12 of endoscope 11, connected to control system 100 and capable of operating the operating segment on the basis of drive signals sll generated from drive control signals slO generated by control system 100 and received as input from drive unit 12. As illustrated, a stream of images si la acquired by endoscope acquisition means 11a is sent to the control system via an appropriate connection, e.g. via drive unit 12.

Control system 100 to control an endoscope device according to the present invention includes an HMD viewer wearable by an operator O, a processing unit 110, first means 120 for acquisition of a control signal sl20 that does not require the use of hands (hereinafter also "hands free"), in particular including a system hm for tracking movements of the operator's head, and further hands-free means 130, 140 for the acquisition of at least one further control signal that do not require the use of hands.

The movement tracking system is designed to detect movements hm of an operator's head with respect to a given reference or horizon. The tracking system is based on the acquisition of movement information by means of sensor means 120 that operate a spatial location system with at least three axes.

Sensor means 120 preferably include position sensor devices such as at least one accelerometer, one gyroscope and one magnetometer, preferably configured to detect, in combination, movements in at least three and preferably at least nine axes. The sensor means may also include detection enhancement systems such as infrared (IR)-based locators or other technology having a similar purpose.

According to a preferred embodiment, a combination of movement components detected by sensor means 120, also called "fusion” in the multimodal environment, allows the tracking system to obtain extremely precise identification of the position and movements of the operator's head in a three-dimensional space. This has a number of advantages, including perfect recognition of angles and inclinations in every direction.

Position/movement signals sl20 acquired by tracking system 120 are a preferred example of a control signal acquired from the first hands free means for the acquisition of a control signal.

Based on this control signal, processing unit 110 can process a motion control signal slO (position and/or orientation) of the endoscope head operating segment 11. The motion control signal slO can be sent to endoscope drive unit 12, for example comprising an electronic control panel connected to the processing unit and actuating means to operate the operating segment of the endoscope on the basis of drive signals generated by the control panel from the motion control signals slO transmitted from processing unit 110 and received as an input from drive unit 12.

The result is a movement control system for the operating segment of the endoscope that does not require the use of hands.

The actuators may for example be three or more servomotors, at least one of which is positioned and configured to move the operating segment of endoscope 11 about the three pan, tilt and roll axes respectively, X, Y, Z.

With regard to the output of motion control signals slO from processing unit 110, these preferably include at least: a signal controlling an angle of rotation about the X axis, a signal controlling an angle of rotation about the Y axis, and a signal controlling an angle of rotation about the Z axis. As will be more evident below, motion control signal slO may also include a zoom control signal, especially if the zoom is of the optical type.

As an example, motion control signals slO for each axis X, Y, Z may be obtained as follows: the data sl20 for the spatial coordinates of the position of the head of operator O collected by sensor means 120 are transformed into digital coordinates in a three-dimensional plane; based on a predetermined resting position (e.g. coordinates relating to the frontal position, eyes parallel to the horizon line), the tracking system detects a movement of the operator's head and records it according to the axis in which it has been detected: horizontal (Y), vertical (X) and rotational (Z). At this point the corresponding angle of rotation about each axis X, Y, Z is digitally calculated and sent as a signal to control the movement of operating segment 10 about the respective X, Y or Z axis. This operation may require a preliminary stage of calibration so that, on the basis of many factors, such as actuator sensitivity and/or endoscope model size, digital values are associated with angular rotation positions for each actuator. When speaking of digital data, a zero shift is equal to 0°, a positive shift (clockwise) is equal to +å° and a negative shift (counter clockwise) is equal to -å° for each actuator in the system.

Transformation of signal slO by control panel 12 and/or the processing of signal slO by processing unit 110 can advantageously apply a proportionahty factor between the detected amplitudes of movement of the operator's head and those desired for the operating segment of the endoscope.

The HMD wearable viewer may be of any known type to display stereoscopic images, being in particular at least of high definition, to simulate a 3D reality on one or more display screens. For example it may be an Oculus Rift CV1, already equipped with Oculus VR tracking sensors that operate a head movement tracking system, such as an HTC Vive or other devices of the same category. The connection to the processing unit, which may for example be a computer comprising at least one processor, can be made via cable 5 or wireless or by any known means.

The HMD viewer is preferably configured to display 3D images in Side-by-Side (SBS) format, i.e. side-by-side and synchronised according to a specific format.

The image stream smhd displayed on the HMD viewer is processed and transmitted to it by processing unit 110.

As shown in Figure 2a, environmental image acquisition means, particularly in the form of two cameras 141,142, are located on a front surface of the HMD viewer to acquire images of the reality surrounding the HMD viewer and send a corresponding stream of environmental images sl40 to processing unit 110.

Cameras 141,142 are preferably placed side by side in the horizontal plane and preferably capture high definition RGB images. The distance between the objectives will preferably be proportional to the human interpupillary distance according to the microscopy (or macroscopy) desired; as known to persons skilled in the art, the greater the focal distance, the greater will be the distance between the two devices. The images captured by cameras 141, 142 on the HMD viewer will therefore be able to correspond exactly to the operator's POV (Point of View) and, therefore, displaying them in SBS format, provide a realistic view of the surrounding environment, as if the viewer were not there.

Video stream sl40 captured by environmental image capture means 140 may be sent to processing unit 110 for processing, for example via the same connection as the HMD viewer or via an independent connection.

According to a preferred aspect, processing unit 110 can control the flow of images smhd displayed by the HMD viewer to switch the display shown on the HMD viewer from images si la from the endoscope to images sl40 of the operator's surroundings from environmental image acquisition means 140, enabling the operator to view the surrounding environment without necessarily having to remove the HMD viewer.

According to a technique known in the art, image acquisition means 140 and the processing unit may be configured to act as a hand detection device, making it possible to acquire a control signal to command such switching by means of a hand gesture captured by acquisition means 140 and detected by a hand recognition algorithm operated by the processing unit.

For example, when the operator's hand passes at a distance of about 3 cm from cameras 141, 142, the processing unit calls up a hand recognition function which, after recognising a predefined gesture, commands a switch and starts sending RAW images from the two cameras 141, 142 directly to the HMD screen.

Although functional in certain circumstances, this image switching control is unsatisfactory for many applications as it is still linked to use of the operator's hands and also leads to rehabihty problems (unwanted switching), as well as the need for a complex system training process to recognise specific hand gestures.

With continuing reference to Figures 1-2 and with further reference to Figure 3, a control system 100 according to the present invention is characterised in that it includes at least second hands-free means 150, 130 for acquisition of a control signal, which do not require the use of hands, are connected to the processing unit and are capable of acquiring and transmitting a corresponding further control signal to the processing unit itself. As will be more apparent below, the at least second means of acquiring a control signal are in fact configured to acquire at least one further control signal in hands-free mode, including one or more of: a control signal to control exclusion of the control signal for movement of the operating end processed on the basis of the tracking of head movements; and/or a signal switching the display of images on the viewer between the images coming from the means for the acquisition of environmental images and images coming from the endoscopy device (or vice versa); and/or an amount of zoom control signal applied to the image acquired by the acquisition means of the endoscopy device.

The second means for hands-free acquisition of a control signal thus make it possible to increase the safety, reliability and/or usability of the endoscopy system, in particular to eliminate or at least reduce the need to use hands and/or unwanted display switching and/or provide additional functionality that can be controlled without the use of hands. This results in a safer, more reliable, usable and better adapted control system for surgical applications where hands should be available to the surgical operator to operate other devices or to operate directly in the field of the endoscope.

As illustrated in Figure 1-2, a first example of second hands-free acquisition means comprises a voice command acquisition system, comprising at least one sound acquisition device, for example a microphone 130, and at least one logical voice command processing unit, configured to process the audio signal sl30 acquired by sound acquisition device 130 and provide a corresponding control signal to processing unit 110 to which the voice command acquisition system is connected. It will be obvious to those skilled in the art that the logical voice command processing unit may be provided within processing unit 110 itself, for example through appropriate programming of the same, so that it processes audio signal sl30 coming from acquisition devices 130 to recognise the speech contained in it, transcribe it into an appropriate file (for example using known Speech-to-text applications) and identify one or more predefined words or phrases (Keywords) in the transcribed text that form corresponding control signals entered by the operator by voice command. It will be obvious to those skilled in the art that the logical unit for processing the audio signal input from acquisition devices 130 may also be an intermediate unit connected between devices 130 and processing unit 110, for example implemented in an audio acquisition card for pre-processing audio signal sl30 emitted by acquisition devices 130. A combination of the two solutions is also possible.

With reference to Figures 1 and 3, a further preferred embodiment of second means for hands-free acquisition of a control signal comprises one or more pedals 151, 152, 153, 154 that can be operated by an operator's foot and are connected to processing unit 110 to which the respective acquired control signal is transmitted. A pedal is any device configured to be operated by an operator's foot.

Advantageously two or more pedals may be combined in one pedal unit 150 (Figure 3), which can be connected to the processing unit, for example, by means of a cable 158 of appropriate length; pedal unit 150 may be placed permanently on a floor in a desired position, providing a convenient operating interface by means of foot pedals 151, 152, 153, which can be easily moved as a single unit and placed in the most convenient position, for example, by means of cable 158 or other appropriate wireless connection.

The acquisition and transmission of signals from the pedal unit to the processing unit can preferably be coordinated by a controller inside the pedal unit, connected to processing unit 110.

Advantageously and with reference to Figure 4, second hands-free acquisition means 150 may include at least one, preferably at least two graduated-action pedals 151, 152 capable of acquiring a respective control signal proportional to a pressure force exerted on pedals 151, 152. For example, each graduated pedal 151 may include a pedal lever 151a capable of being actuated by the pressure exerted by an operator's foot against the force of resilient means 151b (e.g. one or more thrust springs).

Operation of pedal lever 151a may for example be rotation around a respective hinge fulcrum 151b in one direction or another within a certain predefined angle of rotation a. Each pedal 151 may include graduated sensor means 15 Id arranged and configured to detect how much pressure is applied to pedal lever 151a during operation, and output a corresponding proportional control signal sl51that can be transmitted to processing unit 110.

The acquisition of proportional control signals sl51, sl52 by means of one or more graduated pedals 151, 152 makes it possible to control with great precision drives or processes that can be adjusted between two ends of a control scale in a linear or proportional way, and is therefore particularly advantageous for some preferred examples of graduated control that will be described below, such as in particular a zoom factor control with speed/magnitude proportional to the pressure exerted on one of said graduated pedals 151, 152.

According to a further preferred embodiment, second hands-free acquisition means 150 may include at least one, preferably at least two switching pedals 153, 154, that is individual pedals (or push-buttons) of the, for example, switch or ON/OFF push-button type. Each switching pedal 153, 154 may in particular be operated to acquire a control signal for switching between two or more predefined states of a given actuator or signal controlled by processing unit 110.

Switching pedals of this type are particularly advantageous when switching signal acquisition that does not require fine tuning but a fast and reliable response is required. In fact, with rapid pressure of the foot on switching pedal 153 a control signal can be sent to processing unit 110 quickly, conveniently and rehably without using the hands and without having to look at the pedal it is desired to operate. This can be implemented in an even more practical way by conforming an operable part of each pedal 153, 154 with an external surface of a particular shape, preferably different from that of one or more of the other pedals, for example hemispherical like the one shown in Figure 3, so that the operator can recognise which control signal it is configured to acquire, by touch or at least with slight pressure on pedal 153, 154.

With reference to Figures 1 and 5, a preferred configuration of processing unitor 110 and a corresponding preferred example of a control method implemented by system 100 will now be described. As mentioned above, processing unit 110 is configured to process a control signal slO to move operating segment 11 of endoscopy device 10, control signal slO for movement being processed by the processing unit on the basis of a control signal sl20 transmitted by the first hands free acquisition means, in particular by the system tracking movement of the head of operator O, so as to control a proportional actuator drive (e.g. motors) for operating segment 11 about each of at least three axes (X-Y-Z) of movement by means of a control unit 12 for endoscopy device 10.

Processing unit 110 is also configured to control the display of an image stream smhd on the HMD viewer, and in particular to switch the source of this image stream smhd between images from environmental image acquisition means 140 and images si la from acquisition means 11a of endoscopy device 10.

As shown in Figures 1 and 5, the processing unit is also connected to hands-free second means 130, 140 for acquiring an additional control signal that does not require the use of hands and which in the preferred embodiment illustrated include two switching pedals 153, 154, two graduated pedals 151, 152 and a voice command acquisition system 130.

A first switching pedal 153 is used to acquire and send a control signal sl53 to unit 110 signal to control switching of the source of said image stream smhd between the images coming from environmental image acquisition means 140 and images si la coming from acquisition means 11a of endoscopy device 10 (and/or vice versa).

Subsequent pressing of the same pedal 153 will result in the acquisition of a subsequent switching control signal that will switch the image displayed on the viewer back to the one coming from endoscope 11 (or vice versa).

A similar control signal for switching said flow of displayed images smhd can be acquired by voice command acquisition system 130 by saying a corresponding predetermined word or phrase into the respective microphone.

Pressing pedal 153 or the saying of the predetermined word or phrase by operator O will therefore cause processing unit 110 to switch image stream smhd displayed on the HMD viewer without the need to use the hands.

First graduated pedal 151 and/or second pedal 152 may be able to acquire a proportional/graduated hands-free control signal. According to a preferred embodiment, one or both graduated pedals 151, 152 will be able to acquire a control signal for an amount of zoom applied to images from the acquisition means of endoscope 11a or possibly from environmental image acquisition means 140.

In particular, a proportional signal sl51 acquired by first graduated pedal 151 and sent to processing unit 110 may be able to control an amount of close or magnification zoom (Zoom-in) applied to images captured by endoscopy device 10. A proportional signal sl52 acquired by second graduated pedal 152 may instead control an amount of zoom-out applied to images captured by endoscopy device 10.

With regard to control of the amount of zoom applied via processing unit 110, it will be obvious to a person skilled in the art that this may be for example: a digital zoom, which will be controlled by processing unit 110 by means of appropriate processing of flow si la of images Ill captured by means 11a of endoscopy device 10, or a zoom of an optical type, which can be controlled by processing unit 110 by sending a suitable signal to control panel 12 of endoscopy device 10, which will include suitable actuators able to vary optical acquisition arrangement of image acquisition means 11a proportionally so that flow of images si la received by processing unit 110 will already be suitably zoomed by a certain amount. A combination of the two zoom methods is also possible, since the amount of digital zoom is limited by the definition of the original flow of images si la captured by means 11a of the endoscope and the quantity of optical zoom is limited by the physical structure of the endoscope itself.

A similar control signal for the amount of zoom may be acquired by voice command acquisition system 130 by saying a corresponding predetermined word or phrase into the respective microphone. In this case each predetermined word or phrase (e.g. "zoom in" or "zoom out") can be associated with a respective command corresponding to a certain predefined amount of zoom to be applied to the image stream, repetition of the predetermined phrase allowing the amount of zoom the proportionally increased or decreased.

As illustrated diagrammatically in Figure 5, processor 110 is preferably configured to control blocker or enabler s-lk (Figure 5) for control of movement of the operating end of endoscopy device 10 based on tracking head movement signal sl20 from tracking system 120. That is, the processing unit can block or disable the processing and/or sending of signal slO for control of movement of the operating end of the endoscope, processed to control movement of the operating segment based on the detected and tracked movements of the operator's head, when this is not desired or appropriate, and enable it again when it is desired or appropriate.

In particular, the processing unit can be configured to block or disable said control of movement of the operating end of endoscopy device 10 based on tracking movements of the operator's head: automatically when it receives and implements a control signal for switching the source of image stream smhd displayed by the viewer from images si la from acquisition means 11a of endoscopy device 10 to images from environmental image acquisition means 140; and/or when it receives a control signal from second hands-free acquisition means that commands the blocking (disabling) of control of movement of the operating end based on the tracking of head movements.

Similarly and in combination with one or both of the above logical arrangements, processing unit 110 may be configured to (re-)enable control of movement of the operating end of endoscopy device 10 by tracking the operator's head movements: automatically when it receives and acts on a control signal for switching the source of image stream smhd displayed by the viewer from images from environmental image acquisition means 140 to images si la from acquisition means 11a of endoscopy device 10, and or when it receives a control signal from second hands-free acquisition means that commands the (re-)enabling of control of movement of the operating end based on the tracking of head movements. Advantageously, a second switching pedal 154 may be used to acquire said control signal sl54 and send it to unit 110 to control blocking/enabling control of movement of the operating end based on the tracking of head movements (and/or vice versa).

Subsequent pressure on the same pedal 154 or an additional dedicated pedal, if present, will cause a subsequent control signal that will control (re)-enabling of control of movement of the operating end based on the tracking of head movements to be acquired.

A similar control signal for enabling/disabling control of movement of the operating end based on the tracking of head movements may be acquired by voice command acquisition system 130 through saying a predetermined word or phrase into the respective microphone.

Pressure on pedal 154 or saying of the predetermined word or phrase by operator O will therefore bring about the manual enabling or disabling of control of movement of the operating end based on the tracking of head movements, without the need to use the hands.

This simple but effective solution avoids moving the endoscope when it is not necessary or when it can be dangerous to do so, particularly when the operator no longer has the display of images from endoscopy system 10 and would therefore move the operating end "blind".

Preferably, processor 110 is configured so that when in a start-up state and/or default state the control system according to the present invention is in a mode in which control of movement of the operating end based on the tracking of head movement is blocked (disabled).

Advantageously, processing unit 110 may also be configured to block/enable a signal controlling the amount of zoom when it commands the corresponding enabling/disabhng of control of movement of the operating end based on the tracking of head movements.

As shown in Figure 5, a particularly preferred embodiment of the control system according to the present invention provides for processing unit 110 to be configured to exclude a control signal s-esc from one of second hands-free means 150, 130 for acquisition of a control signal, when another of said second hands-free acquisition means 130, 150 is acquiring a corresponding control signal and/or until an action controlled by a corresponding control signal acquired by the other second hands free acquisition means has been completed.

For example, if processing unit 110 is receiving or has just received a signal sl51, sl52 controlling an amount of zoom from one of graduated pedals 151, 152, it will command the exclusion of any signals controlling the amount of zoom ("Zoom-in" or "Zoom out") spoken by the operator and acquired by voice command acquisition system 130 (for example by ignoring them), until the action of changing the amount of zoom controlled by the respective signal acquired via pedal 151, 152 has been completed.

Example of use

A surgical operator connects the control unit to an endoscopy device 10 comprising a mechanical and semi-rigid endoscope with three or more degrees of freedom and switches the control system on, places footswitch 150 next to the operating table, prepares the instruments and places them inside the patient in accordance with practice.

At this point he is ready to proceed with endoscopy, and therefore inserts the endoscope into the surgical area of interest while placing the HMD viewer over the face.

He controls the directional movements of the endoscope with one hand using conventional techniques.

Operating switch button 154 with one foot, it sends a signal sl54 to the processing unit to order the enabling of movement control based on the tracking of head movements; at this point the operator starts to explore the environment inside the patient's body by moving the head and, therefore, by directing the operating segment of the endoscope in proportion to movements of the head.

During the procedure a surgical instrument held in the other hand needs to be replaced and he therefore needs to be able to see the surrounding environment.

By saying a predetermined voice command into microphone 130 (e.g. "EXIT") the operator causes control signal sl30 for switching the display by voice command acquisition system 130 to be acquired and sent to the processing unit which controls switching of the source of image flow smhd sent for display to the HMD viewer from images 11a sourced from the endoscope to images sl40 coming from cameras 141, 142 on the front of the HMD viewer; the operator can see the surrounding world and change the instrument.

Preferably, the system automatically blocks movement command based on the tracking of head movements, or the operator orders the block using button 154.

To control the switch to the display of images from the endoscope (and possibly automatic or manual unblocking of the movement control) and continue the procedure, the operator decides to use pedal 153 (and possibly 154) because he is talking to one of his assistants at the same time and therefore cannot use voice command acquisition system 130 again; he therefore proceeds to press pedal 153 which sends the respective control signal sl53 to processing unit 110, in response to which unit 110 commands switching to images from the endoscope.

The operator can at this point continue with the procedure, but a small detail in a wall of tissue concerns him, so he decides to check it more closely, without moving the endoscope to avoid internal damage to the patient. To do this he uses the zoom function, then gently presses graduated pedal 151 to send a corresponding control signal to the processing unit which processes flow of images si la acquired by the endoscope to provide a magnified view by digital zooming towards the central point of the image.

The operator then releases pedal 151 and the zoom stops; after making the check the operator operates graduated pedal 152 with his foot to control a Zoom-out in order to have a better overview of the endoscope’s environment.

At the end of the entire procedure the operator presses pedal 154 to send a control signal si 54 that controls the blocking of movement control based on head movements, extracts the endoscope from the patient's body, takes off the HMD viewer, continues with the final routine tasks and finally switches off the control system.

A further object of the present invention is a computer program including instructions that implement a method of control according to the present invention when performed by a computer processing unit, such as the processor of a personal computer. Thus a control system according to the present invention for the control of an endoscopy device offers improved reliability, safety and usability in that it is able to implement the acquisition of control signals for all the functions required without using the hands, and also in different ways depending on the specific needs that may arise during surgery.

The system and method of control also make it possible to control auxiliary functions that require proportional control and could not be implemented through control based solely on the tracking of head movements, without the need for the use of hands.

The control system may also be configured to further improve safety of use through means for disabling/enabhng movement control based on head movements that may be either automatic or manually operated, again without requiring the use of the hands.

The system is also easy and economical to implement as it can be implemented using a personal computer on which runs a computer program that implements the method of control according to the invention that is connected to a limited number of peripheral devices (viewer, head movement tracking system and at least a second means of acquiring hands-free control signals, in particular at least one pedal or voice command acquisition system) and is connected through standard connections to any endoscopy device designed to receive movement control signals in at least three axes at right angles.