Introduction

The present invention relates to the field of radio remote controls that allow to command distant devices. These remote controls are connected to the controlled device via a wireless connection, for example with radio waves. To a llow the user to control the distant device more precisely a nd more intuitively, the remote control can tra nsmit one or multiple physical parameters of the device through ha ptic feedback to the user.

State of the art

Remote control (RC) devices in the civil sector have seen a la rge development lately with the apparition of civil drones. However, the interface to interact with these RC devices was already present since 1960 with early remote controls. The interface used two finger actuated joysticks on the remote. At that time, those remote controls were mainly used to control model aircrafts. Today, the technology used i n those remote controls has evolved but the interface has remained the same and has become standard in this sector. Currently, this interface is widely used with model aircraft a nd in the growing sector of drones.

The goal of these remote controls is to allow a human operator to control displacements of a moving aircraft in space. I n addition, the remote controls a llow to control various para meters specific to the controlled machine. Remote controls a re tra nsportable to use them in various environments. Generally, they can control multiple devices by adding a receiver on each of the controlled machines. Thus, the operator does not need to own a specific remote control for each remote-controlled device he owns.

The interface used to control these remotely controlled devices is standard in most sectors: two joysticks on the remote control are actuated by the fingers and allow to control four degrees of freedom (two by joysticks) on the remotely controlled device. Since a standard aircraft has four degrees of freedom (Yaw, Pitch, Roll, Throttle), there a re usually two joysticks on the remote. Generally, they also comprise b utto ns a nd sl ide rs to co ntrol va rio us pa ra m ete rs o n the remotely controlled device. The control input entered by the operator is sent via radio waves towards a receptor located on the controlled machine. This receptor sends the command to the remotely controlled device using a standard protocol. Finally, a gra phic interfa ce o n the remote, on a smartphone or on a computer allows the user to configure his remote to fit his specific needs. Because of their architecture, remote controls are not very intuitive and require training to get used to controlling an aircraft. The movement of both joysticks on the remote control is not directly related to the corresponding movement on the remotely controlled device. The user must therefore get used to the correspondence between the movement of his fingers and the reaction of the remotely controlled device. Moreover, the visual feed back is the on ly information that the pilot receives from the device. If we take the example of a pilot in a plane, on top of the visual feedback, a pilot can feel the plane's reaction in the stick and with the acceleration his body is experiencing. As a result, the pilot can be more precise while piloting. In the case of a remote-controlled aircraft, the lack of touch related feedbacks makes the interaction difficult, especially for small displacements and if they are difficult to observe. All those elements make for a non-immersive, complex and non-intuitive interaction with remote devices through a conventional remote control.

Short description of the invention

The present invention relates to a haptic movement tracking remote control. More precisely, the invention refers to a remote control that allows to command a remotely controlled machine by tracking movements of the handheld remote with sensors and executing a com ma nd on the machi ne that corresponds to the movement executed by the user. Additionally, the remote control receives data from physical sensors on the controlled machine and uses deformations of its shape to induce a haptic stimulus to mimic the reaction of the machine in the hand of the user.

The origin of this invention comes from the desire to conceive an intuitive and immersive remote control by exerting haptic stimuli generated by deformation of the control device that can serve for controlling remotely controlled machine. This system will overcome disadvantages presented in the former state of the art or will bring a useful alternative.

Short description of the drawings

The annexed drawings are included to allow better com prehension of the modes of realization and are part of this specification. The drawings illustrate modes of realization and jointly with the description are used to explain the principles of realization. Identical reference numbers indicate similar corresponding parts.

Modes of realization of the invention will now be described, as an example only, while referring to the annexed drawings, in which:

The Figure 1 is a side view of a remote control device according to the present invention. The Figure 2 is an isometric view of the same device.

The Figure 3 is a cross-section of the internal mechanism of the same device.

The Figure 4 is a side view of the handle of the same device.

The Figure 5 is a top view of the handle of the same device.

The figure 6 is a side view of the internal mechanism in the handle of the same device. The figure 7 is an isometric view of the internal mechanism in the handle of the same device.

The figure 8 is a top view of the internal mechanism in the handle of the same device. The figure 9 is a schematic of the two circle analogy to explain the working principle of the mechanics in the handle of the same device.

The figure 10 is an example of operation with a remotely controlled object and the top view of the device in the hand of a user.

The figure 11 is a box chart of the different steps and main components involved in the system of the device.

Detailed description

Controlling a distant device is often a difficult task. I ndeed, the control interface is usually chosen according to the standards of the industry. They are usua l ly the most straightforward solutions instead of the most intuitive. Through our test in the field of drones, it clearly appeared that people were struggling to associate the inputs on a conventional remote control (two finger actuated joysticks for control) and the corresponding actions taken by the controlled aircraft. Our test users were also struggling to control an aircraft with only visual feedback, especially when the device's displacements were difficult to observe. This is true in many other remote controlled human machine interactions.

Our goal is to propose a solution to this problem by providing a novel solution to interact with remote controlled machines. There are two key elements to this system: first, the control is achieved by tracking movements executed by the handheld remote control in free space and executing similar movements on the controlled machine. Second, the remote control generates haptic stimuli through deformation of its main body to inform the use r of the reaction of the controlled device. By using those two elements in combination, the user is not only able to control the machine intuitively but also feel the machine's reaction, thus creating a real feeling of immersion with the controlled device. Another important object of the invention resides in the fact that the remote control can be described as compact and portable.

In this document, the terms "remote device", "radio controlled device", "controlled device", "controlled machine", "machine" a nd "remotely controlled device" stand for the object which is controlled remotely by the operator. The term "remote control" stands for the object held by the operator that allows him to remotely control the remote device. The term "handle" stands for the part of the remote control which is in contact with the hand of the operator.

I n the fo l lowi ng deta i led descri ption, d rawi ngs i l l ustrate a possi ble ma n ne r of representing the invention. Therefore, words such as « top », « bottom » etc... a re used i n reference to the drawings presented. It should be noted that the components can be arranged in various manners. Thus, the following description must be taken in a broad sense and does not have to be limited to itself. The range of the invention is presented in the claims.

To explain how the system works, we will first explai n each step of the control procedure. Then, we will expose the implementation proposed for such a system. The FIG. 11 is giving an overview of the different steps involved in the control of a distant machine and the main components required.

First, the user generates a command by moving the handheld remote control around a neutral position in space set by the user. Setting the neutral position is achieved by the press of a button. The user can execute a movement in space and a similar command will be sent to the controlled device. For example, in the case of a drone, tilting the remote control on one side will initiate a turn on the corresponding side for the drone. Thus, by moving the remote control i n space, the user ca n di rectly i nteract with his machi ne by m i m icki ng i n his ha nd the movements which should be carried out by the controlled machine. Control is consequently more intuitive for the user.

The user is free to configure the commands as he wishes. He can select the range of movement in which he would like to control his machine, the control sensitivity or a nonlinear sensitivity. Each movement ca n be assigned to a displace ment or a ny com ma nd on the control led machi ne . I n this way, the re mote control ca n i nterface va rious machi nes with specific movements for each controlled platform.

The second step of the control is to receive a feedback from the action performed by the remote-controlled device. This enhances the feeling of control experienced by the user. Indeed, the haptic feedback gives additional information to the visual feedback on how the machine reacted to the command. The haptic feedback is obtained by having the shape of the remote control deforms in the hand of the user. The deformation can take different forms depending on the sensation being simulated: pressures, shear stress frictions or any other feelings which could be felt on the skin, in the tendons and the muscles in a tactile way and kinesthetic way.

In terms of implementation, the remote control uses different sensors such as accelerometers, gyroscopes, magnetometer or any other sensor capable to position the unit in space to generate the command signal. Physical inputs such as potentiometers, pushbuttons, microphones, touchpad or other sensors and other devices which communicate with the remote control such as personal computers or smartphones, can also be used to control the machine. The remote control can extend up to the shoulder. It can also be decomposed into several sub elements placed on the main body. For example, one of the signals sent to the remote device can be obtained by making the difference of two output signals generated by two sensors, each of them disposed on a different part of the body: this would allow to control the rotary motion of a machine and can be computed by making the difference between the output signal of a sensor placed in the hand and the output signal of a sensor placed on the wrist of the user. This allows differentiating between a rotation made by the whole body of the user or just his hand. A transmitter in the remote control sends the command signal towards the remotely controlled device. A receptor unit (called receptor) receives this signal and forwards it to the controlled machine using a standard communication protocol (cPPM, SBus, etc.). Finally, the controlled machine can execute the command.

To actuate the haptic feedback, the receptor on the remotely controlled device is also comprised of sensors such as accelerometers, gyroscopes, barometers or any other sensor capable to position the unit in space that generate indicative data of its attitude, speed and velocity. A transmitter on the controlled device sends this data signal back to the remote control. Finally, a receiver on the remote control collects the signal, processes the data and actuates the haptic feedback actuators. One or multiple actuators on the remote control can induce a deformation, inflexion, torsion or rotation (even a combination of these actions) of one or multiple parts of the remote control in contact with the skin of the user based on the data signal from the remote-controlled device to exert a haptic stimulus to the user. The haptic feedbacks can have various forms: pressures, shear stress frictions or any other sensations which could be felt on the skin, in the tendons and the muscles in a tactile way and kinesthetic way. These ha ptic feedbacks a l low the user to feel the movements a long some or a ll the degrees of freedom of the controlled machine. Each degree of freedom can be felt by the user with one or more types of haptic feedbacks. The goal is to mimic the reaction of the controlled machine as if it was directly in the user's hand.

Two main deformations are proposed in this document: shear stress and pressure.

Pressure can be produced in sections of the hand in contact with the remote control. In FIG. 4 and FIG. 5, we can see the handle 1 deforming. In FIG. 6, FIG. 7 and FIG. 8 we can see the detailed mechanism to modify the shape of the remote control. The thin strips 4 come out of the handle of the remote control with different amplitude and direction. This modifies the surface topography and exerts a pressure in the hand of the user in the di rection of the deformation. This deformation can happen all around the handle of the remote control. The amplitude and direction of the deformation can, for example, correspond to the amplitude and the speed of the rolling and pitching movements of a remotely controlled device. In FIG. 6, we can see that a rotation of rotation around the y axis corresponds to a deformation in the direction a (90 degrees in this case) and with an amplitude Ar that could be proportional to the angular velocity or angular acceleration.

One can observe in the FIG. 3 a cross-section of the mechanism allowing the actuation of the thin strips 4. FIG. 6, FIG. 7 and FIG. 8 details the mechanism situated inside the handle of the remote control. Two brushed DC motors 5 and 6 are used to create a two degree of freedom actuator. The central rotating apparatus composed of 7 and 8 can be compared to two disk, one of which is centered on its rotational axis and the other is off centered and fixed to the first disk, as seen in FIG. 9. The first disk is directly actuated by one of the motor. The second is actuated via two sets of gears 9 from the second motor. By rotating the two motors in the sa me direction, the direction of rotation can be selected. When turning the motors in opposite direction, the second disk extends to the outside (as seen in FIG. 9) and enters in contact with the thin strips, creating the deformation of the remote control surface. When going out of the remote control handle, the strips create pressure zones in the hand of the user. Each strip could also move thanks to an independent actuator. One could also obtain this deformation thanks to pneumatic cushions whose size can vary, electromechanical, mechanical piezo or electromagnetic systems.

Shear stress can be obtained by having the main body of the remote control divided into two or multiple parts. In the FIG. 1 and FIG. 2, a first part of the remote control main body can rotate on its central axis with an angle that is proportionate to a rotation on the controlled machine. This rotation of a first part of the handle will apply a shear stress on the skin of the user. This haptic feedback can be assigned for example with the yaw movements of the remotely controlled device, i.e. when a yaw movement occurs on the remotely controlled device, sensors situated on the latter will detect this yaw movement but also the amplitude and the rate of this movement, then a signal proportional to these two parameters will be emitted by these sensors and sent to the remote control. After evaluation of this signal, the actuators situated in the remote control will generate these rotations in the first part of the main body with a speed and an a mplitude proportional to the physical signal received by the sensors positioned on the remotely controlled device. The surface of the bottom part of the handle can be made of various textures and irregular to amplify the feeling and can be changed according to the user's preferences.

To create a rotation in the bottom part (i.e. the first part) of the remote control compared to the top part (i.e. the second part), one could use a motor which will be connected to the bottom part and with which could perform a rotation according to the desired angle and rotation's rate. Thus, the variation of angle alpha and the rate can be chosen. The part which rotates could be a simple ring connected above and under by a part of the handle. The rotation is perceived by the operator like a friction in the bottom part of his hand.

Example of operation:

For better pla n ning the operation of the remote control, we propose a concrete example of operation. This last one illustrates only one situation in which this technology could be used.

In the FIG. 10, one can observe the effect of a change of direction of a remotely controlled flying device of a drone type on a remote control. The change of direction can either be due to an order sent by the operator or by an independent cause. In the FIG. 10, the wind could have made the aircraft deviate from its slope. The sensors located on the remotely controlled device, in this example an accelerometer and a gyroscope, will detect the change of direction and this information will be sent to the remote control. Thus, the remote control will deform itself in a particular direction, in this example, according to the sa me angle a nd with a n i ntensity Ar proportional to the angular acceleration of the remotely controlled drone. The operator can thus have a precise knowledge of the movement carried out by the drone directly in his hand.