The object of the invention is to provide a controller, to be worn on a human body, used to control devices equipped with a radio receiver, especially in the systems of virtual, mixed or augmented reality, as well as drones and robots.

A game controller consisting of two control units is known from the American patent description US 8267786 B2. The units are equipped with an electronic gyroscope and accelerometer and generate three groups of operational data. The controller is equipped with physical keys and joystick. The controller units are connected to each other by a cable and wirelessly to a computer.

A way of controlling virtual objects using a controller where input data is collected from the controller is known from the American patent description US 9041622 B2. Input data including a request for specific actions to be performed by a virtual object is implemented in augmented reality depending on the user perspective, just as actions performed by the controlled object are displayed with respect to the user perspective. A virtual object controlled by a physical controller can be identified from the input data collected from the natural user interface. The controller is equipped with a force feedback mechanism.

A system for controlling content displayed in virtual environments is known from the American invention description US 20130342572 Al. A user may interact with the displayed content using predefined gestures. The system includes a display device, such as virtual reality goggles for example, and an operationally connected computer.

An input device for use in computer systems to specify a position on a display using radial coordinates, cylindrical coordinates or spherical coordinates is known from the American patent description US 7683883 B2. The device incorporates an ergonomic housing allowing it to be held in a hand, buttons and an accelerometer. The device allows to interact with the 3D environment displayed on a computer display screen. A hand-worn control device dedicated for use with an augmented reality eyepiece is known from the American invention description US 20110221672 Al. The control device contains a control element that generates a control command when activated by the user's finger.

Control moment gyroscopes, used in satellites to provide a reference point and to counterbalance manoeuvring in all directions, are known from the state of the art. Such gyroscopes have rotors of relatively high weight. It is also known that similar gyroscopes are used in an exercise apparatus such as, for example, a gyroscopic exercise apparatus, comprising: at least one control-moment gyroscope; at least one motion sensor for sensing movement of the apparatus; at least one spindle motor for providing rotation to a rotor of the gyroscope; known from the American patent description US 10252151 B2.

In addition, a handheld controller for interacting with physical moving objects and graphical user interfaces, equipped with a gyroscope unit comprising a minimum of one mechanical gyroscope, the rotor of which is coupled to an electric motor is known from the International Application Form WO 2016195503. This enables to simulate motion according to activity and moving objects shown graphical on a screen, which gives the user a three- dimensional physical feedback resulting from the activity of controlled objects. Using the device requires the use of two upper limbs, and due to its size, longer use may be difficult.

A system for control of a drone is known from the American patent description US 20140008496 Al. The commands are executed using a handheld device which includes a motion sensor module with a gyro-sensor and an g-sensor for controlling roll, yaw and pitch under a coordinate system. The drone camera image is displayed on the device screen.

A handheld system for controlling devices, consisting of an audio-video unit and a control unit for controlling the movement of a device is known from the Polish patent description PL 223180 Bl. The AV receiver is fitted to the wrist or forearm, while the control unit is held in a hand and secured with a wrist strap.

The controllers known from the state of the art do not allow comfortable controlling with only one upper limb, even if all fingers, in particular the thumb, are not functioning properly, which is necessary when using most of known handheld controllers. Devices that allow to imitate the user motions in applications, in particular applications of virtual, augmented or mixed reality, as well as when controlling robots or drones, are often characterized by a complex design and, consequently, high production costs. In addition, nowadays, for full control without any significant limitations, it is necessary to use dedicated controllers for specific applications.

A controller to be worn on a human body, comprising a holder for a computer device used in that controller and equipped with at least an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter and receiver, according to the invention, is characterised by the fact that it has a harness for wearing around the hand, where this harness is joined, in its part corresponding to the palm of the hand, to a computer device holder, and this holder has an open part corresponding at least to the part of the touchscreen of the computer device, where this open part is on the side corresponding to the palm of the hand in the area of fingers.

Preferably, the controller comprises a gyroscope unit attached to the harness and comprising at least one mechanical gyroscope.

Further advantages are obtained if its gyroscope unit comprises four mechanical gyroscopes.

Further advantages are obtained if each mechanical gyroscope of its gyroscope unit has a rotor connected to a separate electric motor.

Further advantages are obtained if the holder is connected to the harness by means of an arm, one end of which is connected to the harness at the palm of the hand and the other end is connected to the holder, the arm is connected to the harness and the holder by articulated joints, the gyroscope unit of the controller is fitted between the harness and the arm.

Further advantages are obtained if the articulated joints between the controller arm and the harness at a position corresponding to the palm area of the hand and between the arm and the holder, have at least two axes of rotation each and its arm has an articulated joint at its centre, with the first articulated joint at the centre of the arm and each articulated joint has a pressure screw to lock the arm's position at the ends of the arm.

Further advantages are obtained when the controller's harness is fitted with a spacer in its palm part where the spacer has a rigid base on the outside and an elastic cushion on the inside for supporting on the palm of the hand.

Further advantages are obtained if the controller harness has at least one mounting strap to attach it to the circumference of the hand.

Further benefits are obtained if the controller harness is in the form of a glove with openings exposing fingers.

A method of controlling devices equipped with a radio receiver by means of a controller is, according to the invention, characterised by the fact that, in the first stage, the controller's harness is attached to a hand and the computer device, equipped with at least the following sensors: an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter, is mounted in the controller holder with the touchscreen facing the fingers and connected wirelessly to the radio receiver of the controlled device, after which, in the second stage, the controller is moved and the touchscreen is touched with fingers, and with the use of computer device sensors, a point in the space, in which the controller is located, is established, and then information about the controller's position and touches of the computer device's touchscreen is transmitted wirelessly to the receiver of the controlled device and, on the basis of this information, the device is controlled.

Preferably, in the first stage, a smartphone is mounted in the holder, and when the smartphone is wirelessly connected to the radio receiver of the controlled device, buttons are displayed on the display of the smartphone mounted in the holder, and by touching these buttons, the function assigned to them is activated.

Further advantages are obtained when a virtual, mixed or augmented reality device is controlled.

Further advantages are obtained when, after installing a computer device in the holder, it is connected wirelessly with a smartphone mounted in a virtual reality goggle, then in the second stage a hand, on which the controller is worn, is moved and its touchscreen is touched with fingers, and its sensors are used to determine the point in the space where the controller is located, and then the information about the location of the controller and the touches of the touchscreen is wirelessly send to the receiver of the smartphone mounted in the virtual reality goggle and on the basis of this information a virtual object is controlled.

Further advantages are obtained by displaying a reduced image of the buttons and points, corresponding to where the user touches the touchscreen of the computer device fitted in the controller's holder, on the display of the smartphone fitted in the virtual reality goggles after wireless connection of the computer device fitted in the holder.

Further advantages are obtained if in the second stage the rotor of at least one mechanical gyroscope is rotated.

The application of the invention allows to control a number of devices in a convenient way. The controller is compact and cheap in design and use, especially if the computer device used in it is a smartphone or tablet already in the possession of the user. The solution gives the possibility of precise positioning of the hand in space with easy access to the touch screen. Attaching the holder to the controller's harness by means of an arm allows to position the smartphone used in the controller in an ergonomic position in relation to the hand, which prevents the occurrence of discomfort during long-term use. Because the controller's component is a computer program installed on a computer device used in this controller, it is possible to optimize it in order to perform specific tasks. Controller enables the application of feedback in applications of virtual reality consisting in starting the vibration engine of the smartphone in certain situations, such as a virtual obstacle. For example, when playing tennis in virtual reality, the vibration engine starts when the ball is bounced back. The mechanism of biological feedback also allows the device to be used in the rehabilitation of people with unilateral paresis because the patient does not need to have a fully functional hand to perform exercises in virtual reality. The use of a spacer pad and stiffener also affects the comfort of use, increasing the possibility of setting the smartphone in different positions relative to the hand. In addition to the positive effect on the comfort of use, the spacer pad with its cushion contributes to a more stable fixing of the controller on the hand and prevents it from slipping, which is important, especially when used in games requiring vigorous movements. The construction of the holder and its connection with the harness by means of an articulated arm ensures precise control and prevents the occurrence of unwanted movements occurring in controllers held directly in hands.

If the controller is used to control a drone by hand movement and a specific function is activated using the touch screen, it is possible to conveniently control it in six degrees of freedom by setting the position of a point in space.

On the display of the smartphone mounted in the holder it is possible to display buttons adjusted for controlling a specific device, which gives the possibility of its wide adaptation to many applications. When used in virtual reality, it is possible to display on the display of a smartphone mounted in goggles the preview of buttons and places where the user touches the display mounted in the controller holder. Finger tracking makes it easy to use the controller in virtual reality applications. The application used in the controller allows you to extend its functions and personalize the way you control it, for example by changing the layout of buttons and their size, which allows you to make additional facilitations for people with dysfunction of the upper limbs.

The controller allows for comfortable control of devices not only with one upper limb, but also in case of lack of all fingers being efficient, including a thumb.

In the version in which the controller contains mechanical gyroscopes, they allow not only to stabilize the controller during controlling, but also to obtain physical biological feedback or force feedback. Rotating rotors of mechanical gyroscopes cause the creation of a centrifugal force depending on the given rotational speed controlled by the computer device used in the controller. This allows to change the value of the resulting centrifugal force, the vector of which has its sense opposite to the hand movement performed by the user with the controller attached, which allows to simulate the weight, resistance and speed of objects, which is particularly beneficial in applications of virtual reality.

Feedback will allow precise and safe drone control, allowing not only more stable control, but also more accurate information about the speed of the controlled object and the manoeuvres being performed. Force feedback will provide you with additional stimuli that will positively impact your experience of using the controller in games and applications in virtual reality. This solution also allows, after the development of dedicated sets of exercises, for wide application in rehabilitation. This applies, among others, to patients with upper limb dysfunctions, as well as patients with unilateral paresis. Thanks to the controlled centrifugal force generated by the system of rotating rotors of mechanical gyros, the patient will be able to perform exercises that would not be possible without this system. Additionally, thanks to stimuli informing the patient about the correctness of exercises performed, the effectiveness of therapy is improved.

The controller's insert attached peripherally from the inner side of the harness allows to attach it to other parts of the body, such as for example shank or forearm. The location of the smartphone allows you to use its camera to recognize hand gestures and use them to control devices.

The controller attached to the human body according to the invention in an exemplary embodiment is shown in the drawing, in which Fig. 1 shows a perspective view of the controller in the first embodiment, Fig. 2 - a side view of the same controller, Fig. 3 - a top view with a broken-out section of the same controller with the pad turned eccentrically, Fig. 4 - a top view of the same controller attached to a hand, Fig. 5 - a top view with a broken-out section of the controller in the second embodiment, Fig. 6 - a bottom view with a broken-out section of the controller in the third embodiment, Fig. 7 - a side view with a broken-out section of the same controller, Fig. 8 - a side view of the controller in the fourth embodiment, Fig. 9 - a side view with a broken-out section of the controller in the fifth embodiment, Fig. 10 - a front view the same controller, Fig. 11 - the same controller attached on a forearm, Fig. 12 - a side view of the controller in the sixth embodiment, Fig. 13 - a top view of the controller in the sixth embodiment, Fig. 14 - a bottom view of the same controller, Fig. 15 - preview of buttons displayed on the display of the smartphone mounted in the controller, Fig. 16 - preview of buttons displayed stereo scopically on the display of the smartphone mounted in virtual reality goggles, Fig. 17 - preview of buttons from Fig. 15 shown magnified.

In the first embodiment shown in Figures 1 to 6, a human body worn controller with a computer device comprises a hand worn harness 1, which is attached to a computer device holder 2, in its part corresponding to the palm of the hand. The computer device used in the controller is a smartphone 3 comprising an accelerometer, an electronic gyroscope, a capacitive touchscreen and a camera. The connection of the holder 2 with the harness 1 is made by means of an arm 4. The first end of the arm 4 is connected to the harness 1 and the second end is connected to the holder 2. The connections of the arm 4 with the harness 1 and the holder 2 are made as articulated joints with two axes of rotation. The arm 4 comprises the first cylinder articulated joint 5 in its central part. The harness 1 of the controller comprises in its part, corresponding to the palm of the hand, a spacer pad 6, which comprises, from the inside, a cushion 7 of resilient polyurethane foam and, from the outside, a plastic rigid base 8. The connection between the arm 4 and the harness 1 is placed on the rigid base 8. In its central part, the cushion 7 comprises a recess 9. The cushion 7 is detachably connected by means of Velcro fasteners 14 with a base 8' seated eccentrically on the rigid base 8. This positioning of the cushion 7 ensures a better fit of the harness 1 to various hand shapes and increases the ergonomics of the controller. The holder 2 comprises a spring clip 10 for clamping the smartphone 3. The harness 1 comprises two straps 11 and 12 for fastening at the circumference of the hand, the first of which 11 is fastened at the circumference of the hand between the thumb and other fingers using a buckle 13 and the second strap 12 is fastened at the circumference of the hand near the wrist using a Velcro fastener 14. The harness 1 has a stiffener 15 on the first strap 11, in the part corresponding to the back of the hand, which is lined with elastic foam on the side intended for the back of the hand. The stiffener 15 acts as a slider for adjusting the circumference of the first strap 11 which is locked in the buckle 13. The straps 11 and 12 are connected to each other by means of a connector 16. The use of the stiffener 15 in combination with the spacer pad 6 allows stable positioning of the controller on the hand, preventing it from moving, when making vigorous movements and increasing comfort during long-term use. Articulated joints between the ends of the arm 4 and the harness 1 and the holder 2 allow to place the smartphone 3 in a position allowing for comfortable control with four or five fingers depending on its position in relation to the hand. The first articulated joint 5 and the articulated joints of the arm 4 are fitted with clamping screws 17 that allow the arm 4 and the holder 2 to be locked in a fixed position to prevent the smartphone 3 mounted in the holder 2 from sliding against the hand during use. The harness 1 has a strap 18, between the first strap 11 and the second strap 12, which connects the straps 11 and 12 on the side of the spacer pad 6, additionally stabilizing the attachment of the harness 1 on the hand.

In the second embodiment shown in Figure 5, the spacer pad 6 of the controller comprises a gyroscope unit 19 inside its base 8, in which one mechanical gyroscope 20 is mounted, comprising a rotor 21 supported by a fixed suspension 22. The rotor 21 is driven by an electric motor 23. The speed of the rotor 21 is controlled by the smartphone 3 mounted in the holder 2 of the controller. For the remaining part, the design is the same as in the first embodiment.

In the third embodiment, shown in Figures 6 to 7, the spacer pad 6 of the controller comprises a gyroscope unit 19 inside its base 8, in which four mechanical gyroscopes 20 are mounted, each of which comprises a rotor 21 supported by a suspension 22 comprising a second articulated joint 24 with angle of inclination electromechanically adjustable in two axes. The rotor 21 of each gyroscope is connected to a separate electric motor 23. For the remaining part, the design is the same as in the first embodiment.

In the fourth embodiment, shown in Figure 8, the harness 1 is made in the form of a glove 25. The glove 25 is connected, in its part corresponding to the palm of the hand, to the holder 2 of a computer device. The smartphone 3 is used as a computer device. The glove 25 has a spacer pad 6 in its part corresponding to the palm of the hand. The connection between the glove 25 and the holder 2 is made by means of the arm 4, one end of which is connected to the spacer pad 6 of the glove 25 and the other end to the holder 2. The design of the holder 2, the arm 4 and the spacer pad 6 is the same as in the first embodiment. The glove 25 has a first strap 11 and a second strap 12 to stabilize the glove on the hand and adjust its size.

In the fifth embodiment, shown in Figures 9 to 11, the harness 1 of the controller is additionally equipped with an elastic foam insert 26 placed peripherally from the inside of the straps 11 and 12 of the harness 1. The insert 26 is used after disconnecting the cushion 7 and in addition, to attaching the harness 1 to the hand it also allows its attachment to the forearm. For the remaining part, the design is the same as in the first embodiment.

In the sixth embodiment, shown in Figure 12, the harness 1 has one strap 11. For the remaining part, the design of the harness 1 is the same as in the first embodiment.

In the seventh embodiment, shown in Figures 13 to 14, according to the invention the controller worn on the human body is in the form of a glove 25 detachably connected, in its part corresponding to the palm of the hand, with a silicone holder 2. The smartphone 3 as in the first embodiment is placed in the holder 2. Once attached to the hand, the glove 25 has two holes 27, one for the thumb and one for the other fingers. The silicone holder 2 has a pocket 28 for the smartphone 3 and a wing 29 fastened to this pocket 28 on the circumference of the smartphone 3 with Velcro fasteners 14. The glove 25 has three straps 11, 12 and 30 fastened with Velcro fasteners 14 for attaching to the hand. The straps 11, 12 and 30 are fastened with Velcro fasteners 14. The first strap 11 is fastened around the circumference of the hand in the area between the thumb and the other fingers, the second strap 12 is fastened around the circumference of the wrist and the third strap 30 is fastened around the forearm. The circumference of the glove 25 may be adjusted with the straps 11, 12 and 30. The adjustable circumference of the glove 25, as well as the hole 27 for four fingers, also allows it to be used as a strap for fixing for example, on the shank or shoulder. The controller is equipped with a wrist stabilizer 31 in the form of a rigid tape running from the glove cuff 25, from the side corresponding to the palm of the hand, to the midhand. In addition, the holder 2 has an elastic pressure straps 32, which are attached with Velcro fasteners 14 to the back of the glove 25 to ensure a more stable attachment of the smartphone 3 to the holder 2. In its lower part, the glove 25 has a battery compartment 33, which makes it possible to extend the working time of the controller. The glove 25 is connected to the holder 2 by means of a snap fastener.

In the first embodiment, the way of controlling devices equipped with a radio receiver is done using the controller described in the third embodiment, and the smartphone placed in the virtual reality goggles is controlled. The harness 1 of the controller is attached to the hand, and the smartphone 3 is mounted in the holder 2 so that its touchscreen is facing the fingers. The smartphone 3 is equipped with the following sensors: a camera, electronic gyroscope, accelerometer, magnetometer, capacitive touch screen, as well as a radio receiver and transmitter. Then, the smartphone 3 connects wirelessly with the second smartphone mounted in the virtual reality goggles. The display of the smartphone 3 mounted in the holder 2 shows the shapes acting as the buttons 34 of the controller. The smartphone mounted in the virtual reality goggles displays a preview of the buttons 34 displayed on the smartphone 3 mounted in the holder 2 of the controller. The preview of the buttons 34 is reduced in size and also shows the points 35 of touching the smartphone 3 mounted in the holder 2 of the controller, so that the user can see the position of his/her fingers and there is no problem with finding the buttons 34 on the smartphone 3 of the controller. Then, in the second stage, the hand and the entire upper limb move with the controller attached and the touchscreen is touched with fingers in the area of the displayed buttons 34 as well as predefined gestures are made. The controller's position in the space and the touches and gestures made using the touchscreen are then transmitted wirelessly to a smartphone mounted in the virtual reality goggles. Based on the information sent, virtual reality objects are controlled, for example, a character in a game. If an object encounters virtual obstacles, the vibration engine starts. In addition, in the second stage, gestures are made above the touchscreen of the smartphone 3 and its front camera. These gestures are recorded and recognized by the front camera of the smartphone 3 mounted in the controller holder, and the information about the gestures made is sent to the smartphone mounted in the virtual reality goggles. In addition, in the second stage, the rotors 21 of the mechanical gyroscopes 20 are rotated and the position of the controller is stabilised. By means of the second articulated joints 24, electromechanically controlled, of the suspensions 22 of the mechanical gyroscopes 20, the angle of inclination of the rotors 21 is adjusted. When being rotated, the rotors 21 also allow to obtain a force feedback informing the user about the speed of movement of the controlled object, or about being faced with a virtual obstacle, and also allow to simulate the weight, resistance and speed of objects in virtual reality. The rotational speed of rotors 21 and their inclination is controlled by the smartphone 3 mounted in the holder 2 of the controller. The method is implemented using dedicated applications installed on the smartphone 3 mounted in the holder 2 of the controller and in the virtual reality goggles.

In the second embodiment, according to the invention, a drone is controlled. The controller is used according to the first embodiment. In the first stage, the harness 1 of the controller is attached to the hand, and the smartphone 3 is mounted in its holder 2 as in the first embodiment, and it is wirelessly connected to the drone and the display of the smartphone 3 shows the buttons 34. Then in the second stage, a hand movement is made with the controller attached and the sensors of the smartphone 3 determines the point in the space in which the controller is located. At the same time the touchscreen of smartphone 3 is touched with fingers and the predefined gestures are made or the buttons 34 displayed on the display are touched, which are assigned to specific drone functions. Information about the controller's position in the space and the buttons 34 or gestures touched on the touchscreen are sent wirelessly to the drone. The drone moves in a manner corresponding to the movements made by the user's hand with the controller attached.

List of designations

1 - Harness 18 - Securing strap

2 - Holder 19 - Gyroscope unit

3 - Smartphone 20 - Mechanical gyroscope

4 - Arm 21 - Rotor

5 - First articulated joint 22 - Suspension

6 - Pad 23 - Electric motor Cushion 24 - Second articulated joint

Base 25 - Glove

Base 26 - Insert

Recess 27 - Hole

Clip 28 - Pocket

First strap 29 - Wing

Second strap 30 - Third strap

Buckle 31 - Stabiliser

Velcro fastener 32 - Pressure strap

Stiffener 33 - Compartment

Connector 34 - Button

Screw 35 - Points