The present invention concerns a communication unit, and a method in connection with such a communication unit for communicating with unmanned and autonomous vehicles according to the preambles of the independent claims.

Background of the invention

A vehicle that can be driven without a driver is known as an unmanned ground vehicle, or UGV. There are two types of unmanned ground vehicles, those that are remote-controlled and those that are autonomous.

A remote-controlled UGV is a vehicle that is controlled by a human operator via a communication link. All actions are determined by the operator based on either direct visual observation or by means of sensors such as digital video cameras. A remote-controlled toy car is a simple example of a remote-controlled UGV.

There are major variations among remote-controlled vehicles in use today. These vehicles are often used in dangerous situations and environments that are unsuitable for the presence of humans, such as in disarming bombs and in connection with hazardous chemical spills. Remote-controlled unmanned vehicles are also used in connection with surveillance work and the like.

An autonomous vehicle is essentially an autonomous robot that functions without the need for human control. The vehicle uses its sensors to obtain a sort of limited understanding of its surroundings, which is then used by control algorithms to determine the next step to take with respect to the overall task that an operator has assigned the vehicle.

A fully autonomous vehicle must have the ability to:

» Gather information about its surroundings, such as constructing maps of interiors in mines

* Detect objects of interest, such as rocks, people and vehicles « Move from one location to another without human navigational assistance « Do work over long periods of time without human intervention

» Avoid situations that are dangerous to people, property or the vehicle itself, to the extent that they are not involved in the task

* Repair itself without outside assistance

An autonomous vehicle can also often possess the capacity for autonomous learning, which means: * Self-teaching or creating new abilities without external assistance

* Adjusting strategies based on the surroundings

» Adapting itself to its surroundings without external assistance

Autonomous vehicles naturally require regular service, just as all machines do.

Autonomous vehicles have also been developed for use in dangerous

environments, such as in the defense and war industry, and in the mining industry, both open-pit and underground. If people or normal manually controlled vehicles approach the work area of the autonomous vehicles, they normally cause an interruption in the work for safety reasons. The vehicles are ordered to resume their work once the work area is free again.

An autonomous vehicle thus refers in the present application to a vehicle that is capable of navigating and maneuvering without human control. The vehicle utilizes information about the road, the surroundings and other factors that affect its forward travel in order to automatically control its gas pedal depression, braking and steering.

A careful assessment and identification of the planned forward travel is necessary in order to determine whether a route is passable, and necessary to be able to successfully replace human assessments in terms of driving the vehicle. Road conditions can be complex, and the driver of a normal manned vehicle makes hundred of observations per minute and adjusts the operation of the vehicle based on the perceived road conditions. One aspect of assessing the road conditions is to perceive the road and the surroundings and find a passable route past objects that may be present on the road. The ability to replace the capacity of human perception with an autonomous system involves, among other things, the ability to perceive objects in a precise manner in order to be able to effectively control the vehicle so that it steers past such objects. The technical methods used to identify an object in connection with the vehicle include the use of one or a plurality of cameras and radar to generate images of the surroundings. Laser technologies, both scanning lasers and fixed lasers, are used to detect objects and measure distances. These are often referred to as L!DAR (Light Detection and Ranging) or LADAR (Laser Defection and Ranging). Various sensors are also used in order to sense velocity and accelerations in various directions.

The cameras translate visual images taken in the form of light patterns or infrared radiation into processable data formats. One such format can be pixelated images, wherein a detected image is broken down into series of pixels. Image processing with radar utilizes radio waves generated by a transmitter, which are then detected and used to estimate shapes and objects in front of the transmitter. Different patterns of these reflected shapes and objects can then be analyzed to determine the positions of said objects. GPS and other wireless technologies can also be used to determine when, for example, one is approaching an intersection, a narrowing of the road, or other vehicles.

More specifically, an autonomous vehicle must be able to interpret its

surroundings well enough to be able to perform the task it has been assigned, e.g. "move the block of stone from point A to point B via the mine gallery C." The autonomous vehicle needs to plan and follow a route to the selected destination while detecting and avoiding obstacles in its path. The autonomous vehicle must also perform its tasks as quickly as possible, without making mistakes.

US-201070114416 concerns a system and method for navigating an autonomous vehicle using detection and distance measurements made by means of lasers.

US~2012/0035788 concerns a navigation and control system for autonomous vehicles and comprises sensors, such as laser sensors, configured so as to localize objects in front of the vehicle so that it can be driven without colliding with said objects.

There are many documents that describe vehicles that are steered by means of a handheld unit in proximity to the vehicle, some of which are discussed below. WO-2010/134070 concerns a remote control for a vehicle. The remote control comprises a display screen and a number of buttons and controls. The vehicle is equipped with one or a plurality of cameras that are adapted so as to take pictures of the vehicle surroundings and display them on the screen The operator can remote-control the vehicle by means of the buttons and controls, including activating the horn, determining what equipment is to be used, controlling the equipment, controlling the engine rpm, and controlling the steering of the vehicle. There are also controls for starting the engine and changing gears. The communication between the remote control and the vehicle occurs via a cable or wireiessly, for example via Bluetooth. Using the remote control, an operator can thus manage the vehicle at a distance, which is advantageous in, for example, dangerous surroundings.

FR-2764091 concerns a remote control for a car, such as an electric car. The remote control is intended to be pointed at the car and has a lower section that functions as a joystick, i.e. hand movements can steer the car. A number of buttons are also present that influence various functions of the car, e.g. which gear is to be used. US-2010/0106344 concerns remote control of an unmanned work machine by means of a remote control. The work machine is adapted for autonomous operation in complex and complicated environments. JP-2006256382 concerns a remote control for influencing a vehicle to move itself from a position that is marked by means of a laser beam. A control signal is then transmitted to the vehicle with control instructions for the vehicle to automatically move itself to the marked position. Autonomous vehicles are used today as load carriers in, for example, the mining industry, in both open-pit and underground mines, if people or normal manually controlled vehicles approach the work area of the autonomous vehicles, they normally cause an interruption in the work for safety reasons. The vehicles are ordered to resume their work once the work area is free again.

Autonomous vehicles communicate with a control center, often via a radio interface. If anyone in the vicinity of the vehicles wants to give them an instruction to perform a task, it must be handled via the control center, or quite simply by driving the vehicle oneself, if possible. A future autonomous vehicle may lack the ability to be driven by a human, i.e. there is no cab, no steering wheel or no pedals.

In certain situations it is a disadvantage to have to wait for action by the control center. This can occur, for example, if the vehicle needs to be stopped because work is to be done in proximity to the vehicle, which is then moved to a different location.

One object of the present invention is to facilitate the control of an autonomous vehicle, particularly when one is present near the vehicle. Summary of the invention

The foregoing objects are achieved by means of the invention defined in the independent claims. Preferred embodiments are defined by the dependent claims.

In order for a human to be able to rapidly and easily interact with an autonomous vehicle in its daily work, a communication interface is necessary that is realized according to the present invention by means of a communication unit, and a method in connection with said unit, according to the independent claims.

The communication unit is a handheld unit adapted so as to be used by humans who interact with autonomous vehicles. The unit can, for example, communicate with a vehicle via laser (like common TV remote control, but focused into a narrower beam) or via radio (WiFi, the mobile phone network, etc.), and preferably with the control center of the vehicle via radio as well. The vehicle is equipped with a communication device adapted for two-way wireless communication with the communication unit. According to one embodiment, the communication unit is adapted so as to transfer information to a control center indicating that the vehicle has received instructions from the communication unit, as well as the consequences of said instructions. According to another embodiment, the communication unit is equipped with, for example, a laser pointer adapted so as to generate a laser beam in order to make it easier for the user to point at the correct vehicle or place.

In order to be able to determine its own position, the place at which if is pointing and how it is moving, the communication unit can comprise gyros and/or accelerometers and satellite positioning equipment (in those cases where it is used outdoors). The position can also be determined by monitoring the signal strength from a plurality of WiFi access points in the vicinity. Using the

gyros/accelerometers, the unit can also comprehend gestures that the user makes with it in his hand. The communication unit is preferably equipped with a screen in order to, for example, display information about the vehicle at which the user is pointing.

Using the present invention makes the communication between humans and autonomous vehicles smoother, faster and more practical in daily work as compared to having a control center handle it, since the control of the vehicle can be performed by a person in the vicinity of the vehicle.

The invention is intended in particular to provide simple communication in connection with the control of, for example, heavy autonomous vehicles that are driverless. The point is not to remotely control and maneuver the vehicle, but rather to exploit the built-in intelligence of the vehicle.

For example, a person in the vicinity of the autonomous vehicle can easily move the vehicle by issuing several simple instructions via the communication unit. In simple terms, the invention concerns a local interface adapted so as to issue high-level instructions to an autonomous vehicle. These high-level instructions are normally issued from a control center.

The invention will now be described in detail with reference to the accompanying drawings. The same or similar parts have been given the same reference designations in the figures.

Brief description of the drawing

Figure 1 is a schematic block diagram illustrating one embodiment of the present invention.

Figure 2 is a schematic block diagram illustrating a number of embodiments of the present invention. Figure 3 is a flow diagram thai illustrates a method according to the present invention.

Figure 4 is a flow diagram illustrating a method according to one embodiment of the present invention.

Figure 5 is a schematic block diagram illustrating one embodiment of the present invention.

Detailed description of preferred embodiments of the invention

The invention will now be described with reference to the schematic block diagram in Figure 1.

The present invention concerns a handheld communication unit 2 adapted for wireless communication with an unmanned and autonomous vehicle 4 of the type discussed above in the background portion of the application. The vehicle is, for example, intended to be used in mines, both open-pit and underground. The communication unit 2 comprises one or a plurality of input means 6, such as buttons, levers, controls or input via a touchscreen, first communication means 8 and a processing unit 10, which includes a processor and a memory. The communication unit 2 is also equipped with an energy source, preferably a rechargeable battery, to supply the involved components with operating voltage. To ensure that it will be convenient to use the unit, preferably with one hand, it cannot be too large. It preferably has a longitudinally rectangular shape that fits in the hand, with a maximum length of 15 cm and a maximum width of 5 cm. The thickness can be on the order of a maximum of ca. 2 cm.

The first communication unit 8 is adapted so as to wireiessly transmit an instruction signal 12 in dependence upon inputs via the input means 6, with at least one high-level instruction. A high-level instruction refers to an instruction to the autonomous vehicle 2 to independently perform a high-level task, which is performed without external control.

A high-level task comprises a plurality of different activities that are performed independently by the autonomous vehicle 4 through the use of the "intelligence" thai the autonomous vehicle exhibits in terms of independently being able to react to and make decisions based on events and circumstances in its surroundings, which are sensed by means of sensors on the vehicle. According to one embodiment, the communication unit is equipped with one or a plurality of (e.g. programmable) hotkeys for the most common commands (the high-level instructions) intended to be issued to the autonomous vehicle. The user can use the unit to, for example: 1. Point at a vehicle, give the command "stop". The vehicle stops. Can also be used as an emergency stop.

2. Point to a vehicle, give the command "resume the work". The vehicle goes back to work.

3. Point at a vehicle, give the command "move yourself", point to a location, give the command "there". The vehicle drives to the desired location.

4. Show himself to the vehicle sensors, point to the vehicle, give the

command "follow me". The vehicle follows the user.

5. Point at a vehicle, give the command "return to the depot". The vehicle drives back to the depot/workshop.

6. Point at a vehicle, give the command "who are you?". A display unit on the communication unit displays information about the vehicle.

This list is only an example of various high-level tasks that the vehicle is adapted so as to perform.

According to one embodiment, the high-level instructions comprise a first high- level instruction that is intended to perform a predetermined task. This can be, for example, to drive to a given location, retrieve stone and then leave the stone at another location. A second high-level instruction can, for example, concern driving back to a predetermined location, such as a service depot.

According to another embodiment, the communication unit 2 comprises a motion sensor 22 (see Figure 2) adapted so as to sense motions of the communication unit 2 and transmit a motion signal in dependence upon the sensed motion to the processing means 10. The processing means 10 are adapted so as to determine a third high-level instruction based in part on said motions. The motion signal can, for example, consist of a signal from an acee!erometer and/or a gyro. A sensed motion can then be combined with the simultaneous issuance of a high-level instructions, such as "drive in the direction I am waving" or "follow me toward where i am waving".

According to yet another embodiment, the communication unit 2 comprises a position sensor 24 adapted so as to determine the current position of the communication unit 2 and transmit a position signal to the processing means. This can occur by means of satellite positioning (Global Navigation Satellite System, often abbreviated to GNSS) in those cases where the unit is used outdoors.

GNSS is a collective term for a group of worldwide navigation systems that utilize signals from a constellation of satellites and pseudosatellites to enable position measurement by a receiver. The American GPS system is the most familiar GNSS system, while others include the Russian GLONASS and the future European Galileo. The position can also be determined by monitoring signal strengths from a plurality of access points for wireless networks (WiFi) in the vicinity.

According to one embodiment, the communication unit comprises a light-radiating unit 14 adapted so as to generate a beam of light 16 in a predetermined direction in dependence upon an input via said input means 8. For example, by pressing a button. The light-radiating unit 14 can consist of a laser pointer that generates a laser beam within the visible spectrum. The communication unit 2 is further adapted so as to communicate with a vehicle that is present in said direction, in more detail, the communication unit 2 is adapted so as to first identify a vehicle by pointing at a vehicle with said light-radiating unit and then issuing a high-ievei instruction to the identified vehicle in dependence upon inputs via said input means 6. According to one embodiment, the communication unit 2 comprises second communication means 8 (see Figure 2) adapted so as to send said issued high- level instructions to a control center 20, preferably via a predetermined radio interface. This is illustrated in Figure 2 by a bidirectional arrow between the second communication unit 8 and the control center 20. In Figures 1 and 2 there is also a bidirectional arrow between the vehicle 4 and the control center 20. This indicates the communication that occurs between the control center and the vehicle, and which pertains, for example, to the transfer of high-level instructions from the control center to the vehicle and the transmission of information from the vehicle to the center, it is important for the control center to be informed regarding the activities of the autonomous vehicle. This can occur in part via the second communication means and, naturally, in part in that the vehicle communicates directly with the control center. According to one embodiment, the first communication means 8 are adapted so as to issue the high-level instruction to the autonomous vehicle in the form of an optical signal, preferably a laser signal.

According to another embodiment, the first communication means 8 are adapted so as to issue the high-level instructions to the autonomous vehicle in the form of a radio signal.

The autonomous vehicle is in turn equipped with a communication device adapted so as to communicate with the first communication means 8. This communication device then communicates in turn with a control device in the vehicle that is responsible for controlling the autonomous vehicle in dependence upon the received high-level instructions. The control device must, for example, be able to manage the situation if high-level instructions are also being received directly from the control center, and then be able to prioritize these received instructions. If the communication device has received an instruction signal with a high-level instruction, the reception can, for example, be confirmed by means of a handshake procedure in which the identities of the communication unit and the vehicle are exchanged.

The control device is thus adapted so as to interpret the received high-level instructions and translate them into controllable activities for the vehicle.

One example of this could be that if the high-level instruction "stop" is received, it means that the following procedure will be carried out:

- Reduce gas pedal depression to zero.

- Brake the vehicle.

- Perform this until the velocity is 0 m/s.

The communication unit 2 preferably comprises a display unit 26 adapted so as to display identification information about the autonomous vehicle with which the communication unit 2 is communicating. The display unit can also display information about the high-level instructions that are to be issued, or that have been issued. The display unit can be equipped with a touchscreen and then used for inputting as well.

The invention also concerns a method in connection with a handheld

communication unit adapted for wireless communication with an unmanned and autonomous vehicle. The communication unit has been described in detail above with reference to Figures 1 and 2.

The method according to the invention will now be described with reference to the flow diagram in Figure 3. A special embodiment is illustrated by the flow diagram in Figure 4.

The method thus comprises:

- entering an input via the input means;

- identifying a high-level instruction corresponding to the input;

- issuing a wireless instruction signal containing the high-level instruction, which is an instruction to the autonomous vehicle to independently perform a high-level task that is performed without external control. A high-level task comprises a plurality of different activities that are performed independently by the autonomous vehicle.

According to one embodiment of the method, the communication unit comprises a light-radiating unit adapted so as to generate a beam of light in a predetermined direction in dependence upon an input via the input means, wherein the communication unit is adapted so as to communicate with a vehicle that is present in said direction. More specifically, the method comprises first identifying a vehicle by pointing at a vehicle with the light-radiating unit and then issuing the instruction signal with the high-level instruction to the identified vehicle in dependence upon inputs via said input means. This is illustrated by the flow diagram in Figure 4,

According to another embodiment second communication means are arranged that are adapted so as to transmit the issued high-level instructions to a control center, preferably via a predetermined radio interface.

Examples of various high-level instructions and related high-level tasks for the autonomous vehicle have been discussed above in connection with the description of the communication unit, and reference will be made to that description now as the method is described.

According to one embodiment, the method concerns utilizing a motion sensor arranged in the communication unit and adapted so as to sense motions of the communication unit and transmit a motion signal in dependence upon said sensed motion to the processing means. The processing means are adapted so as to determine a third high-level instruction based in part upon said motions.

According to another embodiment, a position sensor is utilized that is arranged in the communication unit and adapted so as to determine the current position of the communication unit and transmit a position signal to said processing unit. Said signal is then used by the processing unit to determine a high-level instruction. According to one embodiment of the method, the first communication unit is adapted so as to issue the high-level instruction to the autonomous vehicle in the form of an optical signal, preferably a laser signal, According to another embodiment of the method, the first communication unit is adapted so as to issue the high-level instruction to the autonomous vehicle in the form of a radio signal.

As was discussed above, the communication unit preferably comprises a display unit that, according to the method, is adapted so as to display identification information about the autonomous vehicle with which the communication unit is communicating. It can further be used to display information about the high-level instructions. The present invention further comprises a computer program (P) in the vehicle, wherein said computer program (P) contains program code for causing a processing unit 10; 500 or another computer 500 connected to the processing unit 10; 500 to perform the steps according to the method as described above.

The invention further comprises a computer program product containing a program code stored on a computer-readable medium for performing the method steps as described above when said program code is run on a processing unit 10; 500 or another computer 500 computer connected to the processing unit 10; 500.

The computer 500 will now be described with reference to the block diagram in Figure 5.

The program P can be stored in executable form or compressed form in a memory 560 and/or in a read/write memory 550. When it is stated that the data-processing unit 510 performs a given function, it is to be understood that the data-processing unit 510 executes a certain part of the program that is stored in the memory 560, or a certain part of the program that is stored in the read/write memory 550. The data-processing device 510 can communicate with a data port 599 via a data bus 515. The non-volatile memory 520 is intended to communicate with the data- processing unit 5 0 via a data bus 512. The separate memory 580 is intended to communicate with the data-processing unit 510 via a data bus 511. The read/write memory 550 is arranged so as to communicate with the data-processing unit 510 via a data bus 514. The units that are connected to the processing means 10 (see Figure 1 or 2) can be connected to the data port 599.

When data are received at the data port 599, they are stored temporarily in the second memory section 540. Once received input data have been stored temporarily, the data-processing unit 510 is arranged so as to execute code in a manner as described above.

Parts of the methods described herein can be carried out by the device 500 (corresponding to the processing unit in Figure 1 or 2) with the help of the data- processing unit 5 0, which runs the program stored in the memory 580 or the read/write memory 550. When the device 500 runs the program, the methods described herein are carried out.

The present invention is not limited to the preferred embodiments described above. Various alternatives, modifications and equivalents can be used. The foregoing embodiments are consequently not to be viewed as limiting the protective scope of the invention, which is defined by the accompanying claims.