The present invention refers to a method for controlling a moving behavior of an autonomously moving self-propelled robot, from a starting point through a crowded pedestrian environment to a destination by selecting and following one or more particular guide pedestrians.

The present invention also refers to a robot moving support system comprising means for executing the steps of the method.

Furthermore, the present invention refers to a robot using the robot moving support system.

Furthermore, the present invention refers to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to execute the steps of the method.

Furthermore, the present invention refers to a data carrier signal, which the computer program transmits.

Furthermore, the present invention refers to a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to execute the steps of the method.

Robots as vehicles operating in a service environment are often required to operate autonomously in crowded spaces without colliding with pedestrians. A mobile robot that navigates a predefined trajectory through a crowded pedestrian environment is likely to severely impede pedestrian traffic flow, and may lead to collisions with pedestrians. Conversely, a mobile robot that navigates through a crowded pedestrian environment by avoiding surrounding pedestrians will find it difficult, if not impossible, to successfully navigate to a desired endpoint location. For example, a mobile robot employing conventional collision avoidance algorithms may not be able to identify a collision free path because interactions among pedestrians lead to a highly dynamic pedestrian environment. In some examples, conventional collision avoidance algorithms inaccurately assume individual pedestrians move at a constant velocity. Improper navigation of a mobile robot through a crowded pedestrian environment can create a hazardous situation as the mobile robot is operating in close proximity to humans. It is especially concerning if the mobile robot is carrying heavy payloads and is capable of high acceleration maneuvers.

A procedure and a robot moving support system of the type mentioned above are known from the US Disclosure Document US 2018/0129217 A1 , for example. This Disclosure Document discloses a method and a system for navigating a mobile robot through a crowded pedestrian environment by selecting and following a particular guide pedestrian are described herein. In one aspect, a navigation model directs a mobile robot to follow a guide pedestrian based on the position and velocity of nearby pedestrians and the current and desired positions of the mobile robot in the service environment. The mobile robot advances toward its desired destination by following the selected guide pedestrian. By repeatedly sampling the positions and velocities of nearby pedestrians and the current location, the navigation model directs the mobile robot toward the endpoint location. In some examples, the mobile robot selects and follows a sequence of different pedestrians to navigate to the desired endpoint location. In a further aspect, the navigation model determines whether following a particular guide pedestrian will lead to a collision with another pedestrian. If so, the navigation model selects another guide pedestrian to follow. According to this Disclosure Document, the focus is therefore on that the robot follows a guide pedestrian from behind. Specifically, paragraphs [0067] and [0068] disclose that the robot follows at a distance behind the first guide pedestrian. The situation of a road crossing is problematic here. A guide pedestrian can use his intuition to assess a situation in such a way that he is not hit by a vehicle driving along the road. A robot, on the other hand, is exposed to the danger of being hit by a vehicle if it follows the guide pedestrian crossing the road. One reason for this is that the guide pedestrian makes the road crossing safe for himself, but not for a robot following him.

In general, some of situations can be tricky for the robot, for instance moving in a dense respectively crowded environment or even crossing a pedestrian crossing.

It is an object of the present invention to provide a method for controlling a moving behavior of an autonomously moving self-propelled robot, a robot moving support system, a robot, a computer program, a data carrier signal and a computer-readable medium, that will guide the robot safely through a crowded pedestrian environment, as well as make the robot safely cross a crowded road.

This object is achieved by the independent claims. Advantageous embodiments are given in the dependent claims.

In particular, the present invention provides a method for controlling a moving behavior of an autonomously moving self-propelled robot, from a starting point through a crowded pedestrian environment to a destination by selecting and following one or more particular guide pedestrians, comprising the following steps of the robot:

Detecting, by a sensor system of the robot, an environment and determining, by a computing system, one or more guide pedestrians;

Following the one or more guide pedestrians, wherein this takes place along a trajectory, wherein the trajectory is determined by the computing system, and wherein the trajectory (T) oriented on the course of motion of the one or more guide pedestrians; Detecting, by the sensor system, while following the one or more guide pedestrians, the environment and determining, by the computing system, one or more moving obstacles; Determining, by the computing system, the moving parameters of the robot such that one or more guide pedestrians are located, if possible, between the robot and one or more moving obstacles, for avoiding collision with the one or more moving obstacles.

Preferably, the last step of the inventive method is executed in the robot.

According to each embodiment, the robot can have a navigation and/or a communication system.

The present invention also provides a robot moving support system comprising means for executing the steps of the method.

The present invention also provides a robot comprising the robot moving support system.

The present invention also provides a computer program comprising instructions which, when the program is executed by a computer, cause the computer to execute the steps of the method. A computer program is a collection of instructions for performing a specific task that is designed to solve a specific class of problems. The instructions of a program are designed to be executed by a computer and it is required that a computer can execute programs in order to it to function.

The present invention also provides a data carrier signal, which the computer program transmits.

The present invention also provides a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to execute the steps of the method.

The basic idea of the invention is to use a human or an object currently sharing the same direction of motion as a shield to move easily in a crowded or dangerous environment. In this way, the robot avoids being hit by an approaching other pedestrian and/or road user, such as a vehicle. This also allows the robot to move along the side of a pedestrian when moving. For this purpose, it is at least to be assumed that the robot has to move from a starting point to a destination and that the robot has at least one sensor system which is designed to detect other pedestrians and vehicles. The environmental data detected by this sensor system are processed in a computing unit and a trajectory is calculated and travelled on accordingly. The idea is not that the pedestrians serving as shield would be injured first, but that their intuition would not put them in a situation that would cause an unwanted collision. Depending on the safest solution, the guide pedestrian will thus either predefine a trajectory or, for example when crossing a road, serve as a lateral intermediate object between the robot and a supposed obstacle. In other words, the robot moves on in such a way that the guide pedestrian is between the robot and the supposed obstacle. Therefore the protection of the robot is not that it is not overlooked, but that the one or more guide pedestrians are not overlooked.

Preferably the robot is a delivery robot, for example for furniture or food. Delivery robots, which are autonomously moving self-propelled carts that transport a load to a customer, but also other robots falling under the scope of this invention, have to cope with a variety of traffic situations. According to a modified embodiment of the invention, it is provided that in the case of one or more moving obstacles approaching substantially perpendicularly to the trajectory to be moved along, the moving parameters of the robot are adapted in such a way that one or more guide pedestrians are located laterally to the robot. The lateral arrangement of the one or more guide pedestrians, for example, is very helpful when crossing a road. A possible scenario is that a robot should cross a crowded pedestrian crossing. Thus, a moving vehicle approaching the crosswalk would be a moving obstacle. Since zebra crossings are usually perpendicular to a road, the moving vehicle will approach the robot's trajectory substantially perpendicularly. Substantially vertical thus means that two trajectories will cross soon, namely that of the robot and that of at least one moving obstacle. While the one or more guide pedestrians cross the zebra crossing safely, it is important to make sure that the robot is not overlooked. The one or more guide pedestrians can be used as a side shield in such a situation. Therefore the protection of the robot is not that it is not overlooked, but that the one or more guide pedestrians are not overlooked.

According to a modified embodiment of the invention, it is provided that only one guide pedestrian is determined and followed by the robot. This keeps the necessary computing capacities low. It can be assumed that a guide pedestrian always chooses a path that does not harm him. The robot can therefore set its own moving parameters with just one guide pedestrian, which benefit from this basic assumption regarding the guide pedestrian.

According to a modified embodiment of the invention, it is provided that in the case of at least two guide pedestrians, the moving parameters of the robot are adapted in such a way that the robot moves between the at least two guide pedestrians. It turned out that such an arrangement allows a high collision prevention of the robot with regard to other moving obstacles. Even in the situation of crossing a dual road, especially a zebra crossing, the risk of a collision between a vehicle and the robot can be reduced if the two guide pedestrians walk sideways to the robot, so that no vehicle is physically able to hit the robot without a guide pedestrian being hit.

According to a modified embodiment of the invention, it is provided that the moving parameters comprise at least the trajectory and/or the moving speed of the robot. It has turned out that these are the parameters to be considered in order to guarantee a safe movement of the robot with little computing capacity.

According to a modified embodiment of the invention, it is provided that the one or more guide pedestrians are, if possible, continuously kept. This means that no other pedestrian guide will be selected if, for example, a future collision with another pedestrian threatens. Instead, the robot's solution is to position itself skillfully so that the guide pedestrian is positioned between the robot and the other pedestrian. This is a useful solution, for example, if the guide pedestrian wants to carry the robot with him. A possible scenario for this could be that the guide pedestrian has bought a heavy piece of furniture that the robot has to transport from the furniture store to its home. Previous strategies with alternating guide pedestrians have so far overtaxed such a robot. Thus, the robot can also move in densely crowded environments.

According to a modified embodiment of the invention, it is provided that one or more other guide pedestrians are determined dynamically. For example, while one guide pedestrian is kept static, the second guide pedestrian has the advantage that the moving parameters of the robot are adapted to the constantly changing situation of its environment. The static guide pedestrian can be, for example, a person that the robot should follow to a defined destination. The dynamically determined guide pedestrian is optionally used to reduce the risk of collision in a crowded environment, as the robot can calculate with two guide pedestrians. It is also possible that no guide pedestrian is statically determined.

According to a modified embodiment of the invention, it is provided that the one or more guide pedestrians are only replaced by one or more other guide pedestrians, if this is explicitly instructed and/or if the preceding one or more guide pedestrians are no longer detectable by the sensor system. Replacement can be done, for example, to perform a conscious transfer of the robot. For example, when the robot has completed its delivery, the next guide pedestrian can guide the robot back to its base station. Replacing a guide pedestrian if it is no longer detectable by the sensor system is to prevent the robot from wandering aimlessly or stopping if its actual guide pedestrian is suddenly lost in a crowded environment. According to each embodiment, the robot can have a navigation system. Thus, the robot can try to reach the destination through the fallback plane.

According to a modified embodiment of the invention, it is provided that the sensor system comprises at least one camera, preferably a front camera and/or a surround view camera. A front camera has the advantage that it is inexpensive and allows a reliable detection of the area to be moved on. A surround view camera has the advantage that it enables reliable detection around the robot. In addition, it enables a better prediction of the near real-time environment, so that the moving parameters can be adapted to it in high quality. Such a situation can occur if an obstacle approaches from the rear left or right at high speed and the originally planned trajectory of the robot would be cut at risk of collision.

According to a modified embodiment of the invention, it is provided that the computing system comprises a computing unit physically independent of the robot for determining the one or more guide pedestrians. A separate computing unit has the advantage that the computing steps do not have to be carried out in the robot. This saves computing capacities and thus energy. Especially with heavily loaded robots, it is disadvantageous if they stop on the way due to a lack of energy.

According to a modified embodiment of the invention, it is provided that the computing unit is comprised of a mobile device. A mobile device, for example, is a smartphone. This can be operated by a guide pedestrian, for example. It may also be possible for the guide to specify or change the destination. The advantage over a robot controller is that the mobile device can preferably be adjusted while walking. In contrast, an input on the robot would have to be made during standstill. This is time-consuming and therefore disadvantageous. A required determination of the guide pedestrian, for example, can also be carried out in real time on a screen of the mobile device. This reduces the possibility of incorrect determinations.

According to a modified embodiment of the invention, it is provided that each guide pedestrian is a human and/or a moving object. Thus it can be that a human guide pedestrian carries a guiding object with him, which is detected and tracked by the robot. It is also possible that another robot serves as a guide pedestrian. The robot can therefore be used in many different ways, which increases its profitability. According to a modified embodiment of the invention, it is provided that the robot estimates the moving parameters of the one or more particular guide pedestrians and/or of the one or more moving obstacles, so that the robot can position itself foresightedly in such a way that it has with a high probability positioned the one or more particular guide pedestrians between itself and the one or more moving obstacles. This prediction reduces the problem that robots have to stop if moving obstacles cross their trajectory in a collision-prone way. This increases the efficiency and acceptance of robots, since other pedestrians do not feel harassed or threatened by robots. The selection of parameters for data prediction can be set individually. Thus, distance, number of steps and/or speeds can be taken into account. Examples of mathematical methods are approximation, extrapolation and/or other forecasting methods. An approximation is anything that is intentionally similar but not exactly equal to something else. In many cases, a numerical method is based on the idea of approximating a complicated, and often only implicitly known, function by means of a function that is easy to handle. The approximation theory is therefore an integral part of modern applied mathematics. It provides a theoretical foundation for many new and established computer-based solution methods. Extrapolation is the process of estimating, beyond the original observation range, the value of a variable on the basis of its relationship with another variable. It is similar to interpolation, which produces estimates between known observations, but extrapolation is subject to greater uncertainty and a higher risk of producing meaningless results. Extrapolation may also mean extension of a method, assuming similar methods will be applicable. Extrapolation may also apply to human experience to project, extend, or expand known experience into an area not known or previously experienced so as to arrive at a, usually conjectural, knowledge of the unknown, e.g. a driver extrapolates road conditions beyond his sight while driving. Also, other forecasting methods are applicable.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. Individual features disclosed in the embodiments can constitute alone or in combination an aspect of the present invention. Features of the different embodiments can be carried over from one embodiment to another embodiment.

In the drawings: Fig. 1 shows a schematic top view of a crowded pedestrian environment, wherein a robot moves with a stream of pedestrians according to a preferred embodiment of the invention; and

Fig. 2 shows a schematic top view of a crowded pedestrian environment, , wherein a robot crosses a road according to another preferred embodiment of the invention.

Figures 1 and 2 show a robot 10 using a robot moving support system. The figures show the robot 10 in two different situations, using a method according to the invention. This leads to the fact, that the robot 10 can be configured to execute the steps of the method according to both figures according a preferred embodiment of the invention. Also, the robot 10 can be configured to execute the steps of the method according to figure 1 according to another preferred embodiment of the invention or to execute the steps of the method according to figure 2 according to still another preferred embodiment of the invention.

In general, according to both figures, the robot 10 executes a method for controlling a moving behavior of the robot 10 being an autonomously moving self-propelled robot 10, from a starting point A through a crowded pedestrian environment to a destination B by selecting and following one or more particular guide pedestrians 12a, 12b, comprising the following steps of the robot 10:

Detecting, by a sensor system of the robot 10, an environment and determining, by a computing system, one or more guide pedestrians 12a, 12b;

Following the one or more guide pedestrians 12a, 12b, wherein this takes place along a trajectory T, wherein the trajectory T is determined by the computing system, and wherein the trajectory T is oriented on the course of motion of the one or more guide pedestrians 12a, 12b;

Detecting, by the sensor system, while following the one or more guide pedestrians 12a, 12b, the environment and determining, by the computing system, one or more moving obstacles 14a, 14b;

Determining, by the computing system, the moving parameters of the robot 10 such that one or more guide pedestrians 12a, 12b are located, if possible, between the robot 10 and one or more moving obstacles 14a, 14b, for avoiding collision with the one or more moving obstacles 14a, 14b.

The figures 1 and 2 show the situation in a different type. Figure 1 shows the same guide pedestrian 12a and the same robot 10 in two different states, wherein they moved along the trajectory T. For this reason, the robot 10 and the guide pedestrian 12a are shown twice. Figure 2 shows the robot 10 and its guide pedestrians 12a, 12b in just one moment. For this reason, the robot 10 and the guide pedestrians 12a, 12b are shown once.

According to figure 1 , the robot 10 executes the method for controlling a moving behavior of the robot 10 being an autonomously moving self-propelled robot 10, from the starting point A through the crowded pedestrian environment to the destination B by selecting and following one particular guide pedestrian 12a, comprising the following steps of the robot 10:

Detecting, by the sensor system of the robot 10, the environment and determining, by the computing system, the one guide pedestrians 12a;

Following the one guide pedestrians 12a, wherein this takes place along the trajectory T, wherein the trajectory T is determined by the computing system, and wherein the trajectory T is oriented on the course of motion of the one guide pedestrian 12a; Detecting, by the sensor system, while following the one or more guide pedestrians 12a, 12b, the environment and determining, by the computing system, a plurality of moving obstacles 14a, 14b, which are other pedestrians;

Determining, by the computing system, the moving parameters of the robot 10 such that the one guide pedestrian 12a is located, if possible, between the robot 10 and the plurality of moving obstacles 14a, 14b, for avoiding collision with the moving obstacles 14a, 14b.

Figure 1 shows other pedestrians than the guide pedestrian 12a as moving obstacles 14a, 14b. This is a not limiting example of moving obstacles 14a, 14b.

According to figure 1 , only one guide pedestrian 12a is determined and followed by the robot 10. This means, that the robot 10 can move such that the one guide pedestrian 12a is a shield for the robot 10 with regard to the moving obstacles 14a, 14b. For example, but not shown, the robot 10 can move sideward to the one guide pedestrian 12a. So, the invention is not limited to a robot 10 following at a distance behind the guide pedestrian 12a. It has shown, that the flexibility for how to move along with the at least one guide pedestrian 12a maximizes the protection of the robot 10 and keeps it away from any collisions.

Further, the one guide pedestrians 12a of figure 1 is continuously kept, wherein the guide pedestrian 12a is a human.

According to figure 1 , is preferably provided, that the robot 10 estimates the moving parameters of the one particular guide pedestrian 12a and of the moving obstacles 14a,

14b, so that the robot 10 can position itself foresightedly in such a way that it has with a high probability positioned the guide pedestrians 12a, 12b between itself and the moving obstacles 14a, 14b.

The movement of the robot 10, especially the moving along a crowded pedestrian area, can also be executed with more than one guide pedestrian 12a, for example with two guide pedestrians 12a, 12b.

According to figure 2, the robot 10 executes the method for controlling the moving behavior of the robot 10 being an autonomously moving self-propelled robot 10, from the starting point A through a crowded pedestrian environment to a destination B by selecting and following two particular guide pedestrians 12a, 12b, comprising the following steps of the robot 10:

Detecting, by the sensor system of the robot 10, the environment and determining, by the computing system, the two guide pedestrians 12a, 12b;

Following the two guide pedestrians 12a, 12b, wherein this takes place along the trajectory T, wherein the trajectory T is determined by the computing system, and wherein the trajectory T is oriented on the course of motion of the two guide pedestrians 12a, 12b;

Detecting, by the sensor system, while following the two guide pedestrians 12a, 12b, the environment and determining, by the computing system, a plurality of moving obstacles 14a, 14b, which are other pedestrians and vehicles;

Determining, by the computing system, the moving parameters of the robot 10 such that the two guide pedestrians 12a, 12b are located, if possible, between the robot 10 and, as shown in figure 2 for example, two vehicles being the moving obstacles 14a, 14b, for avoiding collision with the two moving obstacles 14a, 14b.

Figure 2 shows other pedestrians than the two guide pedestrians 12a, 12b and vehicles as moving obstacles 14a, 14b. In this example, there are two vehicles shown as the moving obstacles 14a, 14b. It is the aim of the robot 10 to avoid a collision with these two vehicles as moving obstacles 14a, 14b, while the robot 10 crosses the zebra crossing. For this, the robot 10 uses one of the guide pedestrians 12a as a shield in accordance to the lower shown vehicle as a first moving object 14a and the other guide pedestrian 12b as a shield in accordance to the upper shown vehicle as a second moving object 14b. So, This is a not limiting example of moving obstacles 14a, 14b.

The two vehicles, as moving obstacles 14a, 14b, are approaching substantially perpendicularly to the trajectory T to be moved along. Because of this, the moving parameters of the robot 10 are adapted in such a way that the two guide pedestrians 12a, 12b are located laterally to the robot 10.

According to figure 2, referring to the example with the two guide pedestrians 12a, 12b, the moving parameters of the robot 10 are adapted in such a way that the robot 10 moves between the two guide pedestrians 12a, 12b. This way, the robot 10 and its two guide pedestrians 12a, 12b walk in a row. For better understanding, for example, a projected line can be drawn from the first moving obstacle 14a, through the first guide pedestrian 12a, the robot 10, the second guide pedestrian 12b until the moving obstacle 14b. This projected line is just a fiction line and not drawn in figure 2.

Further, the two guide pedestrians 12a, 12b of figure 2 are continuously kept, wherein each guide pedestrian 12a, 12b is a human.

According to figure 2, it is preferably provided, that the robot 10 estimates the moving parameters of the two particular guide pedestrians 12a, 12b and of the moving obstacles 14a, 14b, so that the robot 10 can position itself foresightedly in such a way that it has with a high probability positioned the two guide pedestrians 12a, 12b between itself and the one or more moving obstacles 14a, 14b. The movement of the robot 10, especially the crossing of the crowded zebra crossing can also be executed with only one guide pedestrian 12a.

In both figures the moving parameters comprise at least the trajectory T and the moving speed of the robot 10.

It is also possible, but not shown in any figure, that one or more other guide pedestrians 12a, 12b are determined dynamically by the computing system.

Preferably, the one or more guide pedestrians 12a, 12b are only replaced by one or more other guide pedestrians 12a, 12b, if this is explicitly instructed and/or if the preceding one or more guide pedestrians 12a, 12b are no longer detectable by the sensor system. This is not the case in figure 1 or in figure 2.

Preferably, but not shown in detail, the sensor system comprises at least one camera, preferably a front camera and/or a surround view camera.

Further preferably, but not shown in detail, the computing system comprises a computing unit physically independent of the robot 10 for determining the one or more guide pedestrians 12a, 12b. For example, the computing unit is comprised of a mobile device.

The features according to figure 2 are not limited to a crossing of a zebra crossing, but for example, also for the crossing of any different road.

Reference signs list 10 robot

12a, b one or more guide pedestrians 14a,b one or more moving obstacles

A starting point B destination

T trajectory