COMPUTER-IMPLEMENTED METHOD AND SYSTEM FOR CONTROLLING OPERATION OF AN AUTONOMOUS DRIVERLESS VEHICLE IN RESPONSE TO

OBSTACLE DETECTION

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from U.S. Patent Application No. 13/940,807 filed on July 12, 2013 entitled COM PUTER-IM PLEM ENTED METHOD AND SYSTEM FOR CONTROLLING OPERATION OF AN AUTONOMOUS DRIVERLESS VEHICLE I N RESPONSE TO OBSTACLE DETECTION, which is hereby incorporated by reference.

BACKG ROU N D

[0002] The present application generally relates to autonomous driverless vehicles and, more particularly, to a computer-implemented methods and systems for controlling operation of such vehicles upon detection of hazards.

[0003] Autonomous vehicles operating in unconstrained or lightly constrained environments are typically outfitted with systems that detect obstacles or other environmental hazards in the vicinity of the vehicle. Such hazards may include, e.g., the unexpected presence of a human, animal, or another vehicle. When such a hazard is detected, the autonomous vehicle will alter its behavior to avoid the hazard, either changing course or halting its movement altogether.

[0004] Such safety sensing systems are imperfect, sometimes failing to detect hazards (a false negative event) or signaling a hazard that does not actually exist (a false positive event). The present application is directed to methods and systems for mitigating the impact of false positive events. The methods and systems enable a remote operator to assess whether or not a safety event is a false positive event and, if so, to command the vehicle to ignore it and continue its operation. BRIEF SU M MARY OF TH E DISCLOSU RE

[0005] In accordance with one or more embodiments, a computer-implemented method is disclosed for controlling operation of an autonomous driverless vehicle in response to detection of a hazard in a path of the vehicle. The method is performed by a vehicle control system in the vehicle, and comprises the steps of: (a) detecting presence of a hazard in the path of the vehicle using an obstacle detection system in the vehicle; (b) transmitting a hazard alert message to a computer system operated by a human operator; and (c) receiving a command from the computer system operated by the human operator responsive to the hazard alert message, and operating the vehicle in accordance with said command.

[0006] In accordance with one or more further embodiments, a computer system in an autonomous driverless vehicle is disclosed, comprising: at least one processor; memory associated with the at least one processor; and a program supported in the memory for controlling operation of the vehicle in response to detection of a hazard in a path of the vehicle. The program includes a plurality of instructions stored therein which, when executed by the at least one processor, cause the at least one processor to: (a) detect the presence of a hazard in the path of the vehicle using an obstacle detection system in the vehicle; (b) transmit a hazard alert message to a computer system operated by a human operator; and (c) receive a command from the computer system operated by the human operator responsive to the hazard alert message, and operate the vehicle in accordance with said command.

[0007] In accordance with one or more further embodiments, a system is provided for controlling operation of an autonomous driverless vehicle in response to detection of a hazard in a path of the vehicle. The system comprises a computer system operated by a human operator and a vehicle controller located in the autonomous driverless vehicle. The vehicle controller is programmed to (a) detect the presence of a hazard in the path of the vehicle using an obstacle detection system in the vehicle; (b) transmit a hazard alert message to the computer system operated by a human operator; and (c) receive a command from the computer system operated by the human operator responsive to the hazard alert message, and operate the vehicle in accordance with said command.

BRI EF DESCRI PTION OF TH E DRAWI NGS

[0008] FIG. 1 is a simplified block diagram illustrating a system for controlling operation of an autonomous driverless vehicle in accordance with one or more embodiments.

[0009] FIGS. 2A and 2B are simplified diagrams illustrating the process of applying a region-to-ignore to detected hazards in accordance with one or more embodiments.

[0010] FIG. 3 is a flowchart illustrating the process of handling a received safety alert in accordance with one or more embodiments.

[0011] FIGS. 4A-4C are simplified diagrams illustrating processes for calculating a region- to-ignore in accordance with one or more embodiments.

DETAILED DESCRIPTION

[0012] The present application relates to automated detection of hazards by an autonomous, driverless vehicle. As discussed in further detail below, various embodiments disclosed herein are directed to manually suppressing the automatic response of the vehicle to detection of obstacles that do not present a hazard to the safe operation of the vehicle. A remote operator console provides an override control to a human operator, and communicates with a control module on the vehicle that inter-operates with collision-avoidance and other safety modules on the vehicle.

[0013] FIG. 1 illustrates an exemplary system for controlling vehicle operation in response to detection of obstacles in accordance with one or more embodiments. As shown in FIG. 1, the system resides on two distinct physical components, the vehicle controller (VC) 110 and a remote operator console (ROC 114), which are connected by a wireless computer network. Examples of suitable wireless networks include, but are not limited to, Wi-Fi, WiMAX, and GPRS (General Packet Radio Service). [0014] The VC 110 and the ROC 114 each comprise a programmable computer system including at least one processor and a storage medium readable by the processor. The ROC 114 includes a display interface and input devices such as, e.g., a mouse or keyboard for use by the operator.

[0015] The VC 110, which is located in the driverless vehicle, contains a hazard- detection module (HDM 116), which processes information from sensors 118 and navigation systems 120 in the vehicle in order to determine the presence and location, respectively, of hazards that affect the operation of the vehicle.

[0016] The sensors 118 for detecting obstacles can comprise a variety of sensors including, e.g., LIDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging), RADAR (RAdio Detection And Ranging), SONAR (SOund Navigation And Ranging), and video cameras. The navigation system 120 can comprise any system capable of determining the geographic position and orientation of the vehicle including, e.g., a GPS (Global Positioning System) and AH RS (Attitude and Heading Reference System), which could be based on gyroscopes or accelerometers.

[0017] In one or more embodiments, hazards detected by the HDM 116 are

represented as polygons with absolute geographic coordinates, i.e., polygons in which each vertex denotes a fixed position on the earth. I n accordance with various embodiments, additional functionality is provided in the H DM 116 that enables manual suppression of obstacle detection.

[0018] When the HDM 116 determines a hazard is present in the path of the vehicle, it broadcasts an alert to notify the ROC 114. In addition to simply indicating the existence of a hazard, the alert may also specify the shape and position of the hazard (e.g., its polygon), a measure of confidence in its existence, a description of the sensors 118 that detected the hazard, and any other information that may help a human operator assess the validity of the report. I n return, the ROC 114 may send an 'ignore obstacle' command back to the H DM 116 (as described further below). [0019] If the HDM 116 receives an 'ignore obstacle' command from the ROC 114, it does the following:

1. It constructs and records an 'absolute region-to-ignore' 200 as shown in FIG. 2A. This region is represented as a polygon in which each vertex denotes a fixed position on the earth. (The construction of this polygon is discussed further below.)

2. A timer is started. When the timer has reached a predetermined deadline, an 'ignore obstacle' command received from the ROC 114 is considered complete and the 'absolute region-to-ignore' is cleared.

3. While the HDM 116 has an active 'absolute region-to-ignore' 200, computed hazards 210 (or portions thereof) that lie within that region are ignored and discarded by the HDM 116. In one or more embodiments, this is accomplished by discarding all vertices of a hazard polygon that lie within the 'absolute region-to-ignore' 200 as shown in FIG. 2B. Alternately, the system can compute the geometric subtraction of the area contained by the hazard polygon from the area in the 'absolute region-to-ignore.'

[0020] By way of example, the duration of the 'ignore obstacle' command (i.e., the deadline of the timer) is 20 seconds.

[0021] The ROC 114 receives status alerts from the VC 110. In one exemplary embodiment, the ROC 114 periodically polls the VC 110 to request its current alert state.

[0022] FIG. 3 is a flowchart illustrating exemplary operation of the ROC 114 in accordance with one or more embodiments.

[0023] The ROC 114 waits to receive alerts from the VC 110 at 302. When the ROC 114 receives a safety alert from the VC 110 indicating that the vehicle has detected a hazard in the path of the vehicle, it will display a video stream at 304 to the ROC operator from a camera 122 on the vehicle oriented toward the hazard. [0024] The ROC 114 also overlays a message indicating that a hazard has been detected at 306, and presents a button giving the operator the option to ignore that hazard. The ROC 114 can also draw the hazard's computed polygon as an overlay on the video image, and can display any additional information from the hazard alert, such as a computed confidence measure or an indication of which sensors 118 detected the hazard.

[0025] The ROC 114 then waits for the operator to press the button at 308, while continuing to monitor the state of the safety alert. If the VC 110 happens to clear the safety alert without operator intervention (e.g., a detected vehicle subsequently moves out of the way), then the ROC 114 removes the hazard message and returns to its previous display state at 310.

[0026] If the operator presses the 'ignore hazard' button, then the ROC 114 switches to a 'Ignore Hazard' mode in which:

1. it continues to display video from the same camera; and

2. it presents a message to indicate the new mode, an affirmative query such as "Ignore hazard?" or "Continue driving?", and two buttons labeled 'no' and 'yes' at 312.

[0027] The ROC 114 then waits for the operator to press a button at 314, while continuing to monitor the state of the safety alert. As before, if the VC 110 clears the safety alert without operator intervention, then the ROC 114 removes the message and buttons and returns to its display state prior to the alert at 310.

[0028] If the operator presses the 'no' button at 314, then the ROC 114 returns to the 'hazard detected' display state 306 described previously.

[0029] If the operator presses the 'yes' button, the ROC 114 displays a countdown at 316 and requires the operator to keep the 'yes' button depressed for a short duration to confirm the operator's intent at 318. By way of example, this duration can be five seconds. If the button is kept pressed beyond the timeout, then the 'yes' press is accepted and the ROC 114 sends an "ignore obstacle" command to the VC 110 at 320. The ROC display returns to the 'hazard detected' state and monitors the safety alert status from the VC 110, expecting it soon to be cleared.

[0030] A number of different strategies may be used in the VC 110 to calculate the region-to-ignore. The choice of strategy affects the desired semantics of the 'ignore hazard' action offered to the operator.

[0031] The region-to-ignore may be generated as a polygon of fixed shape and scale corresponding to the field of view of the video camera oriented toward the hazard 210 (i.e., the camera chosen for the ROC video display of the hazard). I n one exemplary embodiment, this shape 400 is an arc of an annulus located in front of the vehicle, typically 90 degrees of arc with a near radius of 5 meters and far radius of 30 meters as shown in FIG. 4A. When the region-to- ignore is generated using this strategy, the 'ignore hazard' action corresponds to the notion of "ignore any hazard currently visible from this camera."

[0032] The region-to-ignore may also be generated with a shape and scale 402 corresponding to the hazard 210 itself. One option is to use a fixed simple shape (e.g., a disc, hexagon, octagon), which is positioned over the hazard and scaled to (over)cover the hazard as shown in FIG. 4B. Another option is to use the polygon 404 of the detected hazard itself, in which case the hazard polygon is outset by a small distance to ensure that the region-to-ignore covers a reasonable amount of variation in the detected shape over time due to imperfect sensing as shown in FIG. 4C. When these strategies are used to generate the region-to-ignore, the 'ignore hazard' action corresponds to the notion of "ignore this specific detected hazard."

[0033] If the polygon is initially generated relative to the vehicle itself (in a reference frame fixed to the vehicle), then it is transformed into absolute geographic coordinates (in a reference frame fixed to the earth) using the current absolute geographic position and orientation of the vehicle provided by the VC navigation module.

[0034] In the particular exemplary embodiments disclosed herein, the vehicle is a driverless, autonomous vehicle that is assigned to execute tasks by an operator stationed at a remote console. It should be understood, however, that the methods and systems disclosed herein are equally applicable to implementations where the operator (and his or her console) are located in the vehicle. In this case, the console may be tethered to the vehicle.

[0035] The processes of the system described above may be implemented in software, hardware, firmware, or any combination thereof. The processes are preferably implemented in one or more computer programs executing on programmable computer systems for the VC and ROC, each including a processor, a storage medium readable by the processor (including, e.g., volatile and non-volatile memory and/or storage elements), and input and output devices. Each computer program can be a set of instructions (program code) in a code module resident in the random access memory of the computer systems. Until required by the computer systems, the set of instructions may be stored in another computer memory (e.g., in a hard disk drive, or in a removable memory such as an optical disk, external hard drive, memory card, or flash drive) or stored on another computer system and downloaded via the Internet or other network.

[0036] Having thus described several illustrative embodiments, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to form a part of this disclosure, and are intended to be within the spirit and scope of this disclosure. While some examples presented herein involve specific combinations of functions or structural elements, it should be understood that those functions and elements may be combined in other ways according to the present disclosure to accomplish the same or different objectives. In particular, acts, elements, and features discussed in connection with one embodiment are not intended to be excluded from similar or other roles in other embodiments.

[0037] Accordingly, the foregoing description and attached drawings are by way of example only, and are not intended to be limiting.