RELATED APPLICATION

[0001] The present patent application claims the benefit of prior filed co-pending U.S. Provisional Patent Application No. 61/746,898, filed on December 28, 2012, the entire content of which is hereby incorporated by reference.

BACKGROUND

[0002] Embodiments of the invention relate to detecting a skidding condition of a vehicle using a forward-facing camera.

[0003] More and more vehicles are equipped with a forward-facing video-camera. The camera is mainly used to provide either "fusion" benefits (e.g., a combination of radar and video information to avoid 'false positive' interventions from adaptive cruise control (ACC) systems) or for use in standalone applications (e.g., for automatic control of vehicle lights (e.g., automatic "high beams") or Forward Collision Warning systems).

[0004] At the same time "standstill management" functions are becoming more and more popular. Examples of standstill management functions include: adaptive cruise control (ACC) stop & go, automatic vehicle hold (AVH), hill hold control (HHC), and controlled deceleration for parking brake (CDP).

SUMMARY

[0005] A "standstill management" function is, in general, included or integrated into an electronic control unit (ECU) of a vehicle. Often, the function is included as part of the electronic stability control (ESC) system. One type of ESC is the ESP system manufactured by Robert Bosch GmbH. The standstill management function becomes active at or during vehicle standstill, either automatically or based on driver input.

[0006] Many existing standstill detection methods only work if wheel speed/yaw-rate sensors are fully functional. More particularly, one of two methods is often used to detect a skidding vehicle.

[0007] The first method involves releasing a brake pressure in one wheel. The ECU looks for indications that could increase the risk of skidding (e.g., ABS activation at low speeds). If the algorithm has determined a higher risk for skidding, the brake pressure of one non-driven wheel is released as soon as standstill has been detected (based on wheel speeds/edges). Skidding is detected if the non-driven wheel starts turning. When the wheel starts turning, the "standstill management" functions are cancelled.

[0008] A disadvantage of the reduced pressure detection method is increased NVH. In this method the return pump in the brake system has to be activated to release the brake pressure from the one wheel.

[0009] The second method involves yaw rate monitoring. The yaw-rate sensor signal is analyzed in standstill (based on wheel speeds/edges) to check for a skidding vehicle. If the yaw-rate sensor signal is above a certain threshold, skidding is assumed (or detected) and all "standstill management" functions are canceled.

[0010] Unlike these two methods, a video camera can be operated independently of wheel speed, brake pressure, and yaw-rate sensors.

[0011] In one embodiment, the invention provides a method of determining "true" standstill in a vehicle. The method includes detecting an object outside of the vehicle via a camera, saving first data on the detected object in a memory, detecting the same object after a delay, saving, after the delay, second data on the detected object in a memory, analyzing, by a controller, the first and second data, and determining "true" standstill has occurred based on the analysis.

[0012] In another embodiment, the invention provides a system for determining a "true" standstill condition of a vehicle. The system includes a camera and a controller. The camera is configured to capture an image of objects outside the vehicle. The controller is configured to receive images of objects from the camera, save images received to memory, compare images saved in memory, determine from the comparison of the images in the memory if the vehicle is moving, and provide an indication whether the vehicle is at a "true" standstill.

[0013] Other aspects of the invention will become apparent by consideration of the detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Fig. 1 is a graph of exemplary AVH operation.

? [0015] Fig. 2 is a block diagram of an embodiment of a vehicle standstill detection system.

[0016] Fig. 3 is a schematic diagram of a vehicle.

[0017] Fig. 4 illustrates the use of a forward-facing video camera to detect a vehicle standstill.

[0018] Fig. 5 is a flow chart of the operation of a standstill detection system. DETAILED DESCRIPTION

[0019] Before any embodiments of the invention are explained in detail, it is to be understood that the invention is not limited in its application to the details of constniction and the arrangement of components set forth in the following description or illustrated in the following drawings. The invention is capable of other embodiments and of being practiced or of being carried out in various ways.

[0020] As an example, Fig 1 shows a graph of AVH to illustrate how "standstill management" functions work (other functions such as ACC Stop and Go, HHC, CDP, etc. also have a need to determine true standstill). The horizontal axis of the graph represents time and the vertical axis of the graph represents velocity. In AVH, the driver decelerates the vehicle by stepping on the brake (shown at point 105). A period of deceleration occurs. Once the vehicle stops, the brake pressure provided by the driver is used to hold the vehicle (point 110). The driver then releases the brake pedal (point 1 15). Following a period of automatic hold, the AVH hands control over to an automated parking brake (point 120), and later the driver steps on the accelerator pedal (requests drive off) and the automated parking brake (APB) is released (point 125). A period of acceleration then follows.

[0021] Vehicle standstill is usually determined inside the ESC-ECU by looking at the wheel speed sensor data. As soon as the w ^r heel speeds of all four wheels are below a certain threshold, it is assumed (or detected) that the vehicle is at a 'standstill. ' If a more precise determination of vehicle standstill is desired an evaluation of the raw- input of the wheel speed sensor edges is performed. If no edges have been received for a certain amount of time (usually between 300-740ms) 'standstill' is assumed. [0022] Point 1 10 on the graph is a critical area where it is necessary to know if the vehicle is in 'true' standstill or is skidding (e.g., with four locked wheels).

[0023] It is difficult to differentiate between 'true' standstill and skidding during braking, especially on low-mue surfaces. On such surfaces, it is possible to lock all four wheels at low speeds or close to standstill. A typical situation where this can happen is at the end of an anti-lock braking system (ABS) stop (where ABS is exiting control but the vehicle has not yet reached standstill). The problem is that, based on wheel speed sensor data alone, it is not possible to differentiate between a 'true' standstill and skidding with locked wheels.

Knowing the difference however, is important. Erroneously activating a "standstill management" function can lead to a non-steerable vehicle and, thus, endanger the vehicle driver and occupants.

[0024] In the case of AVH, for example, this would lead to the current brake pressure being maintained. In AVH, releasing the brake pedal alone does not release the brake pressure. The AVH system releases the brake pressure only if either APB takes over or 'drive off is detected. Thus, the AVH system will maintain braking unless a skid condition is detected, the APB takes over, or a drive off occurs.

[0025] Present AVH systems use the detection methods described above to determine if a skidding condition is occurring such that the brake pressure should be released and control should be returned to the driver.

[0026] Once 'skidding' has been detected, the "standstill management" function is cancelled to allow the driver full control of the vehicle. The differentiation between 'true' standstill and skidding is currently performed inside the ESC-ECU.

[0027] As shown in Fig. 2, one embodiment of the invention includes an ESC-ECU 200 connected to a forward-facing video camera 205, a plurality of wheel speed sensors 210, a yaw rate sensor 215, and a plurality of vehicle brakes 220.

[0028] FIG. 3 shows a schematic view of a vehicle 300. The vehicle 300 has four wheels 305 A, 305B, 305C, and 305D. In some other embodiments, the vehicle 300 can have a different number of wheels. The wheels 305 A, 305B, 305C, and 305D are connected to two axles 310A and 310B, as shown. The four wheels are monitored by a plurality of wheel speed sensors 210A, 210B, 210C, and 210D. The wheel speed sensors 210A, 210B, 210C, and 210D are coupled to an electronic processing unit ("ECU") 315 (i.e., a controller). The vehicle 300 also includes other sensors such as a yaw rate sensor 215, a forward facing camera 205, and one or more object detection devices (e.g., radars) 320. In some

embodiments, the ECU 315 communicates with other devices (e.g., sensors, other controllers) in the vehicle 300 via a bus 330 (e.g., a CAN bus).

[0029] The ECU 315 includes a processor (e.g., a microprocessor, microcontroller, ASIC, DSP, etc.), computer readable media or memory (e.g., flash, ROM, RAM, EEPROM, etc.), which can be internal to the processor, external to the processor, or a combination thereof, and input/output circuitry.

[0030] The invention determines 'true' vehicle standstill by using a forward-facing video- camera. An image processing algorithm of the video-camera determines (based on the video image) if the vehicle is at a 'true' standstill or not. In some embodiments, the system identifies objects, and looks at their trajectory, using this information to determine 'true' standstill. To ensure a short detection time, a fixed object (e.g., a traffic sign) is

selected/identified/evaluated to determine 'true' standstill, hi other embodiments, special lane markings (e.g., "bot-dots") are used because they are already being evaluated by the camera for lane detection.

[0031] For example, as shown in Fig. 4, a vehicle 400 has a camera that detects a stop sign 405. By evaluating the image of the stop sign 405 (e.g., whether the size of the image is changing), the ESP-ECU can determine whether the vehicle 400 is at a standstill.

[0032] A moving object (e.g., a pedestrian crossing the street in front of the vehicle) can also be used. However, since a moving object has more degrees of freedom (e.g., left/right, and forward/backward movements), it is more difficult to evaluate and more time consuming.

[0033] The detection time for determining 'true' standstill can be a fixed time period, or can be based on the type of object being used (e.g., shorter for stationary objects, longer for moving objects) or the distance to the object being used (e.g., the closer the object, the shorter the detection time because it is possible to detect changes in the object quicker when it is closer). The detection time is kept as low as possible to have the system react as quickly as possible, hi some embodiments, a fast detection time is used and multiple images are saved and compared to determine 'true' standstill. For example, images of an object can be captured very fast (e.g., every 1 msec). If the algorithm determines that two consecutive images have changed, skidding is determined. However, if the algorithm is unable to detect a change in the consecutive images, the system can compare the latest image to earlier images, and determine when the last change occurred. This would allow very quick response where possible, but would also catch skidding that the system could not determine using consecutive images (e.g., during a slow skid).

[0034] In some embodiments, 'true' standstill determination is only done below a certain speed threshold. Because wheel lock-up is not very likely at higher speeds (even on low-mue surfaces) it is not necessary to perform 'true' standstill determination until the vehicle's speed is below a predetermined threshold.

[0035] The video camera based 'true' standstill information is provided to other ECUs in two ways: (1) the camera constantly provide a 'true' standstill flag via CAN, or (2) the camera provides a 'true' standstill confirmation only (e.g. when requested by another ECU) via CAN.

[0036] As noted, knowing the difference between 'true' standstill and skidding/sliding with four locked wheels is important for most "standstill management" functions.

[0037] Estimating 'true' vehicle standstill based on the forward- facing camera provides the following advantages: independent verification of 'true' standstill (i.e., a completely different calculation method is being used) estimation works in the case where the ESC system and'Or certain sensors are off due to error (could be beneficial for other ECUs that still require standstill information (e.g. APB)) estimation will also work in case of noise on one or more sensor signals (whereas the ESC-ECU might have difficulties determining 'true' standstill) verification can be performed inside a separate ECU

'true' standstill information automatically available on CAN no noise/vibration''heat (NVH) concern (as compared to a method that requires the ESC return pump to be activated) no extra hardware needed

[0038] Independent verification of 'true' standstill could be beneficial in regards to compliance with safety standards (e.g., IS026262) because being able to prevent/mitigate an unintended brake torque (by recognizing the condition and cancelling the "standstill management" function) may lower the testing requirements of vehicle standstill recognition functions (e.g., by lowering an assigned Automative Safety Integrity Level (ASIL) rating for this function). An improved or second method of determining true vehicle standstill is also beneficial to an "autonomous driving" scenario.

[0039] Fig. 5 shows an embodiment of the operation of a standstill detection system. In the embodiment shown, the system checks if the vehicle is traveling slower than a predetermined threshold (step 500). If the speed is greater than the threshold, the standstill detection system does not perform standstill detection and exits (step 505). If the speed is less than the threshold, the system detects an object, via the camera 205, to use in standstill detection (step 510). As described above, the object is generally a fixed object; however mobile objects can be used as well. Next, the system delays for a time period (step 515). The time period can be detemiined using the speed of the vehicle (e.g., shorter if the vehicle is moving faster), a set time period (e.g., several msec), or variable (e.g., as part of a larger process in the ECU or based on a distance to the object). Following the delay, the same object is again detected (step 520) and analyzed (step 525). The analysis determines if the detected features of the object have changed from the prior detection of the object, and whether this change is indicative of movement of the vehicle. The system then determines if, based on the analysis, the vehicle is truly standing still (step 530). If the vehicle is in true standstill, the system set a standstill flag (step 535), if not, the system resets the standstill flag (step 540) and loops back to delay again (step 515) and continue to check for true standstill. As described above, the standstill flag can continuously be made available (e.g., via CAN bus) or can be provided when requested by another system.

[0040] Thus, the invention provides, among other things, a system and method for determining true standstill of a vehicle.