FIELD OF THE DISCLOSURE

[0001] The present disclosure relates to a driver attention system. More particularly, the present disclosure relates to an automated driving system including a system for attracting driver attention and improving the driver's awareness prior to or during resuming control of the vehicle.

BACKGROUND OF THE DISCLOSURE

[0002] An automated driving system is a motor vehicle driving automation system that is capable of performing part or all of the dynamic driving task (DDT) on a sustained basis.

[0003] An automated driving system can be mounted or is to be mounted in a car or a vehicle (such as a car, a truck, an airplane).

[0004] In the case of road vehicles in particular, it may range in level from no driving automation (level 0) to full driving automation (level 5) according to SAE norm J3016.

[0005] In order to realize the desired functionality, an automated driving system normally comprises at least one sensor, an electronic control unit, and feedback devices which transmit information to the driver and/or act on control member(s) of the vehicle (for instance the steering shaft, the brake, the accelerator pedal or the like) instead of the driver to take some driving load off the driver.

[0006] An automated driving system is at least capable of assuming part of the driving task (for instance, to perform longitudinal control of the vehicle). In particular, many automated driving systems are designed to assist the driver and are therefore called Advanced Driver Assistance Systems (ADAS). Some automated driving systems are capable of assuming the whole driving task, at least during some periods. Such systems are classified at level 3, 4 or 5 according to SAE norm J3016.

[0007] The present invention concerns an automated driving system classified at level 3 or 4 according to SAE norm J3016. [0008] During the periods when the automated driving system is activated, in many cases the driver still has to be able to be ready to take back control of the vehicle. Indeed, in some circumstances the automated driving system can possibly be unable to react properly to the circumstances, for example, a high visual load scenario/period; it is then necessary for the driver to take over within a certain time and to control his or her vehicle by braking, turning, etc., in order to avoid any accident.

[0009] Despite this obligation, during these periods and thanks to the automated driving system, the driver may perform to some extent non-driving- related tasks such as using his/her smartphone, texting, etc., but he or she still remains obligated to be able to safely take over at any moment.

[0010] Various studies have shown that drivers, when performing nondriving related tasks, tend to quickly focus on the non-driving related-task; it then becomes more difficult for them to react and to take over if/when it becomes necessary to do so.

SUMMARY OF THE DISCLOSURE

[0011] The inventors have recognized that there is a need for an automated driving system which makes it as easy as possible, for the driver, to quickly become ready and able to control his vehicle when the driver has not been focused on the driving scene (e.g., texting, reading, etc.)

[0012] According to embodiments of the disclosure, in order to meet the above need, a driver attention system is provided. The driver attention system includes an electronic control unit, and a driver stimulation device configured to draw attention of the driver via stimuli to a desired scene location, wherein the electronic control unit is configured to cause the driver stimulation device to be activated so as to indicate at least one of the presence of a relevant hazard and one or more intended maneuvers to be undertaken by the automated driving system.

[0013] By providing such a system, it becomes possible to quickly obtain the attention of an operator of vehicle, even when the operator may be under the assumption that the automated driving system is handling the driving tasks, and the operator undertaking other tasks unrelated to driving (e.g., cell phone use, conversation, video viewing, etc.). Further the system enables the operator to quickly understand the risks associated with the current driving scene, where those risks exist, and to take control from the automated driving system, and to handle the driving tasks in a safe and efficient manner.

[0014] The driver stimulation device may comprise at least one stimulation signal-emitting element among a light source, a sound-emitting device, and a haptic signal-emitting device.

[0015] The indications of one or more intended maneuvers may include an indication of a change in lateral trajectory and a change in longitudinal trajectory.

[0016] The change in lateral trajectory may correspond to a lane change maneuver.

[0017] The change in longitudinal trajectory may correspond to an acceleration or deceleration of the vehicle.

[0018] The electronic control unit may be configured to indicate at least one of a road position of the relevant hazard and a distance to the relevant hazard.

[0019] The driver stimulation device may comprise a plurality of LED light sources.

[0020] The LED light sources may be arranged according to at least one of the following configurations: 1) substantially linearly on top of a dashboard of a vehicle in which the driver stimulation device is mounted, 2) around the perimeter of a steering wheel of the vehicle in which the driver stimulation device is mounted; and on an interior portion of A-pillars of the vehicle in which the driver stimulation device is mounted.

[0021] A pulse frequency of the light and/or variations in light color may form at least a portion of the indication.

[0022] An intended lateral maneuver indication may comprise sequential illumination of lights from a center of the stimulation device to a lateral portion of the stimulation device consistent with a direction of the lateral maneuver.

[0023] An intended longitudinal maneuver indication may comprise sequential illumination of lights from either a center of the simulation device to both lateral portions of the stimulation device, or from both lateral portions of the stimulation device to the center of the stimulation device.

[0024] A deceleration maneuver may comprise sequential illumination of lights along the stimulation device from both lateral portions of the stimulation device to the center of the stimulation device, and an acceleration maneuver comprises sequential illumination of lights from a center of the simulation device along the stimulation device to both lateral portions of the stimulation device.

[0025] An intended maneuver may be indicated at least 2 times prior to beginning execution of the indicated maneuver.

[0026] According to further embodiments of the present disclosure, a method for improving driver attention during automated driving is provided.

The method includes acquiring surrounding information around a vehicle, determining, based on the surrounding information,

1) the presence of at least one relevant hazard and/or

2) one or more impending maneuvers to be undertaken;

and

[0027] activating a driver stimulation device configured to draw attention of the driver via stimuli to a desired scene location related to the at least one relevant hazard and/or to indicate the one or more impending maneuvers to be taken by an automated driving system of the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention may be better understood and its numerous other objects and advantages will become apparent to those skilled in the art by reference to the accompanying drawing wherein like reference numerals refer to like elements in the following figures and in which :

Fig. 1 is a schematic drawing in perspective of a driver attention system according to a first embodiment of the invention;

Fig. 2 is a schematic drawing in perspective of a driver attention system for attracting attention of a driver to a specific location within a driving scene;

Figs. 3A is a chart showing exemplary stimulation patterns associated with a particular scenario and/or traffic scene;

Figs. 3B-3D are exemplary traffic scenes associated with the stimulation situations patterns in the exemplary decision chart of 3A;

Fig. 4 is an exemplary decision chart for determining when stimuli is to be provided to a driver; and

Fig. 5 is a flowchart highlighting an exemplary method according to embodiments of the present disclosure. DESCRIPTION OF THE EMBODIMENTS

[0029] Fig. 1 is a schematic drawing in perspective of a driver attention system according to embodiments of the disclosure, while Fig. 2 shows another driver attention system of the disclosure that may be used alone or in

combination with the system of Fig. 1. Vehicle 100 may include an automated driving system 20 and a driver attention system 30.

[0030] Vehicle 100 may comprise a dashboard 10, a windshield 12, a steering wheel 14 and two pillars 16A and 16B located on the lateral sides of the windshield 14, among others.

[0031] The automated driving system 20 may comprise an electronic control unit 25 and a driver attention system 30, among others.

[0032] The electronic control unit 25 (ECU 25) may be an automotive-grade control unit comprising one or more processors, and may, for example, comprise a plurality of control units. One of skill understands that, while discussed in the context of being mounted in vehicle 100, control unit 25 could be located (all or in part) in a remote location, with the remote elements being configured to communicate in real time with the elements aboard vehicle 100, for example, via any suitable wireless communication protocol (BLE, 3G, 4G, etc.)

[0033] Control unit 25 may be configured to receive information from the automated driving system 20, sensors associated with the vehicle 100 (e.g., speed sensors, trajectory sensors, cameras, etc.) to enable control unit 25 to carry out algorithms for making decisions related to automated driving. The automated driving system may comprise other electronic control units, actuators, to enable the automated driving system to carry out its main function of driving the vehicle and/or assisting the person seated in the primary position.

[0034] By primary position it is intended to refer to the location in the vehicle in which a passenger/d river seated therein would be expected to resume control of the vehicle from automated driving.

[0035] In the present case, the automated driving system 20 may enable Automated Driving in which all or a majority of the controls of the car are handled automatically by the automated driving system 20. However in this mode, the automated driving system 20 may be unable to handle certain driving situations, and in such situations it becomes desirable that the driver take over control of the vehicle. For example, where a heavy driving load presents itself (e.g., bad weather, traffic congestion, road works, etc.) it may be desirable that a human driver intervene to assist the automated driving system 20. Accordingly, the automated driving system 20 of the present example is a motor vehicle driving automation system classified at level 3 according to SAE norm J3016.

[0036] The driver attention system 30 may comprise one or more

stimulation elements to stimulate an occupant of the primary position in the vehicle in order to develop awareness when the desire to draw such awareness arises. These stimulation elements may include, for example, one or more light- emitting devices 32, one or more sound-emitting devices 36, one or more haptic signal-emitting devices 38, etc.

[0037] Each stimulation element of stimulation device 30 may be

communicatively connected to ECU 25, for example, via a wireless connection or a wired connection as desired.

[0038] The one or more light-emitting devices 32 may comprise, for example, a light strip 32 comprising a plurality of LEDs (Light-emitting diodes) or other suitable light sources, mounted on dashboard 10, windshield 12, and/or steering wheel 14, for example. Alternatively, or in addition, heads-up style lighting may be provided as one or more light emitting devices.

[0039] The one or more sound-emitting devices 36 may comprise one or more loudspeakers 36 located, for example, in a radial arm of the steering wheel 14, in dashboard 10, in ears of a seat headrest (not shown).

[0040] The one or more haptic signal emitting devices 38 is formed essentially by two vibrating plates 38A,38B arranged in the seat bottom 52 of the driver's seat 50 (shown in dashed lines on Figs.l and 2).

[0041] These locations noted above have been selected so that the stimulation signals may be easily perceived by the driver sitting in a primary position of the vehicle 100. One of skill in the art will recognize that these locations are exemplary only.

[0042] The location of the light-emitting device (the strip 32) may be chosen so as to be readily perceived even when the passenger in the primary position performs a non-driving related task, such as operating a smartphone, or glancing to one side of the vehicle or the other, or even behind, to have a conversation with another passenger, for example. [0043] Fig. 2 represents an automated driving system similar to the automated driving system represented by Fig. 1. The primary difference between these two systems is the location of the light-emitting device 32. In the embodiment of Fig.2 the light-emitting device 32 comprises two strips 32A, 32B, each of them comprising a plurality of LEDs. The LEDs are numbered 341A to 347 A on the left-side strip 32A, and 341B to 347B on the right-side strip 32B. These LED strips 32 may replace the single LED strip 32 of the first

embodiment, or may be used in conjunction therewith. Further detail of the structure may be found in the description above with regard to Fig. 1.

[0044] Fig. 3A is a chart showing exemplary stimulation patterns associated with exemplary scenarios and/or traffic scenes, and particularly, for

implementation where one or more LED strips comprise a stimulation element of the stimulation device 30.

[0045] The stimulation elements, for example, light strip 32 may be configured to provide feedback to the occupant of the primary position indicating a state of the Automated Driving function, for example, indicating the automated driving is currently on. In order to accomplish this, for example, when the automated driving system 20 is activated, the electronic control unit 25 may activate the driver attention system 30 according to a periodic stimulation pattern, e.g., a basic pulsed light pattern, to inform the person in the primary position that the automated driving system 20 is in operation. The stimulation pattern chosen may be determined based on, for example, the disclosure of international application no. PCT/EP2017/060840, incorporated herein by reference.

[0046] Further, in the embodiment under consideration, road hazards may be broken out into two categories, global hazards (e.g. general traffic, weather issues, etc.) and specific hazards (e.g. closed lanes, obstacles, strange vehicle behavior, etc.). In addition, intended maneuvers of the automated driving system 20 may be considered separately, for example directional changes of the vehicle, acceleration/deceleration, etc.

[0047] Based on the presence of one or more light-emitting devices 32, various patterns, colors, intensities, and illumination positions may be used for attracting and directing the attention of a person sitting in the primary position of the vehicle. As shown in Fig. 3A when it is desired to indicate a global hazard perceived by the automated driving system 20, all light sources on light- emitting device 32 may be illuminated, with for example, a pulsing variation in intensity of the light sources, as shown at Fig. 3B. In other words, the entire light strip 32 may appear to be pulsating, for example, at a different frequency than a standard, basic frequency indicating operation of the automated driving system 20, where such a standard basic pulse is utilized.

[0048] Where a global hazard is determined to be of low criticality, a lighter color may be chosen, such as white, for example. Where the global hazard is of greater criticality, variations in color may be presented and/or more rapid pulsing of the light sources of light strip 32 may be undertaken. For example, where a critical global hazard is present, e.g. a severe thunderstorm

approaching, the light strip 32 may pulse at a faster rate and with a blue color. One of skill in the art will recognize this as exemplary only, and that other variations may be selected with the intention of convincingly conveying a particular risk of an identified global hazard.

[0049] Regarding specific hazards, when such a specific hazard is identified, a pulse similar to that of the global hazard or may be implemented in a first color, while one or more light sources on light-emitting device 32 may be illuminated at a specific location in a second color which is different than the first color of the basic or global hazard pulse.

[0050] Fig. 3C shows one exemplary representation of such a configuration, with a specific hazard (i.e., a slow-moving tractor-trailer in the driving scene) being highlighted with a pulse of a different color which alternates with the basic pulse of light-emitting device 32, thereby drawing a driver's attention to the location and the driving scene of the specific hazard. In addition, a highlighted area of the light-emitting device may also be caused to appear to increase/decrease as the ego-vehicle approaches/leaves the area near the identified hazard.

[0051] Importantly, the specific location at which the light-emitting device 32 draws the attention of the driver, e.g., using a different color and/or alternating pulse, corresponds to the position in the driving scene where the specific hazard exists from the perspective of (i.e., relative to) the driver, and where the driver should immediately look.

[0052] As shown in the third column of Fig. 4, intentions of the automated driving system 20 with regard to maneuvers of vehicle 100 may also be conveyed to the occupant of the primary position of vehicle 100. For example, where an intention of the automated driving system is to move laterally (e.g. change lanes, turn right, turn left, etc.) ECU 25 may cause light-emitting device 32 to sequentially illuminate lights starting from the center of the light emitting device 32 (i.e., center of the vehicle), in a direction toward a lateral portion of the stimulation device consistent with the direction of the intended lateral maneuver.

[0053] Fig. 3D highlights such an indication with the intended maneuver being a lane change to the right of the vehicle 100. In such a scenario, the light color may be maintained as white, or another suitable color that may be identified with an intended maneuver (e.g. orange).

[0054] According to some embodiments, different colors may be used to differentiate an intended actual "turn" from a lateral "shift" (e.g., a lane change or swerve). In addition, colors, intensity, and/or pulses may be used to indicate an intensity of the maneuver (e.g., emergency swerve).

[0055] Similarly, when an intended maneuver involves a braking operation (i.e., deceleration), light-emitting device 32 may cause sequential illumination of light sources starting at both of the outer lateral portions of light-emitting device 32 toward the center of light-emitting device 32.

[0056] The intensity of the lights and/or the speed with which the sequential illumination occurs, may be indicators of the intended intensity of the braking operation, for example.

[0057] Where an intended maneuver is an acceleration, again sequential illumination may be used, with the sequence being the inverse of that discussed above with regard to deceleration. In other words, the light sequence may commence at the center of light-emitting device 32 with light sources

sequentially being illuminated towards the lateral portions of light-emitting device 32.

[0058] Where for example, the implementation shown at Fig. 2 is used, one of skill in the art will understand that intended lateral maneuvers may be indicated by way of flashing either light-emitting device 32A or 32B based on the intended change of direction or lane change.

[0059] Further longitudinal changes in such a configuration may be indicated by sequentially illuminating light sources of both light-emitting devices 32A and 32B such that the sequence starts at the top or a bottom of the strip and moves either down or up, respectively, based upon the intended maneuver. [0060] Intended maneuvers may be indicated in advance with sufficient time such that such maneuvers may be notified to the person in the primary position with sufficient time to take control of the vehicle should the person so desire. For example, intended maneuvers may be indicated at least twice or possibly more times prior to commencing the intended maneuver, over a period of approximately 1 to 2 seconds.

[0061] In addition or alternatively to the light cues provided by light- emitting device 32, haptic indicators 38A and 38B, as well as audible indicators 36, may be used to indicate the above noted situations. For example, where a global hazard is identified, ECU 25 may cause an announcement to be made via audible indicators 36 noting the presence of such a hazard.

[0062] Presence and location of a specific hazard may also be indicated using audible indicators 36, for example, to announce in a similar manner to that with regard to a global hazard, while haptic indicators 38A and/or 38B indicate a position to which a person should look to identify said hazards in the driving scene. Of course, one of skill will understand that such cues may also be used in conjunction with light-emitting device 32 to provide still further attention directing capabilities.

[0063] Figs. 4 is an exemplary decision chart containing exemplary situations for which stimuli may be provided by driver attention system 30 along the X axis, with an exemplary indication to be provided noted along the y-axis.

[0064] As noted in the decision chart, global hazards of high density traffic, uphill movement, and/or weather concerns (e.g., fog/rain), may be indicated according to the global hazard stimulus described above. Specific hazards such as emergency vehicles stopped on shoulder (e.g. police car) a slow-moving or otherwise strangely operated vehicle, a vehicle located in the acceleration lane, or a road narrowing, may be indicated using the cues noted with regard to a specific hazard, with attention being drawn to a location of the specific hazard in the driving scene.

[0065] With regard to intentions of the automated driving system 20, intended maneuvers may be indicated, for example, in conjunction with a specific hazard, e.g., where roadwork exists, slow or otherwise strangely operated vehicle is present, another vehicle is in an acceleration/passing lane. Such intended maneuvers may be indicative of evasive action to be taken by the automated driving system 20, based on the hazard, e.g., a quick lane change, and/or road narrowing requiring a lane change may be indicated using maneuver intentions as described above.

[0066] As shown in Fig. 4, in such situations, two types of indications may be provided for example both a specific hazard and the automated driving systems intentions for avoiding the specific hazard. In such situations, one or more elements of the driver attention system may be utilized for capturing and directing the driver's attention.

[0067] Fig. 5 is a flowchart highlighting an exemplary method according to embodiments of the present disclosure.

[0068] The driver attention system 30 may be active while the Automated Driving function is active: for example, starting from the moment the

Automated Driving function starts, by a driver's demand (by pushing a specific button for example); and it may terminate when the Automated Driving function stops, as actual driver's control takeover is effective (longitudinal and/or lateral control). During such time, ECU 25 may be configured to receive state information regarding the current vehicle (e.g., speed, heading, etc.) and surroundings of the vehicle (e.g., location, lane lines, driving scene information, etc.) (step 305).

[0069] While the automated driving is currently in operation (step 310), a basic stimulation or "reassurance" pattern may be provided by the driver attention system 20. In order to accomplish this, for example, when the automated driving system 20 is activated, the electronic control unit 25 may activate the driver attention system 30 according to a periodic stimulation pattern to inform the person in the primary position that the automated driving system 20 is in operation. PCT/EP2017/060840 provides examples of how such reassurance may be provided.

[0070] Conversely, when the automated driving system is not in operation, the driver stimulation system may be set to off, and no stimulation pattern may be provided: LED strips 32 may be turned off, loudspeaker 36 silent, vibrating plates 38A, 38B, may not vibrate (step 310: no), and the system returns to step 305.

[0071] During operation of the automated driving system 20, various situations may arise in which it may be desirable to draw the attention of the occupant of the primary position of the vehicle, for example, to prepare the occupant for resuming vehicle control. These situations, for example, those described above with regard to Fig. 4, may be interpreted by ECU 25, and indications caused to be output from the driver attention system 20 (step 315).

[0072] As noted above, during general operation (step 315: general situation) ECU 25 may cause light stimulation device 32 to function as noted above with regard to indicating that the automated driving system is in operation (step 320).

[0073] When a general hazard situation arises (step 315: global hazard), a global stimulus may be initiated by ECU 25 (step 330). For example, as noted above a pulsating of light stimulation device 32 and/or audible indications, and/or haptic indications, may be provided.

[0074] When a specific hazard is identified (step 315: specific hazard), the specific stimulus may be initiated by ECU 25 (step 340), such that a person in the primary position of the vehicle, may be alerted in his attention drawn to a location of the specific hazard in the driving scene, with the option to take over driving up the vehicle.

[0075] When the automated driving system 20 intends to make one or more maneuvers, for example to avoid a specific hazard, or to execute instructions provided by, for example a global positioning system regarding a destination of the vehicle (step 315: maneuver), the maneuver intentions stimuli may be activated by ECU 25, as described above (step 350).

[0076] Under some circumstances, a scene load may extends beyond the limits of ECU 25 and/or automated driving system 20 (step 315: over system limits), and emergency stimuli may be provided to the person the primary position (step 360), such that the occupant of the primary position may immediately resume control of the vehicle and become the driver. In so doing, any number of the stimuli described above may be used such that the driver taking control may become quickly aware of the reasons for the automated driving system 20 passing control to the driver, and the driver may therefore begin to take any actions for maintaining safe and secure operation of the vehicle 100.

[0077] One of skill will understand that no specific "emergency" stimulus has been described but that any combination of stimuli may be implemented. For example announcement via the audible indicator, vibration via the haptic indicators, etc. Such an emergency stimulus may be designed so as not to conflict with the other stimuli provided during the emergency situation so as not to affect the attention drawing effect of the driver attention system 20.

[0078] Although the present disclosure herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure.

[0079] Where any standards of national, international, or other standards body are referenced (e.g., ISO, SAE, etc.), such references are intended to refer to the standard as defined by the national or international standards body as of the priority date of the present specification. Any subsequent substantive changes to such standards are not intended to modify the scope and/or definitions of the present disclosure and/or claims.

[0080] Throughout the description, including the claims, the term

"comprising a" should be understood as being synonymous with "comprising at least one" unless otherwise stated. In addition, any range set forth in the description, including the claims should be understood as including its end value(s) unless otherwise stated. Specific values for described elements should be understood to be within accepted manufacturing or industry tolerances known to one of skill in the art, and any use of the terms "substantially" and/or "approximately" and/or "generally" should be understood to mean falling within such accepted tolerances.

[0081] It is intended that the specification and examples be considered as exemplary only, with a true scope of the disclosure being indicated by the following claims.