VEHICLE EXCEPTION EVENT MANAGEMENT SYSTEMS

Cross Reference To Related Applications

[0001] The present application claims priority to United States non-provisional patent application no. 11/595,015 filed on November 9, 2006, the entirety of which is incorporated herein by reference.

Background

[0002] The present invention relates generally to vehicle event recorders and more specifically to vehicle event recordings including a facility to create discrete data relating to non-discrete data such as audio, video, or other continuous or time sequenced data and to associate the created discrete data with other digital data associated with an event.

[0003] An exception event occurs whenever an extraordinary or unusual condition arises during use of a motor vehicle. When an exception event, or hereinafter simply an 'event', is detected or recognized, information related to the event is recorded. In particular, the recorded information includes information relating to the performance of the vehicle and operator, and the state of the environment in and around the vehicle. [0004] U.S. patent number 5,570,127, describes a video recording system for a school bus. The system has two video cameras, one for an inside bus view, one for a traffic view, and a single video recorder. The system multiplexes the two cameras at appropriate times to the recording device. A switching signal determines which of the two video cameras is recorded by the video recorder so as to view passengers at certain times and passing traffic at other times.

[0005] U.S. Patent 5,586,130, discloses a method and apparatus for detecting fault conditions in a vehicle data recording device so as to detect tampering or unauthorized access. The system includes vehicle sensors for monitoring one or more operational parameters of the vehicle. The fault detection technique contemplates storing a current time value at regular intervals during periods in which the recording device is provided with a source of main power.

[0006] U.S. patent number 5,815,071, discloses a method and apparatus for monitoring parameters of vehicle electronic control units.

[0007] U.S. patent number 5,815,093, discloses a vehicle accident recording system employing a digital camera connected to a nonvolatile memory and an accident sensing interrupter. Unless an accident is detected, the oldest memory is overwritten by the newest images. When an accident is detected the memory is blocked from further overwrites to protect previously recorded images, which may include important information about events leading up to the accident. In preferred embodiments, the system has a wired communication port whereby stored images are downloaded after an accident to a digital device capable of displaying the images.

[0008] U.S. patent number 6,002,326, teaches an antitheft device for a vehicle having both an audible alarm and visual monitoring system. Video monitor operators are responsible for monitoring and handling an emergency situation and informing a 911 emergency station.

[0009] U.S. patent number 6,088,635, discloses a railroad vehicle accident video recorder. In this system, a method and monitoring unit for recording the status of a railroad vehicle prior to a potential accident is presented. The monitoring unit continuously monitors the status of the railroad vehicle's emergency brake and horn. Operation of the emergency brake or horn triggers the system to record video images for a predetermined period of time.

[0010] U.S. patent 6,185,490, discloses a vehicle crash data recorder. The apparatus includes a three stage memory to record and retain information. The system is equipped with a serial or parallel connectors to provide instant on-scene access to accident data. [0011] U.S. patent 6,246,933, describes a traffic accident data recorder and traffic accident reproduction system. The system includes a plurality of sensors for registering vehicle operating parameters, at least one vehicle mounted digital video, audio camera is included for sensing, storing, and updating the operating parameters. A rewritable, nonvolatile memory is provided for storing processed operating parameters and video images and audio signals. A computer can use the data so that an accident can be reconstructed via the collected data.

[0012] U.S. patent 6,298,290, teaches a memory apparatus for vehicle information data. A plurality of sensors including a CCD camera, collision sensor, vehicle speed sensors, steering angle sensor, brake pressure sensor and acceleration sensor, are all

coupled to a control unit. The control unit encodes and records the information to a flash memory and a RAM memory. A wired video output terminal is provided to retrieve the high-bandwidth data.

[0013] U.S. patent number 6,333,759, describes a 360° automobile video camera system. A complex mechanical mount provides for a single camera to adjust its viewing angle giving a 360° range for video recording inside and outside of an automotive vehicle.

[0014] Patent number 6,389,339 teaches a vehicle operation monitoring system and method. Operation of a vehicle is monitored with an onboard video camera linked with a radio transceiver. A mobile modem is designed to transmit live video into a network as the vehicle travels. Using a monitoring service, video data received from the transceiver may be viewed on a home-base computer. The system is aimed at parental monitoring of adolescent driving.

[0015] U.S. patents 6,389,340; 6,405,112; 6,449,540; and 6,718,239 are each directed to automobiles camera systems which capture video images, both of forward-looking and driver views, and store the recorded images locally on a mass storage system. An operator, at the end of the vehicle service day, uses a wired connector to download information into a desktop computer system. Specialized application software is provided to playback and analyze the images and other information a highly integrated user display interface.

[0016] It is not possible in these systems for an administrative operator to manipulate or otherwise handle the captured data at an off-site location without human intervention. It is necessary for a download operator to transfer captured data to a disconnected computer system. While the downloaded files can be e-mailed or otherwise transferred through the Internet, the files are in a proprietary format and can only be read by the proprietary application. Thus, data captured by the vehicles is unavailable to parties having an interest in the data, unless they have purchased the proper software application. A second and major disadvantage is that a person must take action to download the captured data on a frequent basis.

[0017] U.S. Patent number 6,411 ,874 discloses an advanced law enforcement and response system. A highly integrated system provides control for a plurality of detector systems, including video and audio systems, distributed about a police car or other

emergency vehicle. A primary feature included in this device includes an advanced user interface and display system, which permits high level driver interaction with the system. [0018] U.S. patent number 6,421,080 describes a digital surveillance system with pre- event recording. Pre-event recording is important in accident recording systems, because detection of the accident generally happens after the accident has occurred. A first memory is used for temporary storage. Images are stored in the temporary storage continuously until a trigger is activated indicating an accident has occurred. The images then are transferred to a more permanent memory.

[0019] U.S. patent number 6,459,988 teaches methods and systems for detecting a vehicle collision using the global positioning system (GPS). A GPS receiver is combined with wireless technology to automatically report an accident and remotely locate third parties. The system uses the GPS signals to determine when an acceleration value exceeds a preset threshold which is meant to be indicative of an accident having occurred. [0020] U.S. patent number 6,636,790 describes an OBD system coupled to a microprocessor, by way of a standard electrical connector. The microprocessor periodically receives data and transmits using a wireless communications system characterizing the vehicle performance. The patent specifically calls out transmission of data on a predetermined time interval. Thus these inventions do not anticipate nor include processing and analysis steps which result in data being passed at times other than the expiration of a predetermined time period.

[0021] U.S. patent number 6,728,612, teaches an automated telematics test system and method. The invention provides a method and system testing a telematics system in a mobile vehicle a test command from a test center to a call center is based on a test script. The mobile vehicle is continuously in contact by way of cellular communication networks with a remotely located host computer.

[0022] U.S. patent number 6,732,031 describes systems in which motor vehicles are coupled by wireless communications links to remote host servers. [0023] U.S. patent number 6,831,556 describes composite mobile digital information system including a mobile server capable of transmitting captured information from a vehicle to a second location such as a building. In particular, a surveillance system for capturing video, audio, and data information is provided in a vehicle. [0024] U.S. patent number 6,862,524 discloses a system for determining and disseminating traffic information or route information, traffic condition information is

collected from mobile units that provide their location or position information. Further, route information may be utilized to determine whether a mobile unit is allowed or prohibited from traveling along a certain route.

[0025] U.S. patent 6,928,348 discloses a system in which Internet based emission tests are performed on vehicles having special wireless couplings to computer networks. Data may be further transferred to entities of particular interest including the EPA or California Air Resources Board, for example, or particular insurance companies and other organizations concerned with vehicle emissions and environment. [0026] United States patent number 6,947,817 discloses systems for non-intrusive diagnostic testing of oxygen sensor operation. The systems include a wireless communications link between a vehicle and a remote network of server computers. In particular, a WiFi type access point enables an analyzer to communicate by way of the Internet with a server computer hosting an oxygen sensor SOAP (simple object access protocol) service. In essence, the system relates to smog sensors for automobiles which communicate with remote servers by way of a WiFi communications link. [0027] Other U.S. patents that may be relevant to the present disclosure include U.S. patents No. 6,529,159; 6,552,682; 6,594,576; 6,664,922; 6,735,503; 6,739,078; 6,760,757; 6,795,017; 6,810,362; 6,832,140; 6,832,141 ; 6,850,823; 6,867,733; 6,882,313; and 6,922,566.

[0028] U.S. patent application publication 2006/0095175 describes a comprehensive systems having many important components. In particular, the system is a 'crash survivable apparatus' in which information may be processed and recorded for later transmission into related coupled systems. The capability to rate a driver's performance based upon captured data is a particular feature of the system, as is described in some detail.

[0029] U.S. patent publication no. 2005/0099498 titled "Digital Video System- Intelligent Information Management System" describes a digital video information management system for monitoring and managing a system of digital collection devices. A central database receives similar information from a plurality of distributed systems.

[0030] U.S. patent publication no. 2005/0100329 titled "Mobile and Vehicle-

Based Digital Video System" describes a vehicle based video capture and management system with digital recording devices optimized for field use. Because these systems

deploy non-removable media for memory, they are necessarily coupled to data handling systems via various communications links to convey captured data to analysis servers.

[0031] While systems and inventions of the art are designed to achieve particular goals and objectives, some of those being no less than remarkable, these inventions have limitations which prevent their use in new ways now possible. Inventions of the art are not used and cannot be used to realize the advantages and objectives of the inventions taught herein.

[0032] In view of the foregoing it would be desirable to provide methods and apparatus for discretizing non-discrete data associated with a recorded event.

[0033] It would also be desirable to provide analysis of non-discrete data captured in vehicle event recorders.

Summary Of The Invention

[0034] It is, therefore, an object of the present invention to provide methods and apparatus for discretizing non-discrete data associated with a recorded event.

[0035] It is also an object of the invention to provide analysis of non-discrete data captured in vehicle event recorders.

[0036] In accordance with these and other objects of the invention, apparatus and methods are provided to capture, record, analyze, and interpret information relating to or arising from the use of a motor vehicle. In particular, both discrete and non-discrete information is captured by multiple vehicle mounted sensors in response to, or triggered by, the occurrence of a noteworthy event. Non-discrete data is sent to a discretization facility where it is processed and used to produce an interpreted dataset. The interpreted dataset, which is now in a form suitable for automated analysis, is then associated and recombined with the original captured data to form a complete event dataset. The complete datasets are stored in a database.

[0037] An analysis server queries the database for records, which may include both discrete and interpreted data. Because the non-discrete data has been discretized, it is now accessible to or usable by the analysis server. Thus, the information available for

analysis is considerably broader than in previous systems. Consider, the following examples:

[0038] In response to an impulse force detected by an onboard sensor, a vehicle event recorder may capture audio and video data as well as numeric data from a plurality of vehicle sensors . However, numeric data may be insufficient to fully characterize the nature of the event. For example, it may not be possible to determine, from sensor data alone, whether a sideways impulse force is due to an accident, in which the vehicle has collided with another vehicle, or to a curb strike, in which a front wheel has made contact with a curb or another object. A curb strike may be readily distinguishable from an accident upon review of captured video data by noting that the video does not show any other nearby vehicles or that the video appears to show the operator trying to parallel park. An expert event interpreter may be able to determine other important aspects or parameters associated with an event and indicate their occurrence in a discrete manner. For example, the expert can determine that the impact should be characterized as a curb strike and annotate an event data set accordingly.

[0039] An event interpreter's review is facilitated by a purpose-designed, graphical user interface system comprised of control objects which can be set to various values according to the interpretation or judgment of the interpreter. The control object values, having been manipulated by the interpreter, may be associated with a particular event and stored in a database. For example, in a general, daily review of vehicle activity, a computer (analysis server) may determine that a curb strike has occurred. Further, the analysis server may determine a degree of severity by further analyzing force data. And finally may determine that maintenance is necessary. The analysis server transmits an order for a front-end alignment (for example via e-mail) to the fleet maintenance department. When the vehicle is in for maintenance, the necessary alignment will be performed.

[0040] In a second illustrative example, an analysis server analyzes events associated with a particular operator. The operator may have operated only one or many different vehicles in a fleet and may have any number of event records. Each event record having an association with the operator in question is retrieved using an appropriate database query. For example, an analysis server may submit a query for all

events associated with the particular operator which are characterized as an "excess idle time" event. When a vehicle is left idling for extended periods, the operation efficiency of the vehicle is reduced. Accordingly, fleet managers discourage employee operators from idling for extended periods. However, under some conditions, extended idling is warranted. For example, in extremely cold weather, it is necessary for a school bus loading children to run the engine to heat the bus. On the other hand, idling with no one aboard is unnecessary even in cold weather. Discrete data produced by vehicle sensor may be insufficient to properly distinguish these situations. An excess idling event should only be identified after careful interpretation of non-discrete video data. The analysis server can automatically detect if a single operator has accumulated a number of excess idling events that exceed a predetermined threshold such as three events in a month. The analysis server can order a counseling session between a fleet manager and an operator exceeding the limit. In this way, automated systems which depend upon interpreted data are useful for managing operations of fleet vehicles.

[0041] A vehicle event recorder of the present invention captures non-discrete data, including images and audio data, as well as discrete digital or numeric data such as sensor data. The information or data is passed to a processing station or discretization facility. The processing station includes an event record media player for playback of selected datasets and a graphical user interface for adjusting or setting data associated with the event dataset(s).

[0042] Preferred embodiments of vehicle exception event management systems are particularly characterized as including the following elements. A vehicle event recorder is a vehicle mounted system to capture data relating to vehicle use in response to a trigger or exception event. The vehicle event recorder outputs such data to a database and also to a specially arranged discretization facility for interpretation. The discretization facility may be arranged as a part of a computer network to which the vehicle event recorder is similarly coupled. Output from the discretization facility is carefully coupled to stored event records thus preserving their association with the appropriate event. In this way, stored exception event records are far richer with information developed by both discrete and interpretive systems.

Brief Description Of The Drawings

[0043] A better understanding of the present invention can be had with reference to the detailed description with reference to the appended drawings, in which like reference characters refer to like parts throughout and in which:

Fig. 1 is a schematic representation of an exemplary exception event management system according to the principles of the present invention;

Figs. 2 and 3 show the discretization process of Fig. 1 in further detail;

Figs. 4 and 5 show exemplary illustrative embodiments of the graphical user interface of Fig. 3;

Fig. 6 shows an illustrative relationship between graphical user interface controls and the fields of an event record;

Fig. 7 is a schematic representation of a vehicle video data recorder including various vehicle sensors;

Fig. 8 is a block diagram of the structure of an illustrative event record and an interpreted event record; and

Fig. 9 is a block diagram showing an overview of an event management system according to the principle of the present invention.

Detailed Description

[0044] Throughout this disclosure, reference is made to terms which may or may not be exactly defined in popular dictionaries as they are defined here. To provide a more precise disclosure, the following definitions are presented with a view to clarity so that the true breadth and scope of the disclosure may be more readily appreciated. Although every attempt is made to be precise and thorough, language is necessarily imprecise and not all meanings associated with each term can not be completely set forth. Accordingly, each term is intended to also include its common meaning which may be derived from general usage within the pertinent arts or by dictionary meaning. Where the presented definition is in conflict with a dictionary or arts definition, one must consider context of use and provide liberal discretion to arrive at an intended meaning.

[0045] Vehicle Event Recorder — A vehicle event recorder (VER) is a vehicle mounted apparatus including video recording equipment, audio recording equipment,

vehicle system sensors, environmental sensors, microprocessors, application-specific programming, and a communications port, among others. A vehicle event recorder is configured to capture information and data in response to the detection or identification of an abnormal or unusual condition or event.

[0046] Exception Event — An exception event is any occurrence or incident which satisfies predetermined criteria for the identification of an event. The identification of an exception event causes the collection and recording of an event dataset relating to the status and performance of the operator and vehicle. The dataset preferably includes video images of the environment in and around the vehicle. For example, an exception event may be identified or detected when a measured physical parameter exceeds a prescribed threshold or a user operates a 'panic button'.

[0047] A basic understanding of these systems is realized in view of the Fig. and, in particular, the overview of Fig. 1. Motor vehicle 1 includes vehicle event recorder 2 having a video camera, memory, and other systems adapted to record selected data upon the detection or identification of an exception event. More particularly, audio, video and sensor data associated with a period immediately prior to and immediately after the exception event is recorded to memory for temporary storage. In some versions, an OBD 3 is connected to the vehicle event recorder so that data from the OBD may also be captured to the memory.

[0048] Periodically, the vehicle is coupled to a computer network and captured data is transferred from the memory to database 4. For instance, recorded data may be downloaded once every shift, day, week, or other convenient interval to discretization facility 5, and analysis server 6. In preferred embodiments of the invention, the vehicle is connected to a system network merely by returning to a predetermined parking facility, wherein a wireless data communications link is established between the vehicle event recorder and a local wireless access point.

[0049] A single event data record is allocated for each new event data set and each data record is assigned a unique identifier 8, sometimes known as a primary key, so there is a one-to-one correspondence between an event and an event data record stored in the database. Event data record 7 may comprise both non-discrete data portion 9 and

discrete data portion 10. As discussed herein, discrete data portion 9 may include video image series, analog audio recordings, acceleration measurements, and the like, while discrete data portion 10 may include indications of headlights status, numeric speed, steering angle, gear ratio, and the like. An event record transferred from the vehicle is a preliminary data record, because interpreted portion 11 of the event record is not completed, although storage space may have been allocated.

[0050] Preliminary event data record 12 is sent to a discretization facility wherein the discrete and non-discrete data are read, analyzed, and interpreted. In preferred embodiments of the invention non-discrete data is processed by expert systems, "fuzzy logic" rules, and the like to interpret the input data. In other embodiments, a human interpreter reads or views certain event data, including non-discrete data, interprets the data, and adds discrete data values to interpreted portion 11. In still other embodiments of the invention, both machine and human discretization processes are used.

[0051] Automated interpretation is performed by discretization facility 5 which includes interpretation algorithm 14. The image processing routines are specifically designed to "recognize" particular patterns in the video data and yield a discrete output. For instance, the moment of impact is readily discoverable in a video, because the images between one frame and the next tend to change significantly at the moment of impact. Thus, a motion detection routine may be suitable to determine a precise moment of impact.

[0052] Another useful illustrative example is the automated interpretation of traffic light signals. Image analysis can be applied to determine what traffic light color was present when the vehicle approached an intersection. In a more advanced scheme, changes in the traffic light may be automatically quantified by image analysis to determine how much time has passed between a light change and an impact. These and other fully automated image processing modules may be used to read non-discrete image data and produces discrete numeric outputs. Of course, many other image recognition algorithms may be used to produce discrete output from image interpretation.

[0053] Audio data may be processed by discretization algorithms configured to recognize the screech of skidding tires or the crushing of glass and metal to estimate

when panic braking started and the moment of impact. Other algorithms may analyze the energy or spectral content of audio data to estimate vehicle speed and the like. In addition, other kinds of non discrete data may be produced in an analogous manner. For instance, non-discrete data, such as acceleration data captured from an analog signal may be integrated to obtain velocity and position data.

[0054] In addition to automated means for interpretation of non-discrete data, a discretization facility may also include means for manual interpretation of non-discrete data. In some cases, there are no effective algorithms for interpreting non-discrete data. Accordingly, a discretization facility also provides for a human interpreter to review and interpret non-discrete information of an event record. Specifically, a discretization facility is designed to present data captured by vehicle event recorder systems in a dynamic graphical/image presentation. The software provides for the simultaneous display of a graphical user interface including devices for discrete data entry. Such devices or controls are associated with a particular attribute or event record field relating to an event, driver, vehicle, or the environment. Each control has a range of discrete allowed values as well as a present state value. A human interpreter reviews the data via the discretization facility software and manipulates the graphical user interface, to generate discrete interpreted data. Although they are disclosed separately, automatic and human data interpretation may be used together in a serial or parallel manner to produce discrete data based on non-discrete information. The interpreted data is combined with the data in the preliminary event record to form complete event record 15, which is returned to the database for storage.

[0055] Complete event records comprising discrete data, non-discrete data, and interpreted data may be analyzed to identify events requiring actions 16, such as maintenance (wheel re-alignments in response to 'curb strikes'), recognition (an operator exceeds 10,000 hours without an accident), and intervention (driver violations require remedial training), periodic performance reports on drivers/vehicles, among other actions. Possible actions are discussed in more detail below. Analysis servers may run periodic analysis on event data or may run analysis 'on-demand' in response to custom requests formulated by an administrator. In this way, the systems of the present invention provide for advanced analysis of detailed event records which include discretized or interpreted data.

[0056] Figs. 2 and 3 show discretization facility 21 in isolation and better detail.

Configured as a node of a computer network with databases 39, the discretization facility is comprised of event record media player 22 and graphical user interface 23. Media player 22 is preferably a proprietary player operable to play files stored in a predetermined format. Similarly, graphical user interface 23 is designed to support functions particular to discretization and may not be found in general purpose graphical user interfaces. In addition, the discretization facility may include interpretive algorithms 24 which read and interpret non-discrete data to provide a discrete interpreted output. A discretization facility receives as input preliminary event record 25, including non- discrete data. For example, a video or audio recording is non-discrete data which cannot be used in mathematical analysis requiring discrete inputs. The discretization facility outputs an event record including newly added discrete interpreted data.

[0057] The interpretation process is shown in more detail in Fig. 3, which shows media player data inputs, as well as an example of a graphical user interface. Discretization facility 31 includes event record media player 32 and custom graphical user interface 33. Data produced by a vehicle event recorder and an on-board diagnostics system is received by discretization facility 31 in a purpose-designed format and structure. Specifically, a timeline that synchronizes video data and OBD data ensures accurate display and viewing of the data. Common media player standards do not support playing of certain forms of data such as data collected by a vehicle event recorder or OBD system. For example Windows ^M Media Player cannot be used in conjunction with data captured in a motor vehicle, because it is not designed to process digital data 34, which may include data related to speed, acceleration, steering wheel orientation, and the like. Digital data may also include audio data recorded by an event recording system from a vehicle audio transducer, such as an operator compartment microphone or an exterior microphone. Acceleration data 36 may be presented as continuous or non-discrete data subject to interpretation. Video data 37 is captured as a series of sequential images of the environment in or near the vehicle, particularly forward exterior views of traffic and interior views of a vehicle operator. Each of these types of data may be subject to some level of interpretation to extract vital information.

[0058] Illustrative examples of data interpretation follow. Some vehicle collisions include multiple impacts. These multiple impacts might include impacts with fixed

objects such as trees and road signs or impacts with other vehicles. Consider a vehicle that is in an impact with cars both in front of and behind it. An important question may be whether being hit from behind forced the vehicle into the car ahead. A careful review and interpretation of a recording of exterior audio might contribute to a detailed timeline as to when the impacts occurred. Numeric data indicative of an operator's actions, such as panic braking, swerving, or other extreme maneuvers, etc., may be considered in conjunction with an event record timeline to determine operator attention/inattention and other related response factors. Accelerometer data can be used to indicate the start of effective braking. Acceleration data can also be interpreted to indicate a direction of travel following a collision, and can be integrated to derive position and velocity information. Video images can be played back frame-by-frame in slow motion to detect conditions not readily measured by vehicle sensors. Accordingly, a media player of the present invention is particularly designed to receive such data and to present it in a logical manner so a human reviewer can easily interpret the data. A special graphical user interface permits easy quantification and specification of various attributes which may be observed or interpreted during playback. Graphical user interface controls 38 enable the operator to set the value of an attribute from a range of values. The controls are coupled to the database via appropriate programming so that the database preserves the current values of the controls and transfers them as part of an event record.

[0059] An exemplary of graphical user interface further illustrated in Fig. 4 which shows a computer workstation monitor 41. A first portion of the display includes event video player 42 which display video images captured by a vehicle event recorder to provide a detailed visual depiction of the event scene. A video series, necessarily having an associated timeline, may be replayed on video player 42 in several modes including: fast forward, rewind, slow motion, or in actual or 'real-time' speed, among others, as is conventional in video playback systems. The display includes a graphical display field 43 for presenting graphical and numeric information. Time dependent information or data is presented in synchronization with displayed video images. For example, a binary indication of the head light status may be presented as "on" or "1" in a first video frame, but change to "off or "0" after a collision in which the lights are damaged and no longer drawing current as detected by appropriate sensors.

[0060] Another area of the display includes graphical user interface 44. A "tab strip" type graphical user interface control is particularly useful in some embodiments of the invention. Groups of logically related controls may be collected and displayed on separate tabs. Timeline control 46 permits an interpreter to advance or rewind playback at will by sliding a pip along a line. "Start" and "stop" playback controls 47 can be used to freeze a frame or to initiate normal play. Additional, controls may include fast forward, rewind, loop, and the like. Volume control 48 provides for adjusting audio playback intensely (volume). It is important to note that the graphical presentations of display field 43 are synchronized to video playback so that displayed data was captured concurrently with the capture of the displayed video frame.

[0061] Some data information presented may span the entire event period. For example, force data 49 is preferably shown for the entire event period. A "present instant" reference line 410 or time cursor may be used to indicate the moment corresponding to a displayed video frame. Clearly, conventional media players are unsuitable for use in the systems of the present invention, because they do not present event data synchronized to a video. The media player of the present invention is particularly good for displaying and interpreting a broad range of information. While a few interesting and illustrative examples of data types and controls are described above, it should be appreciated that many other types of controls and displays may also be included in a preferred embodiment of the invention. The controls shown are necessarily limited to keep the disclosure clear and the drawing understandable. There may be hundreds of parameters which are captured during an event and which might be displayed in synchronization with a frame-by-frame video of the event. One should realize that any particular parameter may contribute to a valuable understanding of the event.

[0062] The event record media player described above is preferably presented concurrently with graphical user interface 44. Graphical user interfaces may be arranged as a tab strip. For example, a "Driver" tab 412 may have controls which relate specifically to driver characterizations. Useful graphical user interface controls may include checkboxes 413, drop-down listboxes 414, radio buttons 415, sliders 416, command buttons and the like which are well known.

[0063] Checkboxes may be used to indicate binary conditions such as whether a driver was using a cell phone, smoking, alert, wearing sunglasses, made an error, using a seat belt properly, or distracted, for example.

[0064] The value of the driver characterization may be readily realized upon consideration of the difficulty of developing a sensor to automatically determine whether a driver is wearing sunglasses. By providing a means to express such data in a machine readable manner, the detailed nature of an event is realized quite easily. By reviewing an event video, an interpreter can determine that a driver was wearing sunglasses and indicate such by ticking an appropriate checkbox. The checkbox, or more precisely its present state value, is coupled to the specific event record, and stored in the database and able to be processed by computer algorithms. Previously known systems do not accommodate such machine processable data and such data, such as data on sunglass use, is left in a non-discrete form, if it is captured at all. A fleet manager may query the database to get information on the ratio of midday accidents in which drivers were wearing sunglasses versus those in which the drivers were not wearing sunglasses.

[0065] Listbox controls are used to select from a group having a discrete number of distinct members. For example a "crash type" list box may have five elements ('values') identifying five distinct types of crash. For example, a "sideswiped" crash could be indicated after careful review of the media player data. One may appreciate the difficulty of designing a vehicle sensor able to distinguish between a "sideswipe" and a "rear-ender" crash. However the distinction is easily made by someone reviewing a video collected by the video event recorder. Until such information has been reduced to a discrete form it is generally not usable by automated analysis systems.

[0066] An alternative display of record media player 51 and custom graphical user interface 52 is shown in Fig. 5. Image field 53 shows video and image data captured via any of the various cameras of a vehicle event recorder. Numeric or graphical field 54 presents non-image data captured by a vehicle event recorder during an event. The data may be shown in a graphical form, such as arrow indicators 55, to indicate acceleration direction and magnitude, or wheel graphical icon 56, to indicate steering wheel orientation angle. Presenting numeric data in graphical form may help an interpreter better visualize a situation. For example steering wheel indication 56 is far more intuitive

than a numeric value such as "117°". Cursor 57 indicates the time association with the displayed video image and moves to maintain synchronization with the displayed image frame. Graphical field 58 displays a computed path or trajectory of a vehicle as well as collisions with other objects during an event. The sequence of collision and relative severity (indicated by size of star balloon) are readily apparent from the display. Indicator 59 shows the time associated with the displayed video image. This graphic aids an interpreter in understanding details of the event scenario, with particular regard to events having a plurality of impacts.

[0067] An interpreter can manipulate the graphical user interface provided with specific controls associated with the various impacts which may occur in a single event. For illustration, the example in Fig. 5 shows three impacts. Impact 1 and 2 come close together in time, and impact 1 is less severe than impact 2. Impact 3 occurs after impact 2 and is more severe. By ticking appropriate checkboxes, an interpreter may specify the details of the event. For instance drop-down list box 511 may be used to specify the severity of the various impacts. Graphical control 512, a nonstandard graphical user interface control, graphically presents a vehicle and four quadrants A,B,C,D. An interpreter can use the control to indicate the portion of the vehicle in which the various impacts occur. In this manner, graphical user interface 52 and event recorder media player 51 are used to read and interpret both non-discrete and discrete data captured by a vehicle event recorder and to discretize interpretations. It will be appreciated that many other controls can be designed and used to specify event details and the controls described are merely illustrations and not limiting in nature.

[0068] Fig. 6 is another view of the relationships between data source subsystems and a data record structure. Discretization facility 61 may include image processing modules such as pattern recognition systems as well as a combination of specialized event record media players and a custom graphical user interface. Alternatively, a human operator 62 may view image, audio, numeric and graphical data to interpret the event details and enter results via manipulation of graphical user interface controls. In either case, the discretization facility produces an output of machine processable discrete data related to the non-discrete input received there.

[0069] Event data is captured and recorded by vehicle event recorder 63 including camera 68 and vehicle sensors 65. On-board diagnostics system 64 may be connected to video event data recorder 63 by cable. On-board diagnostics system 64 usually presents data continuously. However, on-board diagnostics data is only captured for a period associated with an event detection. As described above the vehicle event recorder produces both numeric/digital data as well as non-discrete data such as video and audio streams. Specifically, transducers or sensors 66 produce discrete data 67. Other discrete data may come from the OBD. Video camera 68 produces non-discrete data 69 comprising a video image series. Copy 610 of the data, including both discrete and non- discrete, is received at the discretization facility 61 for interpretation either by computer interpretive algorithms or by an operator. The data is assembled into an event record in a database structure, including unique identifier or "primary key" 611. A complete event record includes interpreted data 612 output from the discretization facility, non-discrete data 513 captured by the vehicle event recorder and discrete data 514 captured in the vehicle event recorder and not created as a result of an interpretive system.

[0070] Fig. 7 is a schematic representation of vehicle event recorder 71 and onboard diagnostics system 72 and coupling 73 therebetween. Because an event is identified or detected by trigger 74 of the vehicle event recorder, it is desirable when capturing data from the on-board diagnostics system that the data be received and time stamped to be synchronized with system clock 75. Data from the on-board diagnostics system can be properly played back in synchrony with captured video images. Without this timestamp, synchronization is difficult or impossible

[0071] On-board diagnostics system 72 may include transducers coupled to vehicle subsystems, for example steering system 76, engine 77 (including oil pressure sensor or engine speed sensors), transmission 78 (gear ratio) and braking system 79, among others. Today, standard on-board diagnostics systems make available diagnostic data from a great plurality of vehicle subsystems. The vehicle event recorder may collect the sensor data during an event and preserve it in memory 710. Vehicle event recorder 71 may also comprise sensors independent of the on-board diagnostics system such as keypad 711. A keypad permits a vehicle operator to "login" for an assigned use period. Global positioning system receiver 712 and electronic compass 713 similarly may be used with vehicle event recorder 71.

[0072] In addition, vehicle event recorder 71 also may include systems which capture data in a non-discrete form. Video camera 714, microphone 715, and accelerometers 716 are exemplary non-discrete sensors. While several sensors are mentioned, this is not intended to be an exhaustive list merely an illustrative one, and it will be appreciated that a vehicle event recorder may include other discrete and non- discrete data capture subsystems.

[0073] Fig. 8 shows a relationship between preliminary event record 81, taken by on-board hardware, and complete event record 82. Event record 82 includes interpreted data portion 86 having discrete, computer processable data. An event record produced by a vehicle event recorder includes both discrete data portion 83 and non-discrete data portion 84. Data associated with a particular identified event is captured and sent to a discretization facility 85 for processing. At the discretization facility, non-discrete data is analyzed either by humans or machines to produce interpreted data 86, which is amended to the original event record to form a complete event record.

[0074] Finally Fig. 9 is high level block diagram of a system in accordance with the principles of the present invention. Primary system elements mounted in motor vehicle 91 include vehicle event recorder 92 and optional on-board diagnostics system 93. These may be linked together by system clock 94 and event trigger 95. Together, these systems operate to capture discrete and non-discrete data, relating to a deleted event and to pass the capture data to database 96. Discretization facility 97 is comprised of event record media player 98, which is used to present the captured data visually in a time managed system. Discretization facility 97 further includes graphical user interface 99 usable to set or change an interpreted value. Graphical user interface controls are coupled to database 96 and more specifically to a data record associated with an event being played on media player 98 such that the data record reflects interpretations of data made at discretization facility 97. Analysis server 910 includes query generator 911 operative to query event data stored in the database. Preferably, the queries include interpreted data stored as part of complete event record. Result sets 912 returned from the database can be analyzed and can trigger other actions 913. For example, upon meeting a predefined condition, special reports 914 may be generated and transmitted to interested parties. In other systems, vehicle maintenance, scheduling, or other operations may be driven by results produced partly based upon interpreted data in the complete event record.

[0075] One will now fully appreciate a system of the invention configured to process, interpret and analyze data collected in conjunction with vehicle event recorders. Although the present inventions have been described in considerable detail with clear and concise language and with reference to certain preferred versions thereof including best modes anticipated by the inventors, other versions are possible. Therefore, the spirit and scope of the invention should not be limited by the description of the preferred versions contained therein, but rather by the claims appended hereto.