Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
A SYSTEM THAT PERCEIVES, REPORTS FATIGUE AND DISTRACTION OF THE DRIVER AND PROCESSES BIG DATA AND A METHOD OF ITS IMPLEMENTATION
Document Type and Number:
WIPO Patent Application WO/2020/086015
Kind Code:
A1
Abstract:
The invention is a system that detects and reports fatigue and distraction of the driver (204) which is characterized as having the following steps: the driver (204) getting on the vehicle will start the engine (401), the camera (102a, 102b) monitoring the driver (204) will run (402), collection of all data via GPS and transmission of this data via GPS (406) and checking if there is any "incident" by comparing the data collected with the threshold values identified and fatigue, phone usage etc. negative behaviors that are intended to be captured regarding to driver (204) (407).

Inventors:
GUZELSOY HASAN DOGAL (TR)
Application Number:
PCT/TR2018/050778
Publication Date:
April 30, 2020
Filing Date:
December 07, 2018
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
INNOMEL MUHENDISLIK SANAYI VE TICARET LTD SIRKETI (TR)
International Classes:
B60K28/06; B60W40/09; G08B21/06; G08B25/00
Foreign References:
US20140300739A12014-10-09
KR101821496B12018-03-08
KR20160098737A2016-08-19
US20140019167A12014-01-16
Attorney, Agent or Firm:
YILDIRIM, Mehmet Enes (TR)
Download PDF:
Claims:
C L A I M S

1. A system that detects and reports fatigue and distraction of the driver (204) characterized in that; it comprises,

- the software capable of detecting face and periphery, eyes, mouth, hand- arm movements of the driver (204) based on artificial intelligence and deep learning, processing the image that monitors the seatbelt (205) on the left shoulder of the driver (204) from the moment of starting the engine till stopping on the vehicle (306), processing related data if the decibel meter of the device (101) detects loud music and running expected function following it, receiving measurable data such as sudden deceleration, sudden acceleration, sudden cornering, compiling the vehicle (306) location, previous route, future route if desired, ensuring sending of all footage recorded by all cameras (302) in the vehicle (306) from the moment of starting the engine till stopping on the vehicle (306) to the central server (303), associating the incident moment with these footage before and after the requested time frame when a history log or report is required in case of an incident,

the device (101) where aforesaid software is embedded, featuring a memory, decibel meter, GSM, GPS, GYRO, Bluetooh modules on it, capable of obtaining data from the vehicle (306) and cameras (302), a speaker that will warn the driver (204) in case of an incident through the optional management buttons,

cameras (302) not limited with numbers, which are installed on the vehicle and monitor the front, sides and read sections,

the central server (303) where all data is stored and which is suitable for processing big data

and a command and control center (305) which monitors entire system, connects to the vehicle if a warning comes from the vehicles (306), connects to the cameras (302) to the vehicle depending on the identified workflow, reaches to the driver (204) and/or command and control center (305) via call and/or SMS and/or e-mail where and when deemed necessary, compiles periodical reports, shares them with the customer and public, makes comments and referrals when and where deemed necessary and carries out information and report preparation activities on accident analysis and subsequent actions such as trainings.

2. A system mentioned in Claim 1 characterized in that; the device (101), which receives its power from the vehicle's power supply (105), has a battery that feeds the system for energy outages.

3. A system mentioned in Claim 1 characterized in that; the device (101) is positioned in the vehicle (306) in a visible or invisible manner in the glove compartment or in a suitable location on the front panel.

4. A system mentioned in Claim 1 characterized in that; the device (101) monitors the driver's (204) face and the periphery, eyes, mouth, hand movements and whether the driver (204) fastens the seatbelt (205) from the left shoulder continuously from the starting till stopping the vehicle

(306) by using the cameras (l02a, l02b).

5. A method that detects and reports fatigue and distraction of the driver characterized in that; it contains the following process steps,

The driver (204) getting on the vehicle will start the engine (401), the camera (l02a, l02b) monitoring the driver (204) will run (402), checking whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time (403),

starting the driving if the result of software check on whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time is "yes (404), notifying the call center and/or the official of the vehicle owner via call, SMS or e-mail if the result of software check on whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time is "no" (405) collection of all data via GPS with internal and external cameras (302) and transmitting this data via GSM network (307) (406),

and checking if there is any "incident" by comparing the data collected with the threshold values identified and fatigue, phone usage etc. negative behaviors that are intended to be captured regarding to driver (204) (407).

6. A method mentioned in Claim 5 characterized in that; the incidents checked during checking if there is any "incident" by comparing the data collected with the threshold values identified and fatigue, phone usage etc. negative behaviors that are intended to be captured regarding to driver (204) (407) are any, one, more than one and all of following:

Whether the driver (204) is the correct driver (204),

Whether the driver (204) complies with working hours and breaks, Determination of the fatigue of the driver (204) from the facial expression and the length of blinking,

abnormal movements with the head,

increases in mouth movements,

Generation of a phone conversation or messaging movement pattern with his/her hand and arm movements and this movement caught by the camera

(l02a, l02b) and whether the driver (204) uses phone,

moves his/her hands and arms continuously just like eating and drinking something,

Moving the hands and arms continuously just like smoking, whether the diver (204) smokes and other distracting behaviors of the driver (204), if the driver has not fastened the seatbelt (205) on the left shoulder or chest Vehicle's sudden deceleration, sudden acceleration and sudden cornering. Creation of a noisy environment exceeding the defined decibel value.

7. A method mentioned in Claim 5 characterized in that; it comprises process sub-steps of notifying the the command and control center (305) and/or the official of the vehicle (306) owner via call, SMS or e-mail in case of any incident as the result of checking if there is any "incident" by comparing the data collected with the threshold values identified and fatigue, phone usage etc. negative behaviors that are intended to be captured regarding to driver (204) (407) and storing data anyway even if there is no incident (409).

8. A method mentioned in Claim 5 characterized in that; the system identification of threshold values to be used for information and reporting on the incidents checked during checking if there is any "incident" by comparing the data collected with the threshold values identified and fatigue, phone usage etc. negative behaviors that are intended to be captured regarding to driver (204) (407) is composed of following process sub-steps,

the relevant official accesses the relevant screen of the software with the password given to him/her (501).

The official then enters the relevant fields in the system with the parameters of the threshold values that will trigger audible and/or visual alarms, reports, SMS, telephone and e-mail alerts according to company rules and legal requirements (502) and

The system will trigger audible and/or visual alarms, reports, SMS, telephone and e-mail alerts considering these parameters (503).

9. A method mentioned in Claim 5 characterized in that; it comprises following process sub-steps if the driver (204) has not been identified to the system before, The fleet official shall introduce a pas sport- size photograph or the biometric photograph of the driver (204) to the software through a camera (601).

The software will take into account these values defined in telephone, SMS and e-mail alerts according to biometric data (602).

Description:
A System That Perceives, Reports Fatigue and Distraction of the Driver and Processes Big Data and A Method of Its Implementation

Technical Field Related to the Invention

The invention is related to a system for a commanding center, which analyzes the biometric data of the driver by means of a device and a camera embedded on it, detects whether the driver is the one that should drive the vehicle, records compliance with the legal resting periods, perceives the symptoms of fatigue-induced drowsiness, making telephone conversation, messaging, smoking, listening to loud music, distracting elements such as eating and drinking while driving, use of the seat belt, sudden acceleration, the sudden deceleration and sudden comer entrances, saves this data to a database to be created digitally and by means of cameras with desired quantity inside and around the vehicle also in video format continuously during the period when the vehicle remains in operation, interprets the visual and numerical data in this database, informs the manager of the fleet of the driver and vehicle on the events which have been predefined as important and a method in connection to such system.

Known Situation of the Technique

According to data from Turkish Statistical Institute (TUIK), the number of registered vehicles in traffic is more than 22 million as of April 2018.

Again according to (TUIK) data, the number of traffic accidents took place in Turkey in 2017 increased by 1.7 percent compared to 2016 and reached to 1 million 202 thousand and 716. 1 million 20 thousand and 47 of these accidents were with property damage and 182 thousand 669 were accidents with fatality-injuries. 7 thousand 427 people died while 300 thousand 383 people were injured. 43.2% of the people who died in the road network traffic accidents in 2017, were drivers, 34.2% were passengers and 22.6% were pedestrians. When total of 213 thousand 325 faults that had caused traffic accidents with death/injury are examined, it is determined that 89.9 percent is caused by drivers, 8.5 percent is caused by pedestrians, 0.7 percent is caused by road, 0.5 percent is caused by vehicle and 0.4 percent is caused by passengers. Contrary to popular belief, 67.4 percent of the fatal / injured accident in the road network is in daytime, 30 percent in the night and 2.6 percent in the twilight.

According to the report published by the TGNA Petition Sub-Commission in the first months of 2018, the number of people who lost their lives in the traffic accidents in our country is 5 times higher than those in the European Union member states. According to the data shared by the Ministry of Health, death due to traffic accidents is the third most common cause of death.

When the reasons of accidents are examined, very striking results are observed. According to data from Selcuk University Accident Research, Prevention and Application Center, which is another organization studying on the issue, one of every three traffic accidents taking place in Turkey is caused by disorders related to sleep.

Another reason for the accident is the distraction of the driver. Academic studies examining the relationship between traffic accidents and information, communication and entertainment technologies used in vehicles are not sufficient in our country. However, there are many studies on the subject abroad. The Road Safety Survey conducted by the Allianz Technology Center (AZT) in Germany, Austria and Switzerland examined the relationship between the increasing number of traffic accidents and the information, communication and entertainment technologies used in vehicles. As the result of the research conducted in cooperation with the Institute for Applied Psychology and Makam Market Research, performed by interviewing with one thousand 600 drivers, has proven that the risk of accidents has increased considerably due to drivers focusing on technical devices in traffic. In the same study, it was found that 60 percent of the drivers who had had an accident in the last three years had been using their mobile phones during the accident. According to the same study, 46 percent of the drivers, that is to say about one in every two drivers, stated that they violated the rule of not using mobile phones, 39 percent stated that they used the navigation system and 58 percent said they used the radio system through the menu on the front panel. It is another result of the study that 15 percent of the drives write an SMS and 24 percent read their messages by using their smart phones while they are in the traffic. The most important factor affecting the rate of life loss and injury in accidents is the safety belt. The results of the study conducted jointly by the Middle East Technical University the Security General Directorate on the use of seat belt in Turkey are striking. The use of seatbelts that reduce the risk of death in traffic accidents by 50% for driver and front seat passengers and 75% for rear seat passengers has still not reached to the desired level.

According to the study conducted by observing 548 thousand 738 vehicles in city, 342 thousand 960 vehicles outside the city and it total of 892 thousand 496 vehicles throughout Turkey, country wise use of seatbelt is 43.6 percent for the drivers while it is 35.9 percent for the front seat passengers. These rates were 35.7 percent for drivers, 25.2 percent for front-seat passengers for urban areas and 56.2 per cent and 51.1 percent respectively for the country. In countries that have succeeded in combating with traffic accidents, their causes and consequences, this rate has increased to 90%.

In order to reduce traffic accidents, automotive companies and drivers or fleets both in Turkey and around the world assemble products such as Advanced Driver Assistance System- AD AS, Fleet Management System (FMS) on the vehicles as OEM when the vehicle is on the assembly line or after it is released. However, almost all of these systems offer solutions relying on other vehicles and road conditions.

Although fatigue detection systems are mentioned in recent years, most of the systems conclude the tiredness of the driver according to the calculations made based on the vehicle's movements.

However, when the danger warnings perceived by such systems are triggered, it is often too late. In fleet vehicles in the supply chain, the situation is more sensitive. Because, on the one hand, the vehicles are competing against time due to the business related reasons and on the other hand the drivers are expected to drive in accordance with the rules. Fleet managers who use devices and software such FMS etc. can only benefit from these software to create reports after an unwanted event takes place or at the end of the month. However, a support model similar to "air traffic control" serving to aircrafts is not available for land transportation.

Fleet managers request the installation of FMS, internal and external camera, ADAS products while ordering the vehicle or aftermath both inland and abroad. Failing to offer a package which contains all of these products, constitutes a separate integration and coordination problem for the fleets later on. Moreover, the FMS, internal and external vehicle camera, ADAS companies cannot serve as a virtually command center which is able to detect an incident and then to ensure information sharing between the vehicle driver and fleet manager and trigger subsequent preventive actions.

Narrow scope embodiment of the method we propose was thought by real and legal persons both in Turkey and abroad even in the narrow scope and patent or utility model applications have been made. However, patent applications appear to be incomplete or inadequate.

The patent numbered WO2018117782 (Al) and titled "METHOD BASED ON THE ALERTNESS LEVEL OF TRUCK OPERATORS FOR AUTOMATIC TASK ASSIGNMENT IN A FLEET MANAGEMENT SYSTEM" refers to an invention that calculates driver performance by taking into account fatigue in a narrow scope. The invention refers to an innovation that monitors the driver within the fleet management system, articulates, instant readiness (non-exhaustion) data, historical data, accident and failure etc. situations in a single formula together with their coefficients, determines the performance of the driver based on this and assigns the work to the driver automatically based on such performance determination. The system observes the drive at certain intervals, such as 10 minutes. As a result, the driver with less fatigue observed and getting high rating is assigned more and more important work. The invention is a performance analysis system that focuses not on the accidents that may be caused of tiredness but on assigning more and more important work to a driver seem to be more awake. It ignores whether the driver is the right driver and distractions such as telephone calls, smoking, listening to loud music, use of seat belts, sudden acceleration, sudden deceleration and sudden corner entries. A shortcoming of the system is to observe the driver in every 10 minutes. However, tiredness may emerge during the moment when the driver is not observed by the system. However, human biometry sometimes requires interpretation, too. Therefore, establishing a system based solely on biometrics can lead to incorrect scoring. Another point which is not available in the patent but available in our system is the failure of the device to offer a solution of intervention options such as generating a sound warning when driver tiredness is detected, allowing the command center to connect to the vehicle for checking whether the warning is correct, informing the fleet official if necessary and/or contacting directly with the driver at that moment.

The patent numbered CN106228821 and titled “Fatigue Driving Comphrensive Monitoring and Safety Protection and Controlling Method for Highly Dangerous Vehicles” refers to an invention focused solely on fatigue that is composed of a fatigue sensing wrist strap and a fatigue-focused camera. The invention detects fatigue from pulse in the wrist and facial expressions, performs a scoring, and stimulates the driver with a sound or a spray when the corresponding score exceeds 70. It also saves this data to a server. However, human biometry sometimes requires interpretation, too. Therefore, establishing a system based solely on biometrics can lead to incorrect actions and data. Another point which is not available in the patent but available in our system is the failure of the device to offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected. Furthermore, it ignores whether the driver is the right driver and distractions such as telephone calls, smoking, listening to loud music, use of seat belts, sudden acceleration, sudden deceleration and measurement of sudden corner entries. The patent numbered CN204178509 and titled “Fatigue Driving Comprehensive Monitoring and Safety Protection and Controlling Method for Highly Dangerous Vehicles” also refers to a fatigue detection and remote warning system. However it ignores whether the driver is the right driver and distractions such as telephone calls, smoking, listening to loud music, use of seat belts, sudden acceleration, sudden deceleration and sudden comer entries, excessive approaching to the vehicle ahead and lane change.

Again the foreign patents numbered US6097295A titled“Apparatus For Determining The Alertness of Driver” and numbered KR20150005130A titled“System and the method for prevention driving while drowsy” refer to an invention that relies on the period when at least one eye and eyelids of a person remain closed. However, human biometry sometimes requires interpretation, too. Therefore, establishing a system based solely on biometrics can lead to incorrect actions and data. Another point which is not available in the patent but available in our system is the failure of the device to offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected. Furthermore, it ignores whether the driver is the right driver and distractions such as telephone calls, smoking, listening to loud music, use of seat belts, sudden acceleration, sudden deceleration and measurement of sudden comer entries.

Patent numbered CN107784292 and titled“Driver fatigue recognition method based on array lens“ and the patent numbered CN107423684 (A) titled "Rapid face positioning method and system applied to driver fatigue detection“ are about algorithms that detect driver fatigue only and the point which is not available in the patent but available in our system is the failure of the device to offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected. Furthermore, it ignores whether the driver is the right driver and distractions such as telephone calls, smoking, listening to loud music, use of seat belts, sudden acceleration, sudden deceleration and measurement of sudden corner entries.

The patent numbered WO2017143086 with the title“Anti-Distracted Driver System” is intended to prevent the driver from using a mobile phone. The invention is based on a device which detects a mobile phone in a glove compartment. The device does not allow the vehicle to run when it does not detect the mobile phone in the glove compartment. However, a person carrying two mobile phones can easily mislead the system. The patent with the number KR20170016579 (A) and titled“Monitoring and warning method for using phone during driving, and system thereof’ refers to a system that senses and interferes with sound and light when the driver uses a telephone while driving. However, the invention has some shortcomings. First of all it does not offer a solution for hands-free phone talk which is available in our system. Furthermore, it does not offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected as we have in our system. Furthermore, it ignores whether the driver is the right driver, driver fatigue and distractions such as smoking, listening to loud music, use of seat belts, sudden acceleration, sudden deceleration and measurement of sudden corner entries.

The patent with the number CN107944341 and title “Traffic monitoring image-based automatic detection system for safety belt non-fastening behavior of driver” refers to the identification of drivers who are not using a seat belt by means of a camera and software to be installed on the roads. However, this is a public practice that will cover entire city. Furthermore, it does not offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct when the driver does not fasten his/her seatbelt, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected as we have in our system. Furthermore, it ignores whether the driver is the right driver, driver fatigue and distractions such as telephone calls, smoking, listening to loud music, sudden acceleration, sudden deceleration and measurement of sudden corner entries.

The patent with the number CN207208003 and title“It is device and system of inspecting of inspecting that safety belt is worn” refers to an invention that detects whether or not the safety belt is fastened by using NFC (near field communication) method. However, the invention has some shortcomings. First of all it does not offer a solution for hands-free phone talk which is available in our system. Furthermore, it does not offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected as we have in our system. Furthermore, it ignores whether the driver is the right driver, driver fatigue and distractions such as smoking, listening to loud music, sudden acceleration, sudden deceleration and measurement of sudden corner entries.

The patent with the number CN206510764 and title“Driver discretionary security monitoring devices” refers to a system that detects whether the driver has drunk alcohol and fastened his/her seat belt. However, it does not offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct when the driver does not fasten his/her seatbelt, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected as we have in our system. Furthermore, it ignores whether the driver is the right driver, driver fatigue and distractions such as telephone calls, smoking, listening to loud music, sudden acceleration, sudden deceleration and measurement of sudden corner entries.

The patent with the number CN107844783 and title“Method and system for detecting abnormal driving behaviors for operating vehicle” refers to an invention that detects abnormalities of the driver, such as fatigue, smoking, speaking, telephone call by means of a camera, a device and software. However, it does not offer a solution of intervention options such as, allowing the command center to connect to the vehicle for checking whether the warning is correct when the driver does not fasten his/her seatbelt, informing the fleet official if necessary and/or contacting directly with the driver at that moment when driver tiredness is detected as we have in our system. Furthermore, it ignores whether the driver is the right driver and distractions such as listening to loud music, sudden acceleration, sudden deceleration and measurement of sudden corner entries.

As can be seen, all the patents mentioned can provide partial and inadequate solutions with different methods focusing on only different parts of the driver verification, fatigue and distraction. This prevents the efficient use of the inventions resulting from the respective patents. In particular, fleet vehicles become a device dump with devices serving for different purposes, which is a situation that is no longer desired. However, it is possible to detect all these with a single solution method, to be able to intervene immediately when necessary and to report all findings and interventions though the method presented.

Purpose of the Invention

The purpose of the invention is to detect,

Whether the driver is the correct driver,

Whether the driver complies with working hours and breaks,

- Fatigue of the driver from facial expression, yet initially,

Whether the driver uses a telephone,

Whether the driver smokes,

Whether the driver eats food while driving,

Whether the driver uses the seat belt as it should be,

- vehicle's sudden deceleration, sudden acceleration and sudden cornering.

Another purpose of the invention is to ensure that the device warns the drivers with sound signals in case of an incident. Another purpose of the invention is to deliver collected digital and visual data to a command center for instant interventions via GSM / GPRS / GPS, to take the actions of the command center workers to inform the driver and / or the fleet manager immediately at the event moments, thereby removing a significant burden on the fleet managers. . Another purpose of the invention is to reduce the risk of driving violations and associated costs of the fleets. Another purpose of the invention is to monitor the correct driving and violation of the drivers and to plan relevant trainings and take other measures.

Another purpose of the invention is to evaluate collected data as big data, to monitor the intensity of the event locations, to identify the points and routes where frequent fatigue is observed and to share them with the public.

The system and application method developed to realize these objectives are explained in detail below.

Description of Figures

Figure 1: Depiction of the device and cameras on the vehicle.

Figure 2: Depiction of the device and cameras inside the vehicle.

Figure 3: General operation of the system

Figure 4: The main workflow of the software

Figure 5: Workflow on identification of threshold values to be used for information and reports on the system by the fleet manager.

Figure 6: Workflow on introducing the new drive to the system

Numbers and names of main elements mentioned in the figures are given below.

(101) Device

(l02a) Camera

(l02b) Camera

(103) Front camera

(104) Rear camera

(105) Power supply

(204) Driver

(205) Seatbelt

(302) Camera

(303) Server (304a) Telephone and / or other smart devices

(304b) Telephone and / or other smart devices

(305) Command and control center

(306) Vehicle

(307) GSM network

(308) Satellite

Detailed Explanation of the Invention

The invention is related to a system featuring a machine learning and image processing based software that is embedded in the device (101) capable of detecting the driver's (204) hand, arm, face, mouth and eye movements, whether the driver (204) fastens the seatbelt (205), listens to a loud conversation or load music from a noise source, measuring the sudden acceleration, deceleration, cornering and other measurement which are not mentioned here but can be added in the future digitally and visually through on the memory, decibel meter, GSM, GPS, radio, bluetooth modules on board, obtaining data from the vehicle (306) and cameras (302), featuring components that will warn the driver (204) with sound and visually in case of an incident through the optional management buttons and performance of information and report preparation activities on sending the data collected by the device (101) to the central server (303), interpretation of data received by the central server (303) by the command and control center (305) by using a software developed by us, monitoring of the system, connecting instantly to the vehicles (306) if any warning comes from them, connecting to the cameras of the vehicle depending on the identified workflow, reaching to the driver (204) and/or command and control center (305) via call and/or SMS and/or e-mail where deemed necessary, compiling periodical reports and sharing them with the customers and public, making comments and referrals when deemed necessary and other subsequent actions such as accident analysis and trainings. Figure 1 shows the location of the device (101) throughout the vehicle. The device (101) is connected to a driver (204) camera (l02a, l02b) which monitors the driver (204) continuously. Here, the possible locations for placing the driver (204) camera are indicated depending on the vehicle type and seating position of the driver (204). The device (101) has an embedded software and features a memory, decibel meter, GSM, GPS, radio, bluetooth modules on board, is capable of obtaining data from the vehicle (306) and cameras (302) and featuring a speaker that will warn the driver (204) in case of an incident through the optional management buttons. The device (101) will receive visual data from an front camera (103) which also monitors the front of the vehicle and unrestricted number of side and rear cameras (104). The device (101) will receive its energy from the vehicle's own power supply (105), although it has a battery supplying the system in certain situations of interruption.

Figure 2 illustrates the in-cab representation of the device (101) and the cameras (l02a, l02b, 103). The device (101) may be mounted in the vehicle (306) in a visible or invisible manner in the glove compartment or in a suitable location on the front panel. The device (101) is connected to the camera (l02a, l02b) the possible locations of which are depicted. The possible locations for placing these cameras are indicated depending on the vehicle type and seating position of the driver (204). The device (101) will receive visual data from an front camera (103) which also monitors the front of the vehicle. The device (101) shall monitor the driver's (204) face and the periphery, eyes, mouth, hand movements and whether the driver (204) fastens the seatbelt (205) from the left shoulder continuously from the starting till stopping the vehicle (306) by using the the cameras (l02a) and (l02b). When the device (101) detects that the driver (204), is not the correct driver (204) who supposed to dive the vehicle,

does not comply with the legal driving periods,

closes his/her eyes for a period longer than the defined one,

makes abnormal movements with his/her head,

increases mouth movements,

generates a phone conversation or messaging movement pattern with his/her hand and arm movements, moves his/her hands and arms continuously just like eating and drinking something,

smokes,

creates a noisy environment exceeding the defined decibel value,

has not fastened the seatbelt (205) on the left shoulder or chest

makes sudden deceleration, sudden acceleration and sudden cornering, it will give a sound signal and transmit digital and visual records of these situations called "incident" to the central server (303). Figure 3 shows the general operation of the proposed system. The device (101), cameras (302), the server (303) in which the data is collected, phones and/or smart devices (304a, 304b) carried by the driver, fleet managers and other authorities through which this data can be displayed, the command and the control (call) center (305) and the vehicle (306) to be monitored are the main components of the model. Data collected by the cameras (302) will be combined with time etc. numerical values of the vehicle and be processed by the device (101). It will be transmitted to the satellite (308) through the GSM network (307) and then to the server (303). Data coming to the server (303) will be interpreted by the command and control center (305). In case of an incident, the device (101) will warn the driver (204) by giving an audio signal. The command and control center (305) will contact with the vehicle's (306) driver (204) via phone and/or other smart devices (304a) and/or the fleet manager again via phone and/or other smart devices (304b) where and when deemed necessary. The command and control center (305) is also the call and control center. Figure 4 shows the main workflow of the overall operation of the system. The driver (204) getting on the vehicle will start the engine (401). The camera (402) monitoring the driver (204) will run.

The software will check whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time (403).

If the result of software check on whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time (404) is "yes", then the diving can be started. If the result of software check on whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time is "no", then the diving can be started but the call center and/or the official of the vehicle owner will be notified via call, SMS or e-mail (405). The software will collect all data via GPS with internal and external cameras and transmit this data via GSM network (307) (406). The software will check if there is any "incident" by comparing the data collected with the threshold values identified and fatigue, phone usage etc. negative behaviors that are intended to be captured regarding to driver (204) (407).

These incidents can be listed as,

Whether the driver (204) is the correct driver (204),

Whether the driver (204) complies with working hours and breaks,

Determination of the fatigue of the driver (204) from the facial expression and the length of blinking ,

makes abnormal movements with his/her head,

increases mouth movements,

Generation of a phone conversation or messaging movement pattern with his/her hand and arm movements and this movement caught by the camera and whether the driver (204) uses phone,

moves his/her hands and arms continuously just like eating and drinking something,

Moving the hands and arms continuously just like smoking, whether the diver (204) smokes and other distracting behaviors of the driver (204),

has not fastened the seatbelt (205) on the left shoulder or chest

Vehicle's sudden deceleration, sudden acceleration and sudden cornering creation of a noisy environment exceeding the defined decibel value.

If there is an incident, the command and control center (305) and/or the official of the vehicle (306) owner will be notified via call, SMS or e-mail (408). If there is no incident, the driving data will still be stored anyway (409). The incidents that are desired to be paid more attention and their frequencies my differ depending on the company. The official who will identify these incidents will give a separate password to fleet officials. Figure 5 show the workflow on identification of threshold values to be used for information and reports on the system by the fleet manager. For this purpose, the relevant official will access the relevant screen of the software with the password given to him/her (501). The official then enters the relevant fields in the system with the parameters of the threshold values that will trigger audible and/or visual alarms, reports, SMS, telephone and e-mail alerts according to company rules and legal requirements (502). The system will trigger audible and/or visual alarms, reports, SMS, telephone and e-mail alerts considering these parameters. (503).

Figure 6 depicts the workflow on the introduction of fleet drivers (204) to the system. The fleet official shall introduce a passport-size photograph or the biometric photograph of the driver (204) to the software through a camera (601). The software will take into account these values defined in telephone, SMS and e-mail alerts according to biometric data (602).

The invention is a system that detects and reports fatigue and distraction of the driver (204) which is characterized as featuring the software capable of detecting face and periphery, eyes, mouth, hand-arm movements of the driver (204) based on artificial intelligence and deep learning, processing the image that monitors the seatbelt (205) on the left shoulder of the driver (204) from the moment of starting the engine till stopping on the vehicle (306), processing related data if the decibel meter of the device (101) detects loud music and running expected function following it, receiving measurable data such as sudden deceleration, sudden acceleration, sudden cornering, compiling the vehicle (306) location, previous route, future route if desired, ensuring sending of all footage recorded by all cameras (302) in the vehicle (306) from the moment of starting the engine till stopping on the vehicle (306) to the central server (303), associating the incident moment with these footage before and after the requested time frame when a history log or report is required in case of an incident, the device (101) where aforesaid software is embedded, featuring a memory, decibel meter, GSM, GPS, GYRO, Bluetooh modules on it, capable of obtaining data from the vehicle (306) and cameras (l02a, l02b, 103, 104), a speaker that will warn the driver (204) in case of an incident through the optional management buttons, cameras (l02a, l02b, 103, 104) not limited with numbers, which are installed on the vehicle and monitor the front, sides and read sections, the central sever (303) where all data is stored and which is suitable for processing big data and a command and control center (305) which monitors entire system, connects to the vehicle if a warning comes from the vehicles (306), connects to the cameras to the vehicle depending on the identified workflow, reaches to the driver (204) and/or command and control center (305) via call and/or SMS and/or e-mail where and when deemed necessary, compiles periodical reports, shares them with the customer and public, makes comments and referrals when and where deemed necessary and carries out information and report preparation activities on accident analysis and subsequent actions such as trainings.

The invention is a system that detects and reports fatigue and distraction of the driver (204) which is characterized as having the following steps: The driver (204) getting on the vehicle will start the engine (401), the camera (l02a, l02b) monitoring the driver (204) will run (402), checking whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time (403), starting the driving if the result of software check on whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time is "yes (404), notifying the call center and/or the official of the vehicle owner via call, SMS or e-mail if the result of software check on whether the driver (204) was previously introduced to the system or the right driver (204) is the correct driver who must have worked on that time is "no" (405), collection of all data via GPS and transmission of this data via GPS (406) and checking if there is any "incident" by comparing the data collected with the threshold values identified and fatigue, phone usage etc. negative behaviors that are intended to be captured regarding to driver (204) (407).