Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
PREDICTION-BASED MODEL FOR ROAD SAFETY IN A CELLULAR- VEHICLE-TO-EVERYTHING (C-V2X) ENVIRONMENT
Document Type and Number:
WIPO Patent Application WO/2024/018267
Kind Code:
A1
Abstract:
A method, system and apparatus are disclosed. A method implemented in a first connected autonomous vehicle (CAV) configured to connect to at least one of a radio base station and a second CAV is provided. The method includes determining a network connection failure between the first CAV and at least one of the radio base station and the second CAV. The method further includes, in response to the determining of the network connection failure, predicting at least a speed of a first proximate vehicle based on a machine learning model and a vehicle history dataset, where the vehicle history dataset is associated with at least one proximate vehicle. The method further includes adjusting the speed of the first CAV based on the prediction.

Inventors:
BIN SEDIQ AKRAM (CA)
AFANA ALI (CA)
SOROUR SAMEH (CA)
ELBERY AHMED (CA)
HASSANEIN HOSSAM (CA)
Application Number:
PCT/IB2022/056775
Publication Date:
January 25, 2024
Filing Date:
July 21, 2022
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
ERICSSON TELEFON AB L M (SE)
International Classes:
H04W4/44; B60W60/00; G08G1/16; H04W4/02; H04W4/46
Foreign References:
US20200062249A12020-02-27
US20210039664A12021-02-11
Attorney, Agent or Firm:
WEISBERG, Alan M. (US)
Download PDF:
Claims:
WHAT IS CLAIMED IS:

1. A first connected autonomous vehicle, CAV, (23) configured to connect to at least one of a radio base station (16) and a second CAV (23), the first CAV (23) comprising processing circuitry configured to: determine a network connection failure between the first CAV (23) and at least one of the radio base station (16) and the second CAV (23); in response to the determining of the network connection failure, predict at least a speed of a first proximate vehicle based on a machine learning model and a vehicle history dataset, the vehicle history dataset being associated with at least one proximate vehicle; and adjust the speed of the first CAV (23) based on the prediction.

2. The first CAV (23) of Claim 1, wherein the network connection failure is associated with at least one of a dropped and a delayed vehicle state information packet transmitted by one of: the radio base station (16), and the second CAV (23).

3. The first CAV (23) of any one of Claims 1 and 2, wherein the first CAV (23) is configured for front collision avoidance, the first proximate vehicle being at least one of: a leading vehicle in front of the first CAV (23), and the second CAV (23).

4. The first CAV (23) of any one of Claims 1-3, wherein the vehicle history dataset includes at least one of a speed, a direction, an acceleration, and a coordinate of each of the at least one proximate vehicle associated with the vehicle history dataset. 5. The first CAV (23) of any one of Claims 1-4, wherein the at least one proximate vehicle associated with the vehicle history dataset includes up to a maximum number of N vehicles which are closest in proximity to the first CAV (23).

6. The first CAV (23) of Claim 5, wherein N is 10.

7. The first CAV (23) of any one of Claims 1-6, wherein the vehicle history dataset includes information from a time period T prior to the prediction.

8. The first CAV (23) of Claim 7, wherein T is 1.5 seconds.

9. The first CAV (23) of any of Claims 7 and 8, wherein the vehicle history dataset includes vehicle state information which is recorded according to a periodicity of 0.1 ms.

10. The first CAV (23) of any one of Claims 1-9, wherein the at least one proximate vehicle associated with the vehicle history dataset includes only vehicles which are within a distance range R of a location of the first CAV (23).

11. The first CAV (23) of Claim 10, wherein R is 200 meters.

12. The first CAV (23) of any one of Claims 1-11, wherein the processing circuitry is further configured to: receive sensor data collected by at least one vehicle sensor of the first CAV (23); and the vehicle history dataset being based on the received sensor data.

13. The first CAV (23) of any one of Claims 1-12, wherein the first CAV (23) is further configured to: receive vehicle data associated with at least one additional CAV (23), the vehicle data being received from at least one of: a direct connection with the at least one additional CAV (23), and the radio base station (16), the radio base station (16) being in communication with the at least one additional CAV (23); and the vehicle history dataset being based on the received vehicle data.

14. The first CAV (23) of any one of Claims 1-13, wherein the machine learning model is a long short term memory, LSTM, deep neural network.

15. The first CAV (23) of any one of Claims 1-14, wherein the machine learning model is trained at least one of: offline, in a simulated testing environment, and prior to the network connection failure.

16. The first CAV (23) of any one of Claims 1-15, wherein the prediction is further based on a road map associated with a location of the first CAV (23).

17. A method implemented in a first connected autonomous vehicle, CAV, (23) configured to connect to at least one of a radio base station (16) and a second CAV (23), the method comprising: determining (S100) a network connection failure between the first CAV (23) and at least one of the radio base station (16) and the second CAV (23); in response to the determining of the network connection failure, predicting (S102) at least a speed of a first proximate vehicle based on a machine learning model and a vehicle history dataset, the vehicle history dataset being associated with at least one proximate vehicle; and adjusting (S104) the speed of the first CAV (23) based on the prediction.

18. The method of Claim 17, wherein the network connection failure is associated with at least one of a dropped and a delayed vehicle state information packet transmitted by one of: the radio base station (16), and the second CAV (23).

19. The method of any one of Claims 17 and 18, wherein the first CAV (23) is configured for front collision avoidance, the first proximate vehicle being at least one of: a leading vehicle in front of the first CAV (23), and the second CAV (23).

20. The method of any one of Claims 17-19, wherein the vehicle history dataset includes at least one of a speed, a direction, an acceleration, and a coordinate of each of the at least one proximate vehicle associated with the vehicle history dataset.

21. The method of any one of Claims 17-20, wherein the at least one proximate vehicle associated with the vehicle history dataset includes up to a maximum number of N vehicles which are closest in proximity to the first CAV (23).

22. The method of Claim 21, wherein N is 10.

23. The method of any one of Claims 17-22, wherein the vehicle history dataset includes information from a time period T prior to the prediction.

24. The method of Claim 23, wherein T is 1.5 seconds.

25. The method of any of Claims 23 and 24, wherein the vehicle history dataset includes vehicle state information which is recorded according to a periodicity of 0.1 ms.

26. The method of any one of Claims 17-25, wherein the at least one proximate vehicle associated with the vehicle history dataset includes only vehicles which are within a distance range R of a location of the first CAV (23). 27. The method of Claim 26, wherein R is 200 meters.

28. The method of any one of Claims 17-27, the method further comprising: receiving sensor data collected by at least one vehicle sensor of the first CAV (23); and the vehicle history dataset being based on the received sensor data.

29. The method of any one of Claims 17-28, further comprising: receiving vehicle data associated with at least one additional CAV (23), the vehicle data being received from at least one of: a direct connection with the at least one additional CAV (23), and the radio base station (16), the radio base station (16) being in communication with the at least one additional CAV (23); and the vehicle history dataset being based on the received vehicle data.

30. The method of any one of Claims 17-29, wherein the machine learning model is a long short term memory, LSTM, deep neural network.

31. The method of any one of Claims 17-30, wherein the machine learning model is trained at least one of: offline, in a simulated testing environment, and prior to the network connection failure.

32. The method of any one of Claims 17-31, wherein the prediction is further based on a road map associated with a location of the first CAV (23).

Description:
PREDICTION-BASED MODEL FOR ROAD SAFETY IN A CELLULAR-

VEHICLE-TO-EVERYTHING (C-V2X) ENVIRONMENT

TECHNICAL FIELD

The present disclosure relates to wireless communications, and in particular, to C-V2X communication such as for prediction-based vehicle safety.

BACKGROUND

The Third Generation Partnership Project (3 GPP) has developed and is developing standards for Fourth Generation (4G) (also referred to as Long Term Evolution (LTE)) and Fifth Generation (5G) (also referred to as New Radio (NR)) wireless communication systems. Such systems provide, among other features, broadband communication between network nodes, such as radio base stations, and mobile wireless devices (WD), as well as communication between network nodes and between WDs, device-to-device (D2D) communication, and vehicle-to-everything (V2X) communication, which may include any combination of direct communications among vehicles, pedestrians, and infrastructure. Wireless communication networks such LTE and NR networks may use V2X services and support communication for V2X capable user equipment (UE). Cellular V2X (C-V2X) is an example 3GPP standard for V2X communication, which may use and/or be compatible with, for example, 4G and/or 5G standards.

Connected Autonomous Vehicles (CAV) technology has several safety -related applications such as forward collision avoidance, cooperative driving, traffic control and management, and non-signalized intersection management.

Existing safety systems in CAVs may utilize sensors in a CAV to learn about the surrounding environment of the CAV and make appropriate driving decisions. However, these sensors may be unreliable and/or inconsistent, such as in severe weather or when the CAV is driving on a complex road structure.

Existing systems have considered CAVs which rely on V2X to enable the CAVs to collect information about the surrounding environment in real-time and make informed decisions about speeding and maneuvering for decreasing the number of accidents and the associated fatalities. One application for autonomous vehicles such as CAV is a Forward Collision- Avoidance System (FCAS). A typical scenario for FCAS operation is when vehicles follow one another, as shown in FIG. 1, in which a first vehicle 2 is following a leading vehicle 4. Vehicle 2 is followed by vehicle 6. The speed of each vehicle, v n , the length of each vehicle, l n , the gaps between the vehicles, g n , and the distances from the front of each vehicle to the front of the proximate vehicles, h n , are shown in FIG. 1.

The follower vehicle 2 speed v n at time t may limited by three parameters, e.g., as per the following equation: v n < min (v rmax , v n + a n t, v n , afe ) (Equation 1)

The first parameter v rmax is the maximum road speed, which may be a parameter associated with a particular road. A vehicle can drive at this speed when there are no vehicles in front of it, and it therefore may be referred to as the "free-flow speed." The second parameter, a n , is related to the vehicle dynamics and is based on the engine capabilities and the vehicle design. When a vehicle accelerates, it needs time to reach its target speed based on its maximum acceleration a n , which is a function of engine/vehicle parameters such as the engine horsepower, torque, vehicle weight, etc. A typical value for a n is 4.5 m/s 2 . Consequently, over a At time, the vehicle speed may not exceed v n + a n t. The same formula applies to deceleration, e.g., where a n = —4.5 m/s 2 . Additionally, the maximum deceleration rate may be increased in the case of an emergency to avoid accidents, which is known as emergency braking. The third parameter is the vehicle's safe speed v nsafe , which may depend on a variety of parameters and may be computed using car-following models known in the art. A common car-following model is the Krauss model, which is a microscopic, space- continuous model, developed in 1997. In the Krauss model, the safe speed is computed as: (Equation 2) This safe speed is a function of the vehicle’s current speed, where v n is speed in m/s, its maximum deceleration b n in m/s 2 , the gap between the leader and the follower vehicle in meters g n , is the leader speed. In this equation, t r is the reaction time, which is the time needed by the vehicle to react to any event. In human-driven vehicles, a typical value of t r is 1 second, whereas in autonomous vehicles, the value of t r may be smaller.

Existing safety systems in CAVs rely on sensors to learn about the surrounding environment and make appropriate driving decisions. Thus, the safety of CAVs in this setting relies heavily on the reliability of these sensors. Fatal accidents involving existing autonomous vehicles are often caused by sensor failures (e.g., where sensors cannot reliably detect neighboring cars).

Many types of vehicle sensors are prone to errors. For instance, the accuracy of ultrasonic sensors may range from 1 to 4 meters, which may not be sufficient, by itself, to support safety requirements. Moreover, errors from different sensors may accumulate when computing safety parameters. For example, when computing the speeds and acceleration of other vehicles, distance sensor errors may accumulate with localization errors resulting in a large deviation from the actual speeds. In such cases, vehicular communications can reduce such errors by enabling vehicles to communicate their actual parameters (i.e., its accurate speed that is computed based on the engine rotation speed).

5G NR is an emerging technology that is expected to be adopted to enable C- V2X which was developed by the 3rd Generation Partnership Project (3GPP) Release 14. In C-V2X, vehicles can communicate through cellular base stations by leveraging the existing cellular infrastructure. Moreover, since the existence of cellular coverage is not always available, especially in rural environments, other transmission modes are defined in the C-V2X standard to enable vehicles to directly communicate through sidelink channels over a PC5 interface, without the need for base stations/network nodes. Rel. 16 of 5G-NR discusses the requirements for different safety-related CAV systems such as cooperative driving and information sharing. Dedicated short range communications (DSRC) is another existing vehicular communication technology that was initially introduced as a supplementary source of information to improve road safety by enabling vehicles to exchange their state information by periodically sending Basic Safety Messages (BSM). DSRC also allows vehicles to exchange other information such as collision warning and accident reporting messages.

Vehicular communication may serve as a supplementary source of information to complement vehicle sensors, and/or may be a primary or sole source of information in CAVs. For example, there may be situations where vehicular communication is the only available source of information that vehicles can rely on. For instance, on steep roads, measuring the distance to the leader or follower vehicles using laser beams or ultrasound signals might not work because of the lack of line of sight. As another example, some sensors such as Light Imaging Detection and Ranging (LIDAR) sensors and camera sensors are adversely affected by weather conditions, such as fog, that may hinder them from accurately detecting objects. In these scenarios, vehicular communication may provide a more reliable source of information. However, existing systems may be unable to safely adjust a vehicle’s speed when communications between vehicles suffer latency and/or reliability issues.

SUMMARY

Some embodiments advantageously provide methods, systems, and apparatuses for prediction-based road safety for CAVs.

The present disclosure describes techniques for estimating/predicting the speed of a leading vehicle (i.e., leader vehicle 4). One or more embodiments described herein provide techniques for overcoming communication impairments (e.g., latency and/or packet drops). Embodiments of the present disclosure utilize a deep-learning model to predict the leader vehicle 4 speed, e.g., when vehicular communications between leader vehicle 4 and follower vehicle 2 are delayed and/or dropped. The present disclosure describes these techniques with respect to FCAS in CAV environments, but it is to be understood that such techniques may be applied to any scenario where proximate vehicle speed/location/direction/maneuver intention/etc. are to be predicted (e.g., rear-collision avoidance, side-collision avoidance, etc.).

In embodiments of the present disclosure, deep learning/machine learning prediction is used to overcome/compensate for communication delays and/or packet drops to improve road safety and to avoid accidents in CAV environments. For example, a deep learning prediction module may be used to predict the speed of the leading vehicle if the follower vehicle did not receive (e.g., via C-V2X communications) the speed of the leading vehicle (e.g., for 2 consecutive intervals, where each interval spans, e.g., 100 msec.) The deep learning model may be trained using the history of the surrounding car(s) for only 1.5 seconds (15 intervals) prior to making the prediction, which may use only vehicle speed information, and may not necessarily require any other vehicle information. By utilizing a short history and minimizing the amount of information needed, the model may be of low complexity and thus capable of being run in real-time.

Embodiments of the present disclosure may provide one or more of the following advantages:

1. Techniques for overcoming communication problems when they occur by predicting the missing information (e.g., leader vehicle speed), which enhances road safety in CAV environments;

2. The machine learning model may utilize a short history of the speeds only of a limited set of proximate vehicles (e.g., the leader vehicle and the surrounding cars within 200 meters). Therefore, the model may be of low complexity to enable it to be run in real-time;

3. The model may accurately predict leader speed over a long horizon, which may be used in case of severe communication disruption such as burst packet drops or loss of communication for a few seconds; and

4. The model may not need any additional information to be transmitted, and may utilize basic safety messages that are broadcast between vehicles (e.g., via C-V2X or DSRC), and/or may utilize sensor data.

According to one aspect of the present disclosure, a first connected autonomous vehicle, CAV, configured to connect to at least one of a radio base station and a second CAV is provided, where the first CAV includes processing circuitry configured to determine a network connection failure between the first CAV and at least one of the radio base station and the second CAV. In response to the determining of the network connection failure, the processing circuitry is configured to predict at least a speed of a first proximate vehicle based on a machine learning model and a vehicle history dataset. The vehicle history dataset is associated with at least one proximate vehicle. The processing circuitry is further configured to adjust the speed of the first CAV based on the prediction.

According to one or more embodiments of this aspect, the network connection failure is associated with at least one of a dropped and a delayed vehicle state information packet transmitted by one of the radio base station, and the second CAV. According to one or more embodiments of this aspect, the first CAV is configured for front collision avoidance, the first proximate vehicle being at least one of a leading vehicle in front of the first CAV, and the second CAV. According to one or more embodiments of this aspect, the vehicle history dataset includes at least one of a speed, a direction, an acceleration, and a coordinate of each of the at least one proximate vehicle associated with the vehicle history dataset. According to one or more embodiments of this aspect, the at least one proximate vehicle associated with the vehicle history dataset includes up to a maximum number of N vehicles which are closest in proximity to the first CAV. According to one or more embodiments of this aspect, N is 10. According to one or more embodiments of this aspect, the vehicle history dataset includes information from a time period T prior to the prediction. According to one or more embodiments of this aspect, T is 1.5 seconds. According to one or more embodiments of this aspect, the vehicle history dataset includes vehicle state information which is recorded according to a periodicity of 0.1 ms. According to one or more embodiments of this aspect, the at least one proximate vehicle associated with the vehicle history dataset includes only vehicles which are within a distance range R of a location of the first CAV. According to one or more embodiments of this aspect, R is 200 meters.

According to one or more embodiments of this aspect, the processing circuitry is further configured to receive sensor data collected by at least one vehicle sensor of the first CAV and the vehicle history dataset is based on the received sensor data.

According to one or more embodiments of this aspect, the first CAV is further configured to receive vehicle data associated with at least one additional CAV, the vehicle data being received from at least one of a direct connection with the at least one additional CAV, and the radio base station, where the radio base station is in communication with the at least one additional CAV, and the vehicle history dataset being based on the received vehicle data. According to one or more embodiments of this aspect, the machine learning model is a long short term memory, LSTM, deep neural network. According to one or more embodiments of this aspect, the machine learning model is trained at least one of offline, in a simulated testing environment, and prior to the network connection failure. According to one or more embodiments of this aspect, the prediction is further based on a road map associated with a location of the first CAV.

According to another aspect of the present disclosure, a method implemented in a first connected autonomous vehicle, CAV, configured to connect to at least one of a radio base station and a second CAV is provided, where the method includes determining a network connection failure between the first CAV and at least one of the radio base station and the second CAV. In response to the determining of the network connection failure, the method further includes predicting at least a speed of a first proximate vehicle based on a machine learning model and a vehicle history dataset. The vehicle history dataset is associated with at least one proximate vehicle. The method further includes adjusting the speed of the first CAV based on the prediction.

According to one or more embodiments of this aspect, the network connection failure is associated with at least one of a dropped and a delayed vehicle state information packet transmitted by one of the radio base station, and the second CAV. According to one or more embodiments of this aspect, the first CAV is configured for front collision avoidance, the first proximate vehicle being at least one of a leading vehicle in front of the first CAV, and the second CAV. According to one or more embodiments of this aspect, the vehicle history dataset includes at least one of a speed, a direction, an acceleration, and a coordinate of each of the at least one proximate vehicle associated with the vehicle history dataset. According to one or more embodiments of this aspect, the at least one proximate vehicle associated with the vehicle history dataset includes up to a maximum number of N vehicles which are closest in proximity to the first CAV. According to one or more embodiments of this aspect, N is 10. According to one or more embodiments of this aspect, the vehicle history dataset includes information from a time period T prior to the prediction. According to one or more embodiments of this aspect, T is 1.5 seconds. According to one or more embodiments of this aspect, the vehicle history dataset includes vehicle state information which is recorded according to a periodicity of 0.1 ms. According to one or more embodiments of this aspect, the at least one proximate vehicle associated with the vehicle history dataset includes only vehicles which are within a distance range R of a location of the first CAV. According to one or more embodiments of this aspect, R is 200 meters.

According to one or more embodiments of this aspect, the method further includes receiving sensor data collected by at least one vehicle sensor of the first CAV and the vehicle history dataset is based on the received sensor data.

According to one or more embodiments of this aspect, the first CAV is further configured to receive vehicle data associated with at least one additional CAV, the vehicle data being received from at least one of a direct connection with the at least one additional CAV, and the radio base station, where the radio base station is in communication with the at least one additional CAV, and the vehicle history dataset being based on the received vehicle data.

According to one or more embodiments of this aspect, the machine learning model is a long short term memory, LSTM, deep neural network. According to one or more embodiments of this aspect, the machine learning model is trained at least one of offline, in a simulated testing environment, and prior to the network connection failure. According to one or more embodiments of this aspect, the prediction is further based on a road map associated with a location of the first CAV.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present embodiments, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein:

FIG. 1 is a schematic diagram of an example vehicle speed model;

FIG. 2 is a schematic diagram of an example network architecture illustrating a communication system according to principles disclosed herein;

FIG. 3 is a block diagram of a radio base station in communication with a wireless device and CAVs over a wireless connection according to some embodiments of the present disclosure; FIG. 4 is a flowchart of an example process in a CAV for prediction-based road safety according to some embodiments of the present disclosure;

FIG. 5 is a schematic diagram of two proximate CAVs for prediction-based road safety according to some embodiments of the present disclosure;

FIG. 6 is a schematic diagram of a simulation testing environment for proximate CAVs for prediction based road safety according to some embodiments of the present disclosure;

FIG. 7 is an example machine learning model according to some embodiments of the present disclosure;

FIG. 8 is a schematic diagram of a simulation testing environment for proximate CAVs for prediction-based road safety according to some embodiments of the present disclosure;

FIG. 9 is a chart depicting simulation testing results according to some embodiments of the present disclosure;

FIG. 10 is another chart depicting simulation testing results according to some embodiments of the present disclosure;

FIG. 11 is another chart depicting simulation testing results according to some embodiments of the present disclosure; and

FIG. 12 is another chart depicting simulation testing results according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

As discussed above, in CAVs, each vehicle may periodically broadcast its state information to other vehicles. The information may include the vehicle’s ID, speed, acceleration, angle, and/or location. Upon receiving a message from its leader (i.e., leader/leading vehicle 4), the vehicle’s autopilot uses the received leader's speed to compute the vehicle’s safe speed (e.g., according to the Krauss model described above). However, vehicular communications may suffer latency and/or reliability issues, which may affect road safety. For example, if the leader vehicle 4 speed is not received by follower vehicle 2 while leader vehicle 4 is decelerating, the follower vehicle 2 speed will overestimate its safe speed, which may result in a crash/accident if the communication problem persists for too long. Further, existing systems consider the safety of the CAV without considering techniques for overcoming communication problems that may arise.

One or more embodiments of the present disclosure at least in part avoid such an accident and/or similar situations by providing a prediction-based model in a V2X environment.

Before describing in detail exemplary embodiments, it is noted that the embodiments reside primarily in combinations of apparatus components and processing steps related to prediction-based vehicle safety. Accordingly, components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

As used herein, relational terms, such as “first” and “second,” “top” and “bottom,” and the like, may be used solely to distinguish one entity or element from another entity or element without necessarily requiring or implying any physical or logical relationship or order between such entities or elements. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

In embodiments described herein, the joining term, “in communication with” and the like, may be used to indicate electrical or data communication, which may be accomplished by physical contact, induction, electromagnetic radiation, radio signaling, infrared signaling or optical signaling, for example. One having ordinary skill in the art will appreciate that multiple components may interoperate and modifications and variations are possible of achieving the electrical and data communication. In some embodiments described herein, the term “coupled,” “connected,” and the like, may be used herein to indicate a connection, although not necessarily directly, and may include wired and/or wireless connections.

The term “radio base station” used herein can be any kind of base station or network node comprised in a radio network which may further comprise any of base station (BS), radio base station, base transceiver station (BTS), base station controller (BSC), radio network controller (RNC), g Node B (gNB), evolved Node B (eNB or eNodeB), Node B, multi- standard radio (MSR) radio node such as MSR BS, multi- cell/multicast coordination entity (MCE), relay node, donor node controlling relay, radio access point (AP), transmission points, transmission nodes, Remote Radio Unit (RRU) Remote Radio Head (RRH), a core network node (e.g., mobile management entity (MME), self-organizing network (SON) node, a coordinating node, positioning node, MDT node, etc.), an external node (e.g., 3rd party node, a node external to the current network), nodes in distributed antenna system (DAS), a spectrum access system (SAS) node, an element management system (EMS), etc. The radio base station may also comprise test equipment. The term “radio node” used herein may be used to also denote a wireless device (WD) such as a wireless device (WD) or a radio base station. It may also be a road side unit (RSU), e.g., in the case of Dedicated Short Range Communication (DSRC).

In some embodiments, the non-limiting terms wireless device (WD) or a user equipment (UE) are used interchangeably. The WD herein can be any type of wireless device capable of communicating with a radio base station or another WD over radio signals, such as wireless device (WD). The WD may also be a radio communication device, target device, device to device (D2D) WD, machine type WD or WD capable of machine to machine communication (M2M), low-cost and/or low-complexity WD, a sensor equipped with WD, Tablet, mobile terminals, smart phone, laptop embedded equipped (LEE), laptop mounted equipment (LME), USB dongles, Customer Premises Equipment (CPE), an Internet of Things (loT) device, or a Narrowband loT (NB-IOT) device etc.

In some embodiments, the non-limiting terms connected autonomous vehicle (CAV) may be any type of vehicle (e.g., bicycle, car, truck, train, boat, aircraft, etc.) capable of communicating with a radio base station or another CAV over radio signals, such as via V2X communication. A CAV may include, and/or may be considered a type of, a wireless device.

Also, in some embodiments the generic term “radio network node” is used. It can be any kind of a radio network node which may comprise any of base station, radio base station, base transceiver station, base station controller, network controller, RNC, evolved Node B (eNB), Node B, gNB, Multi-cell/multicast Coordination Entity (MCE), relay node, access point, radio access point, Remote Radio Unit (RRU) Remote Radio Head (RRH).

Note that although terminology from one particular wireless system, such as, for example, 3GPP LTE and/or New Radio (NR), may be used in this disclosure, this should not be seen as limiting the scope of the disclosure to only the aforementioned system. Other wireless systems, including without limitation Wide Band Code Division Multiple Access (WCDMA), Worldwide Interoperability for Microwave Access (WiMax), Ultra Mobile Broadband (UMB), Global System for Mobile Communications (GSM), and DSRC may also benefit from exploiting the ideas covered within this disclosure.

Note further, that functions described herein as being performed by a wireless device/CAV or a radio base station may be distributed over a plurality of wireless devices/CAVs and/or radio base stations. In other words, it is contemplated that the functions of the radio base station and wireless device described herein are not limited to performance by a single physical device and, in fact, can be distributed among several physical devices.

In some embodiments, the general description elements in the form of “one of A and B” corresponds to A or B. In some embodiments, at least one of A and B corresponds to A, B or AB, or to one or more of A and B. In some embodiments, at least one of A, B and C corresponds to one or more of A, B and C, and/or A, B, C or a combination thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

Some embodiments are directed to prediction-based vehicle safety.

Referring to the drawing figures, in which like elements are referred to by like reference numerals, there is shown in FIG. 2 a schematic diagram of a communication system 10, according to an embodiment, such as a 3GPP-type cellular network that may support standards such as LTE, NR (5G), and/or C-V2X, which comprises an access network 12, such as a radio access network, and a core network 14. The access network 12 comprises a plurality of radio base stations 16a, 16b, 16c (referred to collectively as radio base stations 16), such as NBs, eNBs, gNBs or other types of wireless access points, each defining a corresponding coverage area 18a, 18b, 18c (referred to collectively as coverage areas 18). Each radio base station 16a, 16b, 16c is connectable to the core network 14 over a wired or wireless connection 20. A first wireless device (WD) 22a located in coverage area 18a is configured to wirelessly connect to, or be paged by, the corresponding radio base station 16a. A second WD 22b in coverage area 18b is wirelessly connectable to the corresponding radio base station 16b. While a plurality of WDs 22a, 22b (collectively referred to as wireless devices 22) are illustrated in this example, the disclosed embodiments are equally applicable to a situation where a sole WD is in the coverage area or where a sole WD is connecting to the corresponding radio base station 16. Note that although only two WDs 22 and three radio base stations 16 are shown for convenience, the communication system may include many more WDs 22 and radio base stations 16.

Also, it is contemplated that a WD 22 can be in simultaneous communication and/or configured to separately communicate with more than one radio base station 16 and more than one type of radio base station 16. For example, a WD 22 can have dual connectivity with a radio base station 16 that supports LTE and the same or a different radio base station 16 that supports NR. As an example, WD 22 can be in communication with an eNB for LTE/E-UTRAN and a gNB for NR/NG-RAN.

A first CAV 23a located in coverage area 18b may be configured to wirelessly connect to, or be paged by, the corresponding radio base station 16b. A second CAV 23b in coverage area 18b is wirelessly connectable to the corresponding radio base station 16b. While a plurality of CAVs 23 a, 23b (collectively referred to as CAVs 23) are illustrated in this example, the disclosed embodiments are equally applicable to a situation where a sole CAV is in the coverage area or where a sole CAV is connecting to the corresponding radio base station 16. Note that although only two CAVs 23 and three radio base stations 16 are shown for convenience, the communication system may include many more CAVs 23 and/or radio base stations 16.

Also, it is contemplated that a CAV 23 can be in simultaneous communication and/or configured to separately communicate with more than one radio base station 16 and more than one type of radio base station 16. For example, a CAV 23 can have dual connectivity with a radio base station 16 that supports LTE and the same or a different radio base station 16 that supports NR. As an example, CAV 23 can be in communication with an eNB for LTE/E-UTRAN and a gNB for NR/NG-RAN.

CAV 23a may communicate with CAV 23b via wireless connection 24, which may be, e.g., a D2D, V2X, C-V2X, etc., connection. For example, in one or more embodiments, one or more CAVs 23 are configured to communicate with radio base station 16 and one or more other CAVs 23. In another example, one or more CAVs 23 are configured to communicate only with one or more other CAVs 23.

A CAV 23 is configured to include a Communication-Aided Safety unit 26, which is configured to perform one or more CAV 23 functions described herein such as, for example, predicting one or more parameters/characteristics (e.g., speed) of a proximate vehicle/CAV of CAV 23 (e.g., a leader vehicle), and is further configured to include a vehicle control unit 27, which is configured to perform one or more CAV 23 functions as described herein such as, for example, controlling one or more functions of a vehicle (e.g., acceleration, braking, etc.), e.g., based on the predictions determined by the Communication- Aided Safety unit 26.

Example implementations, in accordance with an embodiment, of the WD 22, CAV 23, and radio base station 16 discussed in the preceding paragraphs will now be described with reference to FIG. 3.

The communication system 10 includes a radio base station 16 provided in a communication system 10 and including hardware 28 enabling it to communicate with the WD 22 and/or CAV 23. The hardware 28 may include a radio interface 30 for setting up and maintaining at least a wireless connection 32 with a WD 22 and/or CAV 23 located in a coverage area 18 served by the radio base station 16 using wireless connection 63. The radio interface 30 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and/or one or more RF transceivers. The radio interface 30 includes an array of antennas 34 to radiate and receive signal(s) carrying electromagnetic waves.

In the embodiment shown, the hardware 28 of the radio base station 16 further includes processing circuitry 36. The processing circuitry 36 may include a processor 38 and a memory 40. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 36 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 38 may be configured to access (e.g., write to and/or read from) the memory 40, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory).

Thus, the radio base station 16 further has software 42 stored internally in, for example, memory 40, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the radio base station 16 via an external connection. The software 42 may be executable by the processing circuitry 36. The processing circuitry 36 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by radio base station 16. Processor 38 corresponds to one or more processors 38 for performing radio base station 16 functions described herein. The memory 40 is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 42 may include instructions that, when executed by the processor 38 and/or processing circuitry 36, causes the processor 38 and/or processing circuitry 36 to perform the processes described herein with respect to radio base station 16.

The communication system 10 further includes the WD 22 already referred to. The WD 22 may have hardware 44 that may include a radio interface 46 configured to set up and maintain a wireless connection 32 with a radio base station 16 serving a coverage area 18 in which the WD 22 is currently located. The radio interface 46 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and/or one or more RF transceivers. The radio interface 46 includes an array of antennas 48 to radiate and receive signal(s) carrying electromagnetic waves.

The hardware 44 of the WD 22 further includes processing circuitry 50. The processing circuitry 50 may include a processor 52 and memory 54. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 50 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 52 may be configured to access (e.g., write to and/or read from) memory 54, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory).

Thus, the WD 22 may further comprise software 56, which is stored in, for example, memory 54 at the WD 22, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the WD 22. The software 56 may be executable by the processing circuitry 50. The software 56 may include a client application 58. The client application 58 may be operable to provide a service to a human or non-human user via the WD 22.

The processing circuitry 50 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by WD 22. The processor 52 corresponds to one or more processors 52 for performing WD 22 functions described herein. The WD 22 includes memory 54 that is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 56 and/or the client application 58 may include instructions that, when executed by the processor 52 and/or processing circuitry 50, causes the processor 52 and/or processing circuitry 50 to perform the processes described herein with respect to WD 22.

The communication system 10 further includes the CAV 23a already referred to. The CAV 23a may have hardware 60 that may include a radio interface 62 configured to set up and maintain a wireless connection 63 with a radio base station 16 serving a coverage area 18 in which the CAV 23a is currently located, and/or set up a (direct) wireless connection 24 with another CAV 23b. The radio interface 62 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and/or one or more RF transceivers. The radio interface 62 includes an array of antennas 64 to radiate and receive signal(s) carrying electromagnetic waves.

The hardware 60 of the CAV 23a further includes vehicle hardware 66, which may be one or more hardware elements typical of a vehicle, e.g., vehicle sensors (LiDAR, radar, imaging, speed, acceleration, GPS, etc.), engine(s), wheels, brakes, batteries, steering, etc. In applications where the vehicle is a boat, plane, etc., the vehicle hardware 66 would also include appropriate hardware for such vehicle, e.g., rudders, propellers, ailerons, etc.

The hardware 60 of the CAV 23a further includes processing circuitry 68. The processing circuitry 68 may include a processor 70 and memory 72. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 68 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 70 may be configured to access (e.g., write to and/or read from) memory 72, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory).

Thus, the CAV 23a may further comprise software 74, which is stored in, for example, memory 72 at the CAV 23a, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the CAV 23a. The software 74 may be executable by the processing circuitry 68. The software 74 may include a client application 76. The client application 76 may be operable to provide a service to a human or non-human user via the CAV 23 a.

The processing circuitry 68 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by CAV 23a. The processor 70 corresponds to one or more processors 70 for performing CAV 23a functions described herein. The CAV 23a includes memory 72 that is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 74 and/or the client application 76 may include instructions that, when executed by the processor 70 and/or processing circuitry 68, causes the processor 70 and/or processing circuitry 68 to perform the processes described herein with respect to CAV 23a. For example, processing circuitry 68 of the CAV 23a may include Communication-Aided Safety unit 26 which is configured to predict one or more parameters/characteristics (e.g., speed) of a proximate vehicle (e.g., CAV 23b) of CAV 23a, which may be based on a machine learning model stored, e.g., in memory 72, on proximate vehicle data stored, e.g., in memory 72, and/or on road map data stored, e.g., in memory 72. For example, processing circuitry 68 of the CAV 23a may include vehicle control unit 27, which is configured to control one or more functions/vehicle hardware 66 of a vehicle (e.g., acceleration, braking, etc.), e.g., based on the predictions determined by the Communication- Aided Safety unit 26

The CAV 23 a may be in communication with one or more additional CAV 23b via, e.g., V2X connection 24, and/or via connection(s) 63 with base station(s) 16. The inner workings of CAV 23b may be similar or identical to that of CAV 23a.

In some embodiments, the inner workings of the radio base station 16, WD 22, and CAV 23, may be as shown in FIG. 3 and independently, the surrounding network topology may be that of FIG. 2.

Although FIGS. 2 and 3 show various “units” such as Communication- Aided Safety unit 26 and vehicle control unit 27 as being within a respective processor, it is contemplated that these units may be implemented such that a portion of the unit is stored in a corresponding memory within the processing circuitry. In other words, the units may be implemented in hardware or in a combination of hardware and software within the processing circuitry.

FIG. 4 is a flowchart of an example process in a CAV 23 for prediction-based vehicle safety according to some embodiments of the present disclosure. One or more blocks described herein may be performed by one or more elements of CAV 23 such as by one or more of vehicle hardware 66, processing circuitry 68 (including the Communication- Aided Safety unit 26 and/or vehicle control unit 27), processor 70, and/or radio interface 62. A first CAV 23a is configured to determine (Block S100) a network connection failure between the first CAV 23a and at least one of a radio base station 16 and a second CAV 23b. In response to the determining of the network connection failure, the first CAV 23a is configured to predict (Block S102) at least a speed of a first proximate vehicle based on a machine learning model and a vehicle history dataset, where the vehicle history dataset is associated with at least one proximate vehicle. The first CAV 23a is configured to adjust (Block S104) the speed of the first CAV 23a based on the prediction.

In some embodiments, the network connection failure is associated with at least one of a dropped and a delayed vehicle state information packet transmitted by one of the radio base station 16 and the second CAV 23b. In some embodiments, the CAV 23 a is configured for front collision avoidance, where the first proximate vehicle is at least one of a leading vehicle in front of the first CAV 23a, and the second CAV 23b

In some embodiments, the vehicle history dataset includes at least one of a speed, a direction, an acceleration, and a coordinate of each of the at least one proximate vehicle associated with the vehicle history dataset. In some embodiments, the at least one proximate vehicle associated with the vehicle history dataset includes up to a maximum number of N vehicles which are closest in proximity to the first CAV. In some embodiments, N is 10. In some embodiments, the vehicle history dataset includes information from a time period T prior to the prediction. In some embodiments, T is 1.5 seconds. In some embodiments, the vehicle history dataset includes vehicle state information which is recorded according to a periodicity of 0.1 ms. In some embodiments, the at least one proximate vehicle associated with the vehicle history dataset includes only vehicles which are within a distance range R of a location of the first CAV. In some embodiments, R is 200 meters.

In some embodiments, the first CAV 23a is further configured to receive sensor data collected by at least one vehicle sensor (of vehicle hardware 66) of the first CAV 23a, where the vehicle history dataset is based on the received sensor data.

In some embodiments, the first CAV 23a is further configured to receive vehicle data associated with at least one additional CAV 23x, the vehicle data being received from at least one of a direct connection with the at least one additional CAV 23x and the radio base station 16, where the radio base station 16 is in communication with the at least one additional CAV 23x, and the vehicle history dataset is based on the received vehicle data.

In some embodiments, the machine learning model is a long short term memory, LSTM, deep neural network. In some embodiments, the machine learning model is trained at least one of offline, in a simulated testing environment, and prior to the network connection failure. In some embodiments, the prediction is further based on a road map associated with a location of the first CAV 23a.

Having described the general process flow of arrangements of the disclosure and having provided examples of hardware and software arrangements for implementing the processes and functions of the disclosure, the sections below provide details and examples of arrangements for prediction-based vehicle safety.

Some embodiments provide a CAV 23 with a deep learning module (e.g., Communication- Aided Safety unit 26) installed. Referring to FIG. 5, each CAV 23 may be equipped with a Communication- Aided Safety unit 26, which may include, e.g., a Vehicle State Reporter (VSR) module 78; a communication interface module 80 (which may be, and/or may communicate with, and/or may utilize radio interface 62), and which may be configured for 5G-NR communications, DSRC communications, C-V2X communications, etc.; a Vehicle Speed Controller (VSC) module 82 (which may be, and/or may communicate with, and/or may utilize vehicle hardware 66 and/or vehicle control unit 27); deep learning speed prediction (DLSP) module 84; and a vehicle trajectory database 86b (which may be, and/or may communicate with, and/or may utilize memory 72).

The VSR module 78 of the Communication-Aided Safety unit 26 may be responsible for periodically broadcasting the vehicle information (e.g., of CAV 23b which may be the leader vehicle) to the surrounding vehicles. For example, the CAV 23b may periodically (e.g., over 0.1-second interval) broadcast the vehicle information over the shared wireless medium (e.g., C-V2X, D2D, DSRC, etc.). As another example, the CAV 23b may, via C-V2X, utilize the broadcasting capability of the base station, where the VSR module 78 may communicate the CAV 23b vehicle information to the radio base station 16, which may broadcast it to other CAVs 23 in the vicinity of the sender CAV 23b. Or, in the case of side-link/D2D communications, the VSR module 78 may directly broadcast the messages to other vehicles.

The VSC module 82 of the Communication-Aided Safety unit 26 may be responsible for determining parameters used by the vehicle’s autopilot (e.g., by the vehicle control unit 27). For example, the VSC module 82 may be responsible for computing the safe speed of the vehicle, e.g., based on Equations 1 and 2 described herein. Then, the VSC module 82 may instruct the vehicle control unit 27 to control the fuel injection rate to safely and comfortably reach this safe speed. If the VSC module 82 did not receive the required information (e.g., leader speed) for a given time interval, it may request the (predicted) leader speed from the DLSP module 84, and may use this predicted speed to adjust the subject vehicle speed.

The DLSP module 84 may be trained offline and may be running continuously/constantly and may wait for prediction requests, e.g., from the VSC module 82. To perform predictions, the VSC module 82 may require the history data of the leader and/or of the surrounding vehicles, which should already exist in the Vehicle Trajectory Database (VTDB) 86. The DLSP module 84 may read the history of the leader vehicle and the nearest N surrounding vehicles (for example, N=9, thus 10 vehicles in total) over the last T time steps (e.g., 15 time steps, where each step is, for example, 0.1 seconds, for a total period of 1.5 seconds), and may use it to predict the leader speed and/or any other vehicle speed, and may send it back to the VSC module 82.

The VTDB 86 may be a database (e.g., a small database) that may store a history (e.g., a short history) of the speeds (and/or any other vehicle state information) of the surrounding/proximate vehicles. In simulation testing, only 15-time steps were needed to achieve acceptable prediction accuracy or prediction accuracy meeting a predefined threshold. This database may be used as an input for the deep learning model, e.g., to predict the speed of the leading vehicle.

Still referring to FIG. 5, in a first step of one example embodiment of the present disclosure, (Step S106), the VSR module 78b of the leader vehicle (e.g., CAV) 23b sends vehicle state information (e.g., speed, location, direction, acceleration, etc.) to the communication module 80b. In a second step, the communication module 80b transmits (Step S108) the vehicle state information to surrounding vehicles including the follower vehicle CAV 23 a, where such transmission may be via, e.g., one or more of radio interface 62b, antennas 64b, radio interface 62a, antennas 64a, wireless connection 63, V2X connection 24, etc. It also may be broadcast or unicast communication. In a third step (Step S 110), the receiver (communication module 80a) in the follower vehicle receives the vehicle state information and sends the leader speed to the VSC module 82a. It also updates the VTDB. In a fourth step (Step SI 12), the VSC module 82a determines that it did not receive leader speed information for 2 consecutive periods, for example, and sends a request to the DLSP module 84a. In a fifth step (Step S 114), the DLSP module 84a of the follower vehicle CAV 23a reads the trajectory history from the VTDB 86a and uses it to predict the leader vehicle CAV 23b speed. In a sixth step (Step SI 16), the DLSP module 84a sends back the leader vehicle CAV 23b predicted speed to the VSC module 82a. It also updates the VTDB with the predicted information to fill in the missing information. In subsequent steps (not shown), the Communication-Aided Safety unit 26 may send the predicted speed to the vehicle control unit 27, which may adjust one or more parameters of vehicle hardware 66 (e.g., acceleration, braking, steering, etc.) in response to the predicted speed, e.g., in order to attain a safe speed and/or avoid crashing with the leader vehicle CAV 23b.

The impact of communication on the safety of CAV 23 may be modeled according to the example simulation framework shown in FIG. 6, in which network simulator 3 (NS3) and Simulation of Urban Mobility (SUMO) simulators are bidirectionally coupled to synchronize the two simulators in time and to exchange the required information between them. Within this simulation framework, each CAV 23 is represented by two entities, one in the NS3 platform to perform the communication tasks and another in the SUMO platform for mobility and safety assessment.

The simulations are configured as follows: The SUMO simulator reads its inputs (i.e., the road network information and vehicular traffic setting) and starts moving vehicles on the road network considering the traffic conditions on the different road segments, each vehicle’s destination, the traffic signals (traffic lights and stop/yield signs) and the required maneuvering (e.g., changing lanes and taking over other cars). It also computes the vehicle speed based on its parameters, such as its maximum acceleration/deceleration rates. On the other side and in parallel, the NS3 simulator reads the communication settings and applies these settings to vehicles before starting the simulation.

While the simulation is running, NS3 and SUMO synchronize the number of cars and their positions as follows. NS3 periodically pulls the vehicle mobility information from SUMO. Particularly, it reads a list of the vehicles (e.g., CAVs 23) that are currently running on the road network along with their locations. Based on this information, NS 3 updates the vehicles’ locations and moves each vehicle to its new position. This synchronization is performed every time step, with a typical value time step value of 0.1 seconds. If there are new vehicles that entered the network during the previous time step, NS3 activates new ones and moves them to their positions. If a vehicle in NS3 is not on the vehicle list, this means is vehicle has left the road network. Therefore, NS 3 deactivates this vehicle by stopping all the applications (e.g., VSR, VSC) running on it.

In the simulated environment, the forward collision avoidance safety application is modelled by two modules in each vehicle, namely, the Vehicle State Reporter (VSR) 78 and the Vehicle Speed Controller (VSC) 82. The VSR module 78 is responsible for periodically communicating the vehicle state information to other vehicles. The VSC module 82 is responsible for receiving these messages from the leader vehicle and computing the vehicle’s safe speed.

Because state information is exchanged within NS3 while the actual speed control is performed in SUMO, the VSC 82 is divided into two sub-modules, one in NS3 and another one in SUMO. The VSC 82 in NS3 receives messages from other vehicles, if the message sender is the leader vehicle, the VSC 82 communicates this leader speed to the VSC 82 module in SUMO through the communication module established between the two simulators. On the SUMO side, the VSC 82 periodically computes the vehicle’s safe speed based on the most recent received leader speed. Consequently, if a packet is dropped or delayed, the vehicle’s safe speed will be updated using the previous leader's speed. This captures the impact of communication imperfection on the forward collision avoidance performance.

The speed prediction (i.e., DLSP) module 84 is the third component in the simulation system, that the DLSP is used to estimate the leader's vehicle speed. This module may be trained offline and run on every vehicle to provide the prediction service. When the VSC module 82 in the follower vehicle does not receive the speed of the leader, it will request the prediction service from the speed prediction module 84, which estimates the leader speed and sends it back to the VSC module 82.

According to some embodiments of the present disclosure, a deep learning model may be used, in particular a long short term memory (ESTM) deep neural network, to build a model that can use the speed of the surrounding vehicles to predict the leader vehicle speed. For example, in some embodiments, the input for the deep learning model is a 10x15 matrix representing the speed of the 10 nearest cars (including the leader speed) within the range of 200 meters over the last 15 time steps. Other parameters may be used without deviating from the scope of the present disclosure (e.g., fewer or more nearest cars may be considered, the range may be greater or less than 200 meters, the number of time steps may be greater than or less than 15, other vehicle parameters such as location and/or directions may be used, etc.). The 10x15 matrix is shown in Equation 3:

(Equation 3)

Where x t = [x 0 , x r , , x 9 ] is the vector of the surrounding vehicles’ speeds at time t. In this example, the matrix includes the leader history in the first column and the last two values in this column are always 0 since the vehicle did not receive the leader speed in the last two intervals.

An example LTSM model is shown in FIG. 7. The example model has 2 LSTM layers, the first with 100 units, where each unit is 15x10, and the second with 50 units. These parameters are merely exemplary, and other model parameters may be used without deviating from the scope of the present disclosure. Further, other machine learning models may be used without deviating from the scope of the present disclosure.

To generate a dataset for the model training purpose, the SUMO road traffic simulator may be used. Alternatively, any other suitable road traffic simulator known in the art may be used. An example model road network is shown in FIG. 8, in which the network has an intersection with a traffic light. The traffic streams are shown by the dashed arrows. In this example, each traffic stream has 5 CAVs 23 starting at random times at the beginning of the simulation and is repeated after ten seconds, although other parameters may be used without deviating from the scope of the present disclosure.

To simulate a communication problem in the data generation process, a radio base station 16 is simulated located in the middle of the network shown in FIG. 8. While CAVs 23 are moving, they exchange the vehicle state information through the radio base station 16. Whenever a vehicle receives vehicle state message from another vehicle it updates its history (e.g., VTDB). Periodically (e.g., every 0.1 seconds), each vehicle adds a new entry to the data set. In this example, each entry has a 10x15 matrix that represents the speed of the surrounding CAVs 23 over the previous 15- time steps in addition to the actual speed of the leading CAV 23. If a CAV 23 does not receive vehicle data from another CAV 23, it will use its previous received speed. The speed of the leader vehicle (e.g., the variable y in Equation 3) associated with each entry is the actual speed of the leading CAV 23, not the last received one from this leader CAV 23. This way, the data set represents the actual information at each CAV 23 and accounts for any packet drops or delays in its training data.

This data is used to train the LSTM model offline. Then, the trained model is saved, e.g., in memory 72. The simulation shown in FIG. 8 may be run in two cases. The first case is the base case where there is no traffic prediction. In this case, the CAV 23 uses the most recent information received from its leader to compute the safe speed. Thus, if packets are dropped, this data may be obsolete, which may result in accidents. In the second case, the VSC 82 uses the deep learning speed prediction module 84 to estimate the leader speed. In both cases, each CAV 23 is configured to upload data to a server in the network core through the radio base station 16. This traffic may be referred to as the ‘background traffic’. When a CAV 23 starts, an application starts on the CAV 23 (e.g., on processing circuitry 68) which extracts a random number from a uniform distribution between 0 and a maximum Background Traffic Rate (BTR), which simulates background traffic/noise. Higher BTR is typically associated with a higher likelihood of communication failures which may increase the delay and the drop probability of the vehicle state information packets. FIGS. 9 and 10 show a sample of speed prediction for some CAVs 23 in which the LSTM model may accurately estimate the leader speed, demonstrating that this model may be used when the communication network fails to transmit the vehicle state information messages. FIGS. 9 and 10 also shows that the model can predict the speed for a long time that spans a few seconds, which can overcome severe communication problems such as disruption due to handover or signal blockage.

Each of the main cases may be run with different maximum background rates starting from no background traffic to 5 Mpps at a step of 0.5 Mpps. The number of simulated accidents in each case is shown in Table 1 below. In Table 1, the number of simulated accidents that happened in the base case due to the communication problems is contrasted with the second case where deep learning is used to predict the leader speed. The table shows that the packet drops and delays that happen at high BTR (3 Mbps and higher) produced several accidents.

Table 1: Number of accidents with and without the prediction

However, when using the speed prediction instead of the last received leader speed, the results in the table show that using the proposed speed prediction model succeeded in avoiding all the accidents.

FIGS. 11 and 12 show the leader speed (FIG. 11) and the headway distance between the two vehicles (FIG. 12) for one of the simulated accidents that happened in the base case and compares it to that with the prediction case. The accident happened in the base case because the communication started failing at time t= 12 seconds, and packet drops continued for a relatively long time. The follower vehicle continued using the latest received leader speed, which is higher than the actual leader speed, which results in an accident at t= 16 seconds, as shown in FIG. 12. By contrast, when prediction is enabled according to embodiments of the present disclosure, the accident is avoided.

As will be appreciated by one of skill in the art, the concepts described herein may be embodied as a method, data processing system, computer program product and/or computer storage media storing an executable computer program. Accordingly, the concepts described herein may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a “circuit” or “module.” Any process, step, action and/or functionality described herein may be performed by, and/or associated to, a corresponding module, which may be implemented in software and/or firmware and/or hardware. Furthermore, the disclosure may take the form of a computer program product on a tangible computer usable storage medium having computer program code embodied in the medium that can be executed by a computer. Any suitable tangible computer readable medium may be utilized including hard disks, CD-ROMs, electronic storage devices, optical storage devices, or magnetic storage devices.

Some embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, systems and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer (to thereby create a special purpose computer), special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable memory or storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

It is to be understood that the functions/acts noted in the blocks may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

Computer program code for carrying out operations of the concepts described herein may be written in an object oriented programming language such as Python, Java® or C++. However, the computer program code for carrying out operations of the disclosure may also be written in conventional procedural programming languages, such as the "C" programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer may be connected to the user's computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, all embodiments can be combined in any way and/or combination, and the present specification, including the drawings, shall be construed to constitute a complete written description of all combinations and subcombinations of the embodiments described herein, and of the manner and process of making and using them, and shall support claims to any such combination or subcombination.

Abbreviations that may be used in the preceding description include:

CAV Connected Autonomous Vehicles

C-V2X Cellular Vehicle to Everything

DLSP Deep Learning Speed Prediction

DSRC Dedicated Short Range Communication

FCAS Forward Collision Avoidance System

VSC Vehicle State Reporter

VSC Vehicle Speed Controller

VTDB Vehicle Trajectory Database

V2X Vehicle to Everything

3 GPP 3rd Generation Partnership Project

5G-NR 5Th Generation- New Radio

It will be appreciated by persons skilled in the art that the embodiments described herein are not limited to what has been particularly shown and described herein above. In addition, unless mention was made above to the contrary, it should be noted that all of the accompanying drawings are not to scale. A variety of modifications and variations are possible in light of the above teachings without departing from the scope of the following claims.