CROSS-REFERENCE TO A RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Application No. 62/633,366, filed February 21 , 2018, the disclosure of which is incorporated herein in its entirety.

FIELD OF THE INVENTION

[0002] The present disclosure relates to light detection and ranging (LiDAR), and in particular to LiDAR systems and methods for use in a vehicle.

BACKGROUND [0003] Systems exist that enable vehicles to be driven semi-autonomously or fully autonomously. Such systems may use one or more range finding, mapping, or object detection systems to provide sensory input to assist in semi-autonomous or fully autonomous vehicle control. Conventional LiDAR systems can use fiber optic lasers or diode lasers as the light source. Both light sources have their advantages and disadvantages.

BRIEF SUMMARY

[0004] Embodiments discussed herein refer to LiDAR systems that use diode lasers to generate a high-repetition rate and multi-mode light pulse that is input to a fiber optic cable that transmits the light pulse to a scanning system. The fiber optic cable enables the diode lasers to be separated from the scanning system. This separation provides advantages of enhanced temperature control of the diode laser, especially if they are contained in a relatively more temperature controlled environment than that of the scanning system, and further provides design flexibility for the scanning system, which no longer has to house the diode lasers.

[0005] A laser system for use with a fiber optic cable is provided according to one embodiment. The laser system can include control circuitry and a multi-diode laser and optical fiber coupling (MDOFC) coupled the control circuitr ' and the fiber optic cable, the MDOFC operative to generate an optic fiber input beam that is a stack of integrated beams derived from a plurality of diode lasers that emit beams in a multi-mode, wherein the optic fiber input beam is input into the fiber optic cable. [0006] A light detection and ranging (LiDAR) system is provided according an embodiment that includes a control system constructed to be contained within an interior portion of a vehicle. The control system can include control circuitry, and a multi-diode laser and optical fiber coupling (MDOFC) coupled the control circuitry and operative to generate an integrated light beam derived from a plurality of light beams. The system can include a scanning system constructed to be mounted to an exterior portion of the vehicle, and a fiber optic cable coupled to the MDOFC and the scanning system, wherein the fiber optic cable transmits the integrated light beam to the scanning system.

[0007] A light detection and ranging (LiDAR) system is provided according to another embodiment. The system can include a control system contained within a temperature controlled environment. The control system can include control circuitry, at least one diode laser each operative to emit a light beam in response to control signals provided by the control circuitry, and a lens that converges each light beam to have an angle less than a threshold. The system can include a scanning system mounted outside of the temperature controlled environment, and a fiber optic cable coupled to the control system and the scanning system, wherein the fiber optic cable receives a converged light beam from the lens and transmits the light beam to the scanning system.

[0008] A further understanding of the nature and advantages of the embodiments discussed herein may be realized by reference to the remaining portions of the specification and the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG. 1 shows an illustrative vehicle having a LiDAR system that is attached to and/or incorporated therein, according to an embodiment;

[0010] FIGS. 2A and 2B show different illustrative views of multi-diode laser and optical fiber coupling, according to an embodiment;

[0011] FIG. 2C shows an illustrative view of a multi-mode diode emitter according to an embodiment;

[0012] FIGS. 2D, 2E, and 2F each show different integrated beam patterns that can be produced with different arrangements of diode sub-systems, according to various embodiments; [0013] FIG. 3 shows an illustrative simulation result showing irradiance of an integrated laser light signal being provided by a multi-diode laser and optical fiber coupling to a fiber at the coupling, according to an embodiment;

[0014] FIG. 4 shows an illustrative block diagram of a scanning system according to an embodiment;

[0015] FIG. 5 shows an illustrative beam shape at the near and far fields, according to an embodiment; and

[0016] FIG. 6 is a functional block diagram illustrating a vehicle system, according to an embodiment.

DETAILED DESCRIPTION

[0017] Illustrative embodiments are now described more fully hereinafter with reference to the accompanying drawings, in which representative examples are shown. Indeed, the disclosed communication systems and methods may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Like numbers refer to like elements throughout.

[0018] In the following detailed description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of the various embodiments. Those of ordinary skill in the art will realize that these various embodiments are illustrative only and are not intended to be limiting in any way. Other embodiments will readily suggest themselves to such skilled persons having the benefit of this disclosure.

[0019] In addition, for clarity purposes, not all of the routine features of the embodiments described herein are shown or described. One of ordinary skill in the art would readily appreciate that in the development of any such actual embodiment, numerous embodiment- specific decisions may be required to achieve specific design objectives. These design objectives will vary from one embodiment to another and from one developer to another. Moreover, it will be appreciated that such a development effort might be complex and time- consuming but would nevertheless be a routine engineering undertaking for those of ordinary skill in the art having the benefit of this disclosure.

[0020] FIG. 1 shows an illustrative vehicle 100 having a LiDAR system 120 that is attached to and/or incorporated therein according to an embodiment. Vehicle 100 can be genetically classified as having interior portion 101 and exterior portion 102. Interior portion 101 may be portions of vehicle 100 that arc not directly exposed to external environmental factors such as the environment conditions (e.g., water, humidity, sun, ice, wind, etc.) and road conditions (e.g., road debris). Interior portion 101 may be influenced by external environment conditions but to a lesser degree than exterior portion 102. LiDAR system 120 may include, among other features, control system 130, cable system 140, and scanning system 150.

Control system 130 may be contained within interior portion 101, scanning system 150 may be mounted to exterior portion 102, and cable system 140 may exist solely within interior portion 101 or can exist as part of interior portion 101 and exterior portion 102.

[0021] Control system 130 can include controller 132 and laser system 134. Controller 132 and laser system 134 may be coupled to scanning system 150 via cable system 140. Laser system 134 may transmit light pulses through cable system 140 to scanning system 150.

Laser system 134 may use diode lasers to generate light pulses or fiber lasers. Controller 132 may control a current source at which laser system 134 transmits its light pulses. Controller 132 may receive detector signals from scanning system 150 via cable system 140. The detector signals may be the return or consequence signals that are detected by one or more detectors when the transmitted light pulses bounce of an object being observed by scanning system 150. Scanning system 150 may include the appropriate lenses, mirrors, steering optics, and detectors needed to capture an image of a scene existing within a vicinity of vehicle 100.

[0022] LiDAR system 120 separates laser system 132 and scanning system 150 from each other such that laser system 132 is contained within interior portion 101. Keeping laser system 132 (and other components associated with control system 130) within interior portion 101 provides an environment that is less harsh than that of exterior portion 102. This provides cooling advantages over containing laser system 134 as part of scanning system 150, which is located on exterior portion 102. Laser system 134 can use diode lasers or fiber lasers, and the light pulses transmit through cable system 140 to scanning system 150. Cable system 140 can include one or more fiber optic cables for transmitting light pulses from laser system 134 to scanning system 150. Cable system 140 can include one or more electrical conduits for transferring electrical signals between control system 130 and scanning system 150. For example, control system 130 may provide instructions to scanning system 150 to control steering optics. As another example, scanning system 150 may provide detection signals to controller 132 via cable system 140. [0023] FIGS. 2A and 2B show different illustrative views of multi-diode laser and optical fiber coupling (MDOFC) 200 according to an embodiment. MDOFC 200 shows several diode lasers that each emit beams that are coupled into fiber optic cable 290. Fiber optic cable 290 may be connected to a scanning system (not shown) and serves as a transmission conduit of light pulses (originating from the diodes) that are used by the scanning system. MDOFC 200 can include diode sub-systems 210, 220, 230, and 240, mirrors 250 and 252, polarization beam splitters (PBS) 260 and 262, lens 270, and fiber optic cable 290. Each of diode sub-systems 210, 220, 230, and 240 can include a diode laser (21 1 , 221, 231 , or 241), a fast axis collimator (213, 223, 233, and 243), and a slow axis collimator (215, 225, 235, and 245). Light pulses emanating from diode 21 1 passes through fast collimator 213 and slow collimator 215. The combination of fast and slow axis collimators 213 and 215 control the focal length and beam shape of the light emitted by diode laser 21 1. Light pulses emanating from diode lasers 221, 231, and 241 pass through own respective fast and slow axis collimators, which control the focal lengths of the light emitted by those diode lasers. After the light pulses pass through respective slow air collimators, the light pulse interacts with a mirror or a polarization beam splitter.

[0024] Diode laser 21 1 , 221, 231 , and 241 may each be multi-mode laser emitters or emitter stack and not single mode emitters. Multi-mode diode lasers are much cheaper and powerful than signal mode emitters. For example, in one embodiment, diode laser 211 may be an 85mhi diode laser available from Laser Components, Inc. of Bedford, New Flampshire. FIG. 2C shows an illustrative block diagram of diode laser 21 1. Diode laser 211 is operative to emit a multi-mode laser beam out of its X-Y plane, as indicated by the laser beams (shown as arrows). As shown, X is substantially larger than Y. Thus, the width (e.g., X plane ) of the laser beam is substantially larger than its height (e.g. Y plane). MDOFC 200 can input a laser beam that has a height that is substantially the same as its width. MDOFC 200 does this by combining pairs of beams (a beam from two different diode lasers) and stacking those combined beams on top of each other.

[0025] For each of diode lasers 21 1 , 221, 231 , and 241 , their respective slow and fast axis collimators adjust the size, shape, and divergence angle of the beam emanating from each diode laser. That is, the fast and slow _' axis collimators precondition the beam from each laser diode before pairs of beams (each associated wdth a different diode laser) are combined or integrated to form an integrated beam and couple into a given fiber 290. See FIGS. 2A and 2B, which show' the beam changing shape and size from one collimator to the other. In one embodiment, diode lasers 211 and 241 may be one of transverse electric (TE) or transverse magnetic (TM) polarization modes and diode lasers 221 and 231 may be the opposite polarization mode of diode lasers 211 and 241. For example, if diode lasers 211 and 241 are TE diode lasers, diode lasers 221 and 231 are TM diode lasers. As will be explained below, the beams emanating from diode lasers 211 and 221 are combined together and the beams emanating from diode lasers 231 and 241 are combined together. Since diode laser 211 is in the TE configuration and diode laser 221 is in the TM configuration, they can be combined by a polarization beam splitter. The same logic holds true for diode lasers 231 and 241.

[0026] Diode sub-systems 210 and 220 may combine their respective light pulses to form a first integrated beam that is directed towards fiber 290 via lens 270. Light emitted from sub system 210 is reflected by mirror 250 and is combined with light emitted from sub-system 220 at PBS 260 to yield first integrated beam 280. First integrated beam 280 is projected along a first plane defined by the position of mirror 250 and PBS 260. Diode sub-systems 230 and 240 may combine their respective light pulses to form a second integrated beam that is directed towards fiber 290 via lens 270. Light emitted from sub-system 230 is reflected by mirror 252 and is combined with light emitted from sub-system 240 at PBS 262 to yield second integrated beam 282. Second integrated beam 282 is projected along a second plane defined by the position of mirror 252 and PBS 262. The first and second planes are parallel to each other but are offset from each other by' a fixed distance such that the first and second integrated beams 280 and 282 are stacked on top of each other and are directed through optical lens 270, which may be an aspherical lens. Lens 270 can combine beams 280 and 282 to form third integrated beam 284. Third integrated beam 284 may be input directly into fiber 290.

[0027] It should be appreciated that the design and configuration of MDOFC 200 may vary _' depending on a number of different factors. For example, the diameter of the core and the numerical aperture of fiber 290 may dictate the size and operating properties of each diode laser chosen for use in MDOFC 200, the offset distance between paired diode lasers, and how many paired diode lasers can be stacked together. For example, the offset distance between first and second integrated beams 280 and 282 may be set to maximize power transfer of integrated beam 284 into fiber 290. It should be further appreciated that the positioning of diode sub-systems 210, 220, 230, and 240 can be rearranged. For example, diode sub-system 210 can be positioned such that it is aligned adjacent to sub-system 220, and similarly, sub system 240 can also be positioned adjacent to sub-system 230 so that all sub-systems 210, 220, 230, and 240 are on the same side of the beams 280 and 282. Other components such as mirror 250 and PBS 262 may have to be reoriented to accommodate the different positions of sub-systems 210 and 240.

[0028] If desired, any suitable number of diode sub-system pairs may be stacked together to produce the desired integrated beam for coupling with the fiber. For example, if desired, three or more coupled pairs of diode laser beams may be stacked on top of each other. In some embodiments, the final integrated beam input into the fiber optic cable may be the product of a lxN column of diode sub-system pairs (as illustrated in FIG. 2D), a lxM row of diode sub-system pairs (as illustrated in FIG. 2E), or a MxN matrix of diode sub-system pairs (as illustrated in FIG. 2F), where M and N are integers greater than zero. FIGS. 2D, 2E, and 2F each show different integrated beam patterns that can be produced with different arrangements of diode sub-systems.

[0029] Advantages of using diode lasers, as compared to fiber lasers is that they are relatively cheap, exhibit dynamic power control, and have relatively low power consumption. By combining pairs of laser diodes and stacking them together according to embodiments discussed herein, a non-elliptical beam pattern can be produced in the scanning system 150 (e.g., such as a square or circular shaped beam depending on shape of the fiber), the beam pattern is eye-safe, the repetition rate can be increased (e.g., a multiple of the repetition rate of one laser diode), and peak power can be increased.

[0030] Control circuitry (not shown) can sequentially fire each diode laser. Assuming each diode laser operates at a fixed repetition rate of X, a sequential firing of MDOFC 200 can result in a repetition rate that is at least four times X (because there are four laser diodes being fired in sequence). Moreover, because multiple lasers are being used together, the combined power output of the integrated beam coupled to the fiber is greater than any one diode laser. Use of diode emitters, as opposed to fiber lasers, enables the control circuitry to exercise dynamic power control over the power output provided MDOFC. That is, the control circuitry can adjust power output of MDOFC to var ing between 0 and maximum power output in an order of a micro-second. Such dynamic power control is not possible with fiber lasers. In some embodiments, the diode lasers being used in MDOFC 200 may emit light pulses at a wavelength of about 800-1560 nm.

[0031] It should be appreciated that in some embodiments, beam splitters are not needed. For example, the control system can contain just one diode laser that sends its beam into a lens that converges down to the appropriate angle before entry into the fiber optic cable. As another example, if two diode lasers are used, they can be arranged adjacent to each other such that their respective beams form an integrated beam that is input into to the lens before entry into the fiber optic cable. Any suitable number of diodes may be arranged together to form a desired integrated beam.

[0032] FIG. 3 shows an illustrative simulation result showing irradiance of an integrated laser light signal being provided by MDOFC 200 to a fiber at the coupling, according to an embodiment. In one embodiment, the irradiance pattern of FIG. 3 may represent third integrated beam 284 that is input to fiber 290. If the simulation ignores loss from the AR coating, mirror reflectivity, PBS loss, and extinction ratio of the diode lasers, a coupling efficiency of at least 98% can be reached. As shown in FIG. 3, the irradiance pattern is substantially square. This square-like shape is the product of the stacking of the combined pairs of diode laser emissions (e.g., stacking of first and second integrated beams 280 and 282). The divergence angle of the integrated beam provided by MDOFC 200 is also designed to comply with the numerical aperture of coupling fiber 290.

[0033] Placement of MDOFC 200 within the interior portion of a vehicle and not in the scanning system simplifies design constraints of the scanning system. The fiber optic cable transports the laser signals from MDOFC to the scanning system. This frees the scanning system from heat dissipation issues associated with the diode lasers. Because the diode lasers of MDOFC 200 are contained in the interior portion, heat dissipation of the diode lasers can be controlled more easily than convention systems that incorporate diode lasers in the scanning system. In addition, the design of MDOFC 200 use multiple, lower power, laser diodes that each consume less power than a single laser diode designed to provide the same power output as the combined power output of DOFC 200. As a result, the laser diodes in MDOFC 200 do not produce as much heat that need to be mitigated, as compared to the single laser diode designed to provide the same power output as the combined power output of MDOFC 200. The design of MDOFC200 also takes advantages of generous space of the control system 130. A small sized scanning system 150 is preferable, however, minimizing the size compromises the lens design and optical system performance. By embedding MDOFC 200 in control system 130 (within the interior portion), the coupling efficiency to the fiber and system performance can be optimized without constraint of space.

[0034] FIG. 4 shows an illustrative block diagram of scanning system 400 according to an embodiment. Scanning system 400 can include fiber optic cable 410, reflective mirror 420, steering optics 430, rotating polygon 440, receiving optics 450, and detector system 460. Fiber optic cable 410 can include one or more optical fibers, each of which serves as a transmission medium for a MDOFC (e.g., such as MDOFC 200). As illustrated in FIG. 4, cable 410 can include four optical fiber tips 41 1-414. Thus, in this exemplary embodiment, four different MDOFCs exist in the control system contained in the interior portion of a vehicle. Optical fiber tips 41 1-414 are secured within scanning system 400 and are aligned such that laser pulses being emitted by each optical fiber tip are aimed to pass through through-hole 422 of reflective mirror 420 towards steering optics 430. Steering optics 430 re-direct light pulses towards rotating polygon 440, which re-directs light energy out of scanning system 400. The light energy being directed by rotating polygon 440 is cast in accordance with the field of view parameters of scanning system 400. That is, if scanning system 400 has a field of view with range of x, a lateral angle of y, and vertical angle of z, the range x can be controlled by the power of the emitter(s), the vertical angle z can be controlled by the steering optics 430, and the lateral angle y can be controlled by polygon 440. Light energy that is scattered back from objects in the field of view returns to polygon 440 and is redirected to detector system 460 via steering optics 430, reflective mirror 420, and receiving optics 450. Detector system 460 can include any number of detectors, and four detectors 461-464 are shown in FIG. 4. Each one of detectors 461-464 may correspond to one of optical fiber tips 41 1-414.

[0035] Not shown in FIG. 4 are lenses/collimators associated with each of fiber tips 41 1- 414 to shape the beam emanating from each fiber tip towards steering optics 430. The lenses/collimators collimate a relatively divergent beam emanating from each fiber tip. The beam emanating from each fiber tip may be relatively large (at least compared to a single mode sourced laser) because the output of the fiber tip is derived from a stacked combination of multi-mode laser beams. As a result, the totality of the beams aimed towards steering optics 430 can be larger in size than the steering optics 430. Reflective mirror 420 is positioned between fiber tips 41 1-414 and steering optics 430 and sized to reflect receiving beam from steering optics 430 to receiving optics 450, except for the portion of the beams from fiber tips 41 1-414 passing through through-hole 422. Fiber tips 41 1-414 can be aligned and aimed such that the combination of their respective beams converges at through-hole 422. This enables reduction of the diameter of through-hole 422, as it is desirable to minimize diameter of through-hole 422 to maximize the quantity of light energy reflected off reflective mirror 420 (from steering optics 430) to receiving optics 450. The appropriate balance is struck based on desired observable distance and return light capture to yield the appropriately sized through-hole 422. In one embodiment, the size of through-hole 422 may be approximately the same size as the beam emanating from the collimator associated with each fiber tip. For example, if the beam emanating from a collimator is about 10mm, then through-hole 422 may also be about 10mm.

[0036] FIG. 5 shows an illustrative beam shape at the far field using MDOFC coupled with a fiber optic according to an embodiment. More particularly, FIG. 5 shows fiber optic tip 510 is aligned with collimator 520. Illustrative beam pattern is shown at focus plane 530 and far field plane 540. Fiber optic tip 510 may be part of an optical fiber that is coupled to

MDOFC. The beam profile exiting fiber optic tip 510 may be represented by A, the beam profile at focus plane 530 may be represented by B, and the beam profile at far field plane 540 may be represented by C. The beam profile at the far field plane (on an object) has the same shape as the beam existing at the fiber tip, as illustrated. In terms of relationships, assume B is proportional to the Fourier transform of A and C is proportional to Fourier transform of B, as a result, C~A.

[0037] The MDOFC coupled to fiber tip 510 may operate as a multi-mode emitter.

Operating in as multi-mode emitter provides a relatively square-shaped beam (as shown in FIG. 3) at the MDOFC/fiber coupling. The square shaped beam is more desirable compared to the beam resulting from a single-mode emitter or fiber laser, which has a Gaussian shaped beam. Depending on the shape of the optical fiber, the beam exiting the fiber tip can be square (if a square shaped cross-sectional fiber is used) or circular (if a circular shaped cross- sectional fiber is used). If square shaped cross-sectional optic fiber is used, the resulting beam in far field can fill the space without a need for over filling or under filling.

[0038] FIG. 6 is a functional block diagram illustrating a vehicle system 900, according to an example embodiment. Vehicle 900 can be configured to operate fully or partially in an autonomous mode. For example, vehicle 900 can control itself while in the autonomous mode, and may be operable to determine a current state of the vehicle and its environment, determine a predicted behavior of at least one other vehicle in the environment, determine a confidence level that may correspond to a likelihood of the at least one other vehicle to perform the predicted behavior, and control vehicle 900 based on the determined information. While in autonomous mode, the vehicle 900 may be configured to operate without human interaction. [0039] In some embodiments, vehicle 900 can operate under solely control of a human operator, but the various sensors and systems of the vehicle and the road conditions (e.g., road and the path traveled, other vehicles, stop signs, traffic lights, various events occurring outside of the vehicle) can be monitored and recorded

[0040] Vehicle 900 can include various subsystems such as a propulsion system 902, a sensor system 904, a control system 906, one or more peripherals 908, as well as a power supply 910, a computer system 912, and a user interface 916. Vehicle 900 may include more or fewer subsystems and each subsystem can include multiple elements. Further, each of the subsystems and elements of vehicle 900 can be interconnected. Thus, one or more of the described functions of the vehicle 900 may be divided up into additional functional or physical components, or combined into fewer functional or physical components. In some further examples, additional functional and/or physical components may be added to the examples illustrated by FIG. 9.

[0041] Propulsion system 902 may include components operable to provide powered motion for the vehicle 900. Depending upon the embodiment, the propulsion system 902 can include an engine/motor 918, an energy source 919, a transmission 920, and wheels/tires 921. The engine/motor 918 can be any combination of an internal combustion engine, an electric motor, steam engine, Stirling engine, or other types of engines and/or motors. In some embodiments, the engine/motor 918 may be configured to convert energy source 919 into mechanical energy. In some embodiments, the propulsion system 902 can include multiple types of engines and/or motors. For instance, a gas-electric hybrid car can include a gasoline engine and an electric motor. Other examples are possible.

[0042] Energy source 919 can represent a source of energy that may, in full or in part, power the engine/motor 918. That is, the engine/motor 918 can be configured to convert the energy source 919 into mechanical energy. Examples of energy sources 919 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source(s) 919 can additionally or alternatively include any combination of fuel tanks, batteries, capacitors, and/or flywheels. The energy source 919 can also provide energy for other systems of the vehicle 900.

[0043] Transmission 920 can include elements that are operable to transmit mechanical power from the engine/motor 918 to the wheels/tires 921. To this end, the transmission 920 can include a gearbox, clutch, differential, and drive shafts. The transmission 920 can include other elements. The drive shafts can include one or more axles that can be coupled to the one or more wheels/tires 921.

[0044] Wheels/tires 921 of vehicle 900 can be configured in various formats, including a unicycle, bicycle/motorcycle, tricycle, or car/truck four-wheel format. Other wheel/tire geometries are possible, such as those including six or more wheels. Any combination of the wheels/tires 921 of vehicle 900 may be operable to rotate differentially with respect to other wheels/tires 921. The wheels/tires 921 can represent at least one wheel that is fixedly attached to the transmission 920 and at least one tire coupled to a rim of the wheel that can make contact with the driving surface. The wheels/tires 921 can include any combination of metal and rubber, or another combination of materials.

[0045] Sensor system 904 may include a number of sensors configured to sense information about an environment of the vehicle 900. For example, the sensor system 904 can include a Global Positioning System (GPS) 922, an inertial measurement unit (IMU) 924, a RADAR unit 926, a laser rangefinder/LIDAR unit 928, and a camera 930. The sensor system 904 can also include sensors configured to monitor internal systems of the vehicle 900 (e.g., 02 monitor, fuel gauge, engine oil temperature). Other sensors are possible as well.

[0046] One or more of the sensors included in sensor system 904 can be configured to be actuated separately and/or collectively in order to modify a position and/or an orientation of the one or more sensors.

[0047] GPS 922 may be any sensor configured to estimate a geographic location of the vehicle 900. To this end, GPS 922 can include a transceiver operable to provide information regarding the position of the vehicle 900 with respect to the Earth.

[0048] IMU 924 can include any combination of sensors (e.g., accelerometers and gyroscopes) configured to sense position and orientation changes of the vehicle 900 based on inertial acceleration.

[0049] RADAR unit 926 may represent a system that utilizes radio signals to sense objects within the local environment of the vehicle 900. In some embodiments, in addition to sensing the objects, the RADAR imit 926 may additionally be configured to sense the speed and/or heading of the objects. Similarly, laser rangefinder or LIDAR unit 928 may be an ' sensor configured to sense objects in the environment in which the vehicle 900 is located using lasers. Depending upon the embodiment, the laser rangefinder/LIDAR unit 928 can include one or more laser sources, a laser scanner, and one or more detectors, among other system components. The laser rangefinder/LIDAR unit 928 can be configured to operate in a coherent (e.g., using heterodyne detection) or an incoherent detection mode.

[0050] Camera 930 can include one or more devices configured to capture a plurality of images of the environment of vehicle 900. Camera 930 can be a still camera or a video camera.

[0051] Control system 906 may be configured to control operation of vehicle 900 and its components. Accordingly, control system 906 can include various elements include steering unit 932, throttle 934, brake unit 936, a sensor fusion algorithm 938, a computer vision system 940, a navigation/pathing system 942, and an obstacle avoidance system 944.

[0052] Steering unit 932 can represent any combination of mechanisms that may be operable to adjust the heading of vehicle 900. Throttle 934 can be configured to control, for instance, the operating speed of the engine/motor 918 and, in turn, control the speed of the vehicle 900. Brake unit 936 can include any combination of mechanisms configured to decelerate the vehicle 900. Brake unit 936 can use friction to slow wheels/tires 921. In other embodiments, the brake unit 936 can convert the kinetic energy of wheels/tires 921 to electric current. The brake unit 936 may take other forms as well. The brake unit 936 may control braking of the vehicle 900, for example, using a braking algorithm that takes into account input from one or more units of the sensor system 904.

[0053] Sensor fusion algorithm 938 may be an algorithm (or a computer program product storing an algorithm) configured to accept data from the sensor system 904 as an input. The data may include, for example, data representing information sensed at the sensors of the sensor system 904. The sensor fusion algorithm 938 can include, for instance, a Kalman filter, Bayesian network, or other algorithm. The sensor fusion algorithm 938 can further provide various assessments based on the data from sensor system 904. Depending upon the embodiment, the assessments can include evaluations of individual objects and/or features in the environment of vehicle 900, evaluation of a particular situation, and/or evaluate possible impacts based on the particular situation. Other assessments are possible.

[0054] Computer vision system 940 may be any system operable to process and analyze images captured by camera 930 in order to identify objects and/or features in the environment of vehicle 900 that can include traffic signals, road way boundaries, and obstacles. Computer vision system 940 can use an object recognition algorithm, a Structure From Motion (SFM) algorithm, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 940 can be additionally configured to map an environment, track objects, estimate the speed of objects, etc.

[0055] Navigation and pathing system 942 may be any system configured to determine a driving path for the vehicle 900, for example, by referencing navigation data such as geographical or map data. The navigation and pathing system 942 may additionally be configured to update the driving path dynamically while the vehicle 900 is in operation. In some embodiments, the navigation and pathing system 942 can be configured to incorporate data from the sensor fusion algorithm 938, the GPS 922, and one or more predetermined maps so as to determine the driving path for vehicle 900. Obstacle avoidance system 944 can represent a control system configured to identify, evaluate, and avoid or otherwise negotiate potential obstacles in the environment of the vehicle 900. Control system 906 may additionally or alternatively include components other than those shown and described.

[0056] Peripherals 908 may be configured to allow interaction between the vehicle 900 and external sensors, other vehicles, other computer systems, and/or a user. For example, peripherals 908 can include a wireless communication system 946, a touchscreen 948, a microphone 950, and''or a speaker 952. In an example embodiment, peripherals 908 can provide, for instance, means for a user of the vehicle 900 to interact with the user interface 916. To this end, touchscreen 948 can provide information to a user of vehicle 900. User interface 916 can also be operable to accept input from the user via the touchscreen 948. The touchscreen 948 may be configured to sense at least one of a position and a movement of a user's finger via capacitive sensing, resistance sensing, or a surface acoustic w ^r ave process, among other possibilities. Touchscreen 948 may be capable of sensing finger movement in a direction parallel or planar to the touchscreen surface, in a direction normal to the touchscreen surface, or both, and may also be capable of sensing a level of pressure applied to the touchscreen surface. Touchscreen 948 may be formed of one or more translucent or transparent insulating layers and one or more translucent or transparent conducting layers. Touchscreen 948 may take other forms as well.

[0057] In other instances, peripherals 908 may provide means for the vehicle 900 to communicate with devices within its environment. Microphone 950 may be configured to receive audio (e.g., a voice command or other audio input) from a user of vehicle 900.

Similarly, speakers 952 may be configured to output audio to the user of vehicle 900. [0058] In one example, wireless communication system 946 can be configured to wirelessly communicate with one or more devices directly or via a communication network. For example, wireless communication system 946 can use 3G cellular communication, such as CDMA, EVDO, GSM/GPRS, or 4G cellular communication, such as WiMAX or LTE. Alternatively, wireless communication system 946 can communicate with a wireless local area network (WLAN), for example, using WiFi. In some embodiments, wireless communication system 946 can communicate directly with a device, for example, using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicular communication systems, are possible within the context of the disclosure. For example, the wireless communication system 946 can include one or more dedicated short range communications (DSRC) devices that can include public and/or private data communications between vehicles and/or roadside stations.

[0059] Power supply 910 may provide power to various components of vehicle 900 and can represent, for example, a rechargeable lithium-ion or lead-acid battery. In some

embodiments, one or more banks of such batteries can be configured to provide electrical power. Other power supply materials and configurations are possible. In some embodiments, the power supply 910 and energy source 919 can be implemented together, as in some all electric cars.

[0060] Many or all of the functions of vehicle 900 can be controlled by computer system 912. Computer system 912 may include at least one processor 913 (which can include at least one microprocessor) that executes instructions 915 stored in a non-transitory computer readable medium, such as the data storage 914. Computer system 912 may also represent a plurality of computing devices that may serve to control individual components or subsystems of the vehicle 900 in a distributed fashion.

[0061] In some embodiments, data storage 914 may contain instructions 915 (e.g., program logic) executable by processor 913 to execute various functions of vehicle 900, including those described above in connection with FIG. 9. In some embodiments, processor 913 may be operative to run an artificial intelligence (AI) engine, for example, to control the various systems of the vehicle 900. Data storage 914 may contain additional instructions as well, including instructions to transmit data to, receive data from, interact with, and/or control one or more of propulsion system 902, sensor system 904, control system 906, and peripherals 908. In addition to instructions 915, data storage 914 may store data such as roadway maps, path information, among other information Such information may be used by vehicle 900 and computer system 912 at during the operation of vehicle 900 in the autonomous, semi- autonomous, and/or manual modes.

[0062] Vehicle 900 may include a user interface 916 for providing information to or receiving input from a user of vehicle 900. User interface 916 can control or enable control of content and/or the layout of interactive images that can be displayed on the touchscreen 948. Further, user interface 916 can include one or more input/output devices within the set of peripherals 908, such as wireless communication system 946, touchscreen 948, microphone 950, and the speaker 952.

[0063] Port 960 may be a port through which vehicle 900 receives power to charge power supply 910 and to communicate data stored in data store 914.

[0064] Computer system 912 may control the function of vehicle 900 based on inputs received from various subsystems (e.g., propulsion system 902, sensor system 104, and control system 906), as well as from user interface 916. For example, computer system 912 may utilize input from control system 906 in order to control steering unit 932 to avoid an obstacle detected by sensor system 904 and obstacle avoidance system 944. Depending upon the embodiment, computer system 912 can be operable to provide control over many aspects of vehicle 900 and its subsystems.

[0065] The components of vehicle 900 can be configured to work in an interconnected fashion with other components within or outside their respective systems. For instance, in an example embodiment, camera 930 can capture a plurality of images that can represent information about a state of an environment of vehicle 900 operating in an autonomous or manual mode. The environment can include every' conceivable type of data that can be observed and collected by vehicle 900. For example, the environment can include the road and all aspects associated with the road such as temperature, composition of the road (e.g., concrete or asphalt), moisture level, lanes, curbs, turn lanes, cross walks, stop lights, stop signs, yield signs and other traffic signs, and barricades. The environment can include objects such as other vehicles, people, random debris in or adjacent to the road.

[0066] Computer system 912 can monitor and log the environmental inputs in conjunction with operational states of the vehicle. The operational states can refer to operational and control parameters of the vehicle such as speed, trajectory, steering input, acceleration input, and brake input, and also can include results of driver input or AI driver input. This way, regardless of whether the vehicle is operating in autonomous mode or under human control, computer system 912 can simultaneously log the environmental inputs and the operational states to provide a comprehensive vehicle log.

[0067] Although FIG. 6 shows various components of vehicle 900, i.e., wireless communication system 946, computer system 912, data storage 914, and user interface 916, as being integrated into vehicle 900, one or more of these components can be mounted or associated separately from the vehicle 900. For example, data storage 914 can, in part or in full, exist separate from vehicle 900. Thus, vehicle 900 can be provided in the form of device elements that may be located separately or together. The device elements that make up vehicle 900 can be communicatively coupled together in a wired and/or wireless fashion.

[0068] It is believed that the disclosure set forth herein encompasses multiple distinct inventions with independent utility. While each of these inventions has been disclosed in its preferred form, the specific embodiments thereof as disclosed and illustrated herein are not to be considered in a limiting sense as numerous variations are possible. Each example defines an embodiment disclosed in the foregoing disclosure, but any one example does not necessarily encompass all features or combinations that may be eventually claimed. Where the description recites“a” or“a first” element or the equivalent thereof, such description includes one or more such elements, neither requiring nor excluding two or more such elements. Further, ordinal indicators, such as first, second or third, for identified elements are used to distinguish between the elements, and do not indicate a required or limited number of such elements, and do not indicate a particular position or order of such elements unless otherwise specifically stated.

[0069] Moreover, any processes described with respect to FIGS. 1-6, as well as any other aspects of the invention, may each be implemented by software, but may also be

implemented in hardware, firmware, or any combination of software, hardware, and firmware. They each may also be embodied as machine- or computer-readable code recorded on a machine- or computer-readable medium. The computer-readable medium may be any data storage device that can store data or instructions which can thereafter be read by a computer system. Examples of the computer-readable medium may include, but are not limited to, read-only memory, random-access memory, flash memory, CD-ROMs, DVDs, magnetic tape, and optical data storage devices. The computer-readable medium can also be distributed over network-coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. For example, the computer-readable medium may be communicated from one electronic subsystem or device to another electronic subsystem or device using any suitable communications protocol. The computer-readable medium may embody computer-readable code, instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A modulated data signal may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0070] It is to be understood that any or each module or state machine discussed herein may be provided as a software construct, firmware construct, one or more hardware components, or a combination thereof. For example, any one or more of the state machines or modules may be described in the general context of computer-executable instructions, such as program modules, that may be executed by one or more computers or other devices. Generally, a program module may include one or more routines, programs, objects, components, and/or data structures that may perform one or more particular tasks or that may implement one or more particular abstract data types. It is also to be understood that the number, configuration, functionality, and interconnection of the modules or state machines are merely illustrative, and that the number, configuration, functionality, and interconnection of existing modules may be modified or omitted, additional modules may be added, and the interconnection of certain modules may be altered.

[0071] Whereas many alterations and modifications of the present invention will no doubt become apparent to a person of ordinary skill in the art after having read the foregoing description, it is to be understood that the particular embodiments shown and described by way ^? of illustration are in no way intended to be considered limiting. Therefore, reference to the details of the preferred embodiments is not intended to limit their scope.