Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
SYSTEM AND METHOD FOR MAP GENERATING AND VEHICLE PARKING
Document Type and Number:
WIPO Patent Application WO/2021/141833
Kind Code:
A1
Abstract:
A method of generating a map (300) of an area surrounding a vehicle (202) includes sending, by data processing hardware (242) of a vehicle (202), a request for an aerial image associated with the vehicle (202); and receiving the aerial image. The method includes identifying a representation of one or more available parking spaces in the aerial image, and generating a map (300) based upon the aerial image and the identified available parking space representations. The map (300), including the identified available parking space representations, is displayed on a user interface in communication with the vehicle's data processing hardware (242). The method includes receiving a selection of one of the identified available parking space representations, determining a drivable path (306) from the vehicle (202) to a parking space corresponding to the selected parking space representation, and displaying a representation of the drivable path (306) with the map (300).

Inventors:
IP JULIEN (US)
RAMIREZ LLANOS EDUARDO JOSE (US)
YU XIN (US)
Application Number:
PCT/US2020/070928
Publication Date:
July 15, 2021
Filing Date:
December 18, 2020
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
CONTINENTAL AUTOMOTIVE SYSTEMS INC (US)
International Classes:
B62D15/02; G01C21/32; G08G5/04
Domestic Patent References:
WO2019040714A12019-02-28
Foreign References:
DE102018008763A12019-05-02
US20140358424A12014-12-04
Attorney, Agent or Firm:
ESSER, William F et al. (US)
Download PDF:
Claims:
WHAT IS CLAIMED IS:

1. A method of assisting a placement of a vehicle (202) in a parking location, the method comprising: sending, by data processing hardware (242) of a vehicle (202), a request for an aerial image associated with the vehicle (202); receiving the aerial image; identifying a representation of one or more available parking spaces in the aerial image; generating a two dimensional map (300) based upon the aerial image and the identified one or more available parking space representations; and displaying, on a user interface in communication with the data processing hardware (242) of the vehicle (202), the two-dimensional map (300) including the one or more identified available parking space representations.

2. The method of claim 1, wherein the request for the aerial image identifies a geographical area that is not centered at the vehicle (202), and the received aerial image is based upon the geographical area identified. 3. The method of claim 1, wherein identifying the one or more available parking space representations comprises identifying parking space line representations in the received aerial image.

4. The method of claim 1, further comprising selecting or receiving a selection of one of the one or more available parking space representations, determining a drivable path (306) to the parking space corresponding to the selected one of the one or more available parking space representations, and displaying in or with the two dimensional map (300) a representation of the drivable path (306).

5. The method of claim 1, further comprising receiving or determining a scale corresponding to the aerial image, wherein generating and displaying the two dimensional map (300) is based upon the scale corresponding to the aerial image.

6. The method of claim 1, further comprising determining a vehicle (202) pose for the vehicle (202), and displaying with the two-dimensional map (300) a representation of the vehicle (202) pose.

7. The method of claim 1, further comprising identifying in the aerial image one or more representations of static objects in the aerial image, wherein the two dimensional map (300) includes representations of the identified static objects.

8. The method of claim 1, wherein the receiving, the identifying and the generating are performed by the data processing hardware (242) of the vehicle (202).

9. The method of claim 1, wherein the receiving, the identifying and the generating are performed by a server (290) that is remote from the vehicle (202).

10. The method of claim 1, further comprising identifying a plurality of parking space representations based upon the aerial image, and for each parking space representation identified, determining whether the parking space representation is one of the one or more identified available parking space representations or an unavailable parking space representation.

11. A system for assisting in placing a vehicle (202) in a parking location, the system comprising: data processing hardware (242) associated with a vehicle (202); and memory hardware (244) in communication with the data processing hardware (242), the memory hardware (244) storing instructions which when executed on the data processing hardware (242) cause the data processing hardware (242) to perform operations comprising: sending, by the data processing hardware (242), a request for an aerial image associated with the vehicle (202); receiving the aerial image; identifying a representation of one or more available parking spaces in the aerial image; generating a map (300) based upon the aerial image and the identified one or more available parking spaces; and displaying, on a user interface in communication with the data processing hardware (242) associated with the vehicle (202), the map (300) including the representation of the one or more identified available parking spaces.

12. The system of claim 11, wherein the request for the aerial image identifies a geographical area, and the received aerial image is based upon the geographical area identified, wherein the geographical area is not centered at the vehicle (202).

13. The system of claim 11, wherein the instructions for identifying the one or more available parking space representations comprises instructions for identifying parking space line representations in the received aerial image, wherein identifying the one or more available parking space representations is based upon the identified parking space line representations.

14. The system of claim 11, further comprising instructions for selecting or receiving a selection of one of the one or more available parking space representations, determining a drivable path (306) to the parking space corresponding to the selected one of the one or more available parking space representations, and depicting with the display of the map (300) a representation of the drivable path (306).

15. The system of claim 11, further comprising instructions for receiving or determining a scale corresponding to the aerial image, wherein generating and displaying the map (300) is based upon the scale corresponding to the aerial image.

16. The system of claim 11, further comprising instructions for determining a vehicle (202) pose for the vehicle (202) and displaying with the map (300) the vehicle (202) pose.

17. The system of claim 11, further comprising instructions for identifying in the aerial image one or more representations of static objects in the aerial image, wherein the map (300) includes representations of static objects corresponding to the one or more identified static obj ect representations.

18. The system of claim 11, wherein the instructions for receiving, the identifying and the generating are performed by the data processing hardware (242) associated with the vehicle (202).

19. The system of claim 11, wherein the instructions for the receiving, the identifying and the generating are performed by a server (290) that is remotely located relative to the vehicle (202). 20. The system of claim 11, wherein the instructions further comprise instructions for identifying a plurality of parking space representations based upon the aerial image, and for each parking space representation identified, determining whether the parking space representation is one of the one or more identified available parking space representations or an unavailable parking space representation.

Description:
System and Method for Map Generating and Vehicle Parking

TECHNICAL FIELD

[0001] This disclosure relates to a method and system for generating a two dimensional map around a vehicle in a parking area.

BACKGROUND

[0002] Trailers are usually unpowered vehicles that are pulled by a powered tow vehicle. A trailer may be a utility trailer, a popup camper, a travel trailer, livestock trailer, flatbed trailer, enclosed car hauler, and boat trailer, semi-trailer, among others. The tow vehicle may be a car, a crossover, a truck, a semi-truck, a van, a sports-utility -vehicle (SUV), a recreational vehicle (RV), or any other vehicle configured to attach to the trailer and pull the trailer. The trailer may be attached to a powered vehicle using a trailer hitch. The trailer allows the trailer to swivel around the hitch horizontally so that the vehicle- trailer unit is able to move around corners. A receiver hitch mounts on the tow vehicle and connects to the trailer hitch to form a connection.

[0003] In some examples, it is desirable to have a system supported by a vehicle that generates a two-dimensional map of a parking area near the vehicle without having to rely completely on local sensors (e.g., cameras, lidars, radars, etc.) attached to the vehicle for the map generation.

SUMMARY

[0004] Example embodiments and implementations are disclosed of a method and system for assisting in placing a vehicle in a parking space. The method includes sending, by data processing hardware of a vehicle, a request for an aerial image associated with the vehicle, and receiving the aerial image. The method includes identifying a representation of one or more available parking spaces in the aerial image, and generating a two dimensional map based upon the aerial image and the identified one or more available parking space representations. The two-dimensional map is displayed on a user interface in communication with the data processing hardware of the vehicle. The two-dimensional map includes the one or more identified available parking space representations. [0005] The request for the aerial image may identify a geographical area that is not centered at the vehicle, and the received aerial image is based upon the geographical area identified.

[0006] Identifying the one or more available parking space representations includes identifying parking space line representations in the received aerial image.

[0007] The method further includes selecting or receiving a selection of one of the one or more available parking space representations, and determining a drivable path to the parking space corresponding to the selected one of the one or more available parking space representations. A representation of the drivable path is display in or with the two dimensional map.

[0008] The method may include receiving or determining a scale corresponding to the aerial image, wherein generating and displaying the two dimensional map is based upon the scale corresponding to the aerial image.

[0009] The method may also include determining a vehicle pose for the vehicle, and displaying with the two-dimensional map a representation of the vehicle pose.

[0010] Identifying in the aerial image one or more representations of static objects in the aerial image may also be included in the method, wherein the two dimensional map includes representations of the identified static objects.

[0011] In one implementation, receiving aerial image, identifying the representation of the one or more available parking spaces, and generating the two-dimensional map are performed by the data processing hardware of the vehicle. In another implementation, the receiving, identifying and generating are performed by a server that is remote from the vehicle.

[0012] The method may further include identifying a plurality of parking space representations based upon the aerial image, and for each parking space representation identified, determining whether the parking space representation is one of the one or more identified available parking space representations or an unavailable parking space representation.

[0013] The system for assisting in placing a vehicle in a parking location, the system includes data processing hardware associated with a vehicle; and memory hardware in communication with the data processing hardware. The memory hardware stores instructions which when executed on the data processing hardware cause the data processing hardware to perform and/or cause the performance of the method described above. DESCRIPTION OF DRAWINGS

[0014] FIG. l is a schematic view of an example parking system supported by a vehicle.

[0015] FIG. 2 is a schematic view of an example vehicle-trailer system including the parking system of FIG. 1. [0016] FIG. 3 is an image displayed of a map generated by the parking system of

FIG. 1 according to an example embodiment.

[0017] FIG. 4 is a displayed image of the generated map of FIG. 3 with a drivable path determined by the parking system of FIG. 1.

[0018] FIG. 5 is a flowchart illustrating an operation of the parking system of FIG. 1. [0019] FIG. 6 is another map generated and displayed by the parking system of FIG.

1

[0020] Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

[0021] In general terms, example embodiments are directed to a parking system which facilitates a vehicle, such as a tow vehicle towing a trailer having autonomous driving capability, parking in a parking space through use of an aerial and/or satellite image of an area associated with the tow vehicle. The parking system identifies one or more parking space representations in the aerial view, and determines whether the one or more parking space representations is/are a representation(s) of one or more vacant and/or unoccupied parking spaces available in which to park the tow vehicle and trailer (hereinafter “available parking space”). A map generator algorithm of the parking system generates a map based on the aerial view and the one or more identified available parking space representations. The generated map includes the one or more available parking space representations. The generated map is displayed on a user interface display. The parking system receives from the driver of the tow vehicle a selection of one representation of the one or more identified available parking space representations.

A path planning algorithm of the parking system determines a drivable path from a current location of the tow vehicle and trailer to a parking space corresponding to the selected one of the identified available parking space representations. The generated map and a representation of the drivable path are displayed to the tow vehicle driver. The parking system provides to an autonomous driving system of the tow vehicle information pertaining to the selected one of the one or more identified available parking space representations, with which the autonomous driving system drives the tow vehicle and trailer to the parking space.

[0022] Referring to FIGS. 1 and 2, and in some implementations, a vehicle-trailer system 200 includes a tow vehicle 202 attached to a trailer 204. The tow vehicle 202 includes a vehicle tow ball 206 supported by a vehicle hitch bar. The vehicle tow ball 206 is coupled to the trailer 204 by way of a trailer hitch coupler supported by a trailer hitch bar of the trailer 204. The tow vehicle 202 may include a drive system 210 that maneuvers the tow vehicle 202 across a road surface based on drive commands having x, y, and z components, for example. As shown, the drive system 120 includes a front right wheel 212, 212a, a front left wheel 212, 212b, a rear right wheel 212, 212c, and a rear left wheel 212, 212d. The drive system 210 may include other wheel configurations as well. The drive system 210 may also include a brake system (not shown) that includes brakes associated with each wheel 212, 212a-d, and an acceleration system (not shown) that is configured to adjust a speed and direction of the tow vehicle 202. In addition, the drive system 210 may include a suspension system (not shown) that includes tires associates with each wheel 212, 212a-d, tire air, springs, shock absorbers, and linkages that connect the tow vehicle 202 to its wheels 212, 212a-d and allows relative motion between the tow vehicle 202 and the wheels 212, 212a-d.

[0023] The tow vehicle 202 may move across the road surface by various combinations of movements relative to three mutually perpendicular axes defined by the tow vehicle 202: a transverse axis Xv, a fore-aft axis Yv, and a central vertical axis Zv. The transverse axis Xv extends between a right-side and a left-side of the tow vehicle 202. A forward drive direction along the fore-aft axis Yv is designated as Fv, also referred to as a forward motion. In addition, an aft or rearward drive direction along the fore-aft direction Yv is designated as Rv, also referred to as rearward motion. In some examples, the tow vehicle 202 includes a suspension system (not shown), which when adjusted causes the tow vehicle 202 to tilt about the Xv axis and or the Yv axis, or move along the central vertical axis Zv.

[0024] In some implementations, the vehicle 202 includes a sensor system 220 and a driver assist system 247 to provide reliable and robust autonomous driving. The sensor system 220 provides sensor data 222 and may include different types of sensors that may be used separately or with one another to create a perception of the tow vehicle’s environment that is used by the driver assist system 247 of the vehicle-trailer system 200 to autonomously drive and make intelligent decisions based on objects and obstacles detected by the sensor system 220. In some examples, the vehicle 202 supports the sensor system 220; while in other examples, the sensor system 220 is supported by the vehicle 202 and the trailer 204. The sensors may include, but not limited to, one or more imaging devices (such as cameras), and sensors such as, but not limited to, radar, sonar, LIDAR (Light Detection and Ranging, which can entail optical remote sensing that measures properties of scattered light to find range and/or other information of a distant target), LADAR (Laser Detection and Ranging), etc. In addition, the camera(s) and the sensor(s) may be used to alert the driver of possible obstacles when the tow vehicle 202 is traveling in the forward direction Fv or in the rearward direction Rv, by way of audible alerts and/or visual alerts via the user interface 230. Therefore, the sensor system 220 is especially useful for increasing safety by the driver assist system 247 in the vehicle- trailer system 200 which operates under semi-autonomous or autonomous conditions. [0025] The driver assist system 247 includes, among other things, software including program code instructions stored in hardware memory 244 which, when executed by the data processing hardware 242 of the vehicle controller 240, causes the tow vehicle 202 to be autonomously or semi-autonomously controlled during driving operations. In autonomously or semi-autonomously controlling the tow vehicle 202, the driver assist system 247 uses sensor data provided by sensor system 220 and GPS location information.

[0026] Additionally, the sensor system 220 may include a trailer camera 220 that is mounted to the rearward facing side of the trailer 204 to provide a view of a rear-driving path for the trailer 204, or in other words, the rear trailer camera 220 captures images of a rear environment of the trailer 204. It is understood that the trailer 204 may include additional cameras (not shown) mounted to the trailer for providing additional views surrounding the trailer.

[0027] The tow vehicle 202 may include a user interface 230, such as, a display. The user interface 230 receives one or more user commands from the driver via one or more input mechanisms or a touch screen display 232 and/or displays one or more notifications to the driver. In some examples, the user interface 230 is or includes a touch screen display 232; as such, the driver may point his finger and select a representation 236 of an available parking space 1-32. In other examples, the user interface 230 is not a touchscreen and the driver may use an input device, such as, but not limited to, a rotary knob or a mouse to select one of the available parking space representations 236.

[0028] The user interface 230 is in communication with a vehicle controller 240. The vehicle controller 240 includes a computing device (or data processing hardware) 242 (e.g., central processing unit having one or more computing processors) in communication with non-transitory memory or hardware memory 244 (e.g., a hard disk, flash memory, random-access memory) capable of storing instructions executable on the computing processor(s). As shown, the vehicle controller 240 is supported by the tow vehicle 202; however, the vehicle controller 240 may be separate from the tow vehicle 202 and in communication with the tow vehicle 202 via a network (not shown). In addition, the vehicle controller 240 receives images and data 302 from one or more satellites or imaging devices. In some examples, the user interface 230 displays an image of an environment of the vehicle 202 or the vehicle-trailer system 200 based on data 222 received from the sensor system 220 and from images and data 302 received from a remote server. The image may be a top view image or an aerial image.

[0029] Referring again to FIG. 1, the vehicle trailer system 200 and specifically the vehicle controller 240 includes a parking system 246 which includes software or program code having instructions stored in non-transitory memory 244 which are executed by the data processing hardware 242.

[0030] The vehicle trailer system 200 may further include or be otherwise associated with a wireless interface 248 having the capability of communicating, i.e., receiving and transmitting, with entities external to the tow vehicle 202 over the air interface. The wireless interface 248 may utilize any number of existing or future wireless communication technologies for communicating over the air interface, such as technologies compatible with I.E.E.E. standards 802.11(a), (b), (g) or (n), WiMAX and/or cellular network technologies (e.g., 3G, 4G, and 5G). Current wireless communication technologies are well known in the art and will not be described herein for reasons of simplicity.

[0031] In some implementations, the vehicle trailer system 200 also includes a global positioning satellite (GPS) receiver 250 which communicates with vehicle controller 240 and particularly with data processing hardware 242 thereof. The vehicle controller 240 and the GPS receiver 250 combine to provide GPS functionality to the vehicle 202. In general terms, GPS functionality includes receiving geolocation and time information from an array of orbiting satellites for use in a wide variety of commercial and personal applications.

[0032] The vehicle trailer system 200 may further include a machine learning system

255. The machine learning system 255 includes a machine learning algorithm having a trained model that is built on training data. The trained model is able to make predictions or decisions without explicit programming code to do so. In one implementation illustrated in FIG. 1, the machine learning system 255 is one or more neural networks

256, for example, a deep neural network. The neural network 256 pertains to computational approaches used in computer science, among other disciplines, and is based on a large collection of neural unites, loosely imitating the way a biological brain solves problems with large clusters of biological neurons connected by axons. The neural network 256 is self-learning and trained, rather than programed, and excels in areas where the solution feature detection is difficult to express in a traditional computer program. In other words, the neural network 256 is a set of algorithms that are designed to recognize patterns. The recognized patterns are numerical, vectors, into which all-real- world data, such as images, text, sound, or time series is translated. The neural network 256 includes multiple layers of nonlinear processing units 258 in communication with non-transitory memory 260. The non-transitory memory 260 stores instructions that when executed on the nonlinear processing units 258 cause the neural network 256 to provide an output. Each nonlinear processing unit 258 is configured to transform an input or signal using parameters that are learned through training. A series of transformations from input to outputs occurs at the multiple layers of the nonlinear processing units 258. The particular use of the neural network 256 will be described in greater detail below. Even though the machine learning system 255 is shown in FIG. 1 and will be described below as implemented by the neural network 256, it is understood that the machine learning system 255 may have other machine learning algorithm -based implementations.

[0033] The parking system 246 is configured to request an aerial or satellite image of a defined geographical area for purposed of finding a parking location for parking the tow vehicle 202 and corresponding trailer 204. In one implementation, the request is initiated by the driver of the tow vehicle 202, but it is understood that the request may be initiated automatically by the driver assist system 247 of the tow vehicle. With respect to the former implementation, the driver may, for example, initiate the request around the time the driver wishes to park the tow vehicle 202 and corresponding trailer 204. With respect to the latter implementation, the autonomously driven tow vehicle 202 may, for example, determine it is approaching a desired destination and wishes to find a parking location for the tow vehicle 202 and trailer 204, and automatically initiate the request as a result. The geographical area may be is scalable and may vary based on a driver’s request from zero meters to a reasonable distance from each side of the tow vehicle 202. In one implementation, the aerial image is centered at the location of the tow vehicle 202 using GPS functionality of the vehicle controller 240. In another implementation, the aerial map is centered at a location remotely from the tow vehicle 202, such as a remote location specified by the tow vehicle driver or determined by the parking system 246 based upon a planned course of travel by the tow vehicle 202. In this implementation, the tow vehicle 202 is in the geographical area corresponding to the aerial image or is operated to be in the geographical area within a period of time. The request is sent over the air interface to a remote server 290 via the GPS receiver 250 or the wireless interface 248. The remote server 290 may be a server which is configured to generate aerial maps in response to receiving requests therefor. [0034] The remote server 290 is configured to generate an aerial image responsive to receipt of a request for same. The aerial image is based upon the geographical area provided in the request. In one implementation, the remote server 290 sends the requested aerial image to the tow vehicle 202.

[0035] The parking system 246 includes a map generator 252 implemented as an algorithm in software having instructions stored the memory 244 which, when executed by the data processing hardware 242, generate a two-dimensional map based upon the received aerial image. Each pixel forming the two-dimensional map may be a binary bit or a number bits providing a map having a higher resolution. In one implementation, the map generator 252 identifies representations of stationary objects in the aerial image, such as walls, trees, street corners, etc. This identification of stationary objects appearing in the aerial image may be performed by the neural network 256 which, having been previously trained with analyzing numerous aerial images, analyzes the received aerial image and identifies portions of the image (pixels or groups of pixels) pertaining to various stationary objects. These identified image portions are used so that the tow vehicle 202 may perform automated parking maneuvers. It is understood that not all stationary objects are used in generating the two dimensional map. However, lines in the aerial image representing parking lines defining parking spaces, such as lines appearing in a parking lot or appearing along a street, are identified by the neural network 256 for use in the two dimensional map. The neural network 256 is further trained to, based upon the recognition/identification of stationary objects, which may include and is not necessarily limited to representations of parking space lines, parked vehicles, and/or edges appearing in the aerial image, identify parking space representations appearing in the aerial image. The identified parking space representations in the aerial image are then used by the neural network 256 to determine whether the parking spaces are occupied and unavailable or are instead vacant and available for parking therein. Further, the neural network 256 may identify a parking lot representation in which the parking space representations are located. The information pertaining to the parking space representations in the aerial image and their corresponding vacancy/occupancy status is used in generating the two dimensional map. Once the two dimensional map is created by the map generator 252, in one implementation the map is displayed on the user display 232.

[0036] In another implementation, the generated map may be augmented with data coming from the sensors 220 of the tow vehicle 202, which could provide real time mapping.

[0037] In generating the two dimensional map, the map generator 252 may select only a portion(s) of the aerial image which includes one or more available parking space representations identified by the map generator as well as the portion of the map corresponding to the area between the tow vehicle 202 and the identified available parking spaces.

[0038] Fig. 3 illustrates a two dimensional map 300 in which a parking lot representation is depicted by a border 302. Within the border representation 302, map 302 includes a plurality of parking space representations 1-32 identified by the map generator 252. As shown, the map 300 distinguishes between available and unavailable parking space representations by including an “X” or other designation in the unavailable parking space representations while having available parking space representations with no designation. It is understood that the map generator 252 may distinguish between available and unavailable parking space representations in other ways with other designations, such as with color coding or the use of a representation of a vehicle in each unavailable parking space representation. In addition, in instances in which more than one available parking space representations 1-32 are included in map 300, such parking space representations may be numbered so that selection, if performed manually, may be provided orally by the driver simply saying the number of the identified available parking space representation. Map 300 may further include representations 304 corresponding to various identified structures which may be in a path to an available parking space representation.

[0039] Instead of identifying parking space line representations in an aerial image using the neural network 256 or other computer learning apparatus, in another implementation the map generator 252 identifies parking space line representations using other traditional image detection software. [0040] The parking system 246 is configured to allow for human selection of one of the available (i.e., unoccupied) parking space representations appearing in the two dimensional map 300. In particular, the user interface 230 is configured to receive an indication of a selection associated with a parking space representation. For example, the driver of the tow vehicle 202 may make the selection. The selection may be made by touching the touch screen display 232 over the parking space representation 1-32, or by speaking the number associated with the parking space representation 1-32 so that it is received by a microphone (not shown) of the user interface 230 and suitably interpreted thereby.

[0041] The parking system 246 further includes a path planner 254 implemented in software having program code instructions stored the memory 244 which, when executed by the data processing hardware 242, determines a drivable path from the tow vehicle 202 to the parking space corresponding to the selected parking space representation in the map 300. Specifically, the path planner 254 receives the two dimensional map generated by the map generator 252 with the selected parking space representation identified therein and determines a path which the tow vehicle 202 may take in order to reach the parking space corresponding to the selected parking space representation in the two dimensional map. The path planner 254 may also utilize the current vehicle pose and the end target pose (i.e., the parking space corresponding to the selected available parking space representation). In instances in which the tow vehicle 202 must travel on streets or roads before reaching the parking lot having the parking space corresponding to the selected available parking space representation, the map generator 252 may also utilize street information in the geographical area, such as GPS based information. With a path determined, the path planner 254 may depict the determined path representation 306 on touch screen display 232 that overlays the map 300. FIG. 4 illustrates an image displayed on with map 300, the drivable path 306 overlaid on the map, and a representation 308 of the tow vehicle 202 and trailer 204 corresponding to its present position relative to the parking lot. As can be seen, the drivable path 306 extends from the representation 308 of the tow vehicle 202 and trailer 204 to the selected parking space representation. In addition or in the alternative, a roadside optical or radio frequency system may be employed to provide location information to the tow vehicle 202 via one or more beacon signals or the like for guiding the tow vehicle 202 to the parking space corresponding to the selected parking space representation.

[0042] Upon selection of the desired available parking space representation in map 300, the parking system 246 provides to the driver assist system 247 of the tow vehicle 202 information corresponding to the selected parking space representation 1-32. The information may pertain to the location of the parking space corresponding to the selected parking space representation 1-32 as well as the drivable path 306. The driver assist system 247 then controls the tow vehicle 202 to travel to the parking space corresponding to the selected parking space representation 1-32.

[0043] In some examples, the path planner 254 of the parking system 246 instructs the user interface 230 to ask the driver if the driver wants to pull into the selected parking space corresponding to the selected parking space representation 1-32 with the front portion of the vehicle-trailer system 200 (i.e., the tow vehicle 202) entering the selected parking space first, or back into the selected parking space with the rear portion of the vehicle-trailer system 200 (i.e., the trailer 204) entering the selected parking space first. Upon the driver selection, the path planner 254 plans the drivable path 306 accordingly. Therefore, the path 306 may include maneuvering the vehicle-trailer system 200 in a forward direction Fv, FT, a rearward direction Rv, RT, or a combination thereof.

Moreover, the driver assist system 247 positions the vehicle-trailer system 200 in an orientation aligned generally parallel with at least one line defining the selected parking space. In other words, the fore-aft axis Yv of the vehicle 202 and the fore-aft axis YT of the trailer 204 are parallel and aligned generally parallel with at least one line defining the selected parking space.

[0044] As the vehicle-trailer system 200 is autonomously maneuvering towards the selected parking space corresponding to the selected parking space representation 1-32 based on the planned path 306, the path planner 254 continuously updates the planned path 306 based on continuously receiving images and sensor data 222 from the vehicle sensor system 220. In some examples, parking system 246 may identify one or more new objects along the planned path 306 and sends the path planner 254 data relating to the position of the one or more objects. In this case, the path planner 254 may recalculate the planned path 306 to avoid the one or more identified objects. In some examples, the path planner 254 determines a probability of collision and if the probability of collision exceeds a predetermined threshold, the path planner 254 adjusts the path 306 and sends to the driver assist system 247. In one example, the drivable path determined by the path planner 254 is not recalculated if the tow vehicle 202 diverges from the drivable path, such as in order to avoid an obstruction, for example, and subsequently re-enters the drivable path. This assumes that there is a sufficient amount of the drivable path that remains following the re-entry.

[0045] FIG. 5 is a flowchart describing the operation of the parking system 246 according to one example. Initially, the vehicle controller 240 of the tow vehicle 202 instructs wireless interface 248 to send a request at 502 to the remote server 290 for an aerial image of an area corresponding to a location of the tow vehicle 202 or to another area specified in the request. The request is sent over the air interface. The remote server 290, responsive to the received request, generates an aerial image corresponding to the received request, and sends the aerial image to the vehicle controller 240 where it is received by the parking system 246 at 504. Following receipt of the aerial image, the map generator 252 of the vehicle controller 240 generates a two dimensional map 300 at 508. This includes identifying various objects in the aerial image, including parking space representations. Identifying the parking space representations in the aerial image may include identifying parking line representations defining the parking space representations. The identifications may be performed by the neural network 256. The two dimensional map 300 includes the parking space representations 1-32 identified along with other identified structures. The two dimensional map is displayed at 508 on the touch screen display 232. In one implementation, the driver or other occupant of the tow vehicle 202 selects a parking space representation 1-32 appearing in the map 300.

The selection may be by touching the touch screen display 232 over one of the available parking space representations or orally announcing the available parking space representation 1-32 by number, for example. With the available parking space representation 1-32 selected, the path planner 254 determines a drivable path from a location corresponding to the tow vehicle 202 to the location corresponding to the parking space corresponding to the selected parking space representation. The path planner 254 then displays the drivable path on the touch screen display 232, such as an overlay, on the display of the map 300. The drivable path along with a location of a representation of the tow vehicle 202 displayed on the touch screen display 232 are updated as the tow vehicle 202 moves closer to the parking space corresponding to the selected parking space representation 1-32. The control of the tow vehicle 202 along the drivable path may be performed by the driver assist system 247.

[0046] In the foregoing description, vehicle controller 240 receives an aerial image from remote server 290. In an alternative implementation, server 290 performs the actions associated with generating the two dimensional map 302, and sends the server generated map 300 to the vehicle trailer system 200 for subsequent operations including displaying the map on the touch screen display 232, receiving a selection of an available parking space representation in the map, and determining a drivable path from the tow vehicle 202 to the parking space corresponding to the selected parking space representation. One advantage of this alternative implementation is that processing resources by the vehicle controller 240 associated with the map generation are freed for other operations.

[0047] It is understood that the aerial image and the corresponding map 300 generated by the map generator 252 may not always include a parking lot. Fig. 6 illustrates another map 300’ generated by the map generator 252. In this example, the aerial image and the corresponding map 300’ are representations of a section of a city. Map 300’ includes representations of city streets S and city blocks B. Along each street representation S, parking space representations P are displayed. The parking space representations P that are available are visually distinguished from parking space representations P that are unavailable by, in this example, including cross hatching over the unavailable parking space representations P while the available parking space representations P are left empty. When map 300’ is displayed on touch screen display 232, the driver will be able to select an available parking space representation P corresponding to the parking space in which to park the tow vehicle 202. It is further understood that the map generated may be of different settings than those discussed above, including along a highway. In this example, the map generator 252 may identify parking location representations in the aerial image in part by identifying the line along the highway separating a lane in the highway from the shoulder thereof, and identifying an opposite end of the shoulder furthest away from the highway.

[0048] One advantageous feature of the parking system 246 in some implementations is that the system does not rely on vehicle sensors such as cameras or radars, so that weather or light conditions do not affect the precision of the map that is generated. In addition, the tow vehicle 202 does not need to be driven around an area in order to build the map.

[0049] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. [0050] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor. [0051] Implementations of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Moreover, subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The terms “data processing apparatus”, “computing device” and “computing processor” encompass all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus. [0052] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multi-tasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0053] A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims.