Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
A DISPLAY UNIT FOR VISUAL GUIDANCE OF VEHICLES
Document Type and Number:
WIPO Patent Application WO/2021/013532
Kind Code:
A1
Abstract:
The method and system for guiding a vehicle in closed space comprises receiving/capturing one or more images of the closed space from said one or more image capture devices mounted on the vehicle. The images of one or more display units is detected and extracted from the captured images. Each of the extracted image is transformed to correct the perspective. Pattern information is extracted from each of the transformed images. The pattern information decoded and decrypted to obtain vehicle positioning and guidance information. Finally, the positioning and guidance information is transmitted to a control unit of the vehicle for guiding the vehicle within said closed space.

Inventors:
TRIPATHI NIVEDITA (IE)
Application Number:
PCT/EP2020/069088
Publication Date:
January 28, 2021
Filing Date:
July 07, 2020
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
CONNAUGHT ELECTRONICS LTD (IE)
International Classes:
G01C21/36; B62D15/02; G08G1/16
Foreign References:
EP3267153A12018-01-10
US20090172527A12009-07-02
Attorney, Agent or Firm:
JAUREGUI URBAHN, Kristian (DE)
Download PDF:
Claims:
Claims

1. A method for guiding a vehicle in a closed space using a graphical pattern displayed on a display unit, said vehicle comprising a processor and one or more image capture devices, said method comprising:

receiving, by a processor, one or more images of the closed space from said one or more image capture devices;

detecting and extracting, by the processor, images of one or more of said display units from said received one or more images;

transforming, by the processor, each of the extracted image of display units to correct the perspective;

extracting, by the processor, pattern information from each of the transformed images;

decoding and decrypting, by the processor, pattern information from each of the transformed images to obtain vehicle positioning and guidance information; and

transmitting, by the processor, the positioning and guidance information to a control unit of the vehicle for guiding the vehicle within said closed space.

2. The method of claim 1 , wherein the detection of one or more display units comprises detection of a unique pattern displayed on the display units.

3. The method of claim 1 or 2, wherein the display units are active or passive display units.

4. The method of claim 2, wherein passive display units comprise a pattern information attached or painted on or s surface or on any of the interior surfaces of the closed space and wherein active display units comprise flat panel screens controlled by a display processor to display said pattern information.

5. The method of any preceding claim wherein the pattern information further comprises one or more of geometric coordinates of the active display unit with respect to the closed structure, parking space availability, maneuvering directions from the location of the active display unit to a location of suitable parking space within the closed structure, fee for the parking and speed limit in the area.

6. The method of any preceding claim, wherein the display processor is configured to display pattern information based on a detection of a vehicle entering the closed space.

7. The method of any preceding claim, wherein the pattern information is overlaid on an image displayed by the display units. 8. The method of claim 7, wherein the image is one of an advertisement or a human readable sign or information.

9. A system for guiding a vehicle in a closed space using a graphical pattern displayed on a display unit, comprising

a plurality of display units operatively coupled to a display processor for displaying at least a pattern information on said display units; and

a vehicle unit comprising a processor and one or more image capture devices and a transceiver, said processor configured for:

receiving one or more images of the closed space from said one or more image capture devices;

detecting and extracting images of one or more display units from said received one or more images;

transforming each of the extracted image of the display units to correct the perspective;

extracting pattern information from each of the transformed images; decoding and decrypting at least the pattern information from each of the transformed images to obtain vehicle positioning and guidance information; and transmitting using the transceiver, the positioning and guidance information to a control unit of the vehicle for guiding the vehicle within said closed space. 10. The system of claim 9, wherein the detection of one or more display units comprises detection of a unique pattern displayed on the display units.

11. The system of claim 9 or 10, wherein the display units comprise flat panel screens.

12. The system of claim 9, 10 or 1 1 , wherein the pattern information further comprises one or more of geometric coordinates of the display unit with respect to the closed structure, parking space availability, maneuvering directions from the location of the display unit to a location of suitable parking space within the closed structure, fee for the parking and speed limit in the area.

13. The system of any of claims 9 to 12, wherein the display processor is configured to display pattern information based on a detection of a vehicle entering the closed space.

14. The system of any of claims 9 to 13, wherein the pattern information is overlaid on an image displayed by the display unit. 15. The system of claim 14, wherein the image is one of an advertisement or a human readable sign or information.

16. A display unit for displaying graphically encoded information, comprising: an image displayed on the display unit;

a pattern overlaid on the image displayed by the display unit, wherein said pattern comprises of graphically encoded information, wherein said graphically encoded information comprises of one or more of geometric coordinates of the display unit, parking space availability, maneuvering directions from the location of the display unit to a location having suitable parking space, fee for the parking and/or speed limit in the area.

17. The display unit of claim 16, wherein the display unit is a flat panel display, wherein a display processor of the display unit causes the display to display said image and overlays said pattern on said displayed image.

18. The display unit of any of claims 16 or 17, wherein the image displayed is one of: an advertisement or an image providing information.

19. The display unit of any of claim 16 to 18, wherein the graphically encoded information is a quick response code (QR Code).

Description:
Title

A Display Unit for visual guidance of vehicles.

Field

The present disclosure relates to a display unit for visual guidance of a vehicle in a closed space. More particularly, the present disclosure provides the required information to an autonomous vehicle to park a vehicle within a building or an infrastructure using visual guidance.

Background

Autonomous valet parking within an infrastructure (such as a multistorey car parking is reliant on accurate localization/ positioning of the vehicle within the infrastructure. This positioning can happen in two main ways: infrastructure sensing or vehicle sensing. In the infrastructure sensing approach sensors mounted in the environment are used to determine the position of all vehicles and objects within the infrastructure/building. The issue with this approach is the cumbersome purchasing, mounting, calibrating, and maintaining sensors in order to ensure full coverage of the entire infrastructure. This also increases the cost of installing and maintaining infrastructure parking environment.

A solution to the above problem is to allow the vehicle to sense its own position within the infrastructure/building. However, positioning techniques within buildings is non-trivial using on board vehicle sensing technologies such as using global positioning systems (GPS) etc. due to lack of GPS signals inside a building. Also, positioning techniques using only Radar and Lidars are susceptible to inaccuracies and errors as they are based purely on matching a structure of the environment. As a car park can undergo large structural changes very rapidly with cars entering and leaving parking slots these are not appropriate as sole sensors for positioning. Also, it is required that such autonomous vehicles are provided with information critical to the guidance of the vehicle in open spaces. For example, providing the speed limit in the area or the location of a suitable parking lot in the area.

Cameras have a massive advantage over Radars and Lidars for positioning as they can extract semantic scene information (i.e know where cars, walls, road is situated) while also being ideally suited to template matching visual landmarks. It is desirable for autonomous vehicles to maneuver the vehicle in an open space, or to an empty parking space in a parking lot using visual cues detected by the camera. However, due to lack of good lightning conditions inside the parking space, lack of distinctive rich texture surfaces for recognition of patterns and narrow pathways and complex maneuvers, it is difficult for vision systems to extract semantic information from the images captured by a camera installed in the vehicle.

Therefore, there is an unresolved need for a visual guidance system that provides information related to parking spaces and indoor navigation map and other essential information for the maneuvering an autonomous vehicle within a closed parking space or in an open space.

Summary

Embodiments of the present invention are directed to a method and system for visual guidance of a vehicle using a display unit in a closed space, as set out in the appended claims.

The method for guiding a vehicle in closed space comprises receiving/capturing one or more images of the closed space from said one or more image capture devices mounted on the vehicle. The images of one or more display units is detected and extracted from the captured images where the detection of one or more display units comprises detection of a unique pattern displayed on the display units. The invention can be incorporated as a digital or passive sign/advertising billboard as artificial landmarks, which will have the dual purpose of advertising as well as acting as visual landmarks as the surface contains encrypted navigation information for autonomous vehicles and the like. In addition the encrypted billboard based artificial landmarks, if used in an open area, can serve to show other helpful information to autonomous vehicles like speed limit, road signs, traffic alerts, nearby fuel station, or weather in the area.

Each of the extracted image is transformed to correct the perspective. Pattern information is extracted from each of the transformed images. The pattern information is decoded and decrypted to obtain vehicle positioning and guidance information. The pattern information further comprises one or more of geometric coordinates of the active display unit with respect to the closed structure, parking space availability, manoeuvring directions from the location of the active display unit to a location of suitable parking space within the closed structure, fee for the parking and speed limit in the area.

Finally, the positioning and guidance information is transmitted to a control unit of the vehicle for guiding the vehicle within said closed space. The system for guiding a vehicle in closed space comprises a plurality of display units and a vehicle unit. The plurality of display units are operatively coupled to a display processor for displaying at least a pattern information on said display units. The vehicle unit comprises a processor and one or more image capture devices and a transceiver. The processor configured for receiving one or more images of the closed space from said one or more image capture devices and detecting and extracting images of one or more display units from said received one or more images, where the detection of one or more display units comprises detection of a unique pattern displayed on the display units. The processor is further configured for transforming each of the extracted image of the display units to correct the perspective; extracting pattern information from each of the transformed images; decoding and decrypting at least the pattern information from each of the transformed images to obtain vehicle positioning and guidance information, where the pattern information further comprises one or more of geometric coordinates of the display unit with respect to the closed structure, parking space availability, manoeuvring directions from the location of the display unit to a location of suitable parking space within the closed structure, fee for the parking and speed limit in the area.

The processor is further configured for transmitting, using the transceiver, the positioning and guidance information to a control unit of the vehicle for guiding the vehicle within said closed space.

Thereby, the method and system of the present invention provides visual guidance of a vehicle using said display unit in a closed space.

There is also provided a computer program comprising program instructions for causing a computer program to carry out the above method which may be embodied on a record medium, carrier signal or read-only memory.

Brief Description of the Drawings

The invention will be more clearly understood from the following description of an embodiment thereof, given by way of example only, with reference to the accompanying drawings, in which:-

FIG. 1 exemplarily illustrates a plurality of display units arranged in a closed space in accordance with some embodiments of the present invention;

FIG. 2 exemplarily illustrates an image displayed on a display unit as seen by the image capture device and the corresponding transformed image with the correct perspective; FIG. 3 is a flow chart illustrating an embodiment of the inventive method or process for visual guidance of a vehicle in a closed space, in accordance with some embodiments of the present invention; FIG. 4 is a functional block diagram illustrating the primary components of a vehicle unit for visual guidance of a vehicle in a closed space, in accordance with some embodiments of the present invention; and

FIG. 5 is a functional block diagram illustrating the display units and display processor for visual guidance of a vehicle in a closed space, in accordance with some embodiments of the present invention.

Detailed Description of the Drawings

FIG. 1 exemplarily illustrates A plurality of display units (102, 103) within a closed space 101 and a vehicle unit 104 mounted on a vehicle (vehicle not shown) in accordance with one of the embodiments of the present invention. The display units (102, 103) may be mounted on an interior surface or wall of the closed space or mounted on a pillar appearing within the closed space or suspended from the ceiling 106. The vehicle unit 104 detects and processes the image displayed in the display units and provides visual guidance to the vehicle 104 with respect to the closed space based on said detection and processing of the image displayed in the display unit.

FIG. 2 is provided as a mere reference as to how an encrypted guidance information may be displayed 102i on a display unit 102 as seen by the image capture device(s) of the vehicle unit 104. FIG. 2 also exemplarily illustrates as to how the captured image of the display unit is transformed 102t to correct the perspective. Accordingly, FIG. 2 is referenced further in the context of the method and system of the present invention in below paragraphs.

FIG. 3 is a flow chart illustrating an embodiment of the inventive method or process for visual guidance of a vehicle in a closed space using a display unit displaying graphically encoded information, in accordance with some embodiments of the present invention. The method and system for guiding a vehicle in closed space comprises receiving/capturing 201 one or more images of the closed space from said one or more image capture devices mounted on the vehicle. The images of one or more display units is detected and extracted 202 from the captured images where the detection of one or more display units comprises detection of a unique pattern displayed on the display units.

Each of the extracted image is transformed 203 to correct the perspective as shown in FIG. 2 i.e. the image of the display unit to the image capture device is shown as 102i and the image of the display unit captured by the image capture device is transformed to correct the perspective as shown in 102t of FIG. 2. Pattern information is extracted 203 from each of the transformed images. The pattern information is then decoded and decrypted 205 to obtain vehicle positioning and guidance information. The pattern information further comprises one or more of geometric coordinates of the active display unit with respect to the closed structure, parking space availability, manoeuvring directions from the location of the display unit (102, 103) to a location of suitable parking space within the closed structure, fee for the parking and speed limit in the area. A person skilled in the art would appreciate that the locations and directions provided by the display units (102, 103) are in reference to the display unit themselves. Further, the vehicle unit 104 may compute the distance and the orientation of the vehicle with respect to the display unit (102, 103) compute the location and orientation of a vacant parking space with respect to the vehicle or with respect to the enclosed space.

Finally, the positioning and guidance information as may be computed above is transmitted 206 to a control unit of the vehicle for guiding the vehicle within said closed space. In an embodiment the display units (102, 103) are active or passive display units where passive display units comprise a pattern information attached or painted on any of the interior surfaces of the closed space, for example a wall or a billboard. Active display units comprise flat panel screens controlled by a display processor to display said pattern information. In active displays, the display processor is configured to display pattern information based on a detection of a vehicle entering the closed space. In another embodiment, the pattern information is overlaid on an image displayed by the display units where the image is one of an advertisement or a human readable sign or information.

FIG. 4 is a functional block diagram illustrating the primary components of a vehicle unit for visual guidance of a vehicle in a closed space, in accordance with some embodiments of the present invention; and FIG. 5 is a functional block diagram illustrating the display units and display processor for visual guidance of a vehicle in a closed space, in accordance with some embodiments of the present invention.

The system for guiding a vehicle in a closed space comprises a plurality of display units (102, 103) and a vehicle unit 104. The plurality of display units are operatively coupled to a display processor 501 as shown in FIG. 5 for displaying at least a pattern information on said display units (102, 103) where the display units comprise flat panel screens. In an embodiment, the display processor is configured to display pattern information based on a detection of a vehicle entering the closed space within the vicinity of the display unit. Further the display processor 501 is configured to overlay pattern information an image displayed by the display unit where the image is one of an advertisement or a human readable sign or information. The vehicle unit 104 comprises a processor 301 , a memory 302 and one or more image capture devices (304, 305) and a transceiver 303. The processor 301 is configured for receiving one or more images of the closed space from said one or more image capture devices (304, 305) and detecting and extracting images of one or more display units (102, 103) from said received one or more images, where the detection of one or more display units (102, 103) comprises detection of a unique pattern displayed on the display units. The processor 301 is further configured for transforming each of the extracted image of the display units to correct the perspective; extracting pattern information from each of the transformed images; decoding and decrypting at least the pattern information from each of the transformed images to obtain vehicle positioning and guidance information, where the pattern information further comprises one or more of geometric coordinates of the display unit with respect to the closed structure, parking space availability, manoeuvring directions from the location of the display unit to a location of suitable parking space within the closed structure, fee for the parking and speed limit in the area.

The processor 301 is further configured for transmitting, using the transceiver 303, the positioning and guidance information to a control unit of the vehicle for guiding/navigating the vehicle within said closed space. In an embodiment, the memory 302 stores computer code executable by the processor. Further computer code configures the processor 301 to carry out the invention as disclosed above.

Example Embodiment Referring now to Figures 1 to 4, when a vehicle enters a closed space the vehicle comprises a number of algorithms to 'recognize/localize' one or more visual guidance boards (artificial landmark) by identifying the unique patterns from the image embedded in the board displaying pattern like QR-code or advertisement (with some hidden patterns) placed at several locations of parking lot. Algorithms can be based on specific scenarios of the infrastructure and database of unique patterns.

Haar-like features or Local Binary Patterns (LBP) are different features that can be used for those patterns recognition. They are similar to kernels in CNN (deep learning), except the values of the kernels here are manually determined. They are usually used with cascade classifier to form a strong learner or classifier. The classifier is trained with a large number of positive and negative images, and Haar or LBP features are then used as kernels to extract features by traversing through the images. A Decision tree or Random Forest can also be used based on specific scenarios.

The unique patterns which a vehicle's camera captures can have different designs, sizes and views. Although design and size can be pre-determined, but views of the patterns depends on factors like vehicle's entry point, orientation, distance from the landmarks etc. In these kind of scenarios, Deep Learning algorithms like CNN, RPN (wherein, values of kernels are determined by training) can be used. Flaar/LBP based classifier has limits that they can only detect certain features for which they are manually trained.

Once the patterns are localized, any distortions, like perspective warping, fisheye distortion, illumination distortions, are removed. This makes it easier to interpret the correct pattern/code. Once this is done, based on the pattern/code, either a simple template matching algorithm or decryption algorithm can be used to decrypt the vehicle's navigation related messages.

One problem the system has to overcome is pixel level uncertainty handling. A solution the system employs is to decrypt the accurate information based on the detected patterns made of several pixels, in many cases it can be important that there is no pixel level loss or uncertainty (that can be induced while removing the distortions in images) which can lead to a wrong output message. 1 instead of 0, or vice versa, can result in different measurements. Neural Networks or Decision trees are good candidates to be deployed to remove any errors.

Thereby the present invention provides for visual guidance to a vehicle using display units displaying graphically encoded information in closed space. Further, the disclosed system and method acts provides a visual artificial landmark based guidance system in the presence of autonomous vehicles. These artificial landmarks can be advertisements with hidden patterns. Also, the guidance system acts as an advertising or hazard warning or information broadcasting smart displays during the absence of any vehicle. Further, a person ordinarily skilled in the art will appreciate that the various illustrative logical/functional blocks, modules, circuits, techniques/algorithms and process steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or a combination of hardware and software. To clearly illustrate this interchangeability of hardware and a combination of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or a combination of hardware and software depends upon the design choice of a person ordinarily skilled in the art. Such skilled artisans may implement the described functionality in varying ways for each particular application, but such obvious design choices should not be interpreted as causing a departure from the scope of the present invention.

The process described in the present disclosure may be implemented using various means. For example, the process described in the present disclosure may be implemented in hardware, firmware, software, or any combination thereof. For a hardware implementation, the processing units, or processors(s) may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or a combination thereof.

For a firmware and/or software implementation, software codes may be stored in a memory and executed by a processor. Memory may be implemented within the processor unit or external to the processor unit. As used herein the term “memory” refers to any type of volatile memory or nonvolatile memory. Furthermore, although elements of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated. The embodiments in the invention described with reference to the drawings comprise a computer apparatus and/or processes performed in a computer apparatus. However, the invention also extends to computer programs, particularly computer programs stored on or in a carrier adapted to bring the invention into practice. The program may be in the form of source code, object code, or a code intermediate source and object code, such as in partially compiled form or in any other form suitable for use in the implementation of the method according to the invention. The carrier may comprise a storage medium such as ROM, e.g. a memory stick or hard disk. The carrier may be an electrical or optical signal which may be transmitted via an electrical or an optical cable or by radio or other means.

In the specification the terms "comprise, comprises, comprised and comprising" or any variation thereof and the terms include, includes, included and including" or any variation thereof are considered to be totally interchangeable and they should all be afforded the widest possible interpretation and vice versa.

The invention is not limited to the embodiments hereinbefore described but may be varied in both construction and detail.