Title:
MULTIMODAL FUSION OBSTACLE DETECTION METHOD AND APPARATUS BASED ON ARTIFICIAL INTELLIGENCE BLINDNESS GUIDING
Document Type and Number:
WIPO Patent Application WO/2023/015799
Kind Code:
A1
Abstract:
Disclosed in the present invention is a multimodal fusion obstacle detection method based on artificial intelligence blindness guiding, comprising: respectively obtaining an infrared image and a color image of a scene by means of an infrared camera and a color camera; respectively sending the obtained infrared and color bimodal images to a convolutional neural network Q1 and a convolutional neural network Q2, the convolutional neural network Q1 and the convolutional neural network Q2 respectively converting the images into a first multi-channel feature map and a second multi-channel feature map for later flattening into vectors; representing the first multi-channel feature map and the second multi-channel feature map in a vectorized manner, and performing feature vector encoding on a sequence of the first multi-channel feature map and a sequence of the second multi-channel feature map to generate multiple prediction vectors; and classifying the generated multiple prediction vectors and performing position prediction on same. According to the present invention, a Transformer structure is introduced during an obstacle detection process to more effectively achieve multimodal fusion, and a Transformer-block is introduced to fully fuse the features of infrared and color images, such that the obstacle detection accuracy in a low-illumination scene is improved.
More Like This:
Inventors:
QIN WENJIAN (CN)
ZHANG WANG (CN)
ZHANG WANG (CN)
Application Number:
PCT/CN2021/138104
Publication Date:
February 16, 2023
Filing Date:
December 14, 2021
Export Citation:
Assignee:
SHENZHEN INST OF ADV TECH CAS (CN)
International Classes:
G06V10/00
Foreign References:
CN113591770A | 2021-11-02 | |||
CN112926700A | 2021-06-08 | |||
CN112418163A | 2021-02-26 | |||
CN111368118A | 2020-07-03 | |||
US20160180195A1 | 2016-06-23 |
Other References:
ZOU WEI;YIN GUODONG;LIU HAOJI;GENG KEKE;HUANG WENHAN;WU YUAN;XUE HONGWEI: "Low-observable Target Detection Method for Autonomous Vehicles Based on Multi-modal Feature Fusion", CHINA MECHANICAL ENGINEERING, vol. 32, no. 9, 24 June 2021 (2021-06-24), pages 1114 - 1125, XP093035259, ISSN: 1004-132X, DOI: 10.3969/j.issn.1004-132X.2021.09.013
Attorney, Agent or Firm:
BEIJING ZHONG XUN TONG DA INTELLECTUAL PROPERTY AGENCY CO., LTD. (CN)
Download PDF: