Title:
STEREO DEPTH ESTIMATION USING DEEP NEURAL NETWORKS
Document Type and Number:
WIPO Patent Application WO/2019/182974
Kind Code:
A3
Abstract:
Various examples of the present disclosure include a stereoscopic deep neural network (DNN) that produces accurate and reliable results in real-time. Both LIDAR data (supervised training) and photometric error (unsupervised training) may be used to train the DNN in a semi-supervised manner. The stereoscopic DNN may use an exponential linear unit (ELU) activation function to increase processing speeds, as well as a machine learned argmax function that may include a plurality of convolutional layers having trainable parameters to account for context. The stereoscopic DNN may further include layers having an encoder/decoder architecture, where the encoder portion of the layers may include a combination of three-dimensional convolutional layers followed by two-dimensional convolutional layers.
More Like This:
Inventors:
SMOLYANSKIY NIKOLAI (US)
KAMENEV ALEXEY (US)
BIRCHFIELD STAN (US)
KAMENEV ALEXEY (US)
BIRCHFIELD STAN (US)
Application Number:
PCT/US2019/022753
Publication Date:
December 05, 2019
Filing Date:
March 18, 2019
Export Citation:
Assignee:
NVIDIA CORP (US)
International Classes:
G06N3/04; G01S13/931; G01S17/86; G01S17/89; G06N3/063; G06N3/08; G06T7/593; G01S15/931
Foreign References:
| US20170371340A1 | 2017-12-28 | |||
| EP2384009A2 | 2011-11-02 | |||
| US20070182528A1 | 2007-08-09 |
Other References:
ALEX KENDALL ET AL: "End-to-End Learning of Geometry and Context for Deep Stereo Regression", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 13 March 2017 (2017-03-13), XP080756540, DOI: 10.1109/ICCV.2017.17
YIRAN ZHONG ET AL: "Self-Supervised Learning for Stereo Matching with Self-Improving Ability", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 4 September 2017 (2017-09-04), XP080818196
PANG JIAHAO ET AL: "Cascade Residual Learning: A Two-Stage Convolutional Neural Network for Stereo Matching", 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), IEEE, 22 October 2017 (2017-10-22), pages 878 - 886, XP033303533, DOI: 10.1109/ICCVW.2017.108
GODARD CLEMENT ET AL: "Unsupervised Monocular Depth Estimation with Left-Right Consistency", IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION. PROCEEDINGS, IEEE COMPUTER SOCIETY, US, 21 July 2017 (2017-07-21), pages 6602 - 6611, XP033250025, ISSN: 1063-6919, [retrieved on 20171106], DOI: 10.1109/CVPR.2017.699
YIRAN ZHONG ET AL: "Self-Supervised Learning for Stereo Matching with Self-Improving Ability", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 4 September 2017 (2017-09-04), XP080818196
PANG JIAHAO ET AL: "Cascade Residual Learning: A Two-Stage Convolutional Neural Network for Stereo Matching", 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), IEEE, 22 October 2017 (2017-10-22), pages 878 - 886, XP033303533, DOI: 10.1109/ICCVW.2017.108
GODARD CLEMENT ET AL: "Unsupervised Monocular Depth Estimation with Left-Right Consistency", IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION. PROCEEDINGS, IEEE COMPUTER SOCIETY, US, 21 July 2017 (2017-07-21), pages 6602 - 6611, XP033250025, ISSN: 1063-6919, [retrieved on 20171106], DOI: 10.1109/CVPR.2017.699
Attorney, Agent or Firm:
BACON, Kirk, D. et al. (US)
Download PDF: