Title:
VIEWPOINT INVARIANT VISUAL SERVOING OF ROBOT END EFFECTOR USING RECURRENT NEURAL NETWORK
Document Type and Number:
WIPO Patent Application WO/2019/113067
Kind Code:
A3
Abstract:
Training and/or using a recurrent neural network model for visual servoing of an end effector of a robot. In visual servoing, the model can be utilized to generate, at each of a plurality of time steps, an action prediction that represents a prediction of how the end effector should be moved to cause the end effector to move toward a target object. The model can be viewpoint invariant in that it can be utilized across a variety of robots having vision components at a variety of viewpoints and/or can be utilized for a single robot even when a viewpoint, of a vision component of the robot, is drastically altered. Moreover, the model can be trained based on a large quantity of simulated data that is based on simulator(s) performing simulated episode(s) in view of the model. One or more portions of the model can be further trained based on a relatively smaller quantity of real training data.
Inventors:
TOSHEV ALEXANDER (US)
SADEGHI FERESHTEH (US)
LEVINE SERGEY (US)
SADEGHI FERESHTEH (US)
LEVINE SERGEY (US)
Application Number:
PCT/US2018/063843
Publication Date:
July 18, 2019
Filing Date:
December 04, 2018
Export Citation:
Assignee:
GOOGLE LLC (US)
International Classes:
B25J9/16; G05B13/02; G06N3/08
Foreign References:
| US20170334066A1 | 2017-11-23 |
Other References:
FINN CHELSEA ET AL: "Deep visual foresight for planning robot motion", 2017 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), IEEE, 29 May 2017 (2017-05-29), pages 2786 - 2793, XP033127113, DOI: 10.1109/ICRA.2017.7989324
KRAGIC D ET AL: "Vision for robotic object manipulation in domestic settings", ROBOTICS AND AUTONOMOUS SYSTEMS, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 52, no. 1, 31 July 2005 (2005-07-31), pages 85 - 100, XP027792879, ISSN: 0921-8890, [retrieved on 20050731]
SIVIC J ET AL: "Efficient Visual Search for Objects in Videos", PROCEEDINGS OF THE IEEE, IEEE. NEW YORK, US, vol. 96, no. 4, 1 April 2008 (2008-04-01), pages 548 - 566, XP011205584, ISSN: 0018-9219
ROUHOLLAH RAHMATIZADEH ET AL: "Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration", 10 July 2017 (2017-07-10), XP055573860, Retrieved from the Internet [retrieved on 20190325]
GIOVANNI DE MAGISTRIS: "Teaching a Robot Pick and Place Task using Recurrent Neural Network", 1 December 2016 (2016-12-01), XP055570655, Retrieved from the Internet [retrieved on 20190318]
KRAGIC D ET AL: "Vision for robotic object manipulation in domestic settings", ROBOTICS AND AUTONOMOUS SYSTEMS, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 52, no. 1, 31 July 2005 (2005-07-31), pages 85 - 100, XP027792879, ISSN: 0921-8890, [retrieved on 20050731]
SIVIC J ET AL: "Efficient Visual Search for Objects in Videos", PROCEEDINGS OF THE IEEE, IEEE. NEW YORK, US, vol. 96, no. 4, 1 April 2008 (2008-04-01), pages 548 - 566, XP011205584, ISSN: 0018-9219
ROUHOLLAH RAHMATIZADEH ET AL: "Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration", 10 July 2017 (2017-07-10), XP055573860, Retrieved from the Internet
GIOVANNI DE MAGISTRIS: "Teaching a Robot Pick and Place Task using Recurrent Neural Network", 1 December 2016 (2016-12-01), XP055570655, Retrieved from the Internet
Attorney, Agent or Firm:
HIGDON, Scott et al. (US)
Download PDF:
Previous Patent: BACTERIOPHAGE TREATMENT FOR ACNE AND BIOFILMS
Next Patent: MALEATED SOYBEAN OIL DERIVATIVES AS ADDITIVES IN METALWORKING FLUIDS
Next Patent: MALEATED SOYBEAN OIL DERIVATIVES AS ADDITIVES IN METALWORKING FLUIDS