Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
A METHOD ESTIMATING ABSOLUTE ORIENTATION OF A VEHICLE
Document Type and Number:
WIPO Patent Application WO/2011/062525
Kind Code:
A1
Abstract:
The present invention relates to a method estimating absolute orientation of a vehicle by means of camera and GPS. The method requires few components and the principle components are a GPS (11) and a camera (5, 6). The method can be combined with existing navigation methods. According to the method of the invention the absolute orientation is obtained by the use of absolute position from the GPS (11) and relative motion from concurrent images (6) taken above an essentially horizontal ground.

Inventors:
CARLBOM PEIIE (SE)
BECKMAN THOMAS (SE)
ERLANDSSON UIF (SE)
Application Number:
PCT/SE2009/000497
Publication Date:
May 26, 2011
Filing Date:
November 20, 2009
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
SAAB AB (SE)
CARLBOM PEIIE (SE)
BECKMAN THOMAS (SE)
ERLANDSSON UIF (SE)
International Classes:
G01C21/00; G01C21/16; G01S1/00; G05D1/10; G06T7/00; G06T7/20
Domestic Patent References:
WO2006002322A22006-01-05
WO2005100915A12005-10-27
WO1997035166A11997-09-25
Foreign References:
GB2436740A2007-10-03
US20080059065A12008-03-06
EP1898181A12008-03-12
US5894323A1999-04-13
US20060018642A12006-01-26
GB2436740A2007-10-03
Other References:
KATZOURAKIS D ET AL.: "Vision aided navigation for unmanned helicopters.", 17TH MEDITERRANEAN CONFERENCE ON CONTROL AND AUTOMATION, 24 June 2009 (2009-06-24) - 26 June 2009 (2009-06-26), THESSALONIKI, GREECE
TEMPLETON T ET AL.: "Autonomous Vision-based Landing and Terrain Mapping Using an MPC-controlled Unmanned Rotorcraft.", IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, 10 April 2007 (2007-04-10) - 14 April 2007 (2007-04-14), ROMA, ITALY, XP008157832
DAVID G LOWE: "Distinctive Image Feature from Scale-Invariant Keypoints", COMPUTER SCIENCE DEPARTMENT UNIVERSITY OF BRITISH COLUMBIA VANCOUVER, B.C., 5 January 2004 (2004-01-05), pages 1 - 28
HARTLEY RICHARD: "Multiple View Geometry in Computer Vision", 2000, CAMBRIDGE UNIVERSITY PRESS, pages: 364 - 390
See also references of EP 2502025A4
Attorney, Agent or Firm:
FORSBERG, Carl-Göran (Patents and Trademarks, Karlskoga, SE)
Download PDF:
Claims:
Patent Claims 1. A method estimating absolute orientation of a vehicle by means of camera and GPS, characterized in that the absolute orientation is obtained by the use of absolute position from GPS and relative motion from concurrent images taken above an essentially horizontal ground.

A method as claimed in claim 1 , characterized in that three concurrent images used to obtain the absolute orientation.

3. A method as claimed in claim 1 or 2, characterized in that the following steps are carried out:

1) selecting at least three images at a distance from each other,

2) identifying key points in each selected image,

3) obtaining the absolute positions of the images from the GPS defining the image positions when the respective image was taken, and

4) matching key points between selected images.

4. A method as claimed in claim 4, characterized in that the matching of key points between the selected images comprises the following steps:

1) projecting the key points down on a horizontal plane for each key point,

2) calculating the geometrical centre of gravity for the corresponding projections equal to the number of images selected,

3) obtaining an error value for each key point and image by the distance between the back projection of the geometric centre of gravity and its respective key point in the image,

4) obtaining the rotation of the images and the height coordinate of the horizontal plane by minimizing the distances according to item 3).

Description:
A method estimating absolute orientation of a vehicle The present invention relates to a method estimating absolute orientation of a vehicle by means of camera and GPS. In this connection GPS stands for a Global Positioning System.

In the following the expression absolute orientation refers to how the vehicle is directed relative to a system of axes fixed relative to the ground.

In many application fields it is important to get an accurate estimation of the relative orientation of a vehicle. One such application field is production of maps when images can be taken from airborne vehicles such as unmanned airborne vehicles, often abbreviated UAV. Other application fields are as components in navigation equipments for aeroplanes, helicopters and UAVs or included in missiles such as cruise missiles.

There are quite a number of different ways known to estimate the absolute orientation. One way of doing this is to observe landmarks and compare with a map. It is also possible to combine an inertial motion unit, IMU, that can comprise accelerometers and gyros, with a magneto-meter. Another way is to combine an IMU with a GPS. Examples of rather complex navigation methods using camera, GPS and IMU are known from WO 2005100915 Al, US 5894323 A and WO 9735166 Al. Another example is disclosed in US 20060018642 disclosing an infra red camera and GPS used in combination with laser distance measuring and magnetic compass. These by references exemplified methods are component requiring and complex in the treatments of obtained information. Accordingly, these mentioned methods have different advantages and drawbacks differing in under which circumstances they can be used, the accuracy of the results obtained, time delay introduced and so on. The object of the invention is to provide a method estimating absolute orientation requiring few components, namely in principle GPS and camera, so that the costs can be kept down and in addition a method that can be combined with existing navigation methods is available.

The object of the invention is obtained by a method characterized in that the absolute orientation is obtained by the use of absolute position from GPS and relative motion from concurrent images taken above an essentially horizontal ground.

Essential for the invention is that that the vehicle travels above an essentially horizontal ground enabling estimation of the absolute orientation using only a and a GPS.

According to a preferred method three concurrent images are used to obtain the absolute orientation. This is the minimum number of images needed to obtain an estimation of the absolute orientation and that at the same time offers an accurate estimation.

According to a still preferred method the following steps are carried out:

1) selecting at least three images at a distance from each other,

2) identifying key points in each selected image,

3) obtaining the absolute positions of the images from the GPS defining the image positions when the respective image was taken, and

4) matching key points between selected images.

Principles for identification of key points are known since long ago and for example we refer to an article of David G Lowe, Computer Science Department University of British Columbia Vancouver, B.C., Canada: "Distinctive Image Feature from Scale- Invariant Keypoints", pp 1-28, January 5, 2004 and in particular to paragraph 2 on page 2 to page 5.

According to a further development of the still preferred method the matching of the key points between the selected images comprises the following steps:

1) projecting the key points down on a horizontal plane for each key point, calculating the geometrical centre of gravity for the corresponding projections equal to the number of images selected,

obtaining an error value for each key point and image by the distance between the back projection of the geometric centre of gravity and its respective key point in the image,

obtaining the rotation of the images and the height coordinate of the horizontal plane by minimizing the distances according to item 3).

In this connection it can be noticed that such a type of matching per se is previously known from for example an article of Hartley Richard Australian National University Canberra; Australia and Zisserman Andrew University of Oxford ,U : "Multiple View Geometry in Computer Vision, second edition, pp364-390, Cambridge

University Press 2000, 2003. However, according to the invention the matching is applied in a particular application field concerning estimation of absolute orientation of a vehicle. There is no indication to use the matching in such an application field or closely related application fields.

The invention will now be described in more detail with reference to the

accompanying drawings in which:

Figure 1 schematically illustrates reference frames for a vehicle, a camera and horizontal ground.

Figure 2 shows a flow chart illustrating the method according to the invention.

In figure 1 a vehicle 1 , such as an unmanned airborne vehicle, airplane, helicopter or missile, is shown moving above an essentially horizontal ground indicated by a reference frame 2 fixed relative to earth. A reference frame 3 is fixed to the vehicle 1 and a reference frame 4 is fixed to a camera 5. Before applying the method according to the invention the following is assumed to be known regarding the reference frames 2, 3 and 4. Concerning the reference frame 2 fixed relative to earth, position and attitude are assumed to be known. Concerning reference frame 3 fixed to the vehicle, position relative to reference frame 2 is assumed to be known. Concerning reference frame 4 fixed to the camera, it is assumed that position and attitude relative to reference frame 3 are known. The part remaining to be estimated by the method according to the invention is to estimate the attitude of reference frame 3 relative to reference frame 2. The method according to the invention is now described with reference to the flow chat of figure 2.

Based upon images 6 delivered from the camera 5 a key point calculation is carried out in a block 7. This can be done according to known calculation principles as known from the key point reference given above or other similar key point calculation principles. The general principle is to identify key points in the images having particular features. Preferably three images taken at a suitable distance from each other are chosen for this key point process and key points found in all three images are selected. Block 8 comprises selected key points supplied by block 7 and to be projected down to a horizontal ground plane in a block 9. This block 9 is also supplied with position estimates from a block 10 informed from a GPS 11 of the absolute positions, x, y and height, of the images involved in the key point calculations. The position of an image is defined as the position in space where the centre of the camera was located when the image were taken.

In block 9 the key points are projected down on a horizontal plane for each selected key point. Each image defines a position in the horizontal plane for each key point. The position of each key point in space is determined in block 9 by calculating the geometrical centre of gravity for the corresponding projections and equal to the number of images selected. The mutual positions of the key points and the position relative to the image positions are now determined. An error value for each key point and image is obtained by the distance between the back-projection of the geometric centre of gravity and its respective key point in the image. The rotation of the images and the height coordinate of the horizontal plane are determined by minimizing the distances defined in the preceding sentence, block 12 in cooperation with block 9. The minimizing process output orientation estimates, block 13, that in case of key points corresponding to objects essentially on a horizontal ground defines a calculated absolute orientation of the images and when the camera is fixed mounted on or in the vehicle the absolute orientation of the vehicle is also determined. Accordingly, the proposed method takes advantage of the travelling of a vehicle above horizontal ground. Furthermore the method presuppose that the vehicle is equipped with a fixed mounted calibrated camera synchronized with a GPS receiver so that each taken image can be provided with a GPS-position for the camera at the moment when the photo or image is taken.

The invention is not limited to the method exemplified above but may be modified within the scope of the attached claims.