Reading guide

Terms written in italics are found in the glossary.

Field of application of the invention The primary object of the invention is to provide a series of digital images which can be used as input for object VR of an object, which is not suited to be placed on a turntable in a photographic studio.

The state of the art

A few manually operated techniques capable of recording a series of digital images of a large object for the purpose of object VR are known and are described in the following patents.

US2009/0052885

This is a three-wheeled camera dolly in which there is a possibility for adjusting the rear wheel in a direction so that the camera dolly can be moved along a circular path around an object in order to make a recording of an object for the purpose of object VR. The camera dolly is not automatic and there is no feedback on the position of the camera so that the precision of the circular path along which the camera is moved depends on the supporting surface, because the wheels may slip, thereby accumulating an inaccuracy.

EP1209902

This is an image recording method whereby a structure bearing a camera is navigated using sensors placed on a floor. The object of the method is to be able to produce films from different angles in a film studio. The problem with this invention in relation to creating object VR is that physical sensors must be placed to serve as reference marks for controlling the robot. US2009202102 (Al)

This is a system and a method for recording and rendering images in relation to the position from which they were recorded, in order to avoid image stitching in the creation of 360-degree panoramic images. 360-degree panoramic images are recorded from one position and are therefore rotatable with that viewpoint as its centre. This system cannot provide a digital image series usable for object VR, because the images are recorded from one position.

At the present time, there are a number of technologies capable of creating image series of a similar kind, but none of the existing technologies can be used in a simple and mobile manner to create image series of large objects or are sufficiently precise for the images to be reproduced without use of rails, a lengthy calibration process or other time-consuming set-ups. A description follows of some of the closest related existing technologies. When a digital image series to be used for object VR is to be produced, either the recorder unit can be moved around the object, or the object can be rotated and recorded from one fixed angle.

Therefore, use is often made of a rotating platform on which an object is rotated in an interval of degrees determined by the number of images desired in the image series. The rotation is typically performed either automatically by means of a motor or by manual turning according to an angle measuring device. The problems with a rotating platform for large objects are that it can be very difficult to place the object accurately on the centre, and that a rotating platform is difficult to move if it has to carry and rotate an object, such as for instance a combine harvester or a military tank. Moreover, when a rotating platform is used, the background to the object does not change with the changing angles from which the individual images are recorded, which causes the image series to appear artificial to the viewer. The use of a rotating platform therefore requires the background to be manually removed from the images.

Another system used to record image series for object VR are systems where an object is placed on a platform and a mechanism carrying a recorder unit moves around the object at one or more desired angles. Again, a platform with its associated mechanism is not very mobile in the case of large objects, but for small objects this method can be very useful both in terms of reproducibility and mobility. However, a mechanism to move a recorder unit around an object requires the supporting structure to be moved in the course of the recording of an image series if it is not to be visible in an image series.

Today, there are two methods that are generally used when it is desired to produce an image series for object VR of large objects. One is to use a tripod having a set of wheels, where the angle can be adjusted to move the recorder unit along a circular path with a desired radius. The other is to lay out rails around the object to be recorded and move the recorder unit thereupon. The method employing a movable tripod presents a challenging task in placing the object at the centre of the circular path along which it moves, and the method can moreover only be used for moving the recorder unit along a circular path and not along other paths, which in some cases are desirable. The time consumption involved in manually measuring chords of a circle of equal length and moving the tripod precisely accordingly is also problematic. The alternative of using rails can yield reproducible images, but the rails will appear in the images, and the mobility and the time required to set up and calibrate a set of rails is problematic. Brief description of the invention

A method for producing a reproducible digital image series, which can be used for object V , comprising the following steps:

Definition of a point with a navigation unit, which is able to identify its present position continuously and wirelessly in relation to that point;

Programming of a movement path in relation to the defined point for a movable and navigable structure, which path includes a number of halts in the motion for the recording of images with a monocular recorder unit, start acceleration, halt deceleration, distance between each halt in the motion along the path and the duration of the halts in the motion when images are recorded;

Travelling along a path around an object by a movable and navigable structure, where the entire structure carries a monocular recorder unit around an object, characterised in that no use is made of previously physically laid out aids, e.g. mechanical tracks, optical sensors or other forms of visible reference marks, that will be visible in the images recorded by the monocular recorder unit. The travelling is further characterised in that the programmed movement path can be followed without human intervention once it has been initiated; - Setting up for object VR with web application, in which, in principle, it is possible to spool between images from the travelling in chronological or inverse chronological order in order to obtain the effect of rotation by means of e.g. Flash, JavaScript, Silverlight or similar technology; Optional reproduction of the output from said travelling by repetition of the process.

The invention may be characterised in that the movable structure, the purpose of which is to provide a series of digital images that can be used for object VR, uses a navigation system. The invention may moreover be characterised in that a wireless signal can be used from an external unit capable of measuring the distance to a receiver placed on the movable and navigable structure. Said wireless signal can be from an external unit capable of measuring the distance to a receiver placed on a recorder unit, which is placed on a movable structure capable of providing a series of digital images that can be used for object VR, for giving feedback on the direction of the recorder unit in relation to a plane that is parallel to the supporting surface upon which the structure moves. The movable and navigable structure can furthermore position a recorder unit in respect of lens direction and horizontal and vertical angle in relation to a three-dimensional coordinate system. The recorder unit can be navigated along a two-dimensional or a three-dimensional path, wherein a wireless unit constantly transmits coordinate data to a computer, which continuously supplies control input including acceleration and maximum speed to a number of control units, which in turn control a number of motors to move the structure in the x, y plane, and to move the recorder unit in the z plane, the computer sends input and receives feedback on position, acceleration and deceleration to a mechanism, which is able to move the recorder unit so that it maintains its predefined path. A two- dimensional or three-dimensional coordinate system is used to navigate the movable and navigable structure to a desired position on a path from an arbitrary point to a defined point on the desired path so that a point of origin can always be identified. The movable and navigable structure can be halted in its movement path at a number of equidistant points, the distances between these points being further subdivisible into a number of equal distances and so on, so that the structure can be moved by infinitely small distances enabling the density of image angles in an image series to be adjusted. The positions and angles of the images in relation to a defined point in the coordinate system, in relation to which the navigation is generally performed, are stored digitally, enabling the images to be reproduced in the same relation to a point at a desired time. The point can, for example, be the centre of a circle or one of the centres of an oval-shaped movement path. The image series can subsequently be image-processed and the images digitally compared so that the change in pixels can be identified and the portion that has changed in the series can be calculated and stored as images with only the change remaining on the images. If an existing CAD model of the recorded object is found, the changes can be marked on that model since the changes are found in an image series where positions and angles are known for all the images in relation to a point, which can also be marked on the CAD model, and thereby the surface area on the recorded object of any deformations can be precisely calculated. Furthermore, a CAD model can be created on the basis of an image series recorded around an object by analysing how pixels have moved between the images and using the displacement of individual pixels on adjacent images as a vector in a three-dimensional point cloud. Such a point cloud can be used as a dimensionally stable or scalable CAD model and edited in ordinary CAD applications. The recorder unit's interface for programming the recording of an image series is characterised in that the following parameters are entered as input at start:

Number of halts, number of images, specification of close-up image positions, one or more radii of a . movement path, definition of movement path, maximum speed, acceleration and deceleration, light intensity, and time interval between the image recordings. The light intensity can be adjusted during the image recording for object VR so that the object VR image will not appear to flicker during showing. In connection with this lighting adjustment, almanac data can be used to take account of the natural lighting.

Advantages obtained by the invention as compared to the state of art

Reproducible renderings of physical objects with unprecedented accuracy irrespective of where the recording takes place;

The movable and navigable structure is mobile, self-propelled and terrain-going;

Requires no human intervention after the task-specific parameters have been entered; The recorder unit is positioned with high precision in relation to a defined, fixed reference point;

A series of digital images for object VR of a large object can be provided without the use of mechanical tracks that are not part of the structure;

The recorder unit is not restricted to recording a circular image series, but can record several paths around an object, which are not necessarily recorded along a circle around the object. Accordingly, three-dimensional paths across the object can also be made, which can give the object VR image a 3D effect;

The method can be performed by one person;

The external form, colour and surface structure of a physical object can be compared with a higher degree of detail than hitherto irrespective of the placing of the physical object. The novel technical means

With the combination of known mechanical and electronic components in the invention, a novel method is created for providing a series of digital images that can be used for object VR with the advantages mentioned in the foregoing section.

The novel feature of the method is that, by predefining a number of parameters, it is possible to automatically provide a series of digital images with a mobile unit without any human intervention in the actual recording process after the parameters have been defined.

It is new to use a navigation or positioning unit to specify a position in relation to which a monocular recorder unit for object VR is to move and what the direction of the recorder unit has been with respect to vertical, horizontal and vertical and horizontal image direction tilt. Nor has it been possible prior to the present invention to make outdoor object VR image series in which the lighting conditions in the individual images of a series does not vary. This is achieved by collection of light and colour temperature data in the area in which the recording takes place.

Description of the procedure whereby a reproducible series of digital images for object VR is recorded.

The following is an example of how the method for providing a digital image series that can be used for object VR may be carried out.

In order to record a reproducible series of digital images that can be used for object VR, two principal steps are performed:

Step 1 - Automatic recording of image series that can be used for object VR

• A position, in relation to which the recorder unit shall move, is defined in the middle of the object;

• The unit, which is used to define the point, can optionally be mounted on the recorder unit;

• The necessary control parameters are defined by the operator;

• If it is critical for the quality of the images that the lighting is identical in the image series, sensors may be placed in the area. A number of light sources may also be placed, which are able to adjust the lighting between the images; • Recording of digital image series that can be used for object VR is initiated and the recorder unit moves around the object in accordance with the defined control parameters and record the images;

• The image series is stored locally in the recorder unit and is also transmitted wirelessly to an external storage unit. Position information describing from where the individual images in the series were taken in relation to the known position in the middle of the object is stored together with the images, so that the image series can later be reproduced in relation to a different point;

• On the basis of the image series, a dimensionally stable 3D model and an object VR image are created by means of software comparing pixel changes between images in close proximity, using the translation between pixels that show the same location on the edge as vectors in a point cloud and linking their end points with other end points in close proximity so that a large number of points and vectors form a three-dimensional collection of polygons - a 3D model. Step 2 - Reproduction of above-mentioned image series

• The process described above is repeated with the initially defined point being defined in the same place in middle of the object.

To identify the location of the object to be recorded in an imaginary three-dimensional coordinate system, an inertial navigation unit is placed in the centre of the object and a coordinate is saved to serve as reference point for dead reckoning navigation. The inertial navigation unit can now be positioned in relation to this point in three dimensions. The inertial navigation unit is subsequently placed on the unit on a supporting element carrying the recorder unit. Thus, the direction of the recorder unit can now always be positioned in relation to the known point in the centre of the object. Also on the recorder unit is a measuring unit including three gyroscopes, which report the

orientation of the recorder unit horizontally, vertically and vertical and horizontal image direction tilt. Moreover, the inertial navigation unit now placed on the recorder unit receives reference

coordinates from GPS or other wireless communication means capable of determining the position of the recorder unit in the horizontal plane. This reduces the operating error when the inertial navigation unit is moved by dead reckoning. Optionally, a number of wireless lux and colour sensors may be placed in the area along with a number of light sources. The sensors report information on the lighting conditions to the computer, enabling the computer to adjust the voltage to the light sources so that the lighting does not vary between the images. A selection between a number of standard movement paths is made on the computer for moving the complete recorder unit around the object. It might, for example, be an elongated ellipse if the object is a large heavy goods vehicle, or it might be a circle if the object is a private car, and a three-dimensional hemisphere consisting of a number of movement paths if the object is desired to be recorded in three dimensions excluding the underside. After the path has been selected, the unit navigates automatically to a known point on the path and halts there. The recorder unit can now be remotely controlled from the computer and the necessary settings on the recorder unit can be adjusted manually or automatically. The inertial navigation ensures that the recorder unit lens points in the direction of the point on the object that was initially defined. After the recorder unit settings have been adjusted, the desired number of images to be recorded of the object is selected, and if close-up images from certain angles are desired, these are defined by specifying points with the navigation unit on the object to be recorded. It is also selected whether the background should be removed on the images of the object. The unit is then started and it moves along the path, halts and records an image at the specified halting positions. The images are stored in a database in the computer, where also three-dimensional coordinates in relation to the point that was first specified on the object are stored, and the image series is assigned a unique ID. Finally, using a computer script on the basis of the information in the database, the images are set up for an object V display uploadable to an external storage unit. In addition, a point cloud based on pixel movement between individual images from the image series is calculated on the external storage unit. This is achieved by each pixel on the edge of the recorded object moving a fixed number of degrees between the images, this pixel translation being stored as a vector and both end points being stored as coordinates in a point cloud in a three-dimensional coordinate system. The point cloud describes the geometry of the object with a degree of accuracy that depends on the resolution of the images, and can thereafter be processed in ordinary CAD software.

If it is desired to reproduce the image series of an object already recorded, the first step of the process described at the beginning of this section, in which a point in relation to which the image series is to be recorded is defined with the navigation unit, is repeated. In order to position the recorder unit in all directions so that a particular image in the image series is recorded in the same way as in the previous recording, reference points are stored on the object so that these can be marked manually on the image, which is displayed via the recorder unit on the computer. By marking at least three reference points, which are not aligned, on the object it will be made possible to position the recorder unit in relation to the object in such a way that an image can be recorded from the same angle and position as in the previous recording. The ID of the image series contains, in addition to the reference points on a particular image, information on the movement path of the recorder unit and the image direction of the recorder unit at the previously used halt positions, so that the same movement path can be repeated in relation to the defined point. If the point is defined at the same place on the object, the images will be recorded in the same way.

This method therefore permits the recording of a reproducible digital image series of unprecedented precision of an object that can be used for object VR, irrespective of the size of the object.

The technical effect

The following is an example of a mechanical solution that will be capable of using the method for providing a series of digital images for object VR. The components are arbitrarily selected and can be replaced by others having the same effect. This is therefore not the only possible combination of components that will be capable of making a recording of digital images for object VR with the above-mentioned advantages, but merely illustrates the probable effectiveness of the invention. References relate to the drawings.

*

The navigation unit

(7) may be a box containing a circuit capable, by means of a tri-axial gyroscope and a tri-axial accelerometer, of determining a position in relation to another position after movement. (7) has wireless data transfer to (5), which can be accomplished by means of, for example, WiFi, Bluetooth or a wireless broadband connection. In addition, (7) must contain a transmitter that can be .

positioned in relation to some reference points such as, for example, GPS coordinates or coordinates in a three-dimensional coordinate system created by means of a robotic total station. (7) works in two phases, initially, in a first phase before an image is to be recorded from a defined position, navigating (1) into position with a low degree of precision, following which (8), which carries the recorder unit, navigates (6) to the absolute position with a high degree of precision.

Lighting

The light sensors can be small units wherein is found a circuit consisting of a lux sensor, a colour sensor and a wireless communication unit such as, for example, WiFi, Bluetooth or wireless broadband. The light sources can be battery-powered, intelligent photo lights equipped with a wireless communication unit able to communicate with the computer, and an electronic potentiometer able to regulate the power and thereby the light intensity. The movable and navigable structure

All units are physically supported by a bearing structure 1 on which all separate components are mounted and which is carried by wheels being driven by motors 2. The wheels are driven by a number of motors 2, which are in a closed-loop circuit with the navigation unit 7, which is mounted close to the recorder unit. This circuit makes it possible to constantly navigate the recorder unit 6 by means of the motors and wheels 2 and a mechanism 8 having, for example, six degrees of freedom as precisely as the mechanism 8 permits. A laptop computer 5 continuously receives coordinate data from the navigation unit 7 and converts them to movement paths, which are adjusted using regression to match the paths predefined by the operator as far as possible. Encoders mounted on the mechanism 8 continuously transmit position data to the computer 5 so that it can be verified whether the recorder unit 6 is on the correct path. The coordinate data are also used to calculate the shortest possible line from the recorder unit 6 to a fixed, defined point and this line is used as the direction of 6 so that the image direction of the lens of the recorder unit 6 is always directed at that point. On the recorder unit 6 is also mounted a measuring unit containing a gyroscope in the x, y, z directions which continuously reports the orientation of the recorder unit 6 horizontally, vertically and vertical and horizontal image direction tilt. When the positioning takes place prior to the recording of an image, a rough positioning of the supporting structure 1 is performed, after which the mechanism 8 performs fine adjustment of the exact position of the recorder unit 6 for the precise recording of an image. The interface for controlling the control units for the motor 3, the recorder unit 6, the navigation unit 7 and the mechanism 8 is set up from the computer 5. The computer 5 is pre-programmed with a series of movement paths for the recorder unit 6, which can be followed by means of the motor-driven wheels 2 and the mechanism 8. These movement paths are prioritised according to the operator's estimate of the geometry of the object it is desired to record in such a way that a path is chosen, which is best suited for recording an image series of the geometry. The computer 5 is used to set up the recorder unit 6 when an object is to be recorded. The view through the lens of the recorder unit 7 can be seen with the remote control software of the computer 5, so that an image can be evaluated on the display of 5 before it is recorded. When the subject is correctly seen by the recorder unit 6, a calculation is performed by the measuring unit mounted on the recorder unit 6 to determine how the recorder unit 6 must be oriented from the various angles that will be obtained when the recorder unit 6 moves along the predefined path in order to maintain the desired angle towards the defined point in the middle of the object. The image series is assigned a unique ID and is subsequently stored in the computer 5 and uploaded by means of a wireless network unit to a server, where the images are stored in a database together with three-dimensional coordinates describing the position, in relation to a fixed, defined point, of the recorder unit from which the individual images were recorded, and information on the settings of the recorder unit. It would be possible to mount a battery 4 and other components on the supporting structure 1 for additional functions, such as, for example, lighting regulation and distance measurement.

Glossary

Object VR - An interactive representation of an object consisting of a number of digital images, which are recorded around an object in chronological order, where the user can rotate the object. In principle, it can be compared with filming around an object and spooling the film forwards and backwards to achieve an impression that the object is rotated.

Monocular recorder unit - is to be understood as any monocular unit capable of providing a digital image, e.g. a digital single-lens reflex camera, a video camera or a scanner.

Robotic total station - with robotic total station is meant any form of unit capable of sending a wireless signal to a unit and position the wireless unit using that signal. Robotic total stations are typically used in surveying, and can consist of, for example, a GPS transmitter, a laser or ultrasonic station.

Dead reckoning - is the position calculated by navigators as a ship's possible position before GPS became a common technology. As it was not possible to determine longitude, it was necessary to attempt to calculate the ship's position on the basis of a previous known position and the distance travelled from that position. Dead reckoning navigation is used today in certain types of vehicle GPS units, e.g. when the vehicle is running through a long tunnel without satellite contact.

CAD - Computer aided design.

Gyroscope - A body that can be used to measure or maintain orientation using conservation of angular momentum.

Accelerometer - An accelerometer is a transducer that can measure acceleration.

Navigable - by navigable is meant that the unit can be positioned and controlled in relation to its surroundings.