FIELD OF THE INVENTION The present invention relates to an optical device and a method that are adapted in the field of omnidirectional imaging.

BACKGROUND ART Imaging devices that are adapted to capture a wide field of view is often referred as non- perspective imaging devices. They are also generally referred as omnidirectional imaging devices, wide-angle imaging devices or panoramic imaging devices. In practice, these imaging devices can be adapted in a wide variety of usages, which include surveillance, teleconferencing and many other applicable usages.

In the art of these imaging devices, it is considered that a normal camera can only obtain a field of view that is limited from a few degrees up to 180 degrees. As such, light can enter the camera's lens through a semi-sphere thereof, usually at the camera's focal point and therefore, clear images can be produced without any distortion. However, for a non- perspective imaging device, a wide-angle or panoramic imaging device can be adapted to capture a field of view approximately greater than 60 degrees. On the other hand, the omnidirectional imaging devices are generally referred to non-perspective systems that can be adapted to obtain a 360-degree field of view. As such, some wide-angle imaging devices cover only approximately a semi-sphere or more, whilst the omnidirectional imaging devices can cover almost the entire sphere. Although these imaging devices are practically useful in terms of covering wide field of view, as the coverage approaches the full sphere, the captured light rays would not fall into a single focal point of the imaging devices. As a result, images in perspective produced from these systems are often distorted and not meaningful. Besides that, it is still possible to expand the field of view of these imaging devices. A reflector is usually attached in front of the camera at the lens' side, reflecting the field of view, to form a panoramic camera-reflector imaging device. For these imaging devices that are consisted of a panoramic camera with a planar mirror, they may generally be referred as panoramic camera-reflector scanning systems. As for the imaging devices that are consisted of a normal camera but with a substantially flat mirror, they are usually called catadioptric-imaging devices.

However, it should be noted that the transformation for panoramic field of views into a perspective view for these camera-reflector imaging devices is usually required to be precise and is usually determined by a set of parameters. Thus, in these panoramic camera-reflector imaging devices, it is crucial that the camera is correctly aligned with respect to the reflector such that the focus point of the reflector can be merged with the focus point of the camera. Such an alignment of the reflector and the camera would prevent the converted perspective view from being distorted by means of the transformation of the panoramic field of view. As a result, the camera and the reflector are required to be maintained only in a fixed arrangement to ensure a smooth field-of- view transformation.

On a further note, some of these imaging devices may also be required to obtain a further field of view from other direction. As such, in this type of imaging device, the camera and the reflector are therefore allowed to move in a relative manner to each other. However, it is still not yet practical to obtain change of field of view in this instance. The transformation that is restrictively based on the fixed camera-reflector arrangement may inevitably result to unnecessary distortion of the resultant perspective image and production of a non-meaningful image. Therefore, in view of the above, a panoramic camera-reflector imaging device that allows relative displacement of the reflector and the camera and at the same time, allows a swift smooth transformation, that is not based on the fixed camera-reflector arrangement, to generate an undistorted perspective view is very much needed. SUMMARY OF THE INVENTION

Accordingly, to solve the disadvantages and drawbacks of the prior art, there is provided an optical device that is comprised of a panoramic optical unit having a reflector and a visual-capturing component, a regulating unit, and an image-transforming component.

In the present invention, the reflector is provided to reflect a field of view whilst the visual-capturing component is adapted to obtain a visual substantially from the field of view through a visual-receiving portion thereof, and then to generate an omnidirectional visual for the filed of view. The regulating unit is provided to move at least one of the reflector and the visual-capturing component in order to shift the field of view. The present invention also provides the image-transforming component to transform the omnidirectional visual into a perspective visual in accordance to an image transformation.

The optical device of the present invention is further comprised of a correction system that is adapted to implement a visual correction. The correction system is essentially comprised of a sensing component and a correcting component. The sensing component is adapted to acquire data that are associated with at least one of the reflector, the visual- capturing component, and the regulating unit, whilst the correcting component is essentially provided to analyze the data, to determine the visual correction based on the data, and to adapt the visual correction onto the image transformation.

In another aspect of the present invention, a method for implementing a visual correction is also provided. The method is essentially comprised of the steps of actuating at least one of the reflector and the visual-capturing component; acquiring a visual substantially from the field of view by means of the visual-capturing component through the visual- receiving portion thereof, generating an omnidirectional visual by means of the visual- capturing component; and transforming the omnidirectional visual into a perspective visual according to an image by means of an image-transforming component.

The method is further comprised the steps of acquiring data that are associated with at least one of the reflector, the visual-capturing component, and the regulating unit; and analyzing the data and determining a visual correction based on the data by means of the correcting component; and adapting the visual correction onto the image transformation by means of the correcting component. It is an object of the present invention to provide a correction system that allows generation of undistorted images in a rectangular format from the omnidirectional images in a circular format obtained from a omnidirectional imaging device containing a visual-capturing component and a reflector in a flexible arrangement. It is also an object of the present invention to provide a correction system that allows generation of undistorted images in a rectangular format from the omnidirectional images in a circular format particularly when there is a change of field of view.

It is also an object of the present invention to provide a panoramic imaging device that is adapted to allow generation of undistorted images in a rectangular format from the omnidirectional images in a circular format based on the obtained omnidirectional image or the field of view.

It is a further object of the present invention to provide a panoramic imaging device that allows acquisition of data such as the motion or the position of at least one of the reflector or the visual-capturing component, the relative position or the orientation of the reflector and the visual-capturing component, the control output of the regulating unit, or any data that is associated with the visual-capturing component and the reflector; from the regulating unit or from the sensing component; in order to determine the visual correction that converts the omnidirectional image to the undistorted perspective image based on the data.

It is a further object of the present invention to provide a panoramic imaging device that allows generation of undistorted images in a rectangular format from the omnidirectional images in a circular format based on the data acquired independently without human intervention.

It is a further object of the present invention to provide a panoramic imaging device that allows generation of undistorted images in a rectangular format from the omnidirectional images in a circular format on a real-time basis.

It is a further object of the present invention to provide a panoramic imaging device that is adapted for allowing conversion of omnidirectional image in a circular format into a perspective image in a rectangular format to be based on a flexible center of reference, as the center of reference shifts when there is a shift of field of view, to eliminate distortion associated therewith. It is a final object of the present invention to provide a panoramic imaging device that is allowed to be assembled and dissembled substantially in such manner that the panoramic imaging device is portable.

The present invention consists of certain novel features and a combination of parts hereinafter fully described and illustrated in the accompanying drawings and particularly pointed out in the appended claims; it being understood that various changes in the details may be without departing from the scope of the invention or sacrificing any of the advantages of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

For the purpose of facilitating an understanding of the invention, there is illustrated in the accompanying drawings the preferred embodiments from an inspection of which when considered in connection with the following description, the invention, its construction and operation and many of its advantages would be readily understood and appreciated.

FIG. 1 shows the panoramic camera-reflector imaging device of the present invention being movably mounted onto a regulating unit for movement thereof.

FIG. 2 shows the field of view that can be covered by the panoramic camera-reflector imaging device of the present invention.

FIG. 3 shows one of the preferred arrangements of the camera and the planar mirror of the present invention.

FIG. 4 shows the possible new radius for each predetermined angle, θ, ranging from 0° to 360° in the obtained omnidirectional image of the present invention.

FIG. 5 shows the triangle that can be extracted from the omnidirectional image for the calculation of the new radius.

FIG. 6 shows the transformed image in the rectangular format before remapping.

FIG. 7 shows the transformed image in the rectangular format after remapping and stretching.

FIG. 8 shows the effect on the usable area of the omnidirectional image if the separation between the camera lens and the mirror increases.

FIG. 9 shows the camera's centre self-blocking effect of the omnidirectional image if the separation between the camera lens and the mirror increases.

FIG. 10 shows other effects on the omnidirectional image if the mirror is displaced or rotated on at least one axis in relative to the camera lens.

FIG. 11 is shows the results of the transformation based on the center point of the omnidirectional image each according to the corresponding location of the camera lens in the omnidirectional image.

FIG. 12 shows the two omnidirectional images each where a possible transformation may be based on a reference point according to the location of the camera lens therein.

FIG. 13 shows a flowchart of the method for implementing a visual correction of the present invention.

FIG. 14 shows an example of a circle used for a preferred computation for the visual correction.

FIG. 15 shows another example of a circle used for a preferred computation for the visual correction.

FIG. 16 shows one of the examples of a circle used for a preferred computation for the visual correction.

FIG. 17 shows one of the examples of a circle used for a preferred computation for the visual correction.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention relates to an optical device 100 and a method that are adapted in the field of omnidirectional imaging. More particularly, the present invention relates to an optical device 100 and a method that are adapted in the field of omnidirectional imaging. Hereinafter, an optical device and a method therefor shall be described according to the preferred embodiments of the present invention and by referring to the accompanying description and drawings. However, it is to be understood that limiting the description to the preferred embodiments of the invention and to the drawings is merely to facilitate discussion of the present invention and it is envisioned that those skilled in the art may devise various modifications without departing from the scope of the appended claim.

In the panoramic camera-reflector imaging device of the prior art, it is essentially required that the transformation of an omnidirectional image in the circular format into an undistorted image in the rectangular format is based on a fixedly aligned camera- reflector arrangement. The camera and the reflector in these devices are therefore retained in a fixed arrangement to avoid any misalignment therebetween. Besides that, it is also practically preferred in this art that the reflector or the camera can move in such a manner that the reflector is allowed to reflect a field of view from other direction to the camera. However, such a change of field of view would inevitably cause the camera and the reflector to misalign. It should be noted that the transformed perspective image tends to distort due to the misalignment. In view of the above predicaments, the optical device 100 of the present invention is provided to overcome the difficulties to obtain an undistorted perspective image normally associated to panoramic imaging device with flexible camera-reflector arrangement. The optical device 100 of the present invention, in light of the above, is preferably adapted for determining and adapting a transformation onto a substantially omnidirectional image in a substantially circular format 90, based on a predetermined reference point identified from the omnidirectional image, in order to obtain an undistorted perspective image in a rectangular format 110. Although the omnidirectional image is herein described as in substantially circular format 90, it should also be noted that the omnidirectional image might be of elliptical shape as the omnidirectional image obtained in the present invention may be from a misaligned camera-reflector arrangement. Referring now to different figures of the drawings, the optical device 100 of the present invention is shown. The optical device 100 is essentially adaptable into a panoramic imaging device with flexible camera-reflector arrangement. The optical device 100 is also adaptable into such a device that allows change of field of view. In addition to that, it is also preferred that the optical device 100 of the present invention allows acquisition of data that are associated with the motion or the position of at least one of the reflector or the visual-capturing component, the relative position or the orientation of the reflector and the visual-capturing component, the control output of the regulating unit, or any data that is associated with the visual-capturing component and the reflector and can also determine the appropriate transformation based on these data in order to obtain an undistorted perspective image. It should also be noted that the optical device (100) of the present invention is provided to conveniently perform the transformation without human intervention and also on a real-time basis.

Referring now to FIG. 1 and 2, the optical device 100 can be adjusted height-wise to increase the field-of-view coverage. The higher the optical device 100 is adjusted, the larger field of view the optical device 100 can cover. According to FIG. 1, a microphone ring and an Infra-Red Light Emitting Diodes (LED) ring can be attached to the optical device depending to the embodiment of the present invention. The optical device 100 is preferably comprised of a reflector, a visual-capturing component, and a regulating unit. In practice, the reflector is adapted to reflect a field of view whilst the visual-capturing component is adapted to obtain a visual from the reflected field of view through a visual-receiving portion. Normally, from the obtained visual, an omnidirectional visual would be generated therefrom. The regulating unit (not shown) is preferably adapted to move either the reflector or the visual-capturing component in order to shift the field of view.

The optical device 100 of the present invention further comprises an image-transforming component (not shown) and a correction system (not shown). Preferably, the image- transforming component is adapted to transform the omnidirectional visual into a perspective visual. The correction system is essentially adapted for implementing a visual correction. The correction system is essentially comprised of a sensing component and a correcting component. Preferably, the sensing component is adapted to acquire data associated with either the reflector or the visual-capturing component. As aforementioned, the sensing component can also be adapted to acquire data from the output of the regulating unit, as will be hereinafter described in more detail. The correcting component, on the other hand, is preferably adapted to analyze the data, to determine the visual correction based on the data, and to adapt the visual correction onto the image transformation.

Preferably, the visual-capturing component is a camera that can be adapted to obtain either images or videos. It is also preferred that the camera that can record videos is a video camera. The camera preferably constitutes a lens in the visual-receiving portion thereof. In practice, the omnidirectional image would be preferably acquired for the transformation whereas for the video, the transformation would be preferably applied to the omnidirectional visual thereof in order to obtain the perspective video in rectangular format. Alternatively, each successive omnidirectional images or frames of the video would also be preferably acquired for the transformation, depending on the embodiment of the present invention.

Referring now to FIG. 3, according to the most preferred embodiment of the present invention, the camera is preferably a panoramic camera 20 with a wide-angle lens 30 whilst the reflector is a planar mirror 10. The regulating unit is generally provided to move or displace either one of the planar mirror 10 or the panoramic camera 20. With reference to FIG. 1 and 3, either the planar mirror 10 or the panoramic camera 20 is preferably allowed to rotate about at least one axis, and can be displaced in order to shift the field of view by means of the regulating unit. It is also preferred that the regulating unit allows the plane of the planar mirror 10 to lie in the range from substantially parallel to substantially perpendicular to the optical axis of the wide-angle lens 30. In general, the planar mirror 10 and the panoramic camera 20 are allowed to tilt and pan or to be displace in relative to each other depending on the preferred embodiment of the present invention. Furthermore, an application for moderating a controlling mechanism for the physical movement of the mirror 10 (or the camera 20) to achieve different field of view, with the automatic control, is preferably incorporated into the regulating unit. Referring now to FIG. 8 to 12, if the flexible panoramic camera-planar mirror arrangement is adopted, when the mirror size is constant, the increase in separation between the wide-angle lens 20 and the mirror 10 would reduce the usable area in the omnidirectional image, as indicated from left to right in FIG. 8. In addition, when the said separation increases, the centre camera's self-blocking effect would be reduced, as indicated from left to right in FIG. 9. As such, different level of adjustment control of the mirror's reflection and thus, the coverage/field of view of the panoramic camera 20 would be affected.

In FIG. 10, the tilt effect of the mirror 10 with respect to the camera 20 is shown. When the mirror 10 is tilted with respect to a stationary camera 20, the centre of the rotation cannot be adopted as the reference point, as illustrated in FIG. 10. In addition, if the transformation fixedly based on the fixedly aligned camera-mirror configuration is adopted, if the camera 20 and the mirror 10 are both in an aligned position, then the resultant transformed perspective image 94 (in the rectangular format 110) as shown in FIG. 11 can be obtained. If the mirror 10 and the camera 20 either one has been moved to a misaligned position, the reference point in the omnidirectional image 92 can be displaced, as shown in FIG. 11, resulting the resultant transformed perspective image 112 to appear distorted. The said original transformation is based on a reference point that is the centre of the circle (the omnidirectional image in the circular format 90). As shown in FIG. 12, when the mirror is tilted, the centre point would appear to no longer at the centre of the circle, instead there would be an offset. This offset would create complexity to the transformation to the transformation as the radius for each transformation would no longer be a constant and the radius, as it appears, changes according to the angle, as shown in FIG. 12. With that, based on the two constraints above, a new transformation with correction is required and will be introduced hereinafter in more detail. Referring now to FIG. 4 to 7, the correcting component is preferably adapted to determine the substantially center point of the omnidirectional image 70 and the location of the substantially center point of the lens 80 from the omnidirectional image. As such, according to the most preferred embodiment of the present invention, the correcting component utilizes a computational algorithm application to address the above- mentioned constraints. The application preferably takes into account the camera-mirror setup variable and creates a new transformation according to the position of the mirror, as will be hereinafter described in grater detail. Referring now to FIG. 4 to 7 and FIG. 12, when the center of the omnidirectional image 70 is changed, the radius for reference also changes, as shown in FIG. 12. As shown in FIG. 4 and 5, a triangle can be extracted from the omnidirectional image. With reference to FIG. 4, the new radius 60, given as c, of the circle is shown, having different length in each of the directions. As shown in FIG. 5, the distance 40 between the substantially center point of the omnidirectional image 70 and the substantially center point of the lens 80 is given as a, whilst the radius 50 of the circle is given as b. Once the location of the new center point of the lens 80 is identified and determined, the application will acquire said distance 40, a, and the radius 50, b. The application will then calculate the new radius 60, c, with the knowledge of values of the distance 40, a, and the radius 50, b. The new radius 60, c, is acquired, using the formula: c ² = a ² + b ² - 2ab cos θ

for each predetermined angle starting 0° to 360°. The predetermined angle θ, as shown in FIG. 5, is formed between the distance 40, a, and the radius 50, b. The new radius 60, c, for each triangle thereof is calculated at each angle θ increased in stepwise manner from 0° to 360°. Preferably, the circular images 90 are transformed into rectangular images 110 using the new radius 60, c. As shown in FIG. 4 and 6, the new radius 60, c, are remapped to a new buffer. FIG. 6 shows the image after remap, which is now a perspective image in the rectangular format 110. The white part in the image 110 is the lens part, or it is contributed by the new radius 60, c, at the portion near the reference point of the lens 80. This part containing the short radius 60, c, is required to be stretched such that the image 110 will appear undistorted. The image of Region OF Interest (ROI) is then set whereby the image in the ROI will be stretches. The short radius 60, c, will become stretches to fit into the new buffer, as shown in FIG. 7.

Preferably, depending on a preferred embodiment of the present invention, the transformation can betide to an automatic control mechanism, where the output of the rotator is used to control the application's transformation, such that the image transformation will look seamless as the mirror is rotated to cover different field of view. The correcting component of the present invention further comprises an associating component (not shown). The associating component is preferably incorporated in the application. The associating component is adapted to associate the angle, θ with a corresponding image correction (the calculation of the new radius 60, c), and to generate a list of the angles, θ and the corresponding image correction for implementation of the transformation with the said image correction. In addition, the associating component is further adapted to associate the corresponding image correction with the data that are associated with the motion of at least one of the reflector and the camera, the position of at least one of the reflector and the camera, the relative position of the reflector and the camera, the control output of the regulating unit, and the orientation of at least one of the reflector and the camera.

Referring now to FIG. 13, in another aspect of the present invention, a method to convert the omnidirectional image in a circular format 90 into an undistorted perspective image in the rectangular format 110 is shown according to the flowchart. Generally, before the transformation, either the planar mirror 10 or the panoramic camera 20 is allowed to be actuated such that the mirror 10 is allowed to reflect a field of view. Next, a visual substantially from the reflected field of view is preferably acquired through the lens 30 by means of the camera 20. The camera 20 preferably then generates an omnidirectional image substantially in a circular format 90.

During the stage of transformation, firstly, a circular omnidirectional image 90 is inputted or acquired. Next, the substantially center point 70 of the circular image 90 and the location of the substantially center point of the lens 80 from the circular image 90 are preferably acquired by means of the computational algorithm application. The distance 40 between the substantially center point of the omnidirectional image 70 and the substantially center point of the lens 80 is preferably then acquired. The radius 50 of the circular image 90 is also preferably acquired. In the next step, the new radius 60 for each predetermined angle, θ is preferably acquired. With reference to FIG. 5, the predetermined angle, θ, that is formed between the distance (40) and the radius (50), and is increased from 0° to 360° preferably in a stepwise manner. A list of the angles, θ and the corresponding correction, which is according to the new radius 60, is generated. Preferably, the angle, θ is also associated with the said corresponding image correction.

Referring still to FIG. 13, the destination maps from X and Y positions are then preferably initialized. Preferably, the destination maps from X and Y positions are applied according to a Cartesian coordinate system. The circular image 90 is then preferably mapped to the corrected perspective image by using the new radius 60, c. Next, the circular image 90 is preferably separated into two hemispherical portion. Preferably, the circular image 90 is also allowed to be separated into more substantially semi-spherical portions. The said portions are then preferably decomposed and straightened into columnized image data, as indicated in the FIG. 13. Finally, the circular image 90 is converted into a undistortedly corrected perspective image in the rectangular format 110 as shown in FIG. 7.

Referring now to FIG. 14 to 17, a few examples on how the calculation is executed with regard to the optical device 100 and the method as above-disclosed. Particularly in FIG. 17, if the x coordinate of the lens position 80 fulfills the condition as outlined below, then the degree is calculated as below: x > 640, then degree = 180 - degree

x < 640, then degree = 180 + degree

x = 640, then degree = 0

In another preferred embodiment of the present invention, the optical device 100 of the present invention is preferably adapted in a panoramic camera-reflector imaging device (not shown) consisting of a normal camera with a normal lens, and a substantially flat mirror. In this embodiment, the normal camera preferably replaces the panoramic camera 20 and a substantially flat mirror is preferably used. In practice, the regulating unit is allowed to move or displace the camera or the substantially flat mirror. The regulating unit in this embodiment allows the optical axis of the substantially normal lens to lie in a range from substantially parallel to substantially perpendicular to the principal axis of the substantially flat mirror. The regulating unit also allows that either the substantially flat mirror or the normal camera to rotate about at least one axis, and to be displaced in order to shift the field of view. Generally the same controlling mechanism by means of the regulating unit, the same movement of either the mirror or the camera, and the transformation which includes the correction are still preferably applied in this embodiment.

While in the foregoing specification this invention has been described in relation to certain preferred embodiments thereof and many details have been set forth for purpose of illustration, it will be apparent to those skilled in the art that the invention is susceptible to additional embodiments and that certain of the details described herein can be varied considerably without departing from the basic principles of the invention.