Technical Field

The present invention relates to an optical arrangement and control method. More particularly, the present invention relates to an optical arrangement comprising a curved mirror and an adaptive optical element, and a control method of the optical arrangement.

Background

Panoramic cameras have been developed which use a catadioptric design, as shown schematically in Fig. l, to capture a 360 degree panoramic view in real-time using a single camera. This prior art imaging apparatus uses a convex parabolic mirror 101 to capture light from a 360 degree arc in the horizontal plane and a wide range of angles in the vertical plane, such that the field of view comprises a segment of a sphere. A camera 102 located below the mirror is used to capture an image of the surface of the mirror, which can then be de-warped to reconstruct a panoramic field of view. One drawback of such panoramic cameras is that the resolution of the panoramic field of view that is recorded is limited by the resolution of the camera sensor. The invention is made in this context.

Summary of the Invention

According to a first aspect of the present invention, there is provided an optical assembly comprising: a curved reflector disposed on an optical path between a source of electromagnetic radiation and a target onto which the electromagnetic radiation is to be directed; an adaptive optical element disposed on the optical path between the source and the target, wherein the adaptive optical element is controllable to change an angle through which the electromagnetic radiation on the optical path is deflected by the adaptive optical element; and a controller configured to control the adaptive optical element so as to move a point at which the optical path intersects a surface of the curved reflector, thereby to produce a variation in a part of the optical path beyond the curved reflector.

By providing an adaptive optical element which can be controlled so as to move the point at which the optical path intersects the surface of the curved reflector, embodiments according to the first aspect of the invention are able to produce a larger variation in the optical path than could otherwise be achieved by moving or otherwise adapting the adaptive optical element alone at the same rate. That is, the curvature of the curved reflector serves to magnify any deflection in the optical path produced by the adaptive optical element.

In some embodiments according to the first aspect, the source of electromagnetic radiation comprises a scene to be imaged, the target comprises an imaging sensor arranged to capture an image of the scene reflected by the curved reflector, and the adaptive optical element is disposed at a point along the optical path between the curved reflector and the imaging sensor. In this way, the adaptive optical element can be controlled cause the imaging sensor to capture an image of a different part of the scene, by moving the point at which the optical path intersects the surface of the curved reflector.

In some embodiments according to the first aspect, the optical assembly further comprises an image processor configured to de-warp the image captured by the imaging sensor by using a de-warping algorithm to remove distortion according to a known curvature of a region of the curved reflector that is visible in the image, to obtain a de-warped image.

Furthermore, in some embodiments according to the first aspect, the image processor can be further configured to reconstruct a higher-resolution image of the scene from a plurality of lower-resolution images of different parts of the scene, each of the lower- resolution images being captured from a different part of the curved reflector. For example, the image processor maybe further configured to de-warp the plurality of lower-resolution images by applying an appropriate de-warping algorithm for each of the plurality of lower- resolution images, according to the particular part of the curved reflector that is visible in each lower-resolution image. Alternatively, the image processor maybe configured to combine the plurality of lower-resolution images and then de-warp the reconstructed higher-resolution image of the scene.

In some embodiments according to the first aspect, the adaptive optical element is configured to provide a variable focal length, and the controller is further configured to change the focal length of the adaptive optical element so as to provide an optical zoom function. In some embodiments according to the first aspect, the source comprises a point source of electromagnetic radiation and the adaptive optical element is disposed at a point along the optical path between the curved reflector and the point source of

electromagnetic radiation. In such embodiments, the controller can be configured to direct the electromagnetic radiation onto the target via the curved reflector by controlling the adaptive optical element to move the point at which the optical path intersects the surface of the curved reflector.

In some embodiments according to the first aspect, the source comprises a projection source configured to project an image, and the adaptive optical element is disposed at a point along the optical path between the curved reflector and the point source of electromagnetic radiation. In such embodiments, the controller can be configured to control a direction in which the image is projected and/or a size of the projected image by controlling the adaptive optical element to move the point at which the optical path intersects the surface of the curved reflector. Furthermore, the adaptive optical element may optionally be configured to provide a variable focal length, such that the controller can change the focal length of the adaptive optical element so as to increase or decrease the size of the projected image. In some embodiments according to the first aspect, the adaptive optical element comprises one or more of a deformable lens, a deformable mirror, a moveable lens, or a moveable mirror. The deformable or moveable mirror may have a curved surface, or may be a planar mirror. When the adaptive optical element comprises a deformable lens, the controller may further be configured to control the deformable lens to form an image of the scene on the sensor.

According to a second aspect of the present invention, there is provided a method of controlling an optical path between a source of electromagnetic radiation and a target onto which the electromagnetic radiation is to be directed in an optical assembly comprising a curved reflector disposed on the optical path, the optical assembly further comprising an adaptive optical element disposed on the optical path between the source and the target, wherein the adaptive optical element is controllable to change an angle through which the electromagnetic radiation on the optical path is deflected by the adaptive optical element, the method comprising: controlling the adaptive optical element so as to move a point at which the optical path intersects a surface of the curved reflector, thereby to produce a variation in a part of the optical path beyond the curved reflector. In some embodiments according to the second aspect, the source of electromagnetic radiation comprises a scene to be imaged, the target comprises an imaging sensor arranged to capture an image of the scene reflected by the curved reflector, and the method further comprises controlling the adaptive optical element to move the point at which the optical path intersects the surface of the curved reflector, so as to cause the imaging sensor to capture an image of a different part of the scene.

In some embodiments according to the second aspect, the method further comprises emulating a plurality of cameras by controlling the adaptive optical element and the imaging sensor to capture a plurality of images of a plurality of different parts of the scene, and outputting one or more of the plurality of captured images as an emulated camera output for one of the plurality of cameras.

According to a third aspect of the present invention, there is provided a computer- readable storage medium arranged to store computer program instructions which, when executed, perform a method according to the second aspect.

Brief Description of the Drawings

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure l illustrates a prior art panoramic camera;

Figure 2 illustrates an optical arrangement for enabling panning and tilting operations in a panoramic camera, according to an embodiment of the present invention;

Figure 3 illustrates an optical arrangement according to another embodiment of the present invention;

Figure 4 illustrates an optical arrangement comprising two secondary mirrors, according to an embodiment of the present invention;

Figure 5 illustrates an optical arrangement for projecting electromagnetic radiation onto a target, according to an embodiment of the present invention;

Figure 6 illustrates a deformable liquid lens comprising a segmented electrode, according to an embodiment of the present invention; Figure 7 is a flowchart showing a method of dewarping a zoomed image of a region of the curved reflector, according to an embodiment of the present invention; and

Figure 8 is a flowchart showing a method of dewarping and combining a plurality of lower-resolution images to obtain a higher-resolution image, according to an embodiment of the present invention.

Detailed Description

In the following detailed description, only certain exemplary embodiments of the present invention have been shown and described, simply by way of illustration. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive. Like reference numerals designate like elements throughout the specification.

Referring now to Fig. 1, an optical assembly is illustrated according to an embodiment of the present invention. The optical assembly comprises a curved reflector 201 disposed on an optical path between a source of electromagnetic radiation and a target onto which the electromagnetic radiation is to be directed. In the present embodiment the source is the scene which is to be imaged, and the target is an image sensor 203 which can be used to capture an image of the scene.

As shown in Fig. 1, the optical assembly of the present embodiment further comprises an adaptive optical element 202 disposed on the optical path between the scene and the image sensor 203. The adaptive optical element 202 is capable of being controlled to change the angle through which light is deflected by the adaptive optical element 202.

The optical assembly further comprises a controller 210 that is configured to control the adaptive optical element 202 in order to change the angle through which the adaptive optical element 202 deflects light. By controlling the adaptive optical element 202, the controller 210 can shift the optical path as it passes through the adaptive optical element 202, in order to move a point at which the optical path intersects the surface of the curved reflector 201. As shown in Fig. 1, the curvature of the surface of the curved reflector 201 effectively serves to magnify the deflection introduced by the adaptive optical element 202. In this way, a relatively small angular deflection in the optical path at the adaptive optical element 202 can result in a much larger angular deflection in a part of the optical path 221, 222 beyond the curved reflector 201.

Examples of types of adaptive optical elements that may be used in embodiments of the present invention include, but are not limited to: a deformable lens, for example a liquid lens; a deformable mirror, for example a micro-mirror array; a moveable lens; or a moveable mirror. One example of a deformable lens is a liquid lens 600, illustrated schematically in Fig. 6, in which a voltage can be applied to an annular electrode comprising a plurality of segments 601, 602, 603, 604, 605, 606, 607, 608 to control the shape of a liquid/ oil interface, allowing the focal length of the liquid lens to be adjusted. In some embodiments, the deformable lens may comprise a liquid lens with a single annular electrode rather than a segmented electrode as shown in the

embodiment of Fig. 6. In some embodiments, when the adaptive optical element is capable of providing a variable focal length, the controller can be further configured to change the focal length of the adaptive optical element so as to provide an optical zoom function.

In any particular embodiment, the adaptive optical element may include one or more individual elements, in any combination. Figure 3 illustrates an embodiment in which the adaptive optical element 302 comprises a planar mirror capable of being rotated to direct light from a different part of the surface of the curved reflector 301 onto the image sensor 303. Figure 4 illustrates an embodiment in which the adaptive optical element 402 comprises a plurality of planar mirrors 402a, 402b, one or both of which can be rotated to direct light from a different part of the surface of the curved reflector 401 onto the image sensor 403. The adaptive optical element may include one or more transmissive elements such as adaptive lenses, in addition to reflective elements. For example, in one embodiment the adaptive optical element can include a moveable mirror in combination with an adaptive lens to provide a combined pan/tilt and zoom function. The controller can adjust the angle of the mirror to move the point at which the optical path intersects the surface of the curved primary reflector, thereby imaging a different region of the primary reflector, and/or can adjust the focal length of the adaptive lens to provide an optical zoom function, thereby imaging a smaller or larger region of the primary reflector. A moveable lens or mirror may be implemented by mounting a conventional lens or mirror on a mechanism which is configured to move the lens or mirror, for example by rotating or translating the lens or mirror along one or more axes. In addition, when the adaptive optical element comprises a mirror, in some embodiments the mirror may have a curved surface configured to focus light. Furthermore, in some embodiments the adaptive optical element may comprise a convex mirror. The use of a convex mirror for the adaptive optical element can reduce the extent to which the field of view is obstructed by the adaptive optical element, since the mirror can be made smaller for the same function in comparison to a planar or concave mirror.

In the embodiments shown in Figs. 2 to 4, light travels along the optical path from the scene to the curved reflector 201, then to the adaptive optical element 202, and finally on to the image sensor 203. However, in other embodiments light may travel along the optical path in the opposite direction, as shown in Fig. 5. In this embodiment, light travels along the optical path from a source 503, which comprises a point source of electromagnetic radiation such as a light emitting diode (LED) or laser, to the adaptive optical element 502, then to the curved reflector 501, and finally onto a target. For example, in this embodiment the target may be an object in a scene which is reflected in the curved reflector 501, and a controller 510 can be configured to direct the

electromagnetic radiation onto the target via the curved reflector 501 by controlling the adaptive optical element 502 to move the point at which the optical path intersects the surface of the curved reflector 501.

This embodiment applies a similar principle to that used in the embodiment of Figs. 2 to 4, in that a relatively small deflection in the optical path at the adaptive optical element 502 is translated into a much larger variation in part of the optical path 521, 522 beyond the curved reflector 501. However, as stated above, in this embodiment light travels in the opposite direction along the optical path.

As shown in Fig. 5, in this embodiment the controller 510 can omit the image processor since an image is not being formed in this embodiment. The controller 510 may comprise an optical element controller 511 for controlling the adaptive optical element 502, and a memory 512 arranged to store instructions which tell the optical element controller 511 how to configure the adaptive optical element 502 in order to direct light from the source 503 onto any given target location that is visible in the surface of the curved reflector 501. In another related embodiment similar to the one shown in Fig. 5, light from a projection source can be directed onto a scene via the surface of the curved reflector 501 to project an image into the scene. In this embodiment, the point source 503 is replaced with a source capable of forming an image, for example a digital image projector. In such embodiments, the adaptive optical element 502 may be configured to provide a variable focal length, and the controller 510 may be further configured to change the focal length of the adaptive optical element 502 so as to increase or decrease the size of the projected image. In addition, in some embodiments the controller 510 may further include an image pre-processor which applies a pre-distortion to an input image that is to be projected, to account for distortion introduced by the curvature of the curved reflector 501. The pre-distortion can be applied using a transformation that is the inverse of the de-warping algorithms applied in the above-described

embodiments used for imaging.

Additionally, in some embodiments the adaptive optical element 202 can be configured to focus light onto the image sensor 203. When the adaptive optical element 202 is capable of focussing light to form an image on the image sensor 203, a separate objective lens may be omitted to reduce the complexity and cost of the optical assembly. For example, the controller 210 may be configured to control a deformable lens or mirror included in the adaptive optical element 202 to form an image of the scene on the image sensor 203. Alternatively, in other embodiments, a separate objective lens maybe provided between the adaptive optical element 202 and the image sensor 203 to form an image on the image sensor 202. For example, the image sensor 203 may be included in a conventional camera which includes one or more objective lenses.

Continuing with reference to Fig. 2, in the illustrated embodiment light travels along the optical path from the scene to the curved reflector 201, then to the adaptive optical element 202, and finally on to the image sensor 203. Since the adaptive optical element 202 is disposed at a point along the optical path between the curved reflector 201 and the imaging sensor 203, by controlling the adaptive optical element 202 to move the point at which the optical path intersects the surface of the curved reflector 201 the controller 210 can cause the imaging sensor 203 to capture an image of a different part of the scene that is reflected in a different part of the surface of the curved reflector 201. In the present embodiment, as shown in Fig. 2, the controller 210 further comprises an optical element controller 211 to control the adaptive optical element 202, and an image processor 212 configured to de-warp the image captured by the imaging sensor 203. In the present embodiment the controller 210 further comprises a memory 213 arranged to store a plurality of predefined de-warping algorithms each associated with a different configuration of the adaptive optical element 202. The de-warping algorithms are configured to remove distortion according to a known curvature of a region of the curved reflector that is visible in the image, to obtain a de-warped image. Each of the predefined de-warping algorithms can be calculated in advance from a known curvature of the part of the surface of the curved reflector 201 that will be imaged by the image sensor 203 when the adaptive optical element 202 is in a particular

configuration. The memory 213 may further be arranged to store computer program instructions which, when executed by the controller 210, perform any of the methods disclosed herein.

In the present embodiment the image processor 212 may implement a method as shown in Fig. 7 to de-warp an image captured by the image sensor 203. First, in step S701 the image processor 212 receives the image from the image sensor 203. In the as- captured image, the scene will be distorted due to the curvature of the surface of the curved reflector 201. In step S702, the image processor 212 communicates with the optical element controller 211 to check which part of the curved reflector 201 is visible in the current image. Then, in step S703 the appropriate de-warping algorithm associated with the identified part is retrieved from the memory 213, and is applied to the image in step S704 to remove the distortion caused by curvature of the reflector 201. Finally, in step S705 the de-warped image is outputted. For example, the de- warped image maybe sent to a display unit, and/or uploaded to a server, and/or stored in the memory 213.

Although in the present embodiment the controller 210 processes a captured image to remove distortion, in other embodiments the as-captured image may be outputted without applying a de-warping algorithm. For example, the as-captured image may be displayed on a display unit, and/or may be uploaded to a server and/or stored in memory. A stored copy of the as-captured image could still be de-warped at a later stage if a distortion-free image was required. Alternatively, in other embodiments a predefined de-warping algorithm suitable for the current configuration of the optical assembly may be retrieved from remote storage, for example downloaded from an Internet server. As a further alternative, in some embodiments the image processing unit 212 maybe pre-programmed with information defining the surface geometry of the curved reflector 201, and may calculate the de- warping algorithm on an as-needed basis. For example, if the surface of the curved reflector 201 can be described mathematically by sweeping a smooth curve, such as a parabola, around a known axis, the memory 213 of the controller 210 may store equations which define the curve and the axis of rotation, enabling the image processor 212 to compute a suitable de-warping algorithm for a given region of the curved reflector 201. De-warping algorithms for removing such distortion from images are known in the art, and a detailed description will not be provided here so as not to obscure the present inventive concept. Additionally, in some embodiments the image processor 212 can be further configured to reconstruct a higher-resolution image of the scene from a plurality of lower- resolution images of different parts of the scene, each of the lower-resolution images being captured from a different part of the curved reflector 201. A method of constructing a higher- resolution image from a plurality of lower- resolution images is shown in Fig. 8.

First, in step S801 the image processor 212 obtains a plurality of lower-resolution images that have been capture by the image sensor 203. Then, in step S802 the image processor 212 checks which region of the curved reflector 201 is visible in each of the lower-resolution images, retrieves the appropriate de-warping algorithms from the memory 213 in step S803, and applies the relevant algorithm to each image in step S804 to de-warp the plurality of lower-resolution images. Finally, in step S805 the image processor 212 combines the plurality of de-warped lower-resolution images into a higher-resolution image of the scene. In an alternative method the order of the de- warping and combining steps may be reversed, such that image processor 212 first combines the plurality of lower- resolution images into a single higher- resolution image and then de-warps the combined image.

By combining an adaptive optical element with a curved reflector in an optical assembly, as described above, embodiments of the present invention can provide dynamic minimally-mechanical or non-mechanical panoramic view control systems. For example, a minimally mechanical system according to an embodiment of the present invention can be created by using a mirror tilt view controller to move the adaptive optical element in order to provide a pan/tilt function. A non-mechanical system according to an embodiment of the present invention can be created using a deformable lens, such as a liquid lens whose shape is controlled and near-instantly changed by electrowetting as described above in relation to Fig. 6. Such non- mechanical optical assemblies can achieve greater durability, dependability and longer operational system lifespan than mechanical systems. A further advantage arises when a rapid pan/tilt mechanism as described above, such as a moveable secondary mirror, is combined with a rapid zoom mechanism such as a liquid lens to enable a near-instantaneous zoom adjustment. This allows the camera to output a combination of low- resolution large field-of-view images by setting the zoom level to image substantially all of the primary mirror, and high-resolution smaller field- of-view images by setting the zoom level to image a small region of the primary mirror and panning/tilting across the surface of the primary mirror as described above. The combination of low- resolution large field-of-view images, and high-resolution small field-of-view images, can be interleaved at the ultimate frame rate of the image sensor system. This arrangement allows the system to emulate a number of independent lower frame rate pan-tilt-zoom (PTZ) cameras, each capable of scanning the entire field of view and updating position at frame rate.

Whilst certain embodiments of the invention have been described herein with reference to the drawings, it will be understood that many variations and modifications will be possible without departing from the scope of the invention as defined in the

accompanying claims.