This invention relates to a lens system and to an improved arrangement thereof.

It is well known to provide a combination of a camera with a lens for inspection of a portion of an article. It is also known to provide a variable angle mirror in the system in order to change and control a portion of an object or a region of an object that the camera is directed to. Such arrangements have typically been used in product control systems such as for inspection of a surface, confirmation of the presence of a component or reading of a barcode or other information on the article.

It is known to provide means of changing the angle of view of the lens by controlling the position and angle of the mirror. It is a known problem that if the angle of the mirror is changed then a distance to the article may also change. The problem exists for articles having a number of surfaces and a 3D shape but is also a problem with 2D surfaces where the effect of perspective view can increase the distance between the lens and at least some parts of the article or the surface. One solution to the problem of parts of an image being at different distances from the mirror and the lens is to increase a depth of field of the image produced by a camera. This may be carried out by reducing a size of iris in the camera. A disadvantage of this solution is that stopping down the iris may only have a limited effect in increasing the depth of field of the image and in addition will reduce the brightness of the image outputted from the camera. Such a reduction in brightness may not be acceptable in an inspection system in which the light available may be limited.

It is an object of the invention to provide a system that is better adapted to provide images of an object or feature at varying distances from the camera.

According to a first aspect of the invention there is a provided an imaging system in accordance with claim 1 of the appended claims.

The system comprises a lens and a variable angle beam deflector in which the lens comprises a variable focus lens. In some embodiments the sensor may be a camera image sensor. In other embodiments the sensor may be a human eye. The system is preferably arranged to produce an image of an area or region on the sensor.

In other embodiments the sensor may be a 2D array such as an area scan. In other embodiments the sensor may be a ID array such as a linescan sensor.

The variable focus lens may comprise a variable focus lens system. The variable focus lens or lens system is desirably arranged to change the focus for the image sensor.

In some embodiments the variable focus lens may be a mechanical system. The lens system may be motorised. The variable focal lens may comprise a telecentric optical system. The skilled person will appreciate that a telecentric optical system comprises an array of lens and in which an aperture is located at the focal plane of a front optical group of lenses of the telecentric lens [system]. The skilled person will understand that in a telecentric optical systems the distortion degree of the systems has a low distortion degree that may in the range of 0.01% to 1.0% and more preferably the distortion is around 0.1% (from http s : /7 ^' www .opto - engineering.com/index.php?/resources/telecentric-ienses-tuto rial/)

It will be appreciated that the mechanical systems tend to be large and slow. A physically large variable focus lens may make the assembly larger and heavier. A slow focus speed will limit applications in some cases. Such lenses are established technology and the operation of such mechanical motorised variable focus lenses is well known. Further detailed discussion is not required as the arrangement may be conventional and well known to the person skilled in the art.

The skilled person will appreciate that the choice of a particular variable focus lens or lens array is dependent on the use to which the system will be put and the parameters for selecting a suitable variable focus lens or lens array is well known.

In an alternative embodiment the variable focus lens may be a liquid lens. A liquid lens may be arranged to change focus very fast. A liquid lens may change focus within 10ms. A liquid lens may comprise a focussing liquid in a membrane with a control method arranged to change the optical power of the lens.

A typical liquid lens may comprise a core element consisting of a container which is filled with an optical liquid and is sealed with a thin polymer membrane. A control means is provided that shapes the membrane to provide a tunable lens. The membrane may be deflected by movement of the control means which may for example be a ring that deflects the membrane. Alternatively liquid can be pumped in or out of the container in other embodiments. It will be appreciated that a change in the lens radius of several micrometres can have the same optical effect as moving the entire lens several centimetres and therefore liquid lens optical systems can be designed to be more compact. In addition it has been found that liquid lenses have a rapid response time which may be in the order of a few milliseconds.

There are various types of liquid lens technology. For example; Optotune™ use a membrane which is distorted by an electrical coil; Varioptic™ distort a liquid bubble using a voltage; Tag Optics™ oscillates the liquid lens and then precisely times the image acquisition when the lens is at the required optical power.

In some embodiments the Optotune™ and Varioptic™ technologies may be used for image sensor or human eye applications.

In other embodiments the Tag Optics™ technology may be used with an image sensor. The Tag Optics™ is preferably used in combination with an image sensor and control system to control the time at which the exposure of the image sensor occurs.

In some embodiments liquid lenses are preferred because they are compact and can change focus quickly. Liquid lenses have no known maximum number of focus operations.

In some embodiments the variable focus lens system further comprises a fixed focus lens. The fixed focus length may be provided in combination with a variable focus lens which adjusts the focus length of the overall lens system. The system is preferably arranged such that changing an angle of the variable angle beam deflector changes a region or area of view of the camera.

In other systems the region or area of view may be changed by movement of the imaging system. The imaging system may be mounted on or be a part of a robotic arm for example. In other embodiments the imaging system may be mounted on or be a part of a drone.

Desirably the beam deflector is arranged such that it can change a direction of an optical path passing through the system. The beam deflector may be arranged such that adjusting the beam deflector changes the area viewed by the sensor.

In some embodiments the variable angle beam deflector may comprise one or more variable angle mirrors. In a preferred embodiment the beam deflector comprises one or two variable angle mirrors. A beam deflector with one mirror may be arranged to provide a one dimensional or two dimensional deflection of the optical path. A beam deflector with two mirrors may be arranged to provide a two dimensional deflection of the optical path.

Desirably an angle of deflection of the beam deflector may be varied by signals output from a control system. The control system may be arranged to output control signals that adjust a position of the or each variable angle mirror. In a preferred embodiment the beam deflector comprises a mirror angle adjustment mechanism. Desirably the mirror angle adjustment mechanism may comprise a drive mechanism. The drive mechanism may comprise a galvanometer. The galvanometer may comprise a coil with a variable current flowing through the coil in order to change the angle of the mirror.

In one embodiment the system comprises a two mirror system and comprises two variable angle mirrors arranged such that the optical path can be deflected in two dimensions. The system may be arranged such that a first mirror is arranged to deflect the optical path in a first axis. The first mirror may be arranged to be round. The deflected optical path may be arranged to be deflected by a second mirror. The second mirror may deflect the optical path subsequent to the first mirror. The second mirror may be arranged to be elongated in order to cover the range of optical paths from the first mirror. It will be appreciated by the skilled person that the two mirror arrangement is such that the optical path may be altered in two directions so as to provide an image of a larger area to the sensor.

Typically galvanometer controlled variable angle mirrors are arranged to change the angle of the mirror rapidly. A typical speed may be up to 20 000 times per second.

In another embodiment a single mirror system may be utilised. In a preferred single mirror system the system is arranged to change the angle of the single mirror in one or two dimensions. It will be appreciated that a single mirror system may be more desirable in that the overall assembly of the system may be arranged to be relatively more compact than a two mirror system. A single mirror system may be arranged to scan a target area in one or two dimensions depending on whether the mirror can be deflected in one or two directions.

A mirror system may be used in combination with a laser beam. In some embodiments a laser beam may be utilised in a system in which a relatively narrow optical path is deflected.

In another embodiment the variable angle beam deflector may comprise a variable angle prism. An example of a variable angle prism is an Optotune™ tunable prism TP- 12- 16. The skilled person will appreciate that other variable angle beam deflectors may be utilised. It will be appreciated that the system utilised has to be arranged to provide an optical path that is wide enough to allow a full image of the target area to be provided to the sensor for some or all of the deflection positions to the deflector.

Means may be provided for controlling a brightness of an image created on the sensor. In some embodiments a fixed iris may be provided and arranged to reduce a brightness of the image. A fixed iris may provide an additional advantage in increasing the depth of field of the image. It will be understood by the skilled person that the depth of field refers to a range of distances which are in focus when the image of the target area is formed on the sensor.

It will be appreciated that in some systems the brightness of an image may change by a large degree. A change in brightness may be predictable or may be unpredictable. An example of a predictable change is one in which at least a first and a second target area are being inspected. It could be that a first area is dark and does not reflect much light and the second area is reflective. A typical application having predictable brightness changes may be a machine vision application in which first and second areas of a product are being inspected.

Unpredictable changes may occur when the system is used for example in a drone application where a target area may be in sunlight or shade or where sunlight is variable and/or directional.

In some embodiments a fixed iris may be closed such that the brightest images formed are not too bright for the sensor. It will be appreciated that if the image is too bright the sensor may become burnt out or otherwise damaged. A disadvantage of such an arrangement is that many parts of the image may be dark and only a small part of the dynamic ranged of the sensor is utilised.

The system further comprises an adjustable aperture and desirably a diameter of the iris may be controlled by the control system.

In a preferred embodiment the iris may comprise a P-iris. A P-iris may have a mechanism arranged to adjust an aperture of the P-iris. The mechanism may comprise a stepper motor. A stepper motor may provide precise control of an aperture of the iris. Desirably the control system may be arranged to utilise feed-back signals to make an intelligent decision with regard to the brightness of the image created on the sensor. The system may be arranged to optimise brightness in a selected region of the image.

In some systems a brightness of the image may be controlled by changing an exposure time of the sensor.

In some embodiments the variable focus lens system may comprise an optical stack comprising a fixed lens, a variable focus lens, a variable angle beam deflector and an iris in combination with a camera. The focal distance of the fixed lens may be defined as a distance between the fixed lens and the camera sensor. The optical stack may comprise of other optical components which may be arranged in other orders. A selected order for the remaining components may be determined by requirements of the particular application for which the system is to be used. In some embodiments the other optical components may be arranged in one of two orders.

In systems having a fixed lens having a short focal length the fixed lens may be arranged to be located between the variable focus lens and the sensor.

In systems having a fixed lens with a long focal length the variable focus lens may be arranged to be located between the fixed lens and the sensor.

In some systems spacer tubes may be arranged between the variable focus lens and the camera in order to extend the range of the fixed lens focal lengths that can be used in the system.

It may be desirable to place the iris between the variable focus lens and the fixed lens.

The skilled person will appreciate that selection of appropriate lens will be made in view of the diameter and position of the variable angle beam deflector and a size of the camera sensor. It will be appreciated that the selection should also take into account the necessity of avoiding occlusions in the image formed on the sensor.

In other applications the geometry of the lens system may be varied. In another embodiment the mirror may be arranged to be perpendicular to the optical axis of the camera. In such an arrangement the region or area to be viewed may be around the lens assembly. The region or area to be viewed may be 360° around the lens assembly. Such an arrangement may be suitable for inspection of an interior of a pipe for example.

In some embodiments the imaging system may further comprise a light source.

In some embodiments light source may be provided to be“on-axis”. An on-axis light source is arranged to illuminate the region being viewed. Desirably, as the region of view is changed the illuminated region changes with the movement. It may be desirable to use on-axis lighting for imaging systems used in drone systems or as an imaging system for robot guidance. It will be appreciated that on-axis lighting provides“brightfield” illumination. In brightfield illumination light can be directly reflected back into the camera. On axis illumination may be provided by using a half-silvered mirror to insert the light source path into the optical path. Use of a half-silvered mirror can increase the physical size of the assembly and increase the distance from the sensor to the mirror, which may be restrictive to the optical properties of the system.

In an alternative embodiment on-axis lighting may be provided by a light located near to the sensor, for example a ring or square light. Use of a ring or square light around the sensor does not increase the distance to the mirror and has the advantage that the system may be compact. The light ring or square has to be arranged such that the lens system is not in the light path and that sufficient light is reflected by the mirror. In other embodiments it may be desirable to provide other lighting schemes. In some embodiments the lighting may be dark field illumination. Dark field illumination is where light is directed to the subject from a low angle so that only scattered light reaches the sensor. It will be appreciated that dark field illumination has to be provided close to the plane of the subject and therefore might not be a part of the lens system assembly.

It may be desirable to control the wavelength of the light used in the imaging system.

In some applications it may be desirable to use a narrow range of wavelengths. In some applications a contrast between features can be improved by using a narrow range of wavelengths. Certain machine vision systems may use a single colour light to illuminate the features that are needed. For example in applications where a red text is provided on a white background then a blue light may provide better contrast for imaging the text than using a white light.

In other embodiments multiple wavelengths may be used. In some embodiments that different wavelengths may be used together. In other embodiments the different wavelengths may be used separately. In some embodiments a colour image may be created using a monochrome camera by illuminating the region with red, green and blue light in turn and combining the three images. In other embodiments it may be desirable to use a particular wavelength of light. The variable focus lens may be used to ensure that a particular wavelength of light is in focus on the sensor.

In some embodiments non-visible wavelengths of light may be used. Near infra-red wavelengths may be used in some embodiments and applications. For example near infra-red wavelengths may be used in inspection of fruit in order to identify damaged areas of the fruit.

In some embodiments the system comprises a product inspection system. In other embodiments the system may comprise a security system. The system may be arranged carry out interior inspections. In some embodiments the mirror may be arranged to be located perpendicular to the optical axis of the camera. In some embodiments the system may be arranged to carry out interior inspections such as an interior of a pipe. The system may be arranged to view 360° around the assembly.

In other embodiments the system may be arranged to carry out scanning to identify objects. There are many applications where objects need to be tracked by an imaging system. In some embodiments a single object may need tracking over a fairly wide area. In other embodiments multiple objects may be in view at one time. Examples of such systems include components being moved by a conveyor belt; vegetables being sorted as they fall through a grading system; particles moving in a fluid. Conventionally these systems have used one or more high resolution cameras to cover the region of interest. With an imaging system in accordance with the invention a single sensor can be used and the beam deflector used to scan the view across the region of interest. In some embodiments the image processing control system can compensate for the fact that images have been taken at slightly different times. It will be appreciated that this technique may be more appropriate for systems which don’t need a very high frame rate. In some embodiments the imaging system in accordance with the invention may be used in security and surveillance applications. Typically security applications have utilised motorised pan and tilt mechanisms.

The present invention is typically more compact than conventional systems and has fewer moving parts. Accordingly it will be more reliable and able to change viewing direction more quickly than traditional systems.

As with other applications, the variable focus lens preferably allows the iris to be opened so that the system can be much more sensitive to light. It will be appreciated that this facility is very valuable in surveillance applications.

According to another aspect of the present invention there is provided a robot in accordance with claim 17 of the appended claims.

In some embodiments the imaging system may be used for robot guidance. There are many arrangements but a typical one is one in which components arrive on a conveyor belt. The components may arrive in random positions or may arrive in specific orientations. A camera may be attached to the robot arm so that a direction of view of the camera can be directed by the robot. Output from the camera to the robot can be used to locate the component for the robot. The imaging system may be arranged to output orientation data. The robot may be arranged to use the output data from the imaging system to pick up the component and then perform an operation on the component.

In other embodiments an imaging system in accordance with the invention may be mounted at a static position to provide guidance information for a robot. An advantage of the invention is that the area or region that can be imaged may be wider than with conventional systems and further resolution may be higher as the imaging system may be arranged to take a number of images in a range of directions.

It will be appreciated that robot arms have a lot of mass and so move relatively slowly such that it takes time to adjust the direction of the camera to view the component. In conventional systems the camera exposure time will limit the speed at which the arm can move. An imaging system in accordance with the invention may be arranged to compensate for some of the limitations of a robot arm system. The imaging system can change its direction of view much more quickly using the beam deflector. In addition the change of direction of view of the imaging system using the beam deflector is at least in part independent of the direction of the robot arm. A position of the next component can be detected much earlier. In some embodiments the next compound may for example be detected while the previous component is still being handled or while the arm is returning to the conveyor location.

In some embodiments multiple images may be utilised to determine the orientation of a component. For example, when the top surface has been identified, the imaging system must view that surface to establish a rotational orientation, so that a mounting hole can be presented in the right position. In the present imaging system a camera angle can be adjusted much faster, speeding up this operation.

In some embodiments the present imaging system allows images to be taken while the robot arm is moving. The control system may be arranged to utilise a known motion velocity of the motion of the robot arm. The control system can factor the known motion to change a of view of the imaging system and to output control signals to control a focal distance of the imaging system in order to maintain a view of the same location. It will be appreciated that maintaining a view of the same location may be arranged to save time and/or allows exposure times to be longer and/or allows multiple inspections images to be made.

In some embodiments the imaging system may be used to provide guidance information to a drone. In some drones the payload may be a camera arranged to take images of a structure or scene for viewing remotely. A guidance camera providing images for guiding the drone generally need to focus at a long distance. In contrast inspection cameras often need to take images at close range in order to provide a good resolution for detecting flaws. Conventionally two cameras have been needed. A drone comprising an imaging system in accordance with the invention may be arranged such that the imaging system may be used for guidance and for inspection by changing the focus of the variable lens.

In some embodiments, whether the imaging system is used to provide guidance or not, the imaging system can be used to make multiple images of a scene, which are then joined together to provide one composite image which may be of higher resolution than can be achieved with a single image view.

In some embodiments the variable angle mirror may be used to maintain a constant view of the region or area of interest. The control system may be arranged to compensate for any known movement of the drone. Known movement may comprise vibration or movement of the drone at a known speed relative to the view. In this way the imaging system can use a relatively longer exposure resulting in clearer and brighter images. It will be appreciated that stable image technology which removes vibration from camera images is known technology.

In some embodiments the image sensor may comprise a human eye. A single imaging system may provide monocular vision. Two imaging system may provide binocular vision.

The imaging system may for example be used in a machine that is used to direct a user’s view to a certain place. For example a PCB inspection machine might want the operator to view components with potential defects identified by Automated Optical Inspection system. The user’s view could be directed to each area in turn by the imaging system, without the user having to search for the area with a potential defect. The variable focus lens of the imaging system can be used to prevent the user from having to refocus. One problem might be that the system causes feelings of nausea.

In some embodiments the imaging system may be used in combination with traffic monitoring system. In some embodiments the imaging system may be used in car park entry systems and for enforcement of speed limits, special lane enforcement and red traffic lights. Typically vehicles can be in a variety of positions, for example in different lanes, at a variety of distances from the imaging system. Traffic systems are often short of light, especially when working at night. Additional strobe lights are often used, but these are large and any visible wavelengths can distract a driver. For normal road speeds there is a limit to the exposure time when the vehicle is moving, which is typically 1ms.

In a system in accordance with the invention the direction of view can be changed so that vehicles can be viewed in a variety of locations, including on different lanes. As the direction of view can be changed the field of view does not need to be as wide as in conventional systems. Consequently the image resolution can be higher, providing better images and resulting in a higher success rate reading licence plates.

In an imaging system the variable focus lens can be used to change the camera focus, so that vehicles are in focus at different distances from the camera. The change in focus may in some embodiments be an autofocus operation, but more commonly the distance to the vehicle is determined in some way (for example from a vehicle sensor or laser distance sensor or an initial out of focus camera image) and then the lens focus is changed to the known position of the vehicle. It will be appreciated that as the focus can be set, the iris can be opened wider because the depth of field can be shorter and less light is needed to achieve a suitable image. The imaging system may be arranged to track moving vehicles. The imaging system may be arranged to increase an exposure time without introducing motion blur. In addition multiple images can be taken of the same vehicle. The system may be used when separate images of the licence plate, vehicle model and driver are needed.

According to another aspect of the invention there is provided a vehicle in accordance with claim 16 of the appended claims.

Further applications and uses of the system will be described in more detail below with reference to the accompanying Figures in which:

Figure 1 is a sketch of the arrangement of an imaging system in accordance with the invention;

Figure 2 is a sketch of an arrangement of an alternative imaging system;

Figure 3 is a sketch of an alternative embodiment in accordance with the invention;

Figure 4 is typical example of a commercially available beam deflector having a 2D galvanometer mirror with the axes of rotation shown;

Figure 5 is a sketch of an arrangement using a mirror system with 2 axes of rotation; Figure 6 is a sketch of an arrangement using a mirror system with 1 axes of rotation;

Figure 7 is a sketch of an arrangement having on-axis light illumination;

Figure 8a and Figure 8b are perspective and side illustrations of a product arranged for inspection by a machine vision system comprising an imaging system in accordance with the present invention showing image 1 and image 2 at differing heights on the product;

Figure 9 is an illustration of an alternative arrangement to be scanned;

Figure 10 is an illustration of a machine vision system;

Figure 11 is a sketch of an alternative arrangement embodying two imaging systems in accordance with the invention;

Figure 12 is an alternative embodiment of an imaging system;

Figure 13 is an alternative embodiment

The invention comprises an imaging system 1 having a lens 2 and a variable angle beam deflector 4. The lens 2 in the embodiment illustrated in Figure 1 comprises a variable focus lens 2 and the imaging system is arranged to form an image of the subject area 6 on a sensor 8.

In some embodiments the sensor 8 may be a camera image sensor. In some embodiments the camera image sensor may be provided in a camera (not shown), that is an apparatus arranged to capture and record an image. In some arrangements the sensor 8 may be a 2D array such as an area scan.

In other embodiments, such as that illustrated in Figure 2 the sensor 8 may be a 1 dimensional (ID) array such as a linescan 10. The linescan may 10 comprise a ID array of pixels. In some embodiments a linescan 10 can be used to build up an image of an object without the object moving relative to the imaging system. An advantage of this arrangement is that it can provide area scan image sensor functionality using a linescan sensor.

The lens 2 comprises a variable focus lens system, comprising a variable focus lens 3 and a fixed lens 5. The variable focus lens 3 is arranged to change the focal length of the image system for the image sensor 8. In a preferred embodiment the variable focus lens 3 is a liquid lens. The liquid lens may be selected from a number of existing liquid lenses which are commercially available. Optotune™ lenses use a membrane which is distorted by an electrical coil. Varioptic™ lenses distort a liquid bubble using a voltage. Tag Optics™ oscillates the liquid lens and then precisely times an image acquisition to when the lens is at the required optical power.

In this embodiment a variable iris 7 is provided between the variable focus lens and the fixed lens. The variable iris may be a p iris.

The imaging system further comprises a beam deflector 4 which is arranged to change the direction of an optical path between a subject area 12 and the sensor 8. The sensor is a linescan sensor. In the arrangement of Figure 2 the beam deflector 4 is moveable about one axis 14.

In other embodiments such as the arrangement illustrated in Figure 3 the beam deflector 4 comprises an arrangement in which the beam deflector is moveable about a first axis 16 and a second axis 18. In the arrangement of Figure 3 the system comprises a first mirror 20 arranged to rotate about the first axis 16 and a second mirror 22 arranged to rotate about the second axis 18. Movement of the first and second mirrors moves the subject area 24 that is imaged on the sensor 8. Movement of the first mirror 20 moves the subject area 24 in the direction indicated at A and movement of the second mirror moves the subject area in the direction indicated at B.

In many embodiments the moveable deflector is arranged to comprise a galvanometer 26, as illustrated in Figure 4. A galvanometer 26 typically comprises a coil 28 with a variable current passing through the coil 28 in order to change the angle of the mirror 30 about a first axis 32 and a second axis 34. A galvanometer can change the direction of the optical path at speeds that may be up to a typical speed of 20,000 times per second. It will be appreciated that a single mirror can be arranged so as to change the angle of the mirror in one or two dimensions. It will be appreciated that a single mirror system is preferable to a two mirror system as the overall assembly may be more compact.

Figure 5 illustrates a number of imaged areas 12 of a subject area 36 which can be captured by an imaging system in which the 2D mirror system 38 has two axes 40, 42 of rotation.

Figure 5 illustrates a number of imaged 12 areas of a subject area 36 which can be captured by an imaging system in which the ID mirror system 38 has a single axis 44of rotation.

In another embodiment the beam deflection may be achieved by a variable angle prism. A suitable variable angle prism is the Optotune tuneable prism TP- 12- 16. It will be appreciated that using a beam deflector increases the subject area over which an image can be captured. The skilled person will appreciate that the choice of lenses will be dictated by the diameter and position of the variable angle beam deflector and by the size of the camera sensor. It is desirable for example to ensure that a portion of the subject area is not occluded. The skilled person will understand that in some arrangements it is possible for the image to be partially obstructed or an image is not formed and in this sense the image is occluded. In this field the skilled person understands an occlusion to occur when a subject cannot be viewed due to a property of the sensor set up or an event in the subject area. In a comparative example a camera with a 1/3” sensor may be used with an Optotune MR-15-30 variable angle beam deflector. In this arrangement fixed lenses of focal length greater than 15mm are likely to be needed to provide an image free of occlusion.

In an embodiment an Optotune EL-16-40-TC liquid lens may be used within this optical arrangement (1/3” sensor with an Optotune MR-15-30 variable angle beam deflector). In this embodiment the focal point of the overall assembly can be adjusted from a few centimetres to almost infinity providing a significant advantage in the range of focal point that can be achieved within the compact optical arrangement. Other variable focus lenses may be used which will provide other ranges of focal point provided the diameter of the variable angle beam deflector is adequate for the field of view in order to avoid occlusion of portions of the subject area.

Some applications may require high magnification in the centre of the image and the skilled person will appreciate that in these applications the achievement of high magnification in the centre of the image is of more importance than the avoidance of occlusion of portions of the subject area.

In some embodiments the camera may be arranged to have an embodiment for a DC- iris. It will be understood that a DC-iris is open when unpowered and has a galvo mechanism arranged to close the iris. It has been found that such arrangements can be imprecise and usually require a closed feedback system to give accurate results as such systems have been found to be prone to“stiction” which can cause overshoot of the desired opening of the iris. It is well known that the camera acquires an image and outputs a signal indicative of the brightness of the image to the DC-iris. The closed loop solution between the camera and iris is simple but not flexible and the camera is arranged to decide what is or is not an acceptable brightness. It has been found that such arrangements can be confused by bright elements in the image or subject area.

The imaging system comprises a variable iris. It is preferred to use P-irises in which a mechanism adjusting a diameter of the iris is a stepper motor and so provides specific control of the aperture.

The imaging system is arranged to comprise a control system that receives data inputs from sensors in the system and outputs control signals to components of the system. The control system is arranged to utilise feed-back signals from sensors in the system to make an intelligent decision with regard to the brightness of the image created on the sensor. The control system may be arranged to optimise brightness in a selected region of the image.

In some systems a brightness of the image may be controlled by changing an exposure time of the sensor. In some embodiments the imaging system may further comprise a light source. An example embodiment of a system comprising a light source is illustrated in Figure 7. In embodiment the light source 46 is provided to be“on-axis”. An on-axis light source 46 is arranged to illuminate the image area 12 being viewed. Desirably, as the region of view is changed the illuminated region changes with the movement such that the light illuminates the imaged area 12. On-axis lighting may be particularly useful for imaging systems used in drone systems. On-axis lighting may be used in an imaging system for robot guidance.

On-axis lighting provides “brightfield” illumination in which illumination light is directly reflected back into the camera. The on axis illumination is provided by using a half-silvered mirror 48 to insert the light source path into the optical path.

It will be appreciated that in other embodiments on-axis lighting may be provided by a light located near to the sensor, for example a ring or square light. The light ring or square has to be arranged such that the lens system is not in the light path and such that sufficient light is reflected by the mirror.

The control system will now be described in more detail. The control system is arranged to set the positions of all controllable components of the system including the variable focus lens, beam deflector, iris and lighting. Typically the control system is also able to control when the image sensor takes an image and the exposure time.

The control system can be arranged to take no part in the handling of images from the imaging system and maybe concerned solely with the image capture.

Alternatively the control system could be arranged to control image capture and to store the images acquired.

Another alternative is that the processing and interpretation of the images could be an integral part of the control system. In such an arrangement the results of the image interpretation can be used to inform the control system and to control the acquisition of further images. For example, if part of a feature is detected in an image of a subject area, the control system can be arranged to direct the system such that the subject area tracks along the feature to acquire further images of the feature and preferably may be arranged to acquire images of a whole extent of the feature.

The control system comprises data storage means. Images captured by the image sensor are processed and/or stored in the control system. It will be appreciated that in some embodiments the data storage means may be remote from the imaging system. For example a remote PC may comprise the data storage means and the imaging system may be provided with communication means adapted to communicate with the remote PC. In other embodiments the imaging system may be arranged to comprise an image processing means in the imaging system assembly. In some embodiments the image processing means may comprise a processing module which may be arranged to be a part of the control system. In other embodiments the image processing means may be integrated with the image sensor. In some embodiments in which the image processing means is integrated with the image sensor it may be provided in a“smart camera”.

An advantage of the control system is that it provides flexibility in the operation of the whole system. Typically any component can be set to any position within the subject area. Actual operation of the imaging system will depend on the application for which the imaging system is being used. Specific types of operation are given below. Examples of applications for which the imaging system may be used are also provided later. Some general principles and techniques are given here.

In operation the beam deflector and variable focus lens generally operate as a single system. The beam deflector is arranged to provide variable direction of the subject area in one or two dimensions, and the lens is arranged to provide focus at a certain distance from the assembly. The control system can be arranged to be pre programmed. Pre-programmed control is where the assembly needs to image a fixed set of positions, for example to take a number of images of different features of an object. In other embodiments the control system may be arranged to generate images dynamically. Dynamic control is where the imaged positions depend on current circumstances. For example in a drone application the direction and focus of the image might need to vary in order to track an object as the drone moves due to wind. The variable focus lens can be used to adjust the focus for different wavelengths of light. Each wavelength will be in focus at a slightly different distance. The variable focus lens can be used to vary the focus when multiple wavelengths of light are present.

Variable focus lenses can also be used to give increased depth of field, which is a very powerful technique. During an exposure, the variable focus can be changed. The change of focus may be in steps. In other embodiments the change of focus may be changed smoothly between two optical powers.

The effect of the change of focus is to increase the depth of field of the captures image. For part of the exposure each distance will be in sharp focus and in other parts will be out of focus. When in focus the image will be sharp with high frequency features and when out of focus the image will be diffuse with low frequency features. The imaging system may be arranged to combine of the features within a single exposure for the captured image. The result is a compromise where the in-focus features should be easy to see or to automatically process.

The imaging system may use an alternative method for creating an increased depth of field in a captured image of a subject area. The imaging system may comprise a Z- stack. In a Z stack a series of images are taken with the focus being changed by a small amount between each one. Each image will have features at a given distance in focus. The position of a feature in an image gives its X and Y coordinates. The image where a feature is best in focus gives its Z position, so that a 3D profile can be built up in the image processing means or in the control system. Reducing the depth of field by opening the iris increases accuracy of the Z coordinate. The Z-stack technique is particularly advantageous in applications in connection with solid and transparent media, for example particles suspended in a liquid.

The imaging system may be used in applications in which a set of predetermined focus positions as required. The imaging system may be used in applications in which variable focus is needed. An example of an application in which a variable focus may be required is a robot guidance application which focuses on an object which can be at a variable position. In some embodiments the imaging system may be arranged to comprise autofocus routines which are well known and with a fast focusing lens can take as little as 0.5 seconds. In other embodiments the imaging system the imaging system may comprise a distance sensor arranged to measure a distance to the object and the control system is arranged to immediately set the focus of the imaging system to the measured distance. Typically measuring and setting the focus may be done in 10ms.

One feature of changing the focus of the assembly is that the distance to the subject will change. For an image sensor as the distance to the subject changes the effective size of the pixels in the image will also change. The change in size of the pixels in the image may be significant, for example in a measuring application.

Also as the angle of the mirror is changed the angle of the light path to the subject area will change, resulting in distortion of the captured image. Whenever the optical path is not normal to the subject, distortion will occur. A square feature will appear as a trapezium or diamond shape in the captured image.

In some cases it might be necessary to compensate for these effects on the image. Compensation can be done in image processing software. Calibration techniques for carrying our compensation are well documented and will not be further described as they are well known to the person skilled in the art of machine vision.

In principle each optical path angle and each focus distance will need to be calibrated separately. In practice, by calibrating a subset of these conditions, the rest can be estimated by interpolation or extrapolation.

There are many applications of the imaging system.

One application of the imaging system is in machine vision which is a technique where a captured image is processed using software to interpret the image. Machine vision systems are commonly used for inspection of manufactured products on a production line, for robot and drone guidance. In machine vision systems often the system needs to run as fast as possible and the exposure time needs to be kept as short as possible. It will be appreciated that closing an iris to increase the depth of field means that the exposure time needs to be increased to compensate. In some cases the product is imaged while it is moving, such as along a conveyor belt. In this case it is undesirable to increase the exposure time as this will cause motion blur and degrade the image. Typically, motion blur needs to be restricted to the width of a single pixel in the image.

Machine vision systems often inspect many parts of a single product. This can include critical features such as a drilled hole, dimension checking and print inspection, for example OCR checking of a“sell by” date or a barcode. In many cases the features to be inspected will be at different heights HI, H2 on the product, as is illustrated in Figures 8a and 8b. It is often not possible to increase the depth of field enough by closing the iris but the imaging system is able to change the focus in by variation of the variable focus lens in order to capture focused images of area 1 and area 2.

A common application of machine vision is for flat panel and solar cell inspection. A typical arrangement is illustrated in Figure 9a and 9b. Flat panel displays 50, despite their name, are often curved (illustrated in Figure 9b showing a profile or side view of the panel). These objects need inspecting in high resolution to identify flaws during manufacture. A typical prior art solution is to have a mechanical X-Y motion to move the camera or panel to take many small view high resolutions to build up a high resolution image of a large area. Sometimes this motion also needs to move in the Z axis as indicated in Figure 9(a). Using an imaging system in accordance with the invention the X,Y and possibly Z axis movement is no longer needed or does not need high precision positioning so simplifying the system and providing faster and more accurate imaging.

In the example of Figure 9, the variable angle mirror is used to change the view of the camera to image all areas of the panel, typically in a X,Y raster scan. As the angle changes, the image can be quickly refocussed using the variable focus liquid lens to compensate for the change of distance from the imaging system to the panel in the z direction. The distance or change of distance is predictable, so the focus can, in some arrangements, be set to a predetermined calculated value so it is usually not necessary to use an autofocus algorithm. Software algorithms can adjust for the distances changes and the distortion caused by the angle of view not being perpendicular to the panel in a conventional manner. The imaging system in accordance with the invention may also be used in 3D imaging systems. When an object moves relative to a conventional imaging system, the camera normally only gets a view of the object from one direction. With the present imaging system as the object moves, the angle of the camera view can be changed, so that the object is viewed from multiple directions. A schematic illustration of such a system is provided in Figure 10. Such an arrangement allows at least three surfaces of the object to be viewed. Thus the imaging system could be used to inspect an object from multiple directions. For example as the object approaches, the front 52 of the object is in view, when it is adjacent, the near side 54 can be seen and as it leaves, the rear side 56 is in view.

In other applications instead or as well as moving relative to the imaging system, the object could also be rotated. This would allow more, or all, of the object to be viewed from a number of images captured by the imaging system.

In another embodiment, schematically illustrated in Figure 11 two imaging systems are combined in order to view all surfaces of a 3D object as the object passes the imaging system. In this arrangement a first imaging system views a top 58 and first side of the object. A second imaging system is arranged to be diametrically opposite the first imaging system and is arranged to capture images of a bottom 60 and a second side of the object as illustrated Figure 11.

Over the course of the movement of the product past the first and second imaging systems, all surfaces can be inspected, for any convex shape. Without the movement of the product past the first and second imaging systems, there are some angles of surfaces which cannot be imaged with two views (any surface which is parallel to the axis between the two cameras).

It will be appreciated that multiple images can also be used to build up a 3D representation of the object in a conventional manner.

The imaging system can be used to produce a 3D image using a Z-stack, as described above. This technique take a number of images with the focus changed by a small amount between images. For each region of the image area, the image with the best focus can be determined. This gives the distance from the camera to the subject, so allowing the coordinates of each part of an object to be determined in three dimensions. A composite image can be built up which will effectively have a large depth of field.

The imaging system may be used with a linescan camera which is a camera with an image sensor which is one pixel wide and many pixels long, typically 512 to 8192. Each image is a single slice of the subject area. If an object is moving perpendicular to a long axis of the sensor, repeated image acquisitions can be used to build up a 2D image of the object.

An advantage of this arrangement is that for a given cost, a much higher resolution image can be built up of the object. Whereas the width of the image is determined by the number of pixels in the sensor, the length of the image is unlimited. Linescan cameras are normally used for inspecting continuous or very long materials, such as printed paper, glass or an extruded material.

In conventional linescan systems the object must be moving relative to the camera. The present imaging system can extend the capability of a linescan system. An imaging system in accordance with the invention using a linescan can build up an image of an object without the object moving relative to the imaging system. This has advantages such as: it provides area scan image sensor functionality, but using a linescan sensor, with the advantages of cost and resolution. For example a handheld imaging system could use a linescan sensor instead of area scan even where it is beneficial to keep the object static or nearly static.

An imaging system in accordance with the invention, as illustrated in Figure 12, can be used to effectively extend the number of pixels taken in a line. A one dimension beam deflector 58 moveable along the axis 60 of the image sensor 8 can used to move the focussed area so that successive images are all in line and make a longer composite line image. It will be appreciated that with this arrangement there will be a small skew in the position of the imaged areas 12 if the imaged object is moving but this can in many cases be compensated for in any post processing of the image.

If the skew is an issue a 2D beam deflector can be used, as illustrated in the embodiment of Figure 13. A 2 dimensional beam deflector 66 is utilised which is moveable about a first axis 64 and a second axis 66and is arranged to compensate for the movement of the object and ensures all the imaged areas for one composite line are all in line.

The imaging system may also be used to track a moving object and capture multiple images or capture longer exposure images. Alternatively the system may be used to inspect multiple products, for example on multiple conveyor tracks

It has also been found that the variable angle mirror provides a number of other benefits. Like the flat panel example before, a higher resolution lensing arrangement can be used, so that a scene can be imaged by tiling together many images.

This may be particularly advantageous in drone imaging and guidance systems as it allows the drone to take images of an object from a greater distance by for example adjusting the variable focus lens for a clear image. An advantage of the present system is that the imaging system can be adjusted to provide images for drone guidance and for a close up inspection of an object from the same system, contrary to existing arrangements in which the optical qualities would require the drone to carry two imaging systems.

In some embodiments the variable angle mirror may be used to maintain a constant view of the region or area of interest. The control system may be arranged to compensate for any known movement of the drone. Known movement may be detected by one or more sensors or comprise vibration or movement of the drone at a known speed relative to the view. In this way the imaging system can use a relatively longer exposure resulting in clearer and brighter images. It will be appreciated that stable image technology which removes vibration from camera images is known technology.

The imaging system may be used in security and surveillance applications providing a compact and reliable system with fewer moving parts than the conventional systems and one which is able to change viewing direction more quickly than traditional systems.

The imaging system may be used for robot guidance . A camera may be attached to a robot arm so that a direction of view of the camera can be directed by the robot. Output from the camera to the robot can be used to locate a component for the robot and the imaging system may be arranged to output orientation data to enable the robot to pick up the component and then perform an operation on the component.

Alternatively the imaging system can be mounted at a static position to provide guidance information for a robot.

In some embodiments the imaging system can be used for automotive applications. A vehicle may be provided comprising an imaging system according to the invention. Typically driver assistance and self-driving cars have a number of cameras providing a number of viewing orientations around the car. Using the imaging system, it would be possible to change the direction of view of one or more cameras. This could give a number of advantages such as reducing the number of cameras required on a vehicle, increasing the range of angles and directions that can be viewed, or directing two or more cameras to view a particular object or situation of interest.

A robot comprising an imaging system in accordance with the invention can be arranged to compensate for some of the limitations of a robot arm system. The imaging system can change its direction of view much more quickly using the beam deflector and the change of direction of view of the imaging system using the beam deflector is at least in part independent of the direction of the robot arm. In some embodiments the next compound may for example be detected while the previous component is still being handled or while the arm is returning to the conveyor location.

The imaging system may be used in combination with traffic monitoring system or car park entry systems. Typically vehicles can be in a variety of positions, for example in different lanes, at a variety of distances from the imaging system. Traffic systems are often short of light, especially when working at night. In a system in accordance with the invention the direction of view can be changed so that vehicles can be viewed in a variety of locations, including on different lanes. As the direction of view can be changed the field of view does not need to be so wide as in conventional systems. Consequently the image resolution can be higher, providing better images and resulting in a higher success rate reading licence plates. The variable focus lens can be used to change the camera focus, so that vehicles are in focus at different distances from the camera. The imaging system may be arranged to track moving vehicles. The imaging system may be arranged to increase an exposure time without introducing motion blur. In addition multiple images can be taken of the same vehicle. The system may be used when separate images of the licence plate, vehicle model and driver are needed.