Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
APPARATUS, METHOD AND COMPUTER PROGRAM FOR OBTAINING IMAGES
Document Type and Number:
WIPO Patent Application WO/2015/150619
Kind Code:
A1
Abstract:
An apparatus, method and computer program wherein the apparatus comprises: processing circuitry; and memory circuitry including computer program code; the memory circuitry and the computer program code configured to, with the processing circuitry, cause the apparatus at least to perform: obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a secondimage sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images.

Inventors:
MAGNUSSON SIMON (SE)
Application Number:
PCT/FI2015/050127
Publication Date:
October 08, 2015
Filing Date:
March 03, 2015
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
NOKIA TECHNOLOGIES OY (FI)
International Classes:
H04N5/225; H04N1/387; H04N3/14
Domestic Patent References:
WO2012089895A12012-07-05
Foreign References:
US20120188392A12012-07-26
Other References:
WILBURN, B. ET AL.: "High-speed videography using a dense camera array", IEEE COMPUTER SOCIETY CONF. ON COMPUTER VISION AND PATTERN RECOGNITION, vol. 2, 27 June 2004 (2004-06-27), pages 294 - 301, XP010708595
LIU, D. ET AL.: "Efficient space-time sampling with pixel-wise coded exposure for high-speed imaging", IEEE TRANS. ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 36, no. 2, February 2014 (2014-02-01), pages 248 - 260, XP011534227, ISSN: 0162-8828
Attorney, Agent or Firm:
NOKIA TECHNOLOGIES OY et al. (IPR DepartmentKarakaari 7, Espoo, FI)
Download PDF:
Claims:
CLAIMS

1. An apparatus comprising:

processing circuitry; and

memory circuitry including computer program code;

the memory circuitry and the computer program code configured to, with the processing circuitry, cause the apparatus at least to perform:

obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and

comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images. 2. An apparatus as claimed in any preceding claim wherein the determined artefact is used to correct the first image.

3. An apparatus as claimed in any preceding claim wherein the artefact comprises skew.

4. An apparatus as claimed in any preceding claim wherein the rolling shutter parameter comprises at least one of speed, read out direction, read out order.

5. An apparatus as claimed in any preceding claim wherein the obtaining of the images is synchronised so that at least a portion of the first image is obtained simultaneously to at least a portion of the second image.

6. An apparatus as claimed in any preceding claim wherein the obtaining of the images is synchronised so that the obtaining of the first image begins at the same time as the obtaining of the second image.

7. An apparatus as claimed in any preceding claim wherein the first image sensor and the second image sensor are positioned close to each other. 8. An apparatus as claimed in any preceding claim wherein the first image sensor and the second image sensor are configured to capture an image of the same scene.

9. An apparatus as claimed in any preceding claim wherein a plurality of portions in the first image are compared with a plurality of portions in the second image. 10. An apparatus as claimed in any preceding claim wherein the portions of the image correspond to read out lines of the image sensors.

1 1 . An apparatus as claimed in any preceding claim wherein comparing a portion in the first image with a corresponding portion in the second image comprises identifying objects in corresponding portions of the images and measuring the difference in the position of an object in the first image compared to the position of the same object in the second image.

12. An apparatus as claimed in any preceding claim wherein the difference between the first rolling shutter parameter and the second rolling shutter parameter is used to determine the artefact in the image.

13. An apparatus as claimed in any preceding claim wherein the first image and the second image comprise video images. 14. An apparatus as claimed in any preceding claim wherein the first image and the second image comprise still images.

15. An electronic device comprising an apparatus as claimed in any of claims 1 to 14. 16. A method comprising:

obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and

comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images.

17. A method as claimed in claim 15 wherein the method further comprises correcting the first image using the determined artefact.

18. A method as claimed in any of claims 16 to 17 wherein the artefact comprises skew.

19. A method as claimed in any of claims 16 to 18 wherein the rolling shutter parameter comprises at least one of speed, read out direction, read out order. 20. A method as claimed in any of claims 16 to 19 wherein the obtaining of the images is synchronised so that at least a portion of the first image is obtained simultaneously to at least a portion of the second image.

21 . A method as claimed in any of claims 16 to 20 wherein the obtaining of the images is synchronised so that the obtaining of the first image begins at the same time as the obtaining of the second image.

22. A method as claimed in any of claims 16 to 21 wherein the first image sensor and the second image sensor are positioned close to each other.

23. A method as claimed in any of claims 16 to 22 wherein the first image sensor and the second image sensor are configured to capture an image of the same scene.

24. A method as claimed in any of claims 16 to 23 wherein the method further comprises comparing a plurality of portions in the first image with a plurality of portions in the second image.

25. A method as claimed in any of claims 16 to 24 wherein the portions of the image correspond to read out lines of the image sensors.

26. A method as claimed in any of claims 16 to 25 wherein comparing a portion in the first image with a corresponding portion in the second image comprises identifying objects in corresponding portions of the images and measuring the difference in the position of an object in the first image compared to the position of the same object in the second image.

27. A method as claimed in any of claims 16 to 26 wherein the difference between the first rolling shutter parameter and the second rolling shutter parameter is used to determine the artefact in the image. 28. A method as claimed in any of claims 16 to 27 wherein the first image and the second image comprise video images.

29. A method as claimed in any of claims 16 to 28 wherein the first image and the second image comprise still images. 30. A computer program comprising computer program instructions that, when executed by at least one processor, enable an apparatus at least to perform:

obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and

comparing a portion in the first image with a corresponding portion in the second image to determine artefacts in the obtained images.

31 . A computer program comprising program instructions for causing a computer to perform the method of any of claims 16 to 29.

32. A physical entity embodying the computer program as claimed in any of claims 30 to 31 . 33. An electromagnetic carrier signal carrying the computer program as claimed in any of claims 30 to 31 .

Description:
TITLE

Apparatus, Method and Computer Program for Obtaining Images TECHNOLOGICAL FIELD

Examples of the present disclosure relate to an apparatus, method and computer program for obtaining images. In particular, they relate to an apparatus, method and computer program for obtaining images and correcting artefacts such as skew in the obtained images.

BACKGROUND

Image capturing apparatus which use rolling shutters are known. The rolling shutter effect in image sensors is caused by data being read sequentially from the different lines of the image sensor. This leads to different lines of the image sensor being read out at different times. The image sensor may move as the image is being captured. For instance, the hands of the user may cause the image sensor to shake or objects in the scene being imaged may have moved relative to the image sensor. This creates artefacts in the image such as skew. The artefacts may be perceived as wobble in video images.

It is useful to be able to correct the artefacts such as the skew in such images. BRIEF SUMMARY According to various, but not necessarily all, examples of the disclosure there may be provided an apparatus comprising: processing circuitry; and memory circuitry including computer program code; the memory circuitry and the computer program code configured to, with the processing circuitry, cause the apparatus at least to perform: obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images. In some examples the determined artefact may be used to correct the first image. In some examples the artefact may comprise skew.

In some examples the rolling shutter parameter may comprise at least one of speed, read out direction, read out order.

In some examples the obtaining of the images may be synchronised so that at least a portion of the first image is obtained simultaneously to at least a portion of the second image. In some examples the obtaining of the images may be synchronised so that the obtaining of the first image begins at the same time as the obtaining of the second image.

In some examples the first image sensor and the second image sensor may be positioned close to each other.

In some examples the first image sensor and the second image sensor may be configured to capture an image of the same scene.

In some examples a plurality of portions in the first image may be compared with a plurality of portions in the second image.

In some examples the portions of the image may correspond to read out lines of the image sensors. In some examples comparing a portion in the first image with a corresponding portion in the second image may comprise identifying objects in corresponding portions of the images and measuring the difference in the position of an object in the first image compared to the position of the same object in the second image. In some examples the difference between the first rolling shutter parameter and the second rolling shutter parameter may be used to determine the artefact in the image.

In some examples the first image and the second image may comprise video images. In some examples the first image and the second image may comprise still images. In some examples there may be provided an electronic device comprising an apparatus as described above.

According to various, but not necessarily all, examples of the disclosure there may be provided a method comprising: obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images.

In some examples the method may further comprise correcting the first image using the determined artefact.

In some examples the artefact may comprise skew.

In some examples the rolling shutter parameter may comprise at least one of speed, read out direction, read out order.

In some examples the obtaining of the images may be synchronised so that at least a portion of the first image is obtained simultaneously to at least a portion of the second image.

In some examples the obtaining of the images may be synchronised so that the obtaining of the first image begins at the same time as the obtaining of the second image.

In some examples the first image sensor and the second image sensor may be positioned close to each other.

In some examples the first image sensor and the second image sensor may be configured to capture an image of the same scene.

In some examples the method may further comprise comparing a plurality of portions in the first image with a plurality of portions in the second image.

In some examples the portions of the image may correspond to read out lines of the image sensors. In some examples comparing a portion in the first image with a corresponding portion in the second image may comprise identifying objects in corresponding portions of the images and measuring the difference in the position of an object in the first image compared to the position of the same object in the second image.

In some examples the difference between the first rolling shutter parameter and the second rolling shutter parameter may be used to determine the artefact in the image. In some examples the first image and the second image may comprise video images.

In some examples the first image and the second image may comprise still images.

According to various, but not necessarily all, examples of the disclosure there may be provided a computer program comprising computer program instructions that, when executed by at least one processor, enable an apparatus at least to perform: obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and comparing a portion in the first image with a corresponding portion in the second image to determine artefacts in the obtained images.

According to various, but not necessarily all, examples of the disclosure there may be provided a computer program comprising program instructions for causing a computer to perform the method described above.

According to various, but not necessarily all, examples of the disclosure there may be provided a physical entity embodying the computer program as described above. According to various, but not necessarily all, examples of the disclosure there may be provided an electromagnetic carrier signal carrying the computer program as described above.

According to various, but not necessarily all, examples of the disclosure there may be provided examples as claimed in the appended claims. BRIEF DESCRIPTION

For a better understanding of various examples that are useful for understanding the brief description, reference will now be made by way of example only to the accompanying drawings in which:

Fig. 1 illustrates an apparatus;

Fig. 2 illustrates an apparatus;

Fig. 3 illustrates a method;

Fig. 4 illustrates a method; and

Figs. 5A to 5F are representative images which may be used in implementations of the disclosure.

DETAILED DESCRIPTION The Figures illustrate an apparatus 1 , method and computer program 9. In the examples the apparatus 1 comprises: processing circuitry 5; and memory circuitry 7 including computer program code 1 1 ; the memory circuitry 7 and the computer program code 7 configured to, with the processing circuitry 5, cause the apparatus 1 at least to perform: obtaining 31 a first image using a first image sensor 25A and a first rolling shutter parameter; obtaining 33 a second image using a second image sensor 25B and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and comparing a portion in the first image with a corresponding portion in the second image to determine artefacts in the obtained images. This provides the technical effect of enabling the artefacts such as skew in the images to be corrected. In particular the methods used to determine the skew are simple and can be performed quickly. This may enable the image correction to be performed in real time so that a user can see the corrected image immediately after the images have been obtained. The apparatus may be for enabling images to be captured. The images may comprise video and/or still images.

Fig. 1 schematically illustrates an example apparatus 1 which may be used in implementations of the disclosure. The apparatus 1 illustrated in Fig. 1 may be a chip or a chip-set. The apparatus 1 may be provided within an electronic device which may comprise a camera or other image capturing apparatus. The example apparatus 1 comprises controlling circuitry 3. The controlling circuitry 3 may comprise one or more controllers. The controlling circuitry 3 may comprise means for controlling the apparatus 1 . The controlling circuitry 3 may comprise means for controlling one or more image sensors to enable images to be obtained. The controlling circuitry 3 may be implemented using instructions that enable hardware functionality, for example, by using executable computer program instructions in a general-purpose or special-purpose processing circuitry 5 that may be stored on a computer readable storage medium (disk, memory etc) to be executed by such processing circuitry 5.

The processing circuitry 5 may be configured to read from and write to memory circuitry 7. The processing circuitry 5 may comprise one or more processors. The processing circuitry 5 may also comprise an output interface via which data and/or commands are output by the processing circuitry 5 and an input interface via which data and/or commands are input to the processing circuitry 5.

The memory circuitry 7 may be configured to store a computer program 9 comprising computer program instructions (computer program code 1 1 ) that controls the operation of the apparatus 1 when loaded into processing circuitry 5. The computer program code 1 1 , of the computer program 9, provide the logic and routines that enables the apparatus 1 to perform the example methods illustrated in Figs. 3 and 4. The processing circuitry 5 by reading the memory circuitry 7 is able to load and execute the computer program 9.

The apparatus 1 therefore comprises: processing circuitry 5; and memory circuitry 7 including computer program code 1 1 ; the memory circuitry 7 and the computer program code 1 1 configured to, with the processing circuitry 5, cause the apparatus 1 at least to perform: obtaining a first image using a first image sensor and a first rolling shutter parameter; obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images.

The computer program 9 may arrive at the apparatus 1 via any suitable delivery mechanism. The delivery mechanism may be, for example, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a compact disc read-only memory (CD-ROM) or digital versatile disc (DVD), an article of manufacture that tangibly embodies the computer program. The delivery mechanism may be a signal configured to reliably transfer the computer program 9. The apparatus 1 may propagate or transmit the computer program 9 as a computer data signal. Although the memory circuitry 7 is illustrated as a single component in the figures it is to be appreciated that it may be implemented as one or more separate components some or all of which may be integrated/removable and/or may provide permanent/semi-permanent dynamic/cached storage. Although the processing circuitry 5 is illustrated as a single component in the figures it is to be appreciated that it may be implemented as one or more separate components some or all of which may be integrated/removable.

References to "computer-readable storage medium", "computer program product", "tangibly embodied computer program" etc. or a "controller", "computer", "processor" etc. should be understood to encompass not only computers having different architectures such as single /multi- processor architectures and sequential (Von Neumann)/parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific integrated circuits (ASIC), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device whether instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device etc.

As used in this application, the term "circuitry" refers to all of the following:

(a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry) and

(b) to combinations of circuits and software (and/or firmware), such as (as applicable): (i) to a combination of processor(s) or (ii) to portions of processor(s)/software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and

(c) to circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even if the software or firmware is not physically present. This definition of "circuitry" applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term "circuitry" would also cover an implementation of merely a processor (or multiple processors) or portion of a processor and its (or their) accompanying software and/or firmware. The term "circuitry" would also cover, for example and if applicable to the particular claim element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or other network device.

Fig. 2 schematically illustrates another apparatus 21 according to implementations of the disclosure. The apparatus 21 may be configured to enable images to be captured. The apparatus 21 may be an electronic device such as a camera, a mobile cellular telephone, a tablet computer, a personal computer or any other apparatus which may be configured to enable images to be captured. The apparatus 21 may be a handheld apparatus 21 which can be held in a users' hand during use.

Only features referred to in the following description are illustrated in Fig. 2. However, it should be appreciated that the apparatus 21 may comprise additional features that are not illustrated. For example, in some implementations the apparatus 21 could be a mobile cellular telephone or tablet computer. In such implementations the apparatus 21 may also comprise components which enable wireless communications such as a transmitter and/or receiver.

The apparatus 21 of Fig. 2 comprises: controlling circuitry 3 and image capturing apparatus 23. The controlling circuitry 3 and image capturing apparatus 23 may be operationally coupled. It is to be appreciated that any number or combination of intervening elements may be provided between the respective components of the apparatus 21 including no intervening elements.

The controlling circuitry 3 may be as described above in relation to Fig.1. Corresponding reference numerals have been used for corresponding features. The controlling circuitry 3 may be configured to control the apparatus 21 to perform a plurality of different functions. For example the controlling circuitry 3 may be configured to control the apparatus 21 to capture and store images. In some examples of the disclosure the controlling circuitry 3 may also be configured to perform image processing on the captured images. For instance, the controlling circuitry may be configured to determine the skew in an image and adjust the image to correct the skew. In some examples the controlling circuitry 3 may be configured to control the image capturing apparatus 23. For example, the controlling circuitry 3 may be configured to control the shutter arrangement 29 and/or the configuration of the optical arrangement 27. The controlling circuitry 3 may be configured to control the rolling shutter speed of the image sensors 25A, 25B. The rolling shutter speed is the speed at which data is read out from the image sensors 25A, 25B.

The image capturing apparatus 23 may comprise any means which enables the apparatus 21 to obtain images. In the examples of Fig. 2 the image capturing apparatus 23 comprises an optical arrangement 27, a first image sensor 25A, a second image sensor 25B and a shutter arrangement 29. In some embodiments of the disclosure the image capturing apparatus 23 may also comprise other components such as drive components which may enable the positions of the image sensors 25 and/or optical components within the optical arrangement to be adjusted.

The image sensors 25 may comprise any means which may be configured to convert light incident on the image sensor 25 into an electrical signal to enable an image to be produced. The image sensors 25 may comprise, for example, digital image sensors such as charge- coupled-devices (CCD) or complementary metal-oxide-semiconductors (CMOS). The image which is obtained may provide a representation of a scene and/or object which is positioned in front of the image sensors 25.

The controlling circuitry 3 may be configured to read out the electrical signal from the image sensors 25. For example, the processing circuitry 5 may be configured to retrieve an electrical signal comprising image data from the image sensor 25 and store it in the memory circuitry 7. The image data which is obtained from the image sensors 25 may be in the form of a still image or a video image. The image sensors 25 may be positioned close to each other. In some examples the image sensors 25 may be positioned so that there is a very small distance separating the two image sensors 25. In some examples the image sensors 25 may be positioned so that distance separating the two image sensors 25 is less than 1 cm. In some examples the image sensors 25 may be arranged parallel to each other. In other examples the image sensors 25 may be arranged stacked on top of each other. In such examples the top sensor 25 may be at least partially transparent to enable the lower sensor 25 to receive the incident light. The image sensors 25 may be arranged so that they capture an image of the same scene.

In some examples the image sensors 25 may be identical. The image sensors may be identical in that they may have the same sensitivity to incident light. In other examples the image sensors 25 may have different sensitivities and/or different resolutions or differences in any other parameter. For instance one of the image sensors 25 may be configured to detect coloured light and so obtain colour images while the other sensor 25 may only be configured to obtain a black and white image. In some examples the image sensors 25 may have the same size. For instance the image sensors 25 may have the same height and width. In other examples the image sensors 25 may have different sizes.

The optical arrangement 27 may comprise any means configured to focus or deflect incident light from an object or scene onto the image sensors 25. The optical arrangement 27 may receive the incident light from an object or scene external to the apparatus 21 through an aperture in a housing of the apparatus 21 . The optical arrangement 27 may comprise, for example, one or more optical devices such as one or more lenses. In the examples more than one optical arrangement 27 may be provided. For example a different optical arrangement 27 may be provided for each image sensor 25.

The image capturing apparatus 23 may also comprise one or more shutter arrangements 29. The shutter arrangement 29 may comprise any means which enables the exposure of the image sensors 25 to incident light to be controlled. The shutter arrangement 29 may be electronic or mechanical or any other suitable type of shutter arrangement 29. The shutter arrangement 29 may be controlled by the controlling circuitry 3. In some examples a shutter arrangement 29 may be provided for each of the image sensors 25.

In examples of the disclosure the apparatus 21 may be configured to obtain images from the image sensors 25 using rolling shutters. With a rolling shutter the data obtained by each row of the image sensor 25 is read out sequentially. For instance, the controlling circuitry 3 may begin obtaining the image data by reading out the top line of the image sensor 25. Once the read out of data from the top line is completed the data from the next line may be read out. It is to be appreciated that the rolling shutter may begin at any part of the image sensor 25. In examples where a rolling shutter is used different parts of the image sensors 25 may be exposed to incident light at different times. For instance the top row of the image sensor 25 may be exposed to incident light before the lower rows of the image sensor 25. All rows of the image sensor 25 may have the same exposure time even when a rolling shutter is used. The exposure time defines the amount of time each row of the image sensor 25 is exposed for. When using a rolling shutter the rows of the image sensor 25 may be exposed sequentially. The data may be read out of one row while other rows of the image sensor 25 are still being exposed. It is to be appreciated that even though each row of the image sensor 25 is exposed to incident light for the same amount of time the time period during which the rows of the image sensor are exposed may be different. If the image capturing device 23 moves during the time in which any part of the image sensor 25 is exposed then this may lead to skew and other artefacts in the obtained image. Fig. 3 illustrates an example method which may be used to correct such skew and artefacts. The method may be implemented using the apparatus 1 , 21 as described above.

The method comprises, at block 31 , obtaining a first image using a first image sensor 25A and a first rolling shutter parameter. At block 33 the method comprises obtaining a second image using a second image sensor 25B and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter. At block 35 the method comprises comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images. Once the artefact has been determined one or both of the images can be corrected.

Fig. 4 illustrates another example method which may be used to correct skew and other artefacts in images obtained using a rolling shutter. The example method of Fig. 4 may also be implemented using the apparatus 1 , 21 as described above. Figs. 5A to 5F are representative images and portions of images which may be obtained using the method of Fig. 4.

At block 41 a first image 51 is obtained using a first image sensor 25A. The first image 51 is obtained using a first rolling shutter parameter. The first rolling shutter parameter may be the speed at which data is read out from the image sensor 25A. In other examples the parameter could be the direction in which the data is read out or the order in which the data is read out or any other suitable parameter.

Fig. 5A illustrates an example first image 51 which may be obtained at block 41 . The first image 51 had a readout time of 1/40s. The data was read out from top to bottom of the image sensor 25A. The movement of the image capturing apparatus 23 during the readout time leads to skew. The skew comprises a displacement of the objects in the image 51 as can be seen in Fig. 5A. At block 43 a second image 53 is obtained using the second image sensor 25B. The second image 53 is obtained using a second rolling shutter parameter. In this example the rolling shutter parameter comprises the rolling shutter speed. It is to be appreciated that other parameters may be used in other examples of the disclosure. The second rolling shutter speed is different to the first rolling shutter speed.

Fig. 5B illustrates an example second image 53 which may be obtained at block 43. The second image 53 had a readout time of 1/80s. As with the first image data is read out from top to bottom of the image sensor 25B. As the readout time is shorter for the second image there is less skew in the second image. However the skew is still noticeable in the second image as can be seen in Fig. 5B.

In the examples of Fig. 4 and Figs. 5A to 5F the first image 51 was obtained using a slower rolling shutter speed than the second image 53. It is to be appreciated that in other examples the first image 51 could be obtained using a faster rolling shutter speed than the second image 53. Also in the examples of Fig. 4 and Figs. 5A to 5F the first rolling shutter speed is half of the second rolling shutter speed. It is to be appreciated that any other ratios of rolling shutter speeds and/or other variations in rolling shutter parameters may be used in other examples. The obtaining of the images 51 , 53 at blocks 41 and 43 may be synchronized. In some examples the obtaining of the images 51 , 53 may be synchronized so that at least a portion of the first image 51 and at least a portion of the second image 53 are obtained simultaneously. In some examples blocks 41 and 43 may be synchronised so that the obtaining of the first image 51 begins at the same time as the obtaining of the second image 53. This may lead to the top line of the first image 51 matching the top line of the second image 53. This may provide a reference which can be used to calculate skew in the obtained images 51 , 53.

At block 45 corresponding portions of the images 51 , 53 are identified. The corresponding portions of the images may comprise any portions of the images 51 , 53 which represent the same part of the scene which has been captured by the images.

In some examples the portions of the images 51 , 53 may comprise read out lines of the image sensors 25. In some examples the portions of the images 51 , 51 may comprise lines of the image sensor 25 where the lines of the identified in the first image sensor 25A are at the same or a similar position to the lines identified in the second image sensor 25B.

Fig. 5C illustrates an example portion 55 of the first image 51 and Fig. 5D illustrates an example portion 57 of the second image 53. In the examples of Figs. 5C and 5D the portions comprise lines from the middle of the image sensors 25. It is to be appreciated that the lines identified may comprise any portion of the image.

In some examples the top lines of the images may be identified. If the rolling shutter begins at the top of the image sensor 25 then these lines should be the same for both of the images. This may act as a reference which may be used to determine the skew.

At block 47 the skew is determine using the comparison of the two images. Any suitable method may be used to determine the skew in the images. It can be seen by comparing the portion 55 of the first image 51 with the portion 57 of the second image, that objects within the images 51 , 53 are at different horizontal positions. The objects within the images and their relative positions may be detected using pattern recognition, phase detection or any other suitable means. A measurement of the relative positions of identified objects within the portion 55 of the first image 51 and the portion 57 of the second image 53 and the known difference in the rolling shutter speeds may be used to determine the skew in the images 51 , 53.

For instance in the examples of Figs. 5A to 5F the rolling shutter speed used to obtain the first image 51 is half of the rolling shutter speed that was used to obtain the second image 53. If the skew in the image is linear then the skew in the first image 51 will be twice as much as the skew in the second image 53. This information may be used to work out how much the skew has displaced the objects within the respective images 51 , 53.

It is to be appreciated that other methods for calculating the skew may be used in other examples of the disclosure.

At block 49 at least one of the images 51 , 53 is corrected to take into account the skew. Any suitable method may be used to correct the image 51 , 53. In some examples only one of the images 51 , 53 might be corrected. In such examples the second image might be deleted once the skew has been calculated.

Fig. 5E illustrates an example of a corrected image 59. In the corrected image the displacement of objects in the image caused by movement of the image sensors 25A, 25B has been corrected.

At block 50 the corrected image may be cropped. Fig. 5F illustrates an example of a cropped image 61 . In the example of Fig. 4 and Figs. 5A to 5F only two portions 55, 57, one from each image 51 , 53 have been compared. It is to be appreciated that in other examples more than one portion from each image 51 , 53 may be compared. For example, in the implementation described above portions from the middle of the images 51 , 53 are compared. It is to be appreciated that in other examples a plurality of lines form a plurality of different places within the images 51 , 53 may be compared. This may enable a more accurate estimate of the skew to be obtained as it may take into account the fact that the skew might not be the same for all portions of the image.

Comparing multiple portions of the two images 51 , 53 may be useful if the user is moving relative to a scene being imaged. For instance if a user is on a train or vehicle and is taking an image of scene out of a window then objects in the foreground will appear to move more than objects in the background as the image is taken. This may be taken into account by comparing different lines from different parts of the images 51 , 53. Comparing multiple lines of the two images 51 , 53 may also take into account the fact that in some parts of the image it may be difficult to detect any skew. For example if the part of the image comprise a blue sky then it is difficult to identify any artefacts in such an image. If there is no skew detected in one portion of the image then other parts of the image could be compared. Alternatively if no skew detected in one portion of the image then it may be determined not to apply any corrections to the image.

Examples of the disclosure provide for an apparatus and method which enable artefacts in images obtained using a rolling shutter to be corrected or reduced. Using two images of the same scene, where at least part of the images are obtained simultaneously, provides a simple method which may be used to determine the artefacts.

As the method is very simple it enables the correction to be carried out quickly. This may allow the processing to be carried out in real time so that the user of the apparatus 21 can view the corrected image immediately after it has been captured. This may be particularly useful if the use is capturing video images.

As the method uses two different images obtained using different sensors the method may also be used for still images such as photographs.

The blocks illustrated in Figs. 3 and 4 may represent steps in a method and/or sections of code in the computer program 9. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block may be varied. Furthermore, it may be possible for some blocks to be omitted. In the above description the term coupled means operationally coupled. Any number or combination of intervening elements can exist (including no intervening elements).

Where a structural feature has been described, it may be replaced by means for performing one or more of the functions of the structural feature whether that function or those functions are explicitly or implicitly described.

According to various, but not necessarily all, examples of the disclosure, there may be provided an apparatus 1 comprising means for obtaining a first image using a first image sensor and a first rolling shutter parameter; means for obtaining a second image using a second image sensor and a second rolling shutter parameter where the second rolling shutter parameter is different to the first rolling shutter parameter; and means for comparing a portion in the first image with a corresponding portion in the second image to determine artefact in the obtained images.

As used here "module" refers to a unit or apparatus that excludes certain parts/components that would be added by an end manufacturer or a user.

The term "comprise" is used in this document with an inclusive not an exclusive meaning. That is any reference to X comprising Y indicates that X may comprise only one Y or may comprise more than one Y. If it is intended to use "comprise" with an exclusive meaning then it will be made clear in the context by referring to "comprising only one.." or by using "consisting".

In this brief description, reference has been made to various examples. The description of features or functions in relation to an example indicates that those features or functions are present in that example. The use of the term "example" or "for example" or "may" in the text denotes, whether explicitly stated or not, that such features or functions are present in at least the described example, whether described as an example or not, and that they can be, but are not necessarily, present in some of or all other examples. Thus "example", "for example" or "may" refers to a particular instance in a class of examples. A property of the instance can be a property of only that instance or a property of the class or a property of a sub-class of the class that includes some but not all of the instances in the class. It is therefore implicitly disclosed that a features described with reference to one example but not with reference to another example, can where possible be used in that other example but does not necessarily have to be used in that other example.

Although embodiments of the present invention have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the invention as claimed.

Features described in the preceding description may be used in combinations other than the combinations explicitly described.

Although functions have been described with reference to certain features, those functions may be performable by other features whether described or not. Although features have been described with reference to certain embodiments, those features may also be present in other embodiments whether described or not.

Whilst endeavoring in the foregoing specification to draw attention to those features of the invention believed to be of particular importance it should be understood that the Applicant claims protection in respect of any patentable feature or combination of features hereinbefore referred to and/or shown in the drawings whether or not particular emphasis has been placed thereon.

I/we claim