BACKGROUND

[0001] Various imaging devices, such as scanners, photocopiers, and fax machines, take media, such as documents or photographs, as an input and can provide a digitized image of each medium. While digitizing the image of the medium a skew may occur in the digitized image. For instance, if the medium is incorrectly positioned in the imaging device, the output digitized image may be skewed. A skewed image may have to be corrected, for example, for further operations on the image. For instance, the skew may have to be corrected for accurate Optical Character Recognition (OCR) operations and for operations, such as archiving, combining, or for re-printing.

BRIEF DESCRIPTION OF FIGURES

[0002] The detailed description is provided with reference to the accompanying figures, wherein:

[0003] Figure 1 illustrates a schematic of a device for correcting skew in images, according to an example;

[0004] Figure 2 illustrates a detailed schematic of the device for correcting skew in images illustrated in Figure 1 , according to an example

[0005] Figure 3 illustrates a method for correcting skew in images, according to an example;

[0006] Figure 4 illustrates a detailed method for correcting skew in images, according to an example;

[0007] Figure 5 illustrates a network environment using a non-transitory computer readable medium for correcting skew in images, according to an example. [0008] It should be noted that the description and the figures are merely examples of the present subject matter and are not meant to represent the subject matter itself. Throughout the drawings, identical reference numbers designate similar, but not identical, elements. The figures are not to scale, and the size of some parts may be exaggerated to more clearly illustrate the example shown. Moreover, the drawings provide examples and/or examples consistent with the description, and the description is not limited to the examples and/or examples provided in the drawings.

DETAILED DESCRIPTION

[0009] In imaging devices with Automatic Document Feeders (ADF), in which media may be fed to the imaging device at a considerable speed and which can provide a digitized image of each medium, a skew may occur and may have to be automatically corrected for further operations on the digitized image. For example, in imaging devices with ADF, a top skew may occur in which the medium is skewed but the digitized image is substantially rectangular, or a feed skew may occur in which the digitized image of the skewed medium is non-recta ngular. For instance, the skew can be introduced while picking up the medium from an input tray or because of a slippage between one of the rollers feeding the medium to the imaging device or by any other causes of misalignment of the medium.

[0010] Examples of correcting a skew in images are described herewith.

According to an aspect, the present subject matter provides a simple for automatically addressing a skew introduced in an image, for example, due to misalignment of the image. When an image is received, firstly, it is processed for checking whether a skew exists or not. Accordingly, a margin of the image and a reference edge in the image can be identified. For example, the margin can be identified using edge-detection techniques. In addition, as an example, the reference edge can be a straight edge of the image, for instance, in a rectangular image. [0011] Further, based on the margin of the image and the reference edge, it is determined whether the image is skewed or not. For instance, a position of the margin and that of the reference edge may be determined to determine if there is a relativity in the positions of the margin and of the reference edge. In other words, it is determined if the positions of the margin and of the reference edge are same or different. A relative displacement or a difference in position of the margin and of the reference edge indicates the presence of a skew in the image.

[0012] Once the presence of the skew is detected the image is corrected for correcting the skew in the image. This is so achieved by aligning pixels of the image with the reference edge. For example, the pixels of the image can be aligned rowwise or swathe-wise along the reference edge to correct the skew. Upon alignment, the image is checked for presence of any remnant blank space. For example, again using edge-detection techniques, the remnant blank space in the image can be identified. If so, the remnant blank space in the image is filled by estimation techniques, such as interpolation or extrapolation. For instance, a transformation matrix may be applied on the image to transform actual comers of the image to position of the expected corners of the image, for example, identified based on the margins of the skew-corrected image. Accordingly, the present subject matter provides a simple and efficient technique for automatically correcting skew in images which does not involve expenditure of substantial amount of computational resources or time.

[0013] The above aspects are further described in conjunction with the figures, and in associated description below. It should be noted that the description and figures merely illustrate principles of the present subject matter. Therefore, various arrangements that encompass the principles of the present subject matter, although not explicitly described or shown herein, may be devised from the description and are included within its scope. Additionally, the word “coupled” is used throughout for clarity of the description and can include either a direct connection or an indirect connection.

[0014] Figure 1 illustrates a schematic of a device 100 for correcting skew in images, according to an example of the present subject matter. In one example, the device 100 can be an imaging device, such as a scanner, a photocopier, or a fax machine, having automatic document feed which allows media, such as documents or photographs, to be continuously fed to the imaging device without any user intervention. Accordingly, the device 100 can include an imaging sensor which can capture an image of the medium being fed to the device 100. In such a device 100 with automatic feed, the images so captured may be skewed and the device 100 provides for the correction of skew in such images. In another example, the device 100 may be employed as any of a variety of computing devices, including, servers, a desktop personal computer, a notebook or portable computer, a workstation, a mainframe computer, a mobile computing device, and a laptop. In such an example, the device 100 can be coupled to an imaging device and can receive the images to be skew-corrected, i.e., the images can be straightened by removing the skew therein.

[0015] The device 100, in an example illustrated in Figure 1 , includes a processor 102 and a memory 104 coupled to the processor 102. The processor 102 may include microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any other devices that manipulate signals and data based on computer-readable instructions. Further, functions of the various elements shown in the figures, including any functional blocks labeled as “processor(s)”, may be provided through the use of dedicated hardware as well as hardware capable of executing computer- readable instructions. The memory 104 may include any non -transitory computer- readable medium including volatile memory (e.g., RAM), and/or non-volatile memory (e.g., EPROM, flash memory, Memristor, etc.). The memory 104 may also be an external memory unit, such as a flash drive, a compact disk drive, an external hard disk drive, or the like.

[0016] In operation, as mentioned above, the device 100 can identify if the received images are skewed and can, in cases where the images are skewed, correct the skew in images. For example, the image received by the device may include a contained image, such as a digital photograph, which may be skewed in respect to boundaries, i.e., frame, of the image. Accordingly, the device 100 may include instructions 106 which, when the image is received by the device 100, can smooth the image, for instance, to reduce the noise in the image or for reducing the pixilation in the image. Once the image has been conditioned by smoothing, the processor 102 may execute the instructions 108 to identify a margin of the image along with identifying a reference edge in the image. The pre-conditioning of the image facilitates in easy and accurate identification of the margin and the reference edge in the image.

[0017] For instance, in case of a rectangular shaped contained image which is skewed, the four edges of the contained image can be identified as the margins of the image whereas the boundaries of the frame of the image can be identified as the reference edges. In an example, for the purposes of correcting the skew, each margin-reference edge pair can be processed in unison to correct the image. For instance, the margin and reference edge in proximity to each other along their lengths can be considered as the margin-reference edge pair. The description, henceforth, is provided in terms of one pair of the margin and the reference edge. [0018] Further, based on the margin and the reference edge, the processor 102 may execute instructions 110 to determine whether the image is skewed or not. In said example, a relative displacement between the margin and the reference edge in the image can be determined. If there is a relative displacement between the margin and the reference edge, it indicates the presence of a skew in the image and that the image has to be corrected for skew. Accordingly, if the skew is present in the image, the processor 102 may execute instructions 112 to correct the skew in the image by aligning pixels of the image, for instance, the contained image, with the reference edge. Further, if there is any remnant blank space present in the image in which the pixels have been aligned, instructions 114 may be executed by the processor 102 to fill the remnant blank space in the image, for instance, till the reference edge. The operation of the device 100 is discussed in further detail with reference to Figure 2.

[0019] Figure 2 illustrates a detailed schematic of the device 100 to correct skew in images, according to an example. In the illustrated example, the device 100 can be an imaging device and can include, in addition to the processor 102 and memory 104, an automatic document feeder 202 and an image sensor 204. The automatic document feeder 202 can receive a stack of media, such as documents or photographs, and can feed the media one-by-one to the image sensor 204 without having to manually position the medium for being imaged by the image sensor 204. Accordingly, the automatic document feeder 202 can include a delivery mechanism 206 which can deliver the medium from the stack to the image sensor 204. The image sensor 204 can create an image, the image having a contained image of the medium within a boundary of the image so created. In an example, the delivery mechanism 206 can include a set of rollers provided along a delivery path and an exit path for the medium, for first delivering the medium to the image sensor 204 and then picking the medium again and removing it from the image sensor 204 to allow subsequent medium to be imaged.

[0020] The device 100, as mentioned previously, may include the memory 104 having data 208, and engines 210. The data 208 may include data that is generated as a result of the functionalities carried out by any of the engines 210 and saved by the engines 210 to provide various functionalities to the device 100. Further, as illustrated in the present example, the engines 210 may include an image conditioning engine 212, a skew detection engine 214, and a skew correction engine 216. The engines 210 are employed as a combination of hardware and programming (for example, programmable instructions) to use functionalities of the engines 210. In examples described herein, such combinations of hardware and programming may be used in a number of different ways. For example, the programming for the engines 210 may be processor executable instructions stored on a non -transitory machine-readable storage medium and the hardware for the engines 210 may include a processing resource (for example, processors), to execute such instructions. In the present examples, the machine-readable storage medium stores instructions that, when executed by the processing resource, deploy engines 210. In other examples, engines 210 may be deployed using electronic circuitry. The engines 210, among other capabilities, may fetch and execute computer-readable instructions stored in the memory 104.

[0021] In operation, as described above, the device 100 can receive an image of a medium, such as a document or a photograph, from the image sensor 204, determine whether the image of the document or the photograph, referred to as the contained image, is skewed within the boundaries of the image captured by the image sensor 204, and, if skewed, correct the skew.

[0022] To begin with, in an example, the image received from the image sensor

204 is conditioned for facilitating further processing of the image. Accordingly, the image conditioning engine 212 can smooth the image for, for example, reducing the noise in the image or for reducing the pixilation in the image, so that the contained image within the image can be easily identified. In one example, the image conditioning engine 212 can use a Gaussian image smoothing technique, such as a bilateral filter, or any other filter for smoothing the image. The skew detection engine 214 can, then, use the smoothed image for identifying if there is a skew in the image or not, for instance, whether the contained image is aligned with the boundary or not. [0023] In an example, the skew detection engine 212 can first detect a margin of the image, for instance, of the contained image in the image. In said example, the skew detection engine 212 may use a Canny filter for identifying the margins in the image. In one case, a top skew may occur where the medium is skewed but the shape of the image of that medium, i.e., the contained image, remains intact. For instance, fora rectangular medium, the contained image may still be rectangular, but skewed within the frame. In such a case, the skew detection engine 212 can detect the four sides of the rectangle as margins of the image. In another case, a feed skew may occur in which the contained image of the skewed medium is non-rectangular. For example, the contained image of the medium may be straight with respect to the frame for a certain length and, beyond that length, it may be skewed with respect to the frame of the image. In such a case, the contained image may not have the same shape as that of the original medium and may have a polygonal shape instead of a rectangular shape. In such a case, each of the side of the polygon forming the contained image of the medium may be identified as the margin of the image. In one example of the present subject matter, in either of the above cases, each margin may have to be separately corrected as has been explained previously and will be discussed further also. Other implementations are also envisaged by the present subject matter.

[0024] In addition to identifying the margin, the skew detection engine 214 can identify a reference edge for the image. In the present case, again, the skew detection engine 214 can use the Canny filter for identifying the reference edge. For instance, for each margin identified in the image, the skew detection engine 214 can identify a reference edge. Accordingly, in said example, the skew detection engine 214 can identify margin-reference edge pairs, for instance, based on the proximity of the margin and reference edge to each other along their lengths. In other words, the margin and the reference edge closest to each other can be considered as the margin-reference edge pair. Further, the skew detection engine 214 can determine, based on the identified margin and reference edge in a pair as to whether the image is skewed or not.

[0025] In an example, the skew detection engine 214 can determine whether the position of the margin of the contained image is different from the position of the corresponding reference edge. For instance, the skew detection engine 214 can determine whether a relative displacement exists between the margin and the reference edge. The relative displacement indicates that the position of the margin and the position of the reference edge is different, which, in turn, indicates the presence of a skew in the image.

[0026] Once the skew detection engine 214 has detected that a skew is present in the image, i.e., the contained image of the medium in the image captured by the image sensor 204 of the device 100, is skewed with respect to the frame of the image, the skew correction engine 216 can correct the skew in the image. In one example, the skew correction engine 216 can correct the skew by aligning the pixels of the image along the reference edge. For instance, the skew correction engine 216 can translate the pixels of the contained image along the identified margin in a rowwise manner to align the margin with the reference edge, i.e., the frame of the image. In other words, an entire row of pixels of the margin of the contained image can be translated for aligning along the reference edge. [0027] In addition, the skew correction engine 216 can identify if there are any remnant blank spaces in the image after aligning the pixels, referred to as the aligned image. If any remnant blank space is identified, the skew correction engine 216 can fill the remnant blank spaces. In an example, the skew correction engine 216 can fill the remnant blank spaces using estimation techniques, such as interpolation or extrapolation. In another example, the skew correction engine 216 may employ a transformation matrix for filling the remnant blank spaces. In said example, the skew correction engine 216 can identify the actual comers of the aligned image along with the expected corners had there been no remnant blank spaces in the image. Then, the skew correction engine 216 can determine a transformation matrix for the actual corners to be transformed to the expected corners and applies the transformation matrix to the aligned image to fill the remnant blank spaces.

[0028] Figure 3 and Figure 4 illustrate a method 300 and 400 for correcting skew in images, according to an example of the present subject matter. While Figure 3 illustrates the method 300 for correcting skew in images briefly, Figure 4 illustrates the method 400 in detail. The methods 300 and 400 may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, engines, functions, etc., that perform certain functions or employ certain abstract data types. The methods 300 and 400 may also be practiced in a distributed computing environment where functions are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, computer executable instructions may be located in both local and remote computer storage media, including memory storage devices. [0029] The order in which the blocks in the methods 300 and 400 are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to employ the methods 300 and 400, or an alternative method. Additionally, individual blocks may be deleted from the methods 300 and 400 without departing from the scope of the subject matter described herein. Furthermore, the methods 300 and 400 can be employed in any suitable hardware, software, firmware, or combination thereof. In one example, the methods 300 and 400 may be implemented by the device 100 and the various components thereof. For the sake of brevity, the components and details associated with the device 100 as described in Figure 1 and Figure 2 have not been repeated.

[0030] Referring to method 300, at block 302, a margin of an image is detected. For instance, the image can be received from an image sensor of an imaging device, such as a scanner, and can be skewed, and the skew of the image is to be corrected. In some case, the contained image in the received image may be of a rectangular shape but skewed. In another case, the contained image may be of a different shape, such as a polygon, due to the skew. In an example, each of the edges of the detected image can be identified as a margin. For example, in case of a rectangular or polygonal contained imaged, the various sides of the rectangle or the polygon may be detected as the margins of the image.

[0031] At block 304, based on the margin, for instance, each margin, of the image and a reference edge of the image, whether the image is skewed. In one example, the reference edge of the image may be a frame or boundary of the image in which the contained image is present but is skewed. In some cases, the reference edge may be pre-determined for all images.

[0032] Further, at block 306, if the image is detected as skewed, the skew is corrected. The correction is done by aligning pixels of the image, at the skewed margin, with the reference edge of the image.

[0033] Further, at block 308, after correcting the skew in the image, the resulting corrected image is checked for presence of any remnant blank spaces, for example, appearing as a result of aligning the pixels as explained with reference to block 306. The remnant blank space may be detected by using various edge-detection techniques. In case, a remnant blank space is detected, various mechanisms may be employed to fill the remnant space, such as interpolation, extrapolation, transformation matrices. For instance, the expected corners of the image and the actual corners of the image may be aligned so that any remnant blank space present in the image may be filled. [0034] Referring now to the detailed method 400 for correcting skew in images illustrated in Figure 4, at block 402, the image received from the image sensor 204 is smoothed as part of conditioning for facilitating further processing of the image. The image may be smoothed for, for example, reducing the noise in the image or for reducing the pixilation in the image, so that the contained image within the image can be easily identified. In one example, a Gaussian image smoothing technique, such as a bilateral filter, or any other filter can be used for smoothing the image. [0035] At block 404, the smoothed image is used for detecting a margin of the image, for instance, of the contained image in the image. In said example, a Canny filter may be used for identifying the margins in the image. In one case, a top skew may occur where the medium is skewed but the shape of the image of that medium, i.e., the contained image, remains intact. In another case, a feed skew may occur in which the contained image of the skewed medium is non-rectangular. For instance, the contained image of the medium may be straight with respect to the frame for a certain length and, beyond that length, it may be skewed with respect to the frame of the image. In such a case, the contained image may not have the same shape as that of the original medium. In either case, the sides of the contained image can be identified as margins of the image.

[0036] For instance, for each margin identified in the image, a reference edge can be identified. In said example, margin-reference edge pairs can be identified, for instance, based on the proximity of the margin and reference edge to each other along their lengths. In other words, the margin and the reference edge closest to each other can be considered as the margin-reference edge pair.

[0037] Further, at block 406, it is determined, based on the margin and the reference edge, if there is a skew in the image or not, for instance, whether the contained image is aligned with the boundary or not. In the example above where the margin and reference edge pairs are identified, each pair of margin and reference edge can be assessed as to whether the image is skewed or not. For example, the position of the margin of the contained image can be assessed with respect to the position of the corresponding reference edge and it can be determined whether there is a relative displacement between the margin and the reference edge. A relative displacement indicates that the position of the margin and the position of the reference edge is different, which, in turn, indicates the presence of a skew in the image.

[0038] At block 408, once it is detected that a skew is present in the image, i.e., the contained image of the medium in the image, is skewed with respect to the frame of the image, the skew in the image can be corrected In one example, the skew can be corrected by aligning the pixels of the image along the reference edge. For instance, the pixels of the contained image along the identified margin can be translated row-wise to be aligned with the reference edge, for example, the frame of the image. In other words, an entire row of pixels of the margin of the contained image can be translated for aligning along the reference edge, followed by the subsequent row, continuously, until all the rows of pixels of the margin are aligned with the reference edge.

[0039] At block 410, it can be identified as to whether there are any remnant blank spaces in the image after aligning the pixels, referred to as the aligned image. In one example, the remnant blank spaces can be identified using an image processing technique to identify whether all the margins have or have not been aligned with the corresponding reference edge. In case all the margins of the contained image have not been aligned, a remnant blank space may be present. [0040] Accordingly, at block 412, If any remnant blank space is identified in the aligned image, the remnant blank spaces can be filled. In an example, the remnant blank spaces can be filled using estimation techniques, such as interpolation or extrapolation. In another example, a transformation matrix may be employed for filling the remnant blank spaces. In said example, the actual corners of the aligned image are identified and the expected corners, i.e., corners of the aligned image had there been no remnant blank spaces in the image, are also identified. Then, a transformation matrix can be determined for the actual corners to be transformed to the expected corners and the transformation matrix is applied to the aligned image to fill the remnant blank spaces. [0041] Figure 5 illustrates a network environment 500 using a non -transitory computer readable medium 502 for correcting skew in images, according to an example of the present subject matter. The network environment 500 may be a public networking environment or a private networking environment. In one example, the network environment 500 includes a processing resource 504 communicatively coupled to the non-transitory computer readable medium 502 through a communication link 506.

[0042] For example, the processing resource 504 may be a processor of a computing system, such as the device 100. The non-transitory computer readable medium 502 may be, for example, an internal memory device or an external memory device. In one example, the communication link 506 may be a direct communication link, such as one formed through a memory read/write interface. In another example, the communication link 506 may be an indirect communication link, such as one formed through a network interface. In such a case, the processing resource 504 may access the non-transitory computer readable medium 502 through a network 508. The network 508 may be a single network or a combination of multiple networks and may use a variety of communication protocols.

[0043] The processing resource 504 and the non-transitory computer readable medium 502 may also be communicatively coupled to data sources 510 over the network 508. The data sources 510 may include, for example, databases and computing devices. The data sources 510 may be used by the database administrators and other users to communicate with the processing resource 504. [0044] In one example, the non-transitory computer readable medium 502 includes a set of computer readable and executable instructions. The set of computer readable instructions, referred to as instructions hereinafter, may be accessed by the processing resource 504 through the communication link 506 and subsequently executed to perform acts for network service insertion.

[0045] For discussion purposes, the execution of the instructions by the processing resource 504 has been described with reference to various components introduced earlier with reference to description of Figure 1 and Figure 2. [0046] On execution by the processing resource 504, a margin of an image is detected. In one case, the image sensor 204 of the device 100 may be used to obtain the image of any medium, such a photograph or a document. The image may be skewed and the processing resource 504 may execute instructions to correct the skew of the image. The image may include a contained image of the medium, and the contained image may be of any shape, such as a rectangular shape. Due to the skew, the rectangular contained image may either be skewed with respect to the frame of the image captured by the image sensor 204 or it may be aligned upto a length but skewed beyond that length. In the formed case, the skewed contained image may be rectangular whereas, in the latter case, the skewed contained image may be polygonal in shape.

[0047] In the above cases, the margin of the image may be detected as each straight edge of the skewed contained image. For example, in case the detected image is rectangular, margin of the detected image may be the four sides of rectangle. In case the detected image is polygonal, each straight edge of the polygon can be detected as a margin of the image. Further, a reference edge of the image is determined. For instance, the frame or boundary of the image from the image sensor 204 within which the contained image is present may be considered as the reference edges. For instance, for each margin identified in the image, a reference edge can be identified. Accordingly, margin-reference edge pairs can be identified, for instance, based on the proximity, i.e., the margin and the reference edge closest to each other can be considered as the margin-reference edge pair for the purposes of processing the image for correcting the skew.

[0048] In addition, it is determined if the position of the margin of the image is different from a corresponding position of the reference edge of the image. In an example, the assessment of the difference in positions of the margin and reference edge may be, as explained above, performed for each margin-reference edge pair. For instance, a relative displacement between the reference edge and the detected margin can be determined to determine if the image is skewed. For instance, the comparison may be done by mirroring different points on the reference edge and the margin of the image and comparing the points of the margin and the reference edge. If a relative displacement is determined between the reference edge and the margin, then the image is assessed to be skewed. In other words, it means that the position of the reference edge is not the same as that of the margin.

[0049] Accordingly, in case the image is determined to be skewed, the pixels of the image are translated with respect to the reference edge of the image. The translation is done to obtain an aligned image. In an example, the correction may done by aligning pixels of the image along the reference edge in a row-wise manner. In other cases, different transformation mechanisms may be applied to parts of the image where skew may be determined.

[0050] After correcting the skew in the image, the resulting aligned image may have remnant blank space present which can be identified. Once it is identified that remnant blank space is present, the remnant blank space is filled. In an example, for filling the remnant blank space, various techniques, such as estimation techniques or transformation techniques, may be used.

[0051] In one case, a transformation technique may be used for filling the remnant blank space. Accordingly, in the present example, actual corners of the aligned image are determined along with expected corners of the same image. Further, using a transformation matrix-based technique, actual corners of the image may be mapped and transformed to the expected corners. For instance, the transformation matrix may be determined for the actual corners to be transformed to the expected corners and then the transformation matrix can be applied to the aligned image to fill the remnant blank space by transforming the corners of the aligned image. In another case, the remnant blank space can be filled by extrapolating the data points from the filled space to the unfilled space, or by interpolating the data points between the blank space and the reference edge of the image.

[0052] Although aspects of correcting skew in images have been described in a language specific to structural features and/or methods, it is to be understood that the subject matter is not limited to the features or methods described. Rather, the features and methods are disclosed as examples of correcting skew in images.