IMAGES FEATURING MULTIPLE DOCUMENTS

CROSS-REFERENCE TO RELATED APPLICATIONS

[001] This application claims the benefit of U.S. Provisional Application No. 62/1 1 1 ,690 filed on February 4, 2015, the contents of the above referenced applications are herein incorporated by reference.

TECHNICAL FIELD

[002] The present disclosure relates generally to analyzing documents containing images, and more particularly to generating a plurality of files based on images containing multiple documents.

BACKGROUND

[003] The Value-Added Tax (VAT) or sales tax is a broadly based consumption tax assessed on the value added to goods and services. A particular VAT applies to most goods and services that are bought or sold within a given country or state. When a person travels abroad and makes a purchase that requires paying a VAT (or any applicable tax), that person may be entitled to a subsequent refund of the VAT for the purchase. Other taxes applied to purchases may similarly be refunded under particular circumstances. Further, sellers may offer rebates for purchases of products sold in certain locations and under particular circumstances. Such refunds of the purchase price may be reclaimed by following procedures established by the refunding entity.

[004] The laws and regulations of many countries allow foreign travelers the right for reimbursement or a refund of certain taxes such as, e.g., VATs paid for goods and/or services abroad. As such laws and regulations are different from one country to another, determination of the actual VAT refunds that one is entitled to receive often requires that the seeker of the refund possess a vast amount of knowledge in the area of tax laws abroad. Moreover, travelers may seek refunds for VATs when they are not entitled to such refunds, thereby spending time and effort on a fruitless endeavor. Further, availability of the VAT refund may vary based on the type of purchase made and the presence of a qualified VAT receipt.

[005] One procedure to request a refund is to physically approach a customs official at an airport, fill out a form, and file the original receipts respective of the expenses incurred during the visit. This procedure should be performed prior to checking in or boarding to the next destination. Additionally, particularly with respect to goods purchased abroad, the procedure to request a refund may require that the payer show the unused goods to a custom official to verify that the goods being exported match the goods that the payer paid VATs on.

[006] As travelers are not familiar with specific laws and regulations for claiming a refund, the travelers may submit a claim for a refund even though they are not eligible. This procedure further unnecessarily wastes time if the traveler ultimately learns that he or she is not entitled to a refund. It would therefore be advantageous to provide a solution that would overcome the deficiencies of the prior art by providing an effective way to handle VAT refunds electronically and, preferably, over the Internet.

[007] Furthermore, due to the hassles associated with claiming refunds and, in particular, VAT refunds, customers may not be motivated to seek such refunds. Particularly with respect to potentially large refunds, properly managed refunding platforms may be crucial for saving money. As an example, a VAT refunding platform may be important to large enterprises requiring their employees to travel for business purposes. Due to the massive amount of invoices generated by a typical enterprise, many of which may be eligible for VAT refunds, enterprises may be prone to errors during collection and verification of invoices.

[008] Additionally, the large numbers of invoices generated by a typical enterprise ultimately results in creation of a multitude of files corresponding to the invoices. Existing solutions typically require that each invoice is contained in a separate file and, consequently, require individual scanning or otherwise capturing of each invoice. Such manual individual scanning wastes time and resources, and ultimately subject the process to more potential for human error.

[009] It would therefore be advantageous to provide a solution that would overcome the deficiencies of the prior art. SUMMARY

[0010] A summary of several example embodiments of the disclosure follows. This summary is provided for the convenience of the reader to provide a basic understanding of such embodiments and does not wholly define the breadth of the disclosure. This summary is not an extensive overview of all contemplated embodiments, and is intended to neither identify key or critical elements of all embodiments nor to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more embodiments in a simplified form as a prelude to the more detailed description that is presented later. For convenience, the term "some embodiments" may be used herein to refer to a single embodiment or multiple embodiments of the disclosure.

[0011] Some exemplary embodiments disclosed herein include a method for extracting document images from images featuring multiple documents. The method comprises receiving a multiple-document image including a plurality of document images, wherein each document image is associated with a document; extracting a plurality of visual identifiers from the multiple-document image, wherein each visual identifier is associated with one of the plurality of document images; analyzing the plurality of visual identifiers to identify each document image; determining, based on the analysis, an image area of each document image; extracting each document image based on its image area.

[0012] Some exemplary embodiments disclosed herein also include a system for extracting document images from images featuring multiple documents. The system comprises a processing system; and a memory, the memory containing instructions that, when executed by the processing unit, configure the system to: receive a multiple-document image including a plurality of document images, wherein each document image is associated with a document; extract a plurality of visual identifiers from the multiple- document image, wherein each visual identifier is associated with one of the plurality of document images; analyze the plurality of visual identifiers to identify each document image; determine, based on the analysis, an image area of each document image; and extract each document image based on its image area.

BRIEF DESCRIPTION OF THE DRAWINGS [0013] The subject matter disclosed herein is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the disclosed embodiments will be apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0014] Figure 1 is a network diagram utilized to describe the various disclosed embodiments.

[0015] Figure 2 is a flowchart illustrating a method for generating a plurality of invoice images from images featuring multiple invoices according to an embodiment.

[0016] Figures 3A through 3C are flowcharts illustrating methods for extracting invoice images from images featuring multiple invoices according to various embodiments.

[0017] Figures 4A through 4E are exemplary screenshots of images featuring invoices utilized to describe the various disclosed embodiments.

DETAILED DESCRIPTION

[0018] It is important to note that the embodiments disclosed herein are only examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claimed embodiments. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise indicated, singular elements may be in plural and vice versa with no loss of generality. In the drawings, like numerals refer to like parts through several views.

[0019] Fig. 1 shows an exemplary and non-limiting network diagram 100 utilized to describe the various disclosed embodiments. In an embodiment, the network diagram 100 includes a network 1 10 communicatively connected to a server 120, a user device 150, an enterprise device 160, a plurality of web sources 170-1 through 170-n (hereinafter referred to individually as web sources 170 and collectively as web sources 170, merely for simplicity purposes), and a database 180. The network 1 10 may be, but is not limited to, a wireless, cellular or wired network, a local area network (LAN), a wide area network (WAN), a metro area network (MAN), the Internet, the worldwide web (WWW), similar networks, and any combination thereof.

[0020] The user device 150 and each enterprise device 160 may be, but is not limited to, a personal computer (PC), a notebook computer, a cellular phone, a smartphone, a tablet device, a wearable computing device, a scanner, and so on. The user device 150 may include or be communicatively connected to an image sensor 155 utilized to capture images. An enterprise operating an enterprise device 160 may be, but is not limited to, a hotel, a shop, a service provider, and so on.

[0021] In an embodiment, the user device 150 captures an image (e.g., via the image sensor 155) containing multiple invoices and/or other documents. Each invoice typically includes a proof of payment for a potentially refundable purchase. The documents may be in an unorganized form, i.e., the invoices do not need to be arranged, oriented, or otherwise organized in a particular manner so long as information (e.g., words, symbols, numbers, characters, shapes, matrices, labels, barcodes, and so on) in each document is visible in the multiple-invoice image.

[0022] The user device 150 sends the captured multiple-invoice image to the server 120. The server 120 is configured to extract visual identifiers from the multiple-invoice image. The visual identifiers may include, but are not limited to, a document identification number (e.g., an invoice number), a code (e.g., a QR code, a bar code, etc.), a transaction number, a name of a business, an address of a business, an identification number of a business, a total price, a currency, a method of payment (e.g., cash, check, credit card, debit card, digital currency, etc.), a date, a type of product, a price per product, and so on.

[0023] To this end, the server 120 may include or may be communicatively connected to a recognition unit (RU) 125. The recognition unit 125 is configured to execute machine imaging processes. The recognition unit 125 is further configured to enable recognition of the visual identifiers shown in the multiple-invoice image by using one or more computer vision techniques such as, but not limited to, image recognition, pattern recognition, signal processing, character recognition, and the like. The recognition unit 125 may include, but is not limited to, an optical character recognition unit, an image recognition unit, and a combination thereof

[0024] The server 120 is configured to analyze the extracted visual identifiers to identify invoice images illustrated in the multiple-invoice image. The identification may be based on a threshold of visual identifiers required for determining an invoice based on the visual identifiers. The visual identifier threshold may represent the minimum visual identifiers needed for identifying an invoice in the multiple-invoice image. The threshold may include, but is not limited to, a minimum number of visual identifiers, a particular visual identifier, a particular combination of visual identifiers, and so on. For example, a threshold requirement for identifying an invoice in an image based on visual identifiers may include a total price, a merchant identifier, and a type of product. In that example, each identified invoice in the multiple-invoice image will include a total price, a merchant identifier, and a type of product.

[0025] In an embodiment, the server 120 may be configured to determine whether any visual identifiers are required and, if so, retrieving the required visual identifiers. The required visual identifiers may be predefined such that each invoice that is identified will be suitable for further processing. For example, for a value added tax (VAT) reclaim, a location of a purchase may be a required visual identifier. Thus, if an invoice does not include a visual identifier indicating a location of the transaction, the location associated with the invoice may be retrieved.

[0026]The required visual identifiers may be retrieved from, e.g., the enterprise device 160 and/or the web sources 170. The web sources 170 may be, but are not limited to, databases in which data regarding reclaim information is stored. Such databases may include, for example, VAT information exchange systems (VIESs), tax authority databases, rebate sharing systems, and so on. Each web source 170 may be operated by an entity such as, but not limited to, a tax authority, a VAT refund agency, and the like.

[0027]The server 120 is configured to determine an image area associated with each of the identified invoice images based on the analysis. Each image area includes the visual identifiers of its respective invoice image and indicates the boundaries of the invoice image within the multiple-invoice image. The determination may include, but is not limited to, identifying a center of an invoice, identifying boundaries of each invoice image, and so on. In an embodiment, identifying the boundaries of each invoice image may be based on clean areas in the multiple-invoice image, i.e., portions of the captured image where no text appears may be identified as boundaries of the invoice images. Each image area may be a particular shape defined by its boundaries such as, for example, rectangular (i.e., a typical invoice contains text within a rectangular area). The image areas may be the same or different shapes defined by their respective boundaries. [0028] The server 120 is configured to extract each identified invoice image based on its respective determined image area. The extraction may include creating a file for each invoice image. The extraction may further include, but is not limited to, cutting, copying, cropping each identified invoice image. Extraction via cutting may include removing each invoice image from the captured image and generating a new file for each removed invoice image such that, after extraction, the multiple-invoice image does not feature any invoices. Extraction via copying may include generating a new file for each invoice image including a copy of the invoice image such that, after extraction, the multiple-invoice image still contains all copied invoice images. Extraction by cropping includes generating a file containing a copy of the multiple-invoice image for each identified invoice image and shrinking each file based on its respective invoice image such that each file contains only the respective cropped invoice image.

[0029] The server 120 may be configured to store each extracted invoice image separately in, e.g., the database 180. The invoice images stored in the database 180 may be subsequently accessed for processing (e.g., VAT reclaim processing). In an embodiment, the server 120 may be further configured to automatically submit a VAT reclaim for any or all of the extracted invoice images. The VAT reclaim may be submitted to a refund agency via, e.g., one of the web sources 170 as described further in US Patent Application No. 14/836,230, assigned to the common assignee, which is hereby incorporated by reference for all that it contains.

[0030] The server 120 typically includes a processing system 122 coupled to a memory 124.

The processing system 122 may comprise or be a component of a processor (not shown) or an array of processors coupled to the memory 124. The memory 124 contains instructions that can be executed by the processing system 122. The instructions, when executed by the processing system 122, cause it 122 to perform the various functions described herein. The one or more processors may be implemented with any combination of general-purpose microprocessors, multi-core processors, microcontrollers, digital signal processors (DSPs), field programmable gate array (FPGAs), programmable logic devices (PLDs), controllers, state machines, gated logic, discrete hardware components, dedicated hardware finite state machines, or any other suitable entities that can perform calculations or other manipulations of information. [0031] The processing system 122 may also include machine-readable media for storing software. Software shall be construed broadly to mean any type of instructions, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise. Instructions may include code (e.g., in source code format, binary code format, executable code format, or any other suitable format of code). The instructions, when executed by the one or more processors, cause the processing system to perform the various functions described herein.

[0032] It should be noted that the embodiments described herein above with respect to Fig.

1 are discussed with respect to a single user device 150 and a single enterprise device 160 merely for simplicity purposes and without limitation on the various disclosed embodiments. Multiple user devices 150 and/or enterprise devices 160 may be utilized without departing from the scope of the disclosure.

[0033] Fig. 2 is an exemplary and non-limiting flowchart 200 illustrating a method for generating a plurality of invoice image files based on a multiple-invoice image according to an embodiment. In an embodiment, the method may be performed by a server (e.g., the server 120).

[0034] In S210, an image featuring multiple invoices is received. The invoices in the multiple- invoice image may be unorganized such that they are not suitable for immediate processing.

[0035] An exemplary and non-limiting multiple-invoice image may be seen in Fig. 4A, which shows an exemplary and non-limiting screenshot 400A illustrating a multiple-invoice image 410 including a plurality of invoices. The invoices are unorganized such that some of the invoices are upside down, rotated, and positioned at random sections within the multiple-invoice image 410. Each invoice includes information related to a purchase of a good or service.

[0036] In S220, visual identifiers are extracted from the multiple-invoice image. Each visual identifier indicates information related to an invoice in the multiple-invoice image. The visual identifiers may include, but are not limited to, a document identification number (e.g., an invoice number), a code (e.g., a QR code, a bar code, etc.), a transaction number, a name of a business, an address of a business, an identification number of a business, a total price, a currency, a method of payment (e.g., cash, check, credit card, debit card, digital currency, etc.), a date, a type of product, a price per product, and so on. Threshold visual identifier requirements (e.g., a number or particular group of visual identifiers) may be identified based on a type of entity for which the multiple-invoice image was captured.

[0037] In S230, the extracted visual identifiers are analyzed. The analysis may yield identification of metadata associated with the multiple-invoice image. Such metadata may include, but is not limited to, a number of invoice images in the multiple-invoice image, pointer data indicating an invoice image available via one or more storage units (e.g., the enterprise device 160 or the web sources 170), a purchaser of a transaction, and so on.

[0038] In S240, an image area of an invoice image featured in the multiple-invoice image is determined based on the analysis. In an embodiment, the determination may include identifying a boundary of each invoice illustrated in the multiple-invoice image. The image area of an invoice may be defined as the area contained within the boundary of the invoice.

[0039] Exemplary determined image areas may be seen in Fig. 4B, which shows an exemplary and non-limiting screenshot 400B illustrating a multiple-invoice image 410 including a plurality of invoices, with an invoice image of each invoice defined by an image area within boundaries 420-1 through 420-9 (hereinafter referred to individually as a boundary 420 and collectively as boundaries 420, merely for simplicity purposes). In the exemplary screenshot 400B, each boundary 420 is rectangular and occupies a textless border around each invoice.

[0040] In S250, the invoice image is extracted from the multiple-invoice image respective of its image area. The extraction may include generating a new file for the invoice image, and may further include cutting, cropping, and/or copying the invoice image in the captured image. Exemplary methods for extracting invoice images from a multiple-invoice image are described further herein below with respect to Figs. 3A through 3C.

[0041] Extracting invoice images from a multiple-invoice image via cutting may be seen in Fig. 4C, which shows an exemplary and non-limiting screenshot 400C illustrating the multiple-invoice image 410 including the plurality of invoices with invoice images defined by the boundaries 420. In the exemplary screenshot 400C, the invoice image 425-7 enclosed by the boundary 420-7 has been cut from the captured image. Additional invoice images may be further cut from the captured image as demonstrated in Fig. 4E until all invoice images identified in the multiple-invoice image have been removed.

[0042] Fig. 4D shows an exemplary and non-limiting screenshot 400D illustrating the cut invoice image 425-7. A new file including only the cut invoice image 425-7 may be generated based on the cutting.

[0043] In optional S260, the extracted invoice image may be stored as a file in, for example, a database (e.g., the database 180). Stored invoice images may be subsequently processed further. For example, stored invoice images may be analyzed for value added tax (VAT) reclaim eligibility and/or sent to a refund agency.

[0044] In S270, it is determined whether additional invoice images are to be extracted from the multiple-invoice image and, if so, execution continues with S210; otherwise, execution terminates.

[0045] Extraction of an additional invoice image from a multiple-invoice image may be seen in Fig. 4E, which shows an exemplary and non-limiting screenshot 400E illustrating the multiple-invoice image 410 including the plurality of invoices with invoice images defined by the boundaries 420. In the exemplary screenshot 400E, the invoice image 425-9 enclosed by the boundary 420-9 has been cut from the multiple-invoice image in addition to the invoice image 425-7 enclosed by the boundary 420-7. Additional cuts would therefore remove each of the invoice images enclosed by the boundaries 420-1 through 420-6 and 420-8 until the multiple-invoice image contains no invoice images.

[0046] Fig. 3A is an exemplary and non-limiting flowchart S250A illustrating a method for extracting an invoice image from a multiple-invoice image via cutting according to an embodiment.

[0047] In S31 OA, an invoice image featured in a multiple-invoice image is identified based on its image area. In S320A, the identified invoice image is cut from the multiple-invoice image. The cut image is removed from the captured image such that it is no longer featured in the multiple-invoice image. In S330A, a new file including the cut invoice image is generated. In S340A, the generated file may be stored in, e.g., a database.

[0048] Fig. 3B is an exemplary and non-limiting flowchart S250B illustrating a method for extracting an invoice image from a multiple-invoice file via cropping according to an embodiment. [0049] In S31 OB, an invoice image featured in a multiple-invoice image is identified based on its image area. In S320B, a file including the multiple-invoice image is generated. In S330B, the new file is cropped respective of the identified invoice image. The cropping may include shrinking the size of the generated file such that the cropped file only includes the invoice image. In S340B, the cropped new file may be stored in, e.g., a database.

[0050] Fig. 3C is an exemplary and non-limiting flowchart S250C illustrating a method for extracting an invoice image from a multiple-invoice file via copying according to an embodiment.

[0051] In S31 OC, an invoice image featured in a multiple-invoice image is identified based on its image area. In S320C, the identified invoice image is copied from the multiple-invoice image. In S330C, a file including the copied invoice image is generated. In S340C, the generated file may be stored in, e.g., a database.

[0052] It should be noted that the embodiments described herein above are discussed with respect to an image featuring multiple invoices merely for simplicity purposes and without limitations on the disclosed embodiments. Images featuring other documents may be utilized without departing from the scope of the disclosure. It should be further noted that visual identifiers other than those related to VAT reclaims may be utilized to identify documents captured within images according to the disclosed embodiments. It should be further noted that the analyzed image may be either captured and sent (e.g., to the server 120) for invoice image extraction, or may be retrieved from a database, without departing from the scope of the disclosure.

[0053] The various embodiments disclosed herein can be implemented as hardware, firmware, software, or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium consisting of parts, or of certain devices and/or a combination of devices. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units ("CPUs"), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such a computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit. Furthermore, a non-transitory computer readable medium is any computer readable medium except for a transitory propagating signal.

54] All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the principles of the disclosed embodiment and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosed embodiments, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.