Internet computer network

Field of the Invention

[1] This invention relates generally to a system for tracking the communication of image datafiles across an Internet computer network.

Background of the Invention

[2] It would be desirous to be able to track the communication of image data files across an Internet communication network for various purposes, including for marketing analytics purposes.

[3] However, it is currently difficult to track image data including for reasons that webhosted image data files may be manipulated differently by differing web servers, including by way of image cropping and rotation (referred to as a "geometric attack" as is substantially shown in Figures 4 and 5), normalisation and the like.

[4] The present invention seeks to provide a way which will overcome or substantially ameliorate at least some of the deficiencies of the prior art, or to at least provide an alternative.

[5] It is to be understood that, if any prior art information is referred to herein, such reference does not constitute an admission that the information forms part of the common general knowledge in the art, in Australia or any other country.

Summary of the Disclosure

[6] There is provided herein a system for tracking the communication of image data files across an Internet computer network including in ways that addresses or at least substantially ameliorates potential image manipulation by differing web servers.

[7] As such, according to one aspect, the system comprises a tracking server in operable communication with the plurality of electronic devices across the Internet computer network. The system is configured for the tracking server to receive image data files comprising image data and associated image meta data taken by image sensors of electronic devices. The tracking server generates an image identifier using the image data file which, in embodiments, comprises one or more of meta data analysis, image processing including digital watermarking, image identifier signature generation and object recognition.

[8] The electronic device then communicates the image data file to at least one web server across the Internet computer network for hosting. Such web servers may include social platform, microblogging, image database web servers and the like.

[9] The tracking server is then configured for requesting a resource served by the at least one web server and identifying the image identifier from the resource so as to be able to track the communication of the image data file accordingly. The identity of the web server and embodiments, a resource identifier (such as is associated with the user profile) may be provided to the tracking server by the electronic device.

[10] If the associated image file meta data is obtainable from the resource data, the tracking server is able to identify the associated image, thereby offering an image manipulation invariant solution. The tracking server may implement DOM (document object model) tree parsing or traversal to identify image data elements associated with metadata containing elements including in accordance with specific known DOM formats of particular servers or resources.

[11] Additionally, or alternatively if the meta data is not made available in the resource, the tracking server may implement digital watermarking and/or image identifying signature generation, including image manipulation invariant digital watermarking and image identifying signature generation for those web servers manipulating the images to track the image data to a degree of accuracy.

[12] In embodiments, the image identifying signature generation may implement image processing for object recognition to identify objects within image data, so as to be able to track the communication of the image data by identifying those same objects from image data in the resource.

[13] In embodiments, the tracking server is further configured for identifying data indicative of user interaction with a detected/matched image data file, including text matching and analysis techniques. In this way, not only can the communication of the image data be tracked, but also the extent of user interaction therewith.

[14] Other aspects of the invention are also disclosed.

Brief Description of the Drawings

[15] Notwithstanding any other forms which may fall within the scope of the present invention, preferred embodiments of the disclosure will now be described, by way of example only, with reference to the accompanying drawings in which:

[16] Figure 1 shows an interaction between a tracking server and an electronic device within a system for system for tracking the communication of image datafiles across an Internet computer network in accordance with an embodiment;

[17] Figure 2 shows exemplary processing performed by the tracking server and the electronic device in accordance with an embodiment;

[18] Figure 3 illustrates differing types of web servers to which image data files may be communicated in the manner described herein in accordance with embodiments; and

[19] Figures 4 - 5 show various image data file examples. Description of Embodiments

[20] There is provided herein a system for tracking the communication of image data files across an Internet computer network.

[21] Figure 1 shows a tracking server 100 in operable communication with an electronic device 115 across the Internet computer network.

[22] Each computer device comprises a processor 101 for processing digital data. In operable communication with the processor 101 across a system bus is a memory device 108. The memory device 108 is configured for storing digital data including computer program code instructions. As such, during use, the processor 101 fetches these computer program code instructions from the memory device 101 for interpretation and execution. Data results may be restored within the memory device 108.

[23] For illustrative convenience, these computer program code instructions have been shown as having been divided into various computer program code instruction modules.

[24] The electronic device 115 may comprise a wireless processor 102 for communicating across a wireless network with the tracking server 100. In this regard, the tracking server 100 comprises a corresponding network interface 114.

[25] The electronic device 115 may comprise a digital display device 103 for the display of digital data. A touch controller 104 may be overlaid the digital display 103 for haptic interaction with the information presented thereon.

[26] The electronic device 115 comprises image sensors 105 for the capturing of image data in the manner described herein.

[27] The electronic device 115 may further comprise a power controller 106 for managing the power of the electronic device 115 and furthermore, a GPS processor 106 for ascertaining the location of the electronic device.

[28] The computer program code instruction modules of the electronic device 115 may comprise an image capture controller 115 for the capturing of image data 118 utilising the image sensors 105.

[29] Furthermore, the computer program code instruction modules of the electronic device 115 may comprise an image data communication controller 117 for sending image data files to a plurality of web servers across the Internet computer network.

[30] The tracking server 100 is configured for receiving an image data file comprising the image data 118 and associated image meta data from the electronic device 115.

[31] In this regard, the tracking server 100 computer program code instruction modules may comprise an image data analysis controller 115 which may analyse the image data in the manner described in further detail below. The computer program code instruction modules may further comprise an image identifier generator controller 120 configured for generating an image identifier utilising the image data file.

[32] Furthermore, the computer program code instruction modules may further comprise a network communication tracking controller 121 configured for requesting resources served by a plurality of web servers for identifying the image identifier therein for tracking the communication of the image data file across the Internet computer network.

[33] Figure 2 shows exemplary processing 200 implemented by the system.

[34] The processing 200 comprises the capture of image data 201 by the electronic device 115 utilising the image sensors 105. In embodiments, the image data 118 captured maybe self-portrait image data 400 such as which may be captured by a forward facing image sensor 105 of the electronic communication device 115 is a substantially shown in Figure 4.

[35] The electronic device 115 sends an image data file, comprising the image data captured by the image sensor 105 and associated metadata, to the tracking server 100. In embodiments, the associated metadata may be user generated such as image title, image description, image tags and the like. In alternative or additional embodiments, the associated meta data may be image meta data such as EXIF (Exchangeable image file format) meta data and the like.

[36] In embodiments, the image data analysis controller 119 may perform data verification analysis 202 to verify the image data file. Data verification analysis may comprise meta data verification 203 wherein the analysis controller 119 analyses the associated meta data of the image data file.

[37] In one embodiment, the associated meta data comprises location data, including that which may be obtained utilising the GPS processor 106 of the electronic device 115 (or network access location analysis technique). As such, the metadata verification 203 may verify that the location meta data falls within a preconfigured geo-fence region. In this way, the system is configured for tracking image data emanating from specific regions. In embodiments, the server 100 may continuously receive location data from the electronic devices 115 so as to detect when an electronic device 115 enters a specific region so as to be able to send a notification alert to the electronic device 115.

[38] In embodiments, the data verification analysis 202 may comprise object recognition verification 204 wherein object recognition is performed to verify that the image data comprises certain objects. For example, object recognition may employ facial recognition to verify that the image comprises at least one human face. For example, for the exemplary image provided in Figure 6, the object recognition may detect two human faces. In this way, the system is configured for tracking image data comprising certain objects, such as human faces.

[39] Object recognition may further comprise identifying specific types of objects 603 within the image, such as a Coke can like. Such object type identification may be made with reference to an object type reference database, such as, for example, a database comprising exemplary images of Coke can objects such that the Coke can type objects may be identified within the image utilising shape, textual and/or colorimetric comparison or correlation techniques.

[40] The data verification analysis 202 may further comprise image data file classification 205 wherein the image data file is classified according to various parameters.

[41] Such classification may comprise analysis of the associated meta data so as to, for example, classify the image data file according to location.

[42] In further embodiments, such classification 205 may further comprise image analysis of the image data of the image data file. For example, image analysis may comprise the above alluded to facial analysis so as to, for example, analyse the number of human faces within the image data file. As such, in this way, the image data file may be classified according to the number of human faces within the data file.

[43] Further image analysis may be performed wherein, for example, in one embodiment, the image analysis may analyse the gender of human faces or other demographic data from the image data within the image data file. In this way, the classification 205 may, for example, classify an image as comprising three Asian females and one Caucasian male.

[44] The processing 200 may further comprise image identifier generation 206 wherein a unique identifier is generated for the image data file. As will become apparent from the ensuing description, this unique identifier may be utilised for tracking the communication of the image data files for the image data therein across the Internet computer network.

[45] In embodiments, the image identifier 206 may comprise a meta data image identifier generator 216 configured to generate the image identifier from the associated meta data. As such, when the image data file is subsequently disseminated across the Internet computer network with such associated meta data, the image data may be identified from the metadata associated therewith for tracking. The metadata image identifier generator 216 may log the meta data contained therein so as to perform string matching utilising text contain during the subsequently retrieved web resources. Alternatively, the meta data image identifier generator 216 may create a hash of the meta data for matching. As will be described in further detail below, in embodiments, the electronic device 115 may send server or resource identifiers to the server 100 such that the server 100 is able to search or parse information from the particular resources in accordance with a particular format to identify the meta data contained therein.

[46] However, the image identifier generation may comprise image analysis and manipulation techniques. Specifically, in the embodiment shown, the image identifier generation 206 may comprise digital image watermarking 207 wherein a digital watermark is embedded within the image data. [47] In a preferred embodiment, the watermarking is image manipulation invariant watermarking 207 so as to be able to subsequently identify the watermark to a reasonable certainty despite geometric attacks such as rotation and cropping. Specifically, Figure 5 shows an embodiment of image data 500 that has been cropped from image data 400 shown in Figure 4. As such, in accordance with this embodiment, the crop image 500 is recognisable utilising the watermark.

[48] In one embodiment, the cropping invariant digital watermark may comprise addition of noise across the entire image data file at certain frequencies such that a spectral frequency (i.e. Fourier transform analysis) of even a subset region of the image data may identify these signature frequencies.

[49] In one embodiment, an image identifier signature may be generated for the image data. As such, when subsequently retrieving images from the resource, signatures for those retrieved images may be similarly generated and then compared to the initially generated image identifier signature for correlating images.

[50] Similarly, the image identifier signature may be image manipulation invariant.

[51] Image identifier generation may further comprise object recognition 209. Any identified objects may be subsequently recognised with a images of the resource retrieved by the tracking server.

[52] In one embodiment, image analysis is performed to identify a person within the image data. As such, the identified person may be subsequently identified from images across the Internet computer network for correlation.

[53] Object recognition may identify a plurality of objects for increasing correlation reliability. For example, in one embodiment, the object identification may identify a person and an associated object 603 within the image such as, for example, a Coke can object. As such, subsequent images comprising both the same person under same type of object 603 may be identified to a degree of certainty.

[54] The processing 200 further comprises the communication 210 of the image data file, or at least the image data therein to a plurality of web servers across the Internet computer network. As alluded to in Figure 3, these web servers may serve webpages 311, image databases 312, bookmark platforms 313, microblog platforms 314, social platforms 315 and the like.

[55] As such, the tracking server 100 is configured for requesting resources served by these web servers for identifying the image identifier from the resources for tracking the communication of the image data files across the Internet computer network.

[56] As alluded to above, web server identifiers (such as a URL) may be provided by the electronic device 115 to the server 100 for reducing the number of web servers being monitored by the tracking server 100. Additionally, a resource identifier (such as social user profile hand or the like) may further specify the resource served by a particular server further narrowing down the number of resources required to be retrieved by the tracking server 100.

[57] The specification of the web server or resource identifier may further allow the server 100 to parse the resource data in accordance with a known format, including a DOM (document object model) format, specific to the web server or resource.

[58] Where object identification comprises watermarking, the tracking server 100 may send the watermark image to the electronic device 115 which is then communicated to the web servers accordingly.

[59] In accordance with a first embodiment, the electronic device 115 communicates the image to a social media platform 315.

[60] As such, the communication tracking 212 may comprise social platform API authentication and social data feed and social graph inspection 212 for tracking the communication of the image data file or image data.

[61] For example, the electronic device 115 may communicate the image data file to the social platform 315 and furthermore communicate a notification of such to the tracking server 100 including a social profile identifier or authentication credentials such that the tracking server 100 is able to authenticate, via an API 307 with the social media platform 315 for verifying the communication and subsequent publishing/posting by way of social media data feed via the social platform 315.

[62] For example, a social media profile data feed, as identified by a social media user identifier, may be retrieved by the API 307 by the tracking server 100 for the performance of meta data analysis 306 and/or image analysis 305.

[63] In embodiments, the meta data may be displayed within the resource in association with the image data in a particular manner, such as within HTML elements of a specific format. For example, the resource may comprise an HTML DIV element for a post, the DIV element comprising children and, image and paragraph elements for the title meta data, image data and descriptive metadata accordingly. As such, the meta data analysis 306 may comprise DOM traversal to identify the meta data displayed in a particular format. Differing formats may be utilised for differing web servers.

[64] As such, the tracking server 100 may analyse the social media data feed to identify the image meta data of the image data file previously provided to the tracking server 100 to ascertain that the image data file was in fact communicated to the social media platform 315 and communicated accordingly.

[65] In embodiments, as opposed to the tracking server 100 only detecting the serving of the image data file by the social media platform 315 within the social media data feed resource, the tracking server 100 may perform user interaction analysis indicative in relation to the image data file within the social media platform 315. For example, the tracking server 100 may parse and or analyse social media feed data to identify subsequent social media user interaction indicators such as, for example, in one embodiment, the number of likes, number of shares, number of comments and the like.

[66] In embodiments, the tracking server 100 may perform textual analysis of the comments associated with the social media post comprising the image data file to ascertain at least one of user demographic and tonality, such as, for example, positive or negative tones.

[67] In em\bodiments, the server 100 may similarly employ controversial to extract the applicable user interaction indicative information in accordance with specific known formats for specific servers or resources.

[68] In embodiments, utilising the API 307, the tracking server 100 may inspect the social graph 318 of the social media platform 315 to identify associated social media profile so as to be able to similarly inspect such associated social media profiles to further monitor the further dissemination/communication and user interaction of the image data file.

[69] As alluded to above, in embodiments, the tracking server 100 may be configured for requesting resources from other types of web servers for monitoring the communication and associated user interaction with the image data file.

[70] As is shown, these web servers may serve webpages 315 which are crawled utilising a crawler 301 and analysed accordingly other utilising textual meta data analysis 306 or image analysis 305. In this way, the tracking server 100 is configured for detecting the image identifier across a plurality of webpages.

[71] The web servers may further comprise serve image databases 312. In this way, the tracking server 100 may further crawl the image databases 300 using the crawler 312 to perform meta data or image analysis 306, 305 in the manner described herein.

[72] In further embodiments, the tracking server 100 may authenticate with the image database tracking server 312 utilising an API utilising credentials or user profile identifier as provided by the electronic device 115.

[73] In embodiments, the tracking server 100 may ascertain user interaction by monitoring bookmarking platforms 313.

[74] In embodiments, the tracking server 100 may retrieve data from microblog platforms 314, such as by way of a "firehose" API 308 so as to perform meta data analysis thereon. Such meta data analysis may comprise identification of URLs associated with the image data file.

[75] It should be noted that whereas the embodiments described him have been directed to the tracking of image data, the embodiment is provided herein may be equally applicable for the tracking of video data. [76] The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that specific details are not required in order to practice the invention. Thus, the foregoing descriptions of specific embodiments of the invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed; obviously, many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, they thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the following claims and their equivalents define the scope of the invention.