Content identification method

FIELD OF THE INVENTION

The present invention relates to a method and apparatus for identifying content within a received information signal.

BACKGROUND OF THE INVENTION

Many computers are now connected to the Internet over a high-speed Internet connection such as cable or DSL. Faster connections allow users to download files at far greater speeds than previously possible, allowing the download of large files relatively quickly. With this proliferation of fast connections, the Internet has become a convenient mechanism for users to access media content such as video and music files.

Content sharing websites such as YouTube allow a community of users to each upload their own content, thereby providing access to that content to other members of the community. These websites have, over a relatively short period of time, become a popular way for users to distribute and consume media content, resulting in an enormous body of data that continues to grow at an increasing rate.

Amongst uploaded content it is not unusual to find content that a user has taken from a source, e.g. a content owner's website without authorization. Re-distribution of content without proper authorization, attribution and payment typically violates relevant copyright laws, and results in a media owner being deprived of income to which he is properly entitled. This can also prove problematic for operators of content sharing websites as they may attract liability for distributing media files in violation of copyright laws, when the operators of the websites have no knowledge of the copyright status of the files which are being distributed.

One solution is to require the copyright owner to ask that content be removed from the site. Given the vast and growing body of content on public content sharing sites, it is unfeasible to expect a copyright owner to find and report all instances of content that infringes his copyright. A second, more fundamental problem is that the solution is intrinsically reactive; even if, after request, the content is successfully removed, it will

nonetheless have been available on the website and therefore to the community for a period of time.

A more proactive solution is to automatically filter uploaded content using fingerprints. Using this approach, a fingerprint is stored for each multimedia content item which is the subject of copyright protection. Each fingerprint is based upon properties of the content item. When a new content item is uploaded, its fingerprint can be determined and compared with the fingerprints of content items which are the subject of copyright protection.

Depending on the license associated with any matching content item in the database, the website can either refuse the upload, or arrange appropriate attribution and compensation. One such service is known as MediaHedge and is available from Philips BU Content

Identification.

Typical fingerprinting systems store fingerprints of a particular resolution and can identify continuous segments of content of a certain length (around 10 seconds).

Problems arise when two or more contents are aggressively edited and mixed into a composition, a process known as co-creation. Typical fingerprinting systems can not determine the sources of the material when the segments in the composition are shorter than the particular fingerprint resolution in use.

One potential solution to this problem is to maintain databases with finer resolution fingerprints, allowing for shorter segments to be identified. While using finer resolution fingerprints helps with identification, it also increases the search time and the possibility of false matches.

SUMMARY OF THE INVENTION

It is an object of the present invention to obviate or mitigate one or more of the problems outlined above.

According to a first aspect of the present invention, there is provided a method for identifying content within a received information signal, the method comprising receiving an information signal, generating a first data set by selecting a sub-set of data using filter data, and processing the information signal with reference to the first data set to determine whether the information signal comprises particular content.

In this way, processing of the first information signal is performed with reference to a potentially reduced data set based upon the filter data, thereby reducing the processing burden and increasing efficiency. This allows the first data set to comprise data of

a higher resolution than is otherwise practical thereby increasing the likelihood of identifying the existence of a particular content within an information signal.

The method may further comprise, processing said information signal with reference to a second data set to determine whether said information signal comprises particular content. Processing the information signal with reference to the second data set may be carried out in dependence upon the processing with reference to the first data set.

Processing the information signal with reference to both first and second data sets allows the information signal to be processed with reference to data sets of different resolutions. Accordingly, a relatively small high resolution data set may comprise the first data set while a larger low resolution data set may comprise the second data set. In this way it is possible to determine whether a particular content is included within an information signal by processing the information signal first with reference to a small selected high resolution data set and subsequently with reference to a larger low resolution data set.

The filter data may be based upon an identity associated with a source of the received information signal. The filter data may be arranged to select content associated with the identity, for example content downloaded by the identity. The filter data may be stored in a profile associated with an identity. The filter data may be stored in the profile when content is downloaded by the identity.

The filter data may be selected from the group consisting of IP address, geographical data and media genre preference of said identity.

The first data set may comprise a plurality of fingerprints of a first resolution and the second data set may comprise a plurality of fingerprints of a second resolution. Preferably the first resolution is a relatively high-resolution, while the second resolution is a relatively low-resolution. In this way, embodiments of the invention allow a relatively small number of computationally expensive high resolution fingerprints to be processed alongside a larger number of low resolution and computationally less expensive fingerprints.

The processing with reference to the first data set may comprise generating a high-resolution fingerprint of the received information signal and comparing the high- resolution fingerprint with fingerprints of the first data set. The processing with reference to the second data set may comprise generating a low-resolution fingerprint of the received information signal and comparing the low-resolution fingerprint with fingerprints of the second data set.

The method may further comprise, if said information signal comprises said particular content, performing a first at least one action and if said information signal does not comprise said particular content, performing a second at least one action. The first at least one action may comprise any of preventing the upload of said information signal, limiting distribution of the uploaded information signal, arranging for appropriate attribution or arranging for appropriate compensation. The second at least one action may comprise allowing the upload of the information signal and storing it in a database.

According to a second aspect of the present invention, there is provided, a method of providing a requested information signal, the method comprises receiving a request for an information signal, determining an identity associated with the request, generating and storing a high-resolution fingerprint of the requested information signal in response to the request, updating a profile associated with the identity in response to the request and transmitting the information signal.

Updating a profile may comprise storing a reference to the requested information signal in the profile.

Determining an identity associated with the request may comprise determining an identity of the source of the request.

The invention further provides a computer program for carrying out the methods set out above. Such a computer program can be carried on an appropriate carrier medium. Such a carrier medium may be a tangible carrier medium such as a floppy disk, hard disc, CD or DVD or alternatively an intangible carrier medium such as a communications signal.

A further aspect of the invention provides apparatus for carrying out the methods of the present invention as set out above.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which: Fig. 1 is a schematic illustration of a computer network;

Fig. 2 is a flowchart of a process for downloading a file from the server of Fig. 1 in accordance with an embodiment of the present invention;

Fig. 3 is a flowchart of a process for controlling the uploading of files to the server of Figure 1 in accordance with an embodiment of the present invention; and

Fig. 4 is a decision tree for determining the origin of uploaded files in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS Figure 1 shows a plurality of computers connected to the Internet 1. It can be seen that two user computers 2, 3 are connected to the Internet 1, as is a server computer 4. The user computers 2, 3 and the server computer 4 can transmit data between one another over the Internet 1 using methods which will be readily apparent to those of ordinary skill in the art. The server computer 4 has access to a content database 5 storing a plurality of files that can be downloaded to the user computers 2, 3. The server computer 4 is configured to receive such files from the user computers 2, 3 and store such files in the content database 5. The server computer 4 also has access to a low-resolution fingerprint database 6. The low- resolution fingerprint database 6 stores low-resolution fingerprints of all the content stored in the content database 5. A fingerprint for a particular content item is data generated by processing the particular content item and is based upon characteristics of the particular content item. Low-resolution fingerprints are generated from relatively large sections of content (e.g. 5 or 10 seconds of a video file). The stored fingerprints may then be used to identify whether a received file contains a particular content item by generating a low- resolution fingerprint of the received file and comparing it with each of the low-resolution fingerprints stored in the low-resolution fingerprint database 6. A fingerprint can be used to identify sections of content at least as large the resolution of the fingerprint. For example, given a set of low-resolution fingerprints having a resolution of 5 seconds, identification of particular content in a video file using the low-resolution fingerprints would require the inclusion of at least 5 seconds of the particular content in the video file.

The server computer 4 also has access to a high-resolution fingerprint database 7. High-resolution fingerprints are generated to correspond to relatively small sections of content (e.g. one frame of a video file). Each high-resolution fingerprint can therefore be used to identify particular content based upon a smaller section of that content than is possible using low-resolution fingerprints. For example, identification of particular content in a video file would require the inclusion of only one frame of the particular content in the video file if a high-resolution fingerprint of the described type was used.

The server computer 4 also has access to a database of user profiles 8 for each user of the user computers 2, 3 who accesses the server computer 4. Before communicating

with the server computer 4, a user using one of the user computers 2, 3 first registers their identity with the server computer 4 using methods well known to those skilled in the art. This registration process generates a user profile for the user which is stored in the database of user profiles 8. While the registration process may require a user to actively establish a user profile, alternative techniques involving cookies or profiling of IP addresses may be used.

Figure 2 is a flowchart showing processing carried out when a file is downloaded from the server computer 4 to the user computer 2. It will be appreciated that the process is identical if the file is downloaded to the user computer 3. The processing shown in the flowchart of Figure 2 is carried out by the server computer 4. At step SI a request for a file is received from a user using the user computer

2. At step S2 the server computer 4 determines the identity of the user that has made the request. This may be achieved, for example, by requiring the user to provide a username and password before they are able to download a file. Once the user's identity has been determined, processing passes to step S3 where the corresponding user profile is updated in the database of user profiles 8. The updated corresponding user profile will now indicate that the user has downloaded the requested file and processing passes to step S4. At step S4 the server computer 4 generates a high-resolution fingerprint of the requested file and stores this fingerprint in the high-resolution fingerprint database 7. Processing now passes to step S5 where the server computer 4 sends the requested file to the user. In this way the server computer 4 has access to high-resolution fingerprints of each file that is downloaded from the content database 5.

Figure 3 is a flowchart showing a process for controlling the uploading of a file by a user from the user computer 2 to the content database 5 associated with the server computer 4. It will be appreciated that the process is identical if the file is uploaded from the user computer 3. The processing shown in the flowchart of Figure 2 is carried out by the server computer 4.

At step S6 a file for upload is received from the user computer 2. At step S7 a high-resolution fingerprint of the uploaded file is generated. At step S 8, a subset of high- resolution fingerprints from the high-resolution fingerprint database 7 is selected based upon filter data. In this embodiment the filter data is the list of files previously downloaded by the uploading user, obtained from the user profile of that user stored in the database of user profiles 8. At step S9 the generated high-resolution fingerprint is compared with each high- resolution fingerprint in the subset of high-resolution fingerprints generated at step S8. If the uploaded fingerprint matches one or more fingerprints in the generated subset, then the

uploaded file contains content from one of the files that the user has previously downloaded from the content database 5. The file is therefore rejected and processing terminates at step SlO. If however the generated high-resolution fingerprint does not match a fingerprint in the generated subset processing passes to step SI l. The upload of a particular file may be prohibited for a variety of reasons, for example the upload may be prohibited if the uploading of a particular file would violate relevant copyright laws. In the present case any content included in the content database 5 is considered to be restricted and it is for this reason that uploading is prevented at step SlO.

At step Sl I a low-resolution fingerprint of the uploaded file is generated. At step S 12 the generated low-resolution fingerprint is compared with all of the low-resolution fingerprints in the low-resolution fingerprint database 6. If the generated low-resolution fingerprint matches one or more fingerprints in the low-resolution fingerprint database 6, then the uploaded file contains content from one of the files stored in content database 5. The file is therefore rejected and processing terminates at step S13. If however the generated low- resolution fingerprint does not match a fingerprint in the low-resolution fingerprint database 6, then the upload is allowed and the uploaded file is stored in the content database 5 and the low-resolution fingerprint of the uploaded file is stored in the low-resolution fingerprint database 6.

From the preceding description it can be seen that a check is made based upon a selected sub-set of high-resolution fingerprints. The use of high-resolution fingerprints allows relatively small portions of particular content to be identified, while the use of a subset reduces the time taken to carry out the search. The use of low-resolution fingerprints provides a second level of determination as to whether particular content is contained within uploaded content. Although such low-resolution fingerprints are only able to detect relatively large portions of the particular content, their use provides a backup which can locate copyright protected content not downloaded from the content database 5 by the uploading user and consequently not having a high-resolution fingerprint in the sub-set of high-resolution fingerprints.

The previous description has been concerned with an embodiment in which a determined subset of detailed fingerprints is selected based upon an uploading user's download history stored in the database of user profiles 8. It will be appreciated that other criteria may be used. Indeed, the present invention may use one of, or any combination of criteria, including but not limited to, IP address, demographic information and preferred media genres.

Further, the previous description concerned an embodiment of the present invention in which an uploaded file is rejected if it contains content from a file stored in the content database 5. It will be appreciated that in alternative embodiments of the present invention, the system may take a number of actions depending upon the copyright owners requirements. Potential actions may include, but are not limited to, arranging for the correct attribution, arranging for any necessary payment, or simply allowing or denying the upload.

Figure 4 is a decision tree showing the decisions taken to identify the origin of uploaded files in accordance with a generalized embodiment of the present invention. A file, C ₁ , is received at the server computer 4. At decision 9 it is then determined whether:

that is, whether the high-resolution fingerprint of the file C ₁ is a member of the set of high- resolution fingerprints stored in the high-resolution fingerprint database 7. If the high- resolution fingerprint of C ₁ is contained within the high-resolution fingerprint database 7, then the file C ₁ has been downloaded from the content database 5, and is a member of the set of downloaded files Dd. It is therefore necessary to check the license of the file C ₁ to determine an appropriate course of action, for example arranging appropriate attribution.

If however the high-resolution fingerprint Of C ₁ is not contained within the high-resolution fingerprint database 7, at decision 10 it is determined whether:

F _L (C ₁ ) e D _L

that is, whether the low-resolution fingerprint a member of the set of low-resolution fingerprints stored in the low-resolution fingerprint database 6. If the low-resolution fingerprint Of C ₁ is contained within the low-resolution fingerprint database 6, then the file is contained within the content database 5. It is therefore necessary to check the license of the file C ₁ to determine an appropriate course of action.

If the low-resolution fingerprint is not contained within the low-resolution fingerprint database 6, then C ₁ is not contained within the content database 5, and it is therefore assumed that C ₁ is new content.

The low and high-resolution fingerprints can be obtained from content in a variety of ways known to those skilled in the art. For example, a particular embodiment which processes audio-visual content, may calculate the low-resolution fingerprints as a fixed

length binary string representing each group of frames corresponding to 5 seconds of video, while the high-resolution fingerprints may be another fixed length binary string obtained from each frame. An embodiment arranged in this way would allow the system to identify individual frames from a content that a client had previously downloaded from the server, while the system would require 5 seconds of content for correct identification of material that the client had not previously downloaded from the server.

It will be appreciated that the embodiments of the invention described above can be used to control the upload of files of any type to a database. For example, embodiments of the invention can be used with audio, video, image, graphics and multimedia files.

It will be appreciated that the computer network of Figure 1 is merely exemplary. In many embodiments of the invention the network would include more than two user computers 2, 3.

The term "database" as that term is used herein is to be construed broadly to cover any means for storing data such that the stored data can be retrieved. That is, while the term covers formalized database management systems it also includes any data storage system such as for example a file storage system of the type provided by many operating systems.

Embodiments of the present invention have been described above. It will however be appreciated that various modifications can be made to the described embodiments without departing from the scope of the appended claims.