Instant messenger with intelligent music selection

FIELD OF THE INVENTION

This invention relates to a system, a method of operating the system, a server device for use in the system, and to a computer program product on a computer readable medium.

BACKGROUND OF THE INVENTION

Currently, many young people spend a lot of time chatting behind their PC, using messaging software such as Instant Messenger. As a separate activity, they also spend time watching music videos via the television. Observe that both functions are not combined, that watching music videos offers little interaction (see below) and is not personalized, and that chatting via messaging software is perceived by many to be more fun because the individual is undertaking the activity with his/her friends.

There are many television programs which broadcast music videos and offer a small amount of interactivity. For example, there are some services that allow people to send SMS messages, from their mobile phone, which are then displayed on the screen. Although this is a form of interaction, it is rather primitive. Moreover, these programs are not personalized. That is, everyone is presented the same videos, and everyone sees the same messages.

Messenger programs are popular because they offer interaction with the user's friends. However, they don't directly support interaction in relation to a joint activity or experience, such as listening to music. Hence, except for chatting, they are programs with a very limited functionality, which are typically used in combination with some other activity, such as gaming or doing homework. Therefore, typically, a user sits behind his PC to chat with his friends and the television is switched on, to listen to music and to watch music videos. If the user doesn't like the music, he can become irritated and switch the TV off (temporarily).

Attempts to improve the existing chat (messenger) programs are known. For example, Korean Patent Application Publication 2003/0067649 discloses a system for transferring music using a wired/wireless Internet messenger, which is designed to enable a

user to work or use a messenger service while hearing a favorite song. The document discloses an instant messenger system which comprises an instant messenger and a messenger database server. A messenger music streaming system comprises a music media server, a music web server, and a music database server. The instant messenger requests the member information, the friend information, and the member certification to the music web server in real-time. After the member certification, the music web server finds out an effect sound and a music source file from the music database server, and sends it to the music media server. The music media server sends the music source file to the messenger database server. This system provides a music sample through an instant messaging program, but does so only in response to a specific selection by a single user.

A combined music and chat service is also disclosed in United States of America Patent Application Publication 2006/0224971, which describes a system and method for facilitating discovery and promotion of musical works via an electronic network. Broadly, the method comprises the steps of receiving the musical works from artists; establishing a virtual room; allowing a user to enter the virtual room and select one or more of the musical works to be played; and allowing the user to chat with a second user within the virtual room. To this broad idea can be added, as desired, any one or a plurality of a large number of enhancement features that further facilitate the discovery and promotion of the works. This system allows users to choose a musical work and listen to that work together while chatting in a "virtual room". Again, all of the music choices are made by an individual user.

SUMMARY OF THE INVENTION

It is therefore an object of the invention to improve upon the known art. According to a first aspect of the present invention, there is provided a system comprising a plurality of client devices and a server device, the server device being arranged to operate a messaging service, and each client device being arranged to operate a messaging client and being connected to the server device, wherein the server device comprises a messenger component arranged to operate the messaging service, and to identify connected client devices, and a content selector component arranged to access data relating to a plurality of content files, to access a plurality of user profiles, at least one user profile representing a connected client device, and to select a content file, based upon the user profiles of the connected client devices.

According to a second aspect of the present invention, there is provided a method of operating a server device, comprising operating a messaging service, identifying client devices connected to the server device, accessing data relating to a plurality of content files, accessing a plurality of user profiles, at least one user profile representing a connected client device, and selecting a content file, based upon the user profiles of the connected client devices.

According to a third aspect of the present invention, there is provided a server device comprising a messenger component arranged to operate a messaging service, and to identify connected client devices, and a content selector component arranged to access data relating to a plurality of content files, to access a plurality of user profiles, at least one user profile representing a connected client device, and to select a content file, based upon the user profiles of the connected client devices.

According to a fourth aspect of the present invention, there is provided a computer program product on a computer readable medium, the product serving for operating a server device and comprising instructions for operating a messaging service, identifying client devices connected to the server device, accessing data relating to a plurality of content files, accessing a plurality of user profiles, at least one user profile representing a connected client device, and selecting a content file, based upon the user profiles of the connected client devices. By virtue of the invention, it is possible to provide a service that allows people to play content files such as music clips while chatting via a messaging service. The system combines on-demand-content, messenger-like functionality, and user profiles into a new service. With this service, a user can chat with his friends (just like Microsoft Messenger TM), and watch music videos, television programs, or listen to music tracks. An advantage of the invention is that the content selection is based on the taste of the people who are connected to the messenger service.

Since the network of people that an individual user is connected to varies, a user will get a diverse music selection. Each user can give an evaluation score to the music that he hears, which will affect the music selection and optionally update the personal user profile. This service is an improvement over the known systems, because it allows people to listen to music together and to interactively react with their friends on the music. Furthermore, the system provides a really personalized and dynamic experience because the music selection changes depending on the people who are online or taking part in a specific

chat session. Finally, by giving evaluation scores to the music, a user can influence the music selection.

The system has commercial opportunities, which may provide additional services to both clients and content providers, or service providers, for example because the people on-line will together pay a small fee for the music to which they are listening. This is only one model, other alternatives include by way of example: commercials spliced into the audio/video content, additional commercial info/links etc. displayed in a window in parallel with the chat session, and/or music industry members sponsoring content by providing (free) streaming audio of new tracks. The music selection algorithm operated by the content selector component together with the scoring mechanisms, gives feedback to the service provider about the user's music preferences. This forms useful information for the music industry. The music selection can (in part) be influenced by the music industry. By "plugging" a video (by paying money for it), the likelihood that a particular music video will be selected can be increased. Another alternative is the use of data mining in the chat rooms by building profiles based on participant profiles and personal tastes, in order for example to provide the music industry with input for targeted advertisement campaigns. The data mining results could be used by the service provider to increase the hit rates/click-through rates of advertisements. The system can also be used to construct playlists for each individual user from the selected music. In this scenario the music can be presented for free, but when the users selects a song again from the playlist, a fee is requested (or he/she is directed to a purchasing service).

This can be achieved by additionally providing the server in the system with a business profile database arranged to store a plurality of business profiles, the content selector component being arranged to select the content file from the content database, based additionally upon one or more business profiles. These business profiles can be used to subtly adjust the selection choices, favoring certain content files.

Preferably, the content selector component is further arranged to transmit the selected content file to the connected client devices. Once the selection of a content file has been made by the server, this can be transmitted to the connected client devices to be played to those users who are chatting via the messaging client located on their local client device. This transmission can be in the form of streaming data or as a single download. In an alternative embodiment, the content selector component is arranged to transmit an identifier to the connected client devices, the identifier identifying the selected content file. This may be because the desired content file is already present on at least one of the client devices,

stored on a local storage medium. Alternatively, the piece of music that has been selected may be found at a location remote from the server running the messaging service, and the client devices will retrieve the selected content from that other location.

As mentioned above, the content selector component is advantageously further arranged to receive feedback on the selected content file from one or more connected client devices, and to adjust future selections accordingly. This improves the user experience of the system, as the users can rate the music that has been selected, and this will influence future selections made on the server. This feedback mechanism can be used to update the user profile of the user who gives the feedback. It can also be used as a method of creating a user profile for a user who does not yet have one. As the users make further feedback decisions in the future, their new user profile is refined with this data. The feedback can also be gathered (following the obtaining of the necessary permissions as required by law) and used to rate specific content files such as music tracks, which have potential value to actors in the chain such as record companies and television channels. Ideally, the content selector component, when selecting a content file from the content database, based upon the user profiles of the connected client devices, is arranged to access an individual user profile for each connected client device and to generate a community user profile by combining the accessed individual user profiles.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Fig. 1 is a schematic diagram of a system comprising a server and multiple client devices, Fig. 2 is a more detailed schematic diagram of the system of Fig. 1,

Fig. 3 is diagram of message flow between the components of the system of Fig. 1,

Fig. 4 is a flowchart of a method of operating the server of Fig. 2, Fig. 5 is a schematic diagram of a screenshot of a client device, and Fig. 6 is a schematic diagram showing further details of the server of Fig. 2.

DETAILED DESCRIPTION

Figure 1 shows a system that comprises a plurality of client devices 10 and a server device 12. A network 14, such as the Internet, supports the connection between the

various components. The client devices 10 are shown as personal computers (PCs), but could equally be portable devices such as mobile phones or PDAs. The server device 12 is arranged to operate a messaging service, and each client device 10 is arranged to operate a messaging client and is connected to the server device 12. The messaging service allows two or more users to communicate with each other. Each client device 10 runs a messaging client in which text is entered. The server 12 runs the messaging service that communicates the entered text to the connected client devices 10, for display locally on those devices 10. This all occurs in real-time. The server 12 manages the connections of the client devices 10 to the messaging service and ensures that the users are identified and if necessary authenticated. The server 12 also maintains a storage function of the various "threads" that are currently being run. Multiple users can be connected, and a single user can participate in multiple "chat" sessions. Although the embodiment is implemented in a text-based chat service, it can also be used in other services, for example in audio/video chat services. Although adding background audio/video content is well suited for text-based chat sessions, the present system is not limited to text-based messenger services and can be used advantageously to complement or enhance any messenger services with (further) audio/video feeds.

A more detailed view of the system is shown in Figure 2. Four different users 10 are connected to the server 12. The users are split into two logical communities 16. This means that userl and user2 are chatting together in one community 16 and user3 and user4 are chatting together in a second community 16. The server 12, as discussed above, can handle very large numbers of connected client devices simultaneously, and can process the message traffic in real-time, running multiple concurrent sessions. The server 12 differs from a conventional instant messaging server in that it also has a content selection function, which is described in Figure 2 with reference to music selections, but which could equally be video files or indeed any kind of content that can be streamed or provided as a discrete file.

The server device 12 comprises a messenger component 18, which is arranged to operate the messaging service with the client devices 10, and is also arranged to identify connected client devices 10. The messenger 18 could be distributed across several different machines. The server 12 also includes a content database 20 arranged to store a plurality of content files, in this example music videos, and a profile database 22 that is arranged to store a plurality of user profiles, at least one user profile representing a connected client device 12. An initial user profile can be automatically generated, thus allowing new users to seamlessly connect to the system. Access to the content database 20 is managed by a music manager

component 24, and access to the profile database 22 and a business database 30 is managed by a profile manager component 26.

The example embodiment shown in Figure 2 shows the server device 12 provided with the databases 20 and 22, storing content and user profiles, respectively. However, it is also possible that the functions of the server 12 are performed with respect to remote or distributed sources of the content and user profiles.

The server 12 also includes a content selector component 28 that is arranged to select a content file from the content database 20, based upon the user profiles of the connected client devices 10. The content selector component 28 is arranged to access data relating to a plurality of content files and to a plurality of user profiles. In the embodiment of Figure 2, the data concerning the content files and the user profiles are found locally, stored on the respective databases 20 and 22 within the server 12. However, the selector component 28 could access the data and user profiles from remote data functions and/or from distributed sources of data. For example, the content selector 28, when accessing the user profiles, may actually acquire them directly from the client devices 10 as and when the users connect to the messenger 18, rather than storing the profiles locally in the database 22. In a similar way, the accessing of the data about the content files may involve accessing data from multiple sources including files stored on the client devices 10 and/or data stored on proprietary databases or web servers, for example. In one embodiment, the selector component 28 is arranged to access an individual user profile for each connected client device 10 and to generate a community user profile by combining the accessed individual user profiles. This is then used to intelligently select a suitable music file from the database 20, which is likely to be well received by the individual users that make up a specific chat community 16. Alternatively or additionally, novel matching content could be introduced to the session by content providers, which might also provide a click to buy function. The novel content could be based on mutually positively rated content, or content which matches the most recent taste, and/or mood settings from the users. The files that are stored by the database 20 include metadata that define the contents of the individual music files. This is discussed in more detail below, with reference to Figure 6. The server 12 also comprises the business profile database 30 that is arranged to store a plurality of business profiles. The content selector component 28 can be configured so that when selecting the content file (whether from the content database 20 or from another source), this can be based additionally upon one or more business profiles. The selector component 28 runs an algorithm to select a piece of content based upon an amalgamated

profile of all the users in a current community 18. This algorithm can be biased towards certain content files on the basis of the business profiles stored in the database 30. Optionally, the server 12 can be run on a commercial basis where specific originators of content can pay to have their artist's work chosen by the algorithm run by the selector 28 on a more regular basis.

Once a selection has been made, the content selector component 28, if the selected content file is stored locally in the database 20, is further arranged to transmit the selected content file to the connected client devices 10, via the messenger component 18. The content can be provided on a streaming basis, or the content can be downloaded to the client device 10 for playback. As regards the users at the client devices 10, they see nothing more than that they are chatting with their friends and they are supplied with some content (a piece of music or music video) that accompanies their chat session. Alternatively, the content selector component 28 can be arranged to transmit an identifier to the connected client devices 10, the identifier identifying the selected content file, rather than actually sending the chosen content to the client devices 10. This can be used to save bandwidth, if the specific piece of content is already present at the connected client device 10, or indeed if the user is going to acquire the content from another source such as a proprietary service.

The content selector component 28 is also further arranged to receive feedback from one or more connected client devices 10 on the selected content file, and to adjust future selections accordingly. As shown in Figure 2, there is a feedback loop marked

"getMusic/rateMusic" which allows the users to rate the content that is being supplied to them. This could be in a simple thumbs up/thumbs down manner or could be a more complicated numerical scale, for example. The choices made by the users can be incorporated into the algorithm executed by the music selector 28 and/or could be used to update the user profiles of those making the feedback. Mood-based selection is also possible, on the basis that, for example, first thing in the morning a user does not wish his normal preference for heavy metal to be indulged. This allows a user to temporarily adjust his profile, which can be dropped when a session is finished. An alternative implementation of the same idea would be for the individual user profile to be configured to support one or more moods as sub-profiles. Users can also make explicit music/content suggestions, allowing them to "plug" their own preferred titles.

Figure 3 shows an example of how a message sequence chart would look for a simple example of two users chatting in a community 16. The message flow is also illustrated in the system diagram of Figure 2, which shows the names of messages against the

connections between specific components of the system. In this example, the community 16 is composed of two client devices 10, being userl and user2.

At the beginning of the process, userl will contact the messenger component 18 of the server 12, and their presence and identity will be ascertained. The same steps are repeated for user2, at which point, the server 12 is now running an instant message session for the two users, via messaging clients present on their respective client devices 10. Userl now decides that they would like to accompany their chat session with some music, so the "getMusic" message flows from the specific client device 10 to the server 12. In the system diagram of Figure 2, this message is shown as traveling via the feedback loop directly to the music selector component 28, but the system could equally be configured so that the message is received by the messenger 18 and passed to the music selector 28.

The music selector 28 makes a request "getCommunity" to ascertain the client devices 10 that are currently connected in the chat session in which userl (who made the "getMusic" request) is participating. The answer is then returned from the messenger component 18, which is running the chat session, that the current members of the session are userl and user2. This will now be used by the music selector 28 as a basis for selecting a content file to be outputted at the client devices 10 presently within the community 16 that contains userl .

Once the music selector 28 knows the current users, "getProfile" requests are made by the selector component 28 of the profile manager 26 for each of the users within the community 16. The profile manager 26 has continuous access to the user profile database 22, and recalls the respective user profiles for the two users, userl and user2. These are then passed to the selector component 28. The selector component 28 also makes a "getBusinessprofiles" request to the profile manager 26, and receives back one or more business profiles.

The content selector component 28 is arranged to select a content file from the content database 20, based upon the user profiles of the connected client devices 10. The selector 28 runs an algorithm that applies selection criteria based upon the user profiles of userl and user2, and any business profiles that are used. The selector 28 communicates with the music manager 24 to obtain metadata about the content stored in the database 20, and this is used in conjunction with the profile information to select a music clip. This music file is obtained by the music manager 24 from the database 20 and passed to the music selector 28. The selector 28 then supplies the chosen music file to the connected client devices 10 within the current community 16, being userl and user2.

The users have the option to rate the music that is being delivered to them (either streamed or downloaded as a file) and this can be fed back to the selector 28, shown in Figure 2 and 3 as the "rateMusic" message. The content selector 28 can then use this to issue an "updateProfile" message to the profile manager 26, which can adapt the user profiles of those users who participate in the feedback loop. The sequence of messages shown in Figure 3 is terminated by a "goodbye" message when the current chat thread is ended.

The processing carried out by the selector occurs in real-time, and is a continuous process. A chat thread can have multiple participants, not just the two shown in the illustration of Figure 3. In the case when there are more than two participants, a single user leaving a session will not end that session, but it will affect the operation of the algorithm that the selector 28 is running. The user profile of the user that leaves the session will no longer be used to make the content selection, and in a similar fashion, as new members join the chat thread, then their user profiles will be included in the choice of content. A dynamic calculation of the content selection is occurring, as the members of the current community 16 change, caused by users leaving and joining.

The method of operating the server device 12 is summarized in Figure 4, which comprises, step Sl, operating the messaging service, step S2, identifying the client devices 10 connected to the server device 12, step S3, accessing data relating to a plurality of content files, step S4, accessing a plurality of user profiles, (at least one such user profile representing a connected client device 10), and finally at step S5, selecting a content file based upon the user profiles of the connected client devices 10. In the preferred embodiment of the invention, if the content file is stored locally, the selected content file is then supplied to the connected client devices 10 in the current community 16.

In relation to the accessing of data relating to content it is sufficient for the content selector 28 to know where the content resides, which might be in a central database, or stored at the client device of a user, or located somewhere on a peer2peer network. In a similar fashion, with regard to the profiles that the content selector 28 accesses, it is likewise sufficient to have the relevant information available at the actual selection time, so users might provide their respective profile when entering a session, rather than the server 12 storing the profiles. It is therefore true that there is no essential need for a central content/profile database located at the server 12.

Any distributed and/or centralized content databases that are accessible by the selector 28 can be combined with a central (sponsored) database of content that exists (from for example, a music vendor) and that is interleaved between the content to which the

participants have access. This supports a wide range of content being available to the end users from a wide variety of sources, and additionally can be used as a platform to provide users with the opportunity to buy any such content from the vendor, for example with a simple user interface interaction such as a mouse click. The profiles accessed by the content selector 28 can be combined with further information such as, for example, additional user input or profiles such as a mood profile; for example, "although I do like thrash metal I am now more in the mood for Rachmaninov". This allows the users to tailor the content that accompanies their chat sessions to their current specific mood. This could be supported by data such as time of the day or day of the week, or previous listening patterns of one or more users. Other parameters such as parameters in the dynamics of the chat can also affect the selection process. For example, the activity measure of the participants in the selection can be used to adjust the selection algorithm(s) run by the server 12 (e.g. lurkers/dormant users are not taken into account for selection). It is also possible to use a default selection "profile" that might be attributed to a bot, in relation to chat rooms that relate to a predefined topic.

To illustrate the end user experience, an example of the display 38 of a client device 10 is shown in Figure 5. The user that is accessing the messaging service via this client device 10 will see a window on his display device 38 that is maintained by the local messaging service client. The two users that are currently participating in the chat session are userl and user2, and their communications are displayed in the window of the chat client. In addition, the selected content file will be being outputted from both of their client devices 10. In this case the track "Thrash" by "Global warming" (fictional) is being played, and a button 40 is provided that allows the user of the client program to rate the current track being played. Infinite possibilities exist for the structure of the button 40. As shown in Figure 5, it is designed to bring up a sub-window, in which the user will rate the track on a scale of 1 to 10, but a simpler interface, such as the use of thumbs up and thumbs down icons could be used instead.

The display of the client program can be enhanced with the presentation of advertisements which are visible/interleaved with the chat to, for example, pay for the content being shown/played as an alternative. In the case of audio background, the advertisements could be ads in an ad window, or alternatively provided as audio messages instead of the content, resembling a radio station, thereby providing free audio background (albeit with advertisements).

Figure 6 shows in further detail the server 12 of the system. The music database 20 is shown as storing multiple content files 32, each file 32 having metadata 34 associated therewith. The server 12 may have multiple databases 20, each storing different types of media files that can be used to supply content to the users who are participating in the chat session that is being managed by the messenger component 18 of the server 12. The profile database 22 contains multiple profiles 36, each of which relates to a user of a client device 10.

Each music clip 32 is associated with meta information 34 (attributes) describing the clip. This might be video, video genre, artist, album, song, and song genre. User profiles 36 and business profiles can specify scores for each attribute from -5 through 0 (neutral) to +5 in integer intervals. For example, a specific music clip may have, in its metadata 34, the song genre of "modern jazz". Each user profile 36 will have a score for this attribute ranging from -5 to +5, although 0 may be the score if the specific user has not made any judgment about modern jazz. The score of a community 16 for an attribute is the average score of all members of the community 16 for that attribute. If no score is defined for an attribute, the neutral score (0) is used.

The selection algorithm operated by the selector component 28 works by sequentially reducing the set of content clips 32 to choose from, using the community and business profiles. Given is a set C of clips 32 and an attribute A, having values a! ... a _n . Initially there is created a probability distribution where each value a, has a chance of 1/n, illustrated as follows:

Based on a business profile, the distribution can be adapted to increase the likelihood that a particular value will be chosen. For example, the chances will be linearly increased/decreased depending on a positive or negative score:

Similarly, the distribution will further be changed by the value for the attribute in the community profile. Given such a distribution, a value a of an attribute A is selected.

The set C of clips is then reduced by removing all clips having a value different from the value a for the specific attribute A. These steps are repeated for the other attributes. From the resulting set of clips, a clip is then selected either randomly or based on other factors such as time since last used. This is the clip that will be offered to the users of the community 16. A user can then score the clip, and this is fed back into the decision making process. For instance, by indicating that he does not like this type of video, this particular video, this artist, this music genre etc., the user can change the make-up of their profile 36.

The business profiles stored in the business profile database 30 can also be used in the algorithm executed by the content selector 28. For example, the attribute "song genre" may have values: jazz, pop, dance, classic and blues. A business profile for this attribute may be [jazz -5, pop 2, dance 2, classic -4 and blues O]. The default distribution for this attribute is:

Applying the business profile to this attribute gives (based on a conversion of the attribute scale from -5 to +5 to a 0/5 to 10/5 scale):

Adjusting the distribution to sum to 1 gives:

And simplifying this table gives:

If the community profile for the current users is [jazz 0, pop 4, dance 2, classic 0 and blues 0], then applying this profile to the above table (again converting the -5 to +5 scale over to a 0/5 to 10/5 scale) gives the following result:

which simplifies to:

multiplying through by 50/7, and cancelling, to give a distribution summing to 1 gives:

Given this probability distribution, a value for the attribute "song genre" is selected. In this example it means that Jazz will not be selected, and that Pop has the highest chance of being selected. Once a genre has been selected, the content (such as a collection of videos) is reduced by removing all videos of genres other than the one selected. This algorithm is repeated for all attributes of the community profile. From the resulting collection of videos, a video is selected. This final selection from the shortlist can be made randomly, or based on some other criteria, such as business rules, or the least recently selected.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.

In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.

It will be clear that within the framework of the invention many variations are possible. It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove. The invention resides in each and every novel characteristic feature and each and every combination of characteristic features. Reference numerals in the claims do not limit their protective scope.

Use of the verb "to comprise" and its conjugations does not exclude the presence of elements other than those stated in the claims. Use of the article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.