METHOD FOR DYNAMICALLY GENERATING CONTENT THAT IS USER- CUSTOMISED AND CONTEXTUALISED BASED UPON AUDIO AND VISUAL INSPECTION

Field of the Invention

The present invention falls within the scope of systems for making digital content available . More particularly, the present invention relates to methods for dynamic programming of digital content , applicable to the various audio-visual consumption platforms - both live and on-demand content - with the user as a fundamental element in the formula for selecting, generating and making this content available on their digital devices .

Background of the Invention

In traditional TV broadcasting systems , two types of content are made available - live content and on-demand content . The users have at their disposal the tools , more or less advanced, depending on the platform, which allow them to search or discover the content of their interest . Examples are : ( i ) the Programming Guide (EPG) with the list of programmes in the line-up, including basic information of the category or classi fication o f the programme ; ( ii ) the programme catalogues built from the channel line-up and divided by topics ; ( iii ) the purely on-demand content catalogues , such as the shops of previously categorised content ( films and series ) to rent , buy and consume , which also include the so-called Over The Top digital platforms (Netflix, Disney+ , among others ) .

Although the user experience with television plat forms has evolved signi ficantly, with advanced search engines and content recommendation, the user still spends some time in the process of spotting the programme , segment or set of segments that satisfies his or her interests or curiosities at the moment .

The TV ecosystem has more recently incorporated, among others , components for contextualised advertising, as well as for A/V ( audio/ video ) analysis , which will be the basis of innovative services in a wide universe of platforms ( e . g . STBs , Smart TVs , Apple , Android, Web ) .

However, the solutions available to date rely only on the base information (metadata ) loaded along with the media elements that make up the content , and do not perform dynamic and active analysis of the media elements themselves .

Technical problem to be solved

With a content inspection architecture in A/V streams , orchestrating a set of analyses that take advantage of advances in machine learning and computer vision, it is possible to detect certain visual or audio elements in real time , identi fy and segregate in a linear channel the advertising blocks from the content itsel f , select and classi fy imagery based on a set of criteria, or collect metadata that can be used, for example , for the creation of visual summaries , indexing and advanced search of topics on the news , checking the participants of a program, or clipping of multi-segment content .

The combination of these techniques , together with an engine capable of creating and feeding the user profile , both explicitly (preferences indicated by the user ) and implicitly ( track record of content consumption, acceptance of content suggested by the system, browsing and viewing patterns ) , opens the opportunity to create a solution that allows the dynamic generation of a unique programming, focused and oriented to the user in each moment .

Summary of the Invention

The present invention consists of a method, which can be employed in any television content reproduction platform ( STBs , Smart TVs , Apple and Android devices , Web ) , which interacts with an Audio and Video inspection subsystem for active assessment of the contents broadcast live or previously recorded, combining it with the user ' s preferences and the viewing context , aiming at the dynamic and intelligent generation of customised and contextualised programming .

In other words , the present invention has the users as a fundamental element in the formula for selecting, generating and making available the tailored content on their digital devices , a formula that is based on the dynamic and active analysis of the media elements themselves that comprise the content , unlike the solutions available to date which rely upon the base information (metadata ) , loaded together with the media elements , associated with the content .

The present invention for creating unique , user-oriented and focused programming at each moment from the global set of available content ( on-demand) , also describes a digital system that implements an intelligent method for switching channels of digital content ( on live broadcasts ) - iZapping .

Drawings

Figure 1 - Example of a preferred embodiment of the architecture of this invention . The reference signs mean the following :

1 - media stream of live broadcasts from all channels ;

2 - media stream of programmes previously recorded and/or available on-demand;

3 - stream of events coming from the digital device in use by the user ;

4 - information stream aggregated by the audio/video inspection module ;

5 - information stream aggregated by the user profile creation module ;

6a deferred stream of contents from a channel or programs on demand;

6b - information stream for switching channels on live broadcasts ;

7 - user feedback stream; i - audio/video inspection module ; ii - user profile creation module ; iii - dynamic content creation module ; iv - cDVR; v - digital device for playing digital television contents .

Detailed description

The present invention is based on any digital device for multichannel reproduction of digital television contents (v) , such as a Set-top Box ( STB ) , a personal computer, a tablet , a mobile phone , etc . (hereinafter referred to as digital device ) .

The proposed architecture , as illustrated by figure 1 , implies the interaction between at least one audio/video inspection module ( i ) , at least one user profile creation module ( 11 ) and at least one dynamic content creation module ( iii ) . Each of said modules comprises computing means adapted to perform the functions for which they are intended, and which will be detailed below .

The audio/video inspection module ( "Deep Video Content Insights" ) is configured to perform audio and video inspection on all contents transmitted or available to the user (8) through the digital device (v) , such as for example:

- live broadcasts from all channels (actively evaluated in real time) - stream (1) ; or

- programmes previously recorded and/or available on- demand - stream (2) .

In module (i) several combined A/V content analysis techniques are applied with the following objectives:

Detection of specific clips (video and/or audio segments ) ;

- Detection of advertising blocks and segregation of the content of these blocks;

- Capture of representative programme imagery (e.g. with celebrity presence, or scene indicative of the topic) ;

Metadata extraction for content indexing and inference of the topics covered;

- Clipping of distinct segments within the same content;

- Identifying the key players in each segment.

The user profile creation module ("Profile Engine") is responsible for building a user's profile. This module is fed by a certain set of events coming from the digital device (v) in use by the user (8) - stream (3) , such as:

- Preferences (namely content categories and genres) indicated by the users themselves (for example, at the time of the service enrolment process) ;

- Evolution of content consumption track record;

Content browsing and viewing patterns (variable depending on the context, such as location, type of device (v) used, day of the week or time of the day) .

To implement the dynamic and intelligent generation of customised and contextualised programming, the information aggregated by modules (i) and (ii) will feed continuously - streams (4) and (5) - the dynamic content creation module ("Machine Learning Intelligent Content Generator") , responsible for dynamically creating the content to be presented to the user (8) . The deep knowledge resulting from the dynamic and active analysis of the media elements which compose the content, combined with the knowledge that the architecture allows to be obtained from the user (8) , enables the generation and availability of customised content. This architecture can be applied in the two main scenarios of television content consumption on digital platforms:

(a) The deferred consumption of content from a digital content channel, or the consumption of programmes on demand: aims at presenting unique programming, which is user-oriented and focused at any given moment. This programming may be comprised of one or more programmes, or one or more segments from different programmes and/or channels, or even of contents from an on-demand catalogue. When curating the programming, the system will take into consideration the time usually available for content consumption, user (8) preferences, previous consumption of content and the viewing context. For instance: the user (8) gets home at 10 p.m., and within half an hour (time that the user (8) usually spends on those contents or at his request) , he watches the most important news of all the evening news, without having to tune to 5 or more channels and without having to look for the news that interest him from each one of them. Stream - 6a

(b) On channel switching (on live broadcasts) . The system can interact directly with the digital device (v) for the implementation of an intelligent method of channel switching - iZapping. The referred techniques may be used so that, in this zapping process, the channels are tuned not in a linear sequence (e.g. channel 1, channel 2, channel 3, ...) , but taking into consideration the user (8) , their preferences, and the current viewing context. In this scenario (iZapping) channels whose broadcast content is not interesting or suitable for the user (8) at the current viewing moment would be ignored. Example: the user (8) at dinner time watches news, so the zapping will give priority to channels that are currently broadcasting news, later on and already in the living room, the user (8) prefers series and movies, so the zapping will be altered in order to accommodate his preferences prioritizing the series and movies channels, although it could also suggest, e.g. an interview with the Mayor that is being broadcast at the moment. Stream - 6b.

The method of the present invention for intelligent and prompt finding of relevant content may be further improved, through a process that allows the system to sense the acceptance of programming curated by the end user (8) , with the transmission of events from the digital device (v) relating to user ( 8 ) behaviour regarding the content displayed and viewed . This automated feedback process - stream ( 7 ) - allows module ( ill ) to continuously improve the results of the dynamic content generation in relation to changes that will naturally occur .

Embodiments of the invention

In one embodiment of the invention, a method is described for dynamically and intelligently generating customised and contextualised programming, and delivering it to the user via a digital television content reproduction device . The method is based upon the Audio and Video Inspection over all the contents transmitted or available to the user through said device (v) , and the User Prof ile Creation, so that the result is perfectly aligned with its preferences and viewing context . To this end, the developed method may be comprised of the following steps :

— Reception of the live tuning signal of regular and premium channels that can be subscribed by users ;

— Access to the contents recorded or available in catalogue for on-demand consumption;

— Assessment of digital content , carried out in realtime on the streams of available channels , for inspection of associated audio and video , allowing the identi fication of various elements for a rigorous and enriched classi fication of the content , its division into segments or chapters , and the indexing and mapping of topics and participants throughout the broadcast ;

— Managing and searching of user profiles , with their preferences , consumption track record and viewing patterns ;

— Dynamic creation of customised programming and delivery of the respective digital content to the user, which may consist of one or more programmes , or one or more segments from di fferent programmes and/or channels .

In another embodiment of the developed method, additional information for creating the user profile may be provided through external systems , integrated in the present solution subj ect to informed consent .

In one embodiment of the developed method, in case the user profile is unknown, the digital programming to be proposed may take into account the trends of content consumption by the community of users of the solution .

In one embodiment of the developed method, the analysis of the digital content may be done in post-processing over previously recorded and/or available on-demand programs , as an alternative , or as a complement , to the real-time active inspection over the broadcast channels streams .

In another embodiment of the developed method, the evaluation of the digital content, done in real-time over the broadcast channels streams , in conj unction with relevant data from the user profile , may be used for intelligent channel switching iZapping - by creating a dynamic channel alignment , excluding channel s from the grid whose broadcast content is not interesting or suitable to the user at the current viewing moment .

The present invention is also related to a digital system for making digital contents available . The said system may comprise the following elements :

— at least one digital device for digital content reproduction (v) ;

— at least one audio/video inspection module ;

— at least one user profile creation module ;

— at least one dynamic content creation module , wherein, said modules are configured to operate according to the method of the present invention . Additionally, the system may further comprise :

— a subscription management subsystem;

— a subsystem for transmitting a communication operation;

— a subsystem for aggregating indicators of activity and viewing of digital content .

In an alternative embodiment of the system, the digital multichannel reproduction device is one of the following devices : computer, mobile phone , tablet ; satellite , cable , terrestrial or IP digital video decoder .

Practical scenarios

Considering that the invention herein described applies to a speci fic example in which the operator of a STB-type digital device (v) , a user who is a sports fan, e . g . of football , accesses a feature with segment suggestions about his team, its athletes and coach, when a game is on, the channel which is broadcasting the game is then added to his curated list of channels .

I f the user has chosen to follow their favourite artist , they receive suggestions of segments from the morning or afternoon show where they were j ust performing, or an interview on an evening show .

The user can choose to be noti fied ( Push Noti fication to their mobile device ) each time their company is mentioned on any of the channels they have access to .

The user can get a visual summary and see the corresponding segments of a programme in which he has several guests and/or several topics .