System and method for instantaneous access to dynamic media production data

Field of invention:

The presented invention relates to improvement of media production systems and methods, and, more particularly, to systems and methods for capture, storage and exchange of digital content and metadata to enable more efficient media production management, distribution, exploitation and archiving.

Background Invention:

A typical media production generates significant amount of information about the produced content or elements thereof, as well as about the processes involved in preparation, execution, management and distribution of the production. This is especially the case with more complex media productions, such film, television or music production. In a film production, for example, a wealth of production documents, such as scripts, logs, calls sheets and progress reports, are generated daily and are manually shared between the various production departments and crew members. These documents comprise metadata describing the generated content and/or related production process.

Some production systems digitize the documentation, which to some extent simplifies the exchange of generated information and facilitates later easy retrieval of the documents and related metadata [WO 2013070802]. However, this approach does not guarantee reliable metadata due to the limited accuracy of OCR technology used to digitize text from the printed documents.

There are, however, some systems that enable inherently digital capture of media production metadata, but are tied to specific subset of devices, such as cameras [WO 2007064357], smart pens [US 20130222610] or other film set devices [US 8457484] and thus can only capture a small fraction of all metadata categories that can be generated in the media production process.

The systems that focus solely on the media production workflow by capturing inherently digital metadata have successfully automated and streamlined media processing stages such as rendering, encoding and post-production [US 8615517]. This approach is very effective because it involves capture of digital information and its utilization in the digital processing stages of media production. However, it does not address streamlining and optimizing of production management nor does it offer access to this invaluable information for other stakeholders in the media lifecycle.

In addition, these systems lack support for real-time exchange of production metadata across the range of production departments. This functionality could streamline production management and enable new methods of media production.

Another feature of media production metadata is that the structure of data for a given media production process is typically fixed. For example, the metadata categories in film production documents, such as call sheets and shooting schedules, are pre-defined and tend to be standardized. This approach leaves no flexibility to the production

management, thus further hindering the efficiency of the overall production process. In addition, there is no possibility to introduce a new metadata category and thus capture non-standard or specific information during the production process. Given the rapid developments in the way media is being produced, distributed and consumed, the flexibility of data structure and facilitation of dynamic data categories is of a vital importance.

The present invention seeks to provide an efficient system for capturing and utilizing media production metadata through instantaneous exchange of relevant information to all stakeholders in the media lifecycle, as well as the capability of dynamically assigning (or removing) data categories, whether it is a specific type of content, data that describes the produced content or elements of the production process. The invention aims to provide more streamlined production management; dynamic and scalable system for metadata capture, management and utilization as well as novel ways to engage end-users in the media production process.

Summary:

According to the present invention from Claim 1 and the succeeding claims, a system and related methods provide means of instantaneous capture and delivery of media production data, where the production data refers to entities of digital media content such as video, audio, image or text data, and digital metadata that describes the content or any part of the media production process. In certain embodiments of the system, the means of controlling production workflow and stored data structure are dynamically creating and deleting content and metadata categories, thus facilitating adaptation to different types and/or scales of media production, distribution, exploitation and archiving. The presented method of instantaneous capture and delivery of metadata and/or content, as well as the dynamic structure of the media production data, facilitates new ways of active engagement by a wide range of stakeholders in all stages of the media lifecycle, including producers, financiers, archivists and end-users. In some embodiments, the content and/or metadata are propagated globally over existing global computer networks, such as the Internet, while in other embodiments, the exchange of data happens over a local network of computers and other connected devices.

Detailed description:

Figure 1 shows a schematic diagram of main elements of the present invention. In this embodiment, the notion of user 100 is generalized to any user of the media management system 124 according to the embodiment of the present invention. In a typical media production scenario, the user can be a cameraman, director or assistant producer, who uses a device or a plurality of devices 101 to capture data 110 and/or retrieve data 111 from the presented media management system. A plethora of production devices such as cameras, sensors and communication devices, can be connected to the media

management system that orchestrates complex and time-critical management of human, technological and material resources required for a production.

In addition to content and metadata describing the content and/or the production process, user device 101 can send control signals 120 to the control module 104 and/or receive control signals 120 from the control module 104. The control signals 120 in conjunction with the control module 104 implement control signaling between the media management system 124 and the user device 101, such as requests by user 100 to create a new metadata category by the data structure control module 119, or sending the identification of the user 100 to the access control module 106. The captured data 110 and retrieved data 111 exchanged between the user device 101 and the media management system 124 comprise both media content (video, audio, images, text, etc.) and/or the metadata describing the content (timecode, GPS locations, camera settings, etc.) or elements of the production process (call sheets, progress reports, etc.).

In some embodiments, the application programming interface 105 comprises three modules: capture module 102, delivery module 103 and control module 104, and acts as the interface between different types of user devices 101, and the media management system 124. In some embodiments, the exchange of data 110, 111 and control signals 120 can be implemented, but it is not limited to, using wired or wireless networking technology. In some other embodiments, the capture module 102, delivery module 103 and control module 104 are implemented as hardware modules that facilitate direct connection of devices, such as user device 101.

The control signals 121 implement control signaling within the media management system 124, exchanging control signals between application programming interface 105, access control module 106, database 107 and workflow control module 109.

The capture module 102 receives data 110 from the user device 101, adapts it to the format compatible with the database 107 and sends the formatted data 113 to the access control module 106. The access control module checks access rights for the user

100. If the user 100 has got permission to write the captured and formatted data 113, the database 107 will store the data.

In some embodiments, the user device 101 can be a media production device, such as camera or microphone; or a user input device such as tablet or a laptop computer.

The media production data 110 can be captured instantaneously in digital format from a production device 101 capable of:

capturing digital content natively, or

capturing analogue content and digitizing it in real-time, or

generating digital data natively, or

analyzing captured content and generating digital data.

The user input data 110 can be captured instantaneously in digital format from the user device 101 capable of:

capturing digital data natively, or

analyzing user interactions and generating digital data, or

analyzing user behavior and generating digital data.

If the user 100 requests retrieval of the data 111 from the database to the user device

101, the application programming interface 105 will send a request through control signal 121 to the database 107 to dispatch data 116, and if access control module 106 grants the access, the data 114 will be delivered to the delivery module 103 of the application programming interface 105. The delivery module 103 will adapt the data 114 and dispatch the adapted data 111 to the user device 101. Following the same procedure, the delivery of content and/or metadata from the database 107 to the user device 101 can be initiated by any module of the media management system 124 through the control signaling 121. The data 111 delivered to the user device 101 is a tangible result of the presented innovation, capable of, for example, informing the user 100 in real-time of changes to the production process, or requesting timely user input for a specific technical data that cannot be captured at a later stage in the production. According to the present invention, the dynamic structure of metadata in the presented media management system 124 is implemented in the data structure control module 119. The data structure control module 119 manages the entire database schema, which enables creation and deletion of metadata and/or content categories used in any particular embodiment of the presented system. Examples of the metadata categories are: text fields of the call sheet such as type of transport required for a given day of shooting, or numerical values of main camera's GPS coordinates. The changes to the existing data structure can be triggered by requests from the application programming interface 105 and thus implicitly the user 100, access control module 106, database 107 or workflow control module 109, using the control signaling 121. In order to deploy the changes in the database schema and add or delete data categories, the data structure control module 119 exchanges information with the database 107 through the data control signaling 122.

The instantaneous propagation of content and/or metadata is implemented through data delivery push that can be initiated by any module connected to the control signaling 121. The access control module 106 controls users' access to data depending on the user's access rights for a given content and/or metadata.

If user 100 retrieves specific metadata 111 through delivery module 103, changes its value and saves it to the database 107 through the capture module 102, the control module 104 of the application programming interface 105 then requests workflow control module 109 to propagate this update to all relevant users. The workflow control module 109 triggers the data processing module 108, and mines the database for all instances of content and/or metadata that require update. After each update, the data processing module 108 informs the workflow control 109 to send the update to relevant users. The workflow control 109 then requests the delivery of the updated content to all relevant users by following the procedure of data delivery from the database 107 to the user device 101 described above. The data processing module 108 can retrieve the data 117 from the database 107, as well as write the processed data 118 to the database 107. Here, the data refers to both media content and metadata. The workflow control module 109 controls any data processing through the processing control signaling 123. The data processing module 108 can exchange information with the workflow control module 109 using the processing control signaling 123.

Figure 2 shows embodiment of the presented invention utilized in a film production scenario that implements dynamic allocation of metadata categories where users of the system can access the media management system 207 over a global computer network 204, such as the Internet. In this embodiment, the professional archivist 200 uses a desktop computer 205 to access the application programming interface 206 of the media management system 207 over a communication link 208 that can be

implemented using, but not limited to: direct wired link, a computer networking system such as local area network [LAN), wide area network (WAN) or virtual private network [VPN).

A remote end-user 201 interacts with the media management system 207 through a mobile handheld device 202 capable of connection to the Internet 204 over a wireless network 203, such as 3G, LTE or Wi-Fi. A typical end user interested in the production process can be a film fan, a hobbyist or a film student. The instantaneous propagation of production data that comprises produced content and/or metadata enables end-users with appropriate access rights to retrieve information about the production as they are captured, i.e. in real-time, such as the production stills, location details or progress reports. In addition to retrieving the data, the presented invention facilitates capture of the content and metadata by end-users, thus enabling their contributions to the production process. This functionality of the presented invention facilitates exploitation of the user generated content and user generated metadata in the media production process.

In the depicted scenario, the archivist 200 requests creation of new metadata category that will enable more effective long-term preservation of the produced video content. An example of the requested category can be a 'type of chroma subsampling' used in the video compression standard during the production. This type of metadata is very technical and specific, and cannot always be envisaged by the production management or crew members. Therefore, another stakeholder in the lifecycle of media product, in this case the archivist 200, can influence the structure of captured metadata. The application programming interface 206 passes on the request to the dynamic metadata module 119 which creates a new metadata category.

On the other hand, an internal control module of the media management system 207, such as the workflow control module 109 described in Figure 1, can detect that a specific metadata category is not utilized in a particular production process, and thus delete that category from the database schema.

The creation of metadata categories triggered by system users and an automated purgation of unused metadata categories results in a dynamic optimization of the database schema complexity, thus producing an optimal scale of production

management and the supporting media management system 207. Furthermore, the presented invention facilitates structural changes of the production metadata at any time initiated by either system users or control modules within the system, thus achieving fully dynamic allocation of metadata categories.

Figure 3 presents a local deployment of the media management system 307 on a locality of a film production. Very often, the locations of film and TV productions have no infrastructure to connect to the Internet or other communication channel that would enable connection between the production team and a remote media management system. In order to maintain the real-time access to content and/or metadata updates across the production team a local instance of the media management system 307 is deployed. The users can access the media management system 307 by, but are not limited to: direct wired links 300, over a locally deployed wireless computer network 305, or otherwise. User 302 is a cameraman operating a digital film camera 304 that is directly connected to the application programming interface 306 of the media management system 307. The user 301 is a camera assistant who uses a tablet device 303 to access the application programming interface 306 of the media management system 307 over a wireless computer network, such as Wi-Fi.

In this embodiment of the presented invention, the cameraman 302 does not manually generate metadata, but the digital camera 304 automatically sends metadata describing the camera parameters and miscellaneous sensor readings, such as GPS location of the camera, directly to the application programming interface 306 of the media

management system 307. However, the camera assistant 301 interacts with the tablet device 303 to input additional metadata related to the camera. For example, the camera assistant 301 might be asked to populate a new metadata category 'type of chroma subsampling' that has been requested by archivist 200 employed by the production studio to secure long-term preservation of material filmed during this particular production. This scenario demonstrates benefits of the presented invention to the production process through streamlined and effective communication between various stakeholders at different stages in the lifecycle of a media product.

Figure 4 shows a certain embodiment of the presented invention in a specific

configuration of connection 414 between two media management systems 401 and 403. This configuration facilitates, but it is not limited to, the following processes:

synchronization of data between two media management systems, backup of one media management system to another or mirroring of one media management system by another. The connection 414 can be, but is not limited to: direct wired link, over a dedicated communication channel or a computer network.

In the embodiment of the present invention depicted in Figure 3, a locally deployed media management system 307 captured and stored production data at a given location. Once the shooting at location has finished, the media management system 403 is connected to the main media management system 401 that holds all previously produced content. The delivery module 410 of the application programming interface 402 in the media management system 403 sends all data from database 412 to the capture module 404 of the application programming interface 400 in the media management system 401. The handshake protocol between the two systems is managed by the control signaling 408 and 413. Finally, data can be transferred from database 407 to database 412 through exchange between the delivery module 405 and the capture module 409.

In certain embodiments of the presented invention, the media management system 510, as depicted in Figure 5, can be accessed by a variety of users over the Internet 508 and utilized in applications such as social media sharing and technology enhanced education. User 500 is a celebrity film director invited by the production team to generate a specific collection comprising content and/or metadata stored in the database of the media management system 510 or uploaded by the celebrity user 500 using a personal computer 503 connected to the Internet 503. The media management system 510 will treat this collection as newly produced content with associated metadata, where some metadata categories might need to be created anew. In certain embodiments of the invention, the media management system 510 facilitates creation of such collections by any user.

User 502 is keen film follower that retrieves content, metadata and curated collections from the media management system 510 using a handled mobile device 505 over a wireless mobile network 506 that is connected to the Internet 508. Using widely available communication tools and social media deployed over the Internet, user 502 shares the collection curated by the celebrity user 500 with user 501. User 501 interacts with a tablet device 504 that is connected to the Internet 508 over the wireless computer network 507. User 501 is a student, and uses production data from the media management system 510 in his student project.

This example demonstrates wider implications of the presented invention to education and entertainment. In addition, having access to the user generated content, the production team can exploit the value added content at various stages of the media lifecycle and in various domains, such as media distribution, advertising or archiving. Brief Description of Drawings:

The presented invention will be described by referring to the accompanying drawings:

Figure 1 is a schematic diagram of the system according to the embodiment of the present invention.

Figure 2 demonstrates functionalities of dynamic metadata category creation and access to the media management system over the Internet.

Figure 3 presents a local deployment of the media management system on a locality of a film production and a combination of manual and automated ingest of digital metadata during the production process.

Figure 4 shows functions of data exchange between two media management systems that facilitates updating, mirroring or backing up the media production data.

Figure 5 demonstrates functionalities of the media management system that enable users to create, curate and share new content and metadata inline with the existing production data, thus facilitating social media and educational applications of the invention.