Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHOD AND A DEVICE FOR RECEIVING AND MODIFYING PICTURE DATA
Document Type and Number:
WIPO Patent Application WO/2010/131977
Kind Code:
A1
Abstract:
A method and a device for receiving and modifying picture data, e.g. video data for film production. Picture data, e.g. video data in a compressed raw format, is received from a camera. Metadata representing camera functions or settings is received from a first wireless metadata recording device such as mobile phone. Metadata representing script data is received from a second wireless metadata recording device such as a portable computer. Modified picture data is generated based on the received picture data and the received metadata, e.g. by color correcting the picture data in dependence on the metadata.

Inventors:
ROSENLUND JOHN CHRISTIAN (NO)
VAALER AUDUN (NO)
BAKKE ARNE MAGNUS (NO)
Application Number:
PCT/NO2010/000170
Publication Date:
November 18, 2010
Filing Date:
May 05, 2010
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
DRYLAB R & D (NO)
ROSENLUND JOHN CHRISTIAN (NO)
VAALER AUDUN (NO)
BAKKE ARNE MAGNUS (NO)
International Classes:
G11B27/034; G06F17/30; G06T3/40; G11B27/32; H04N1/60; H04N5/76; H04N5/926; H04N9/79; H04N9/804
Domestic Patent References:
WO2007064357A12007-06-07
WO2007149444A22007-12-27
Foreign References:
US20080089580A12008-04-17
US20090231459A12009-09-17
EP1102271A12001-05-23
EP1349164A12003-10-01
US20060244845A12006-11-02
Other References:
None
Attorney, Agent or Firm:
ONSAGERS AS et al. (St. Olavs plass, Oslo, NO)
Download PDF:
Claims:
CLAIMS

1. Method for receiving and modifying picture data, comprising

- receiving picture data from a camera;

- receiving metadata from a metadata recording device; - generating modified picture data based on the picture data and the metadata.

2. Method according to claim 1, wherein the generating of the modified picture data includes color correcting the picture data in dependence on the metadata.

3. Method according to claim 1 or 2, wherein the metadata includes camera functions or settings.

4. Method according to claim 3, wherein the metadata recording device is a wireless terminal, and wherein the step of receiving the metadata includes receiving the camera functions or settings via a wireless interface. 5. Method according to one of the preceding claims, wherein the metadata includes script metadata.

6. Method according to claim 5, wherein the script metadata identifies at least one of a day, a scene, a slate and a take, and wherein the color correcting is performed based on a day, scene, slate or take identified by the script metadata.

7. Method according to claim 5 or 6, wherein the metadata recording device is a wireless terminal, and wherein the step of receiving the metadata includes receiving the script metadata via a wireless interface.

8. Method according to one of the preceding claims, wherein the metadata is stored in a database.

9. Method according to one of the preceding claims, wherein the generating step includes an introductory de-Bayering or demosaicing step performed on the received picture data.

10. Method according to one of the preceding claims, further comprising receiving additional metadata from the camera.

11. Method according to one of the preceding claims, wherein the camera is a digital film camera.

12. Method according to one of the preceding claims, wherein the picture data is video data in a compressed raw format. 13. Method according to one of the preceding claims, further comprising outputting the modified picture data and the metadata.

14. Method according to claim 13, wherein the modified picture data and the metadata are provided to a post- production facility. 15. Method according to claim 13, wherein the metadata provided to the post-production facility comprises XML data.

16. Method according to one of the preceding claims, wherein the received picture data includes a first set of time stamps, and the metadata includes a second set of time stamps, and wherein the generating of modified pictures includes comparing the first and second sets of time stamps.

17. Method according to one of the preceding claims, further comprising encoding the modified picture data to multiple formats.

18. Method according to claim 17, wherein the encoding to multiple video formats comprises

- a common decompression substep;

- a common de-Bayering or demosaicing substep; and

- a common color correction substep; and

- separate encoding substeps, each adapted for a particular video format. 19. Device for receiving and modifying picture data, comprising

- first input circuits for receiving picture data from a camera;

- second input circuits for receiving metadata from a metadata recording device; and

- processing circuits for generating modified picture data based on the received picture data and the metadata. 20. Device according to claim 19, adapted to perform a method as set forth in one of the claims 1-18.

Description:
METHOD AND A DEVICE FOR RECEIVING AND MODIFYING PICTURE DATA

Field of the invention

The present invention relates to a method and a device for receiving and modifying picture data, e.g. video data received from a digital film camera. The method and device may particularly be used in film production or television production.

Background of the invention

The process of filmmaking is usually divided into five stages:

- development, including writing a script; - pre-production; including preparations for the shoot;

- production; wherein the raw elements for the film are recorded ("on-set"); and

- post-production, which includes film editing, soundtrack editing, adding of effects; etc.; and

- sales and distribution. The present method and device are particularly useful for the production phase indicated above.

In current film or television production, in particular in the "on set" phase of the production, the film script (the "story") is the basis for decisions made in the workflow. On a film set, the director and director of photography materialize the script by their visual interpretation. Film production has for about 130 years been shot on analogue film. There is now an emerging trend in the industry, changing to digital film recording. Film production, e.g. of drama film, involves high cost in a limited shooting period. The recorded material has a high economical value. While conventional analogue film provides a protected, fixed medium for the original recording, digital film recording involves more volatile original data which must be safely preserved.

The digital file provided by a digital film camera is typically recorded in a RAW format. This format needs manipulation before it can be displayed or transformed into a format suitable for viewing. Such manipulation needs some kind of metadata to know how to present the image. During production, metadata is provided by, i.a., the camera assistant and the script supervisor. Currently, metadata created by the crew on the film set is usually written down on paper and follows the production workflow in paper form or in digital formats that does not communicate with the digital workflow.

Products in the background art that make use of manual input of production crew's meta data are known as Iridas Speedgrade OnSet and 3Cp from Gamma & Density.

Thus, there is a general need for methods and devices that may enhance the overall film production process, or details in the process.

Summary of the invention

The method and device of the present invention are defined by the appended patent claims.

Brief description of the drawings

Embodiments of the invention will be described below with reference to the drawings.

Fig. 1 is a schematic block diagram illustrating principles of a device and a method for receiving and modifying picture data.

Fig. 2 is a schematic flow chart illustrating principles of a first embodiment of a method for receiving and modifying picture data.

Fig. 3 is a schematic flow chart illustrating principles of a second embodiment of a method for receiving and 'modifying picture data. Detailed description of embodiments

Fig. 1 is a schematic block diagram illustrating principles of a device and a method for receiving and modifying picture data.

A device 100 for receiving and modifying picture data is arranged to receive picture data from a camera 110. The received picture data may, e.g., be video data in a compressed raw format.

The camera 110 may be a digital film camera, i.e. a professional digital video camera with very high resolution, suitable for professional movie production. Other digital video cameras may also be applicable.

Video data in compressed raw format contains minimally processed data (i.e. with minimal loss) from the image sensor of the digital camera. Other video/image formats, including, e.g., an uncompressed raw format, may also be applicable.

The device 100 for receiving and modifying picture data is further arranged to receive metadata from at least one metadata recording device 120, 130. To this end, the device 100 includes input circuits 140 for receiving raw picture data from the camera 110 and for receiving metadata from a metadata recording device; and processing circuits (not illustrated) configured for generating modified picture data based on the raw picture data and the metadata. The at least one metadata recording device may include a camera metadata recording device 120, intended to be operated by, e.g., a camera assistant, or a script metadata recoding device 130, intended to be operated by, e.g., a script supervisor, or both.

The device 100 for receiving and modifying picture data is further arranged to generate modified picture data based on the received picture data and the received metadata.

To this end, the device 100 for receiving and modifying picture data includes processing circuits (not illustrated), a memory (not illustrated) and associated input/output circuits (not illustrated). In an aspect, generating of the modified picture data performed by the device 100 includes color correcting, or color grading, the received raw picture data in dependence on the content of the metadata.

In an aspect, the received metadata includes camera functions or settings, indicated as "technical information" in fig. 1. Such metadata may include one or more elements selected from the following elements: Position data (e.g. data representing GPS coordinates) acceleration data, e.g. gyro data, time data.

In this case, the metadata recording device 120 may be a wireless terminal, and the receiving of metadata may include receiving the camera functions or settings via a wireless interface.

Alternatively, a wired or another type of interface may be used for transferring the metadata.

The camera functions or settings metadata may be received from a metadata recording device in the form of a wireless terminal, e.g. a portable computer, a PDA, a mobile phone or the like, e.g. provided with WLAN communication capability or another wireless or wired communication capability. In this case, the receiving of the metadata includes receiving the camera functions or settings metadata via a wireless interface such as a WLAN interface.

In a particular embodiment, a wireless terminal 120 which has built-in features for providing at least one of the following data may be used: Position data (e.g. provided by a built-in GPS receiver), acceleration data (e.g. provided by a built-in gyro sensor), and time data (e.g. provided by a built in GPS receiver). This will be particularly useful if the wireless terminal is physically attached to the camera, since the metadata provided by the wireless terminal's built-in features will be representative for the position and movement of the camera, and for the absolute time.

As an example, the wireless terminal 120 may be an Apple iPhone. Other types of terminals, including portable computers, PDA's, smartphones, mobile telephones may be applicable.

In another aspect, the received metadata may include script metadata. Such script metadata may identify at least one of a day, a scene, a slate and a take.

The script metadata may also be received from a metadata recording device in the form of a wireless terminal 130, e.g. a portable computer, a PDA, a mobile phone or the like, e.g. provided with WLAN communication capability or another wireless or wired communication capability. In this case, the receiving of the metadata includes receiving the script metadata via a wireless interface such as a WLAN interface.

Alternatively, a wired or another type of interface may be used for transferring the metadata.

As an example, the wireless terminal 130 may be a portable Macintosh computer OSX. Other types of terminals, including portable computers, PDA's, smartphones, mobile telephones may be applicable.

In the device 100 for receiving and modifying picture data the received metadata may be stored in a database 160, e.g. together with storage of the received picture data.

Although the database 160 has been illustrated as an integral element of the device 100, it should be understood that the database 160 may be external to the device 100. Also, the database 160 may be embodied either as a central database or distributed among any suitable number of physical database elements.

The received metadata and the received picture data may also be stored in a back-up subsystem 170, which may be provided with a verification subsystem 172, further connected to external (or internal) storage devices such as a magnetic tape storage (indicated as "LTO tape") 174, a disk array (indicated as RAID; Redundant Array of Inexpensive Disks) 178, or another magnetic, optical or solid-state storage (indicated as "Harddisk") 176, or any combination of these.

The generating step, performed by the device 100, may include a de-Bayering or demosaicing step, performed on the received picture data. An element representing the de-Bayering or demosaicing step is illustrated at 180. De-Bayering or demosaicing involves a pre-transformation of the raw data to a viewable form.

The generating of the modified picture data, in particular the color correcting, or color grading, of the received raw data, may be performed based on a day, scene, slate or take identified by the script metadata. This color correcting may be performed by the color correcting element 190.

The substep of color correcting (or color grading) based on meta data, in particular script meta data, provides a useful connection of the story (identified by the script data) to the digital workflow. As a result, a user is given the possibility to display and digitally color grade the work in context to the story, in an efficient and time- saving way.

The color correcting element 190 is further connected to the color profile element 210, which is further connected to the digital display 220.

In addition to the metadata received from the camera recording device 120 and the script data recording device 130, additional metadata may also be received from the camera 110.

Such additional metadata may include camera identification data, lens data, shutter data, camera time data, etc.

The device 100 for receiving and modifying picture data may also be arranged for outputting the modified picture data and the metadata, e.g. using the output rendering element 200.

The modified picture data and the metadata may, e.g., be further forwarded to a post-production facility 270. The metadata provided to the post-production facility 270 may comprise XML data, as illustrated at 260. In an aspect, the picture data received by the device 100 for receiving and modifying picture data includes a first set of time stamps, and the metadata includes a second set of time stamps. The generating of modified pictures may then include comparing the first and second sets of time stamps.

In an aspect, the device 100 for receiving and modifying picture data may further comprise a sub-process of encoding the modified picture data to multiple output formats.

The multiple output formats may include formats with various resolutions, and they may be useful for different purposes, such as previewing, editing, storage, transmission, etc. The sub-process of encoding the modified picture data to multiple video formats may comprise a common decompression substep; a common de-Bayering or demosaicing substep; and a common color correction substep; followed by separate encoding substeps. In this context, "common" step denotes a step that is performed only once in the overall encoding sub-process. Each encoding substep may be adapted for a particular video format. The division of the overall encoding sub- process into common pre-substeps (decompression, de-Bayering or demosaicing, and color correction), which are common to various resulting file formats, and separate encoding substeps, each particularly adapted to the file format in question, results in a substantial reduction of the overall processing/rendering time, compared to the traditional approach of decompression, de-Bayering/demosaicing, color correction and encoding, all steps made separately for each required output format.

The output data may be transferred to one or more of the following: the XML output 260, a network element 259 connected to the Internet, a storage element 240 such as a portable hard disk drive, a display device 230 such as an optical film display.

Fig. 2 is a schematic flow chart illustrating principles of a first exemplary embodiment of a method for receiving and modifying picture data.

The method starts at step 400.

Next, in step 410, picture data is received from a camera. The picture data may be video data in a compressed raw format.

Next, in step 420, metadata is received from a metadata recording device, e.g. including camera functions or settings, or script metadata, or both.

Next, in step 430, modified picture data is generated based on the received picture data and the received metadata, e.g. by color correcting the picture data in dependence on the metadata.

The method terminates at step 490.

Fig. 3 is a schematic flow chart illustrating principles of a second exemplary embodiment of a method for receiving and modifying picture data.

The same reference numerals have been used for those elements which also appear in fig. 2. The written description of elements shown on fig. 2 equally applies for the description of the same elements depicted in fig. 3.

The method starts at step 400.

Next, in step 410, picture data is received from a camera, e.g. a digital film camera. Next, in step 420, metadata is received from a metadata recording device, e.g. including camera functions or settings, or script metadata, or both.

In substep 422, metadata including camera functions or settings is received from a first wireless terminal, and the camera functions or settings are received via a wireless interface associated with the first wireless terminal.

Next, in the substep 424, metadata including script data is received from a second wireless terminal. The script metadata is received via a wireless interface associated with the second wireless terminal.

Next, in step 430, modified picture data is generated based on the received picture data and the received metadata. The received picture data may include a first set of time stamps, and the metadata includes a second set of time stamp. The generating of the modified picture data may include comparing the first and second sets of time stamps.

First in the generating step 430, in substep 432, the received picture data is subject to an introductory de-Bayering or demosaicing process.

Next, in substep 434, at least one of a day, a scene, a slate and a take is identified from the script metadata.

Next, in substep 436, a color correcting is performed on the picture data in dependence on the metadata. The color correcting is performed based on a day, scene, slate or take identified by the script metadata.

Next, in substep 440, the modified picture data and the metadata is output and further provided to a post-production facility. The output metadata may include XML data.

The modified picture data may be encoded to multiple formats. The encoding may comprise a common decompression substep; a common de-Bayering or demosaicing substep; and a common color correction substep; as well as separate encoding substeps, each adapted for a particular video format, in the same way as already explained with reference to fig. 2.

The metadata may also be stored in a database. The received picture data may also be stored in the database. The received picture data may also be stored in a backup facility, in the same way as already explained previously in the present specification.

Various aspects of the invention may, amongst other things, provide at least some of the following advantages: - the total workflow of the movie production process may be made more efficient, - a user of a movie production facility may gain better control over the creative process,

- the safety and reliability is increased, compared to previous solutions handling digital raw data in movie production,

- the camera assistants' and script supervisors' work may be made more efficient, since previous manual operations and tasks are effectively substituted by use of wireless communications;

- managing of files resulting from a movie production set is made more efficient, making it possible to make quick and well-reasoned decisions based on an early viewing of recorded material, - digital post-processing may be made more efficient.

The invention has been described herein by examples. Numerous variations and alternatives will be evident to the skilled person. Thus, the scope of the invention is set forth by the appended claims and their equivalents.