POSITIONING A SUBJECT WITH RESPECT TO A BACKGROUND SCENE IN A DIGITAL CAMERA

Background

[0001] This invention relates to the pre-positioning of multiple images in a still digital camera for subsequent compositing onto the image of a background scene.

[0002] Blue screen image compositing technology makes possible the generation of a number of special video effects such as believable giants, or shrunken people as in the movie, "Honey I Shrunk The Kids".

[0003] Another useful effect uses multiple image layers. Subjects are photographed separately and composited one at a time to become a multilayer composite image. This process allows one actor to play several roles in the same scene.

[0004] Not all subjects must be people. If one of the subjects, in a multilayer composite image is a wall of flames, then previously composited subjects will be seen to be behind the flames, while subsequent composited subjects will be seen to be on the camera side. To be effective, those subjects behind the fire should be made smaller, while those subjects on the camera side of the fire are enlarged to provide distance perspective.

[0005] Although the blue screen image compositing process was developed for motion picture and television productions, it works equally well for still pictures. Another compositing technique employs the detection of luminance levels, but is not as successful as the blue screen process. Even a double exposure has been used to create a composite image to achieve an artistic effect in still photography.

[0006] With the advent of the digital camera, many of its users would like to achieve some of these special effects for themselves, without requiring the skills of an image processing specialist.

Summary of the Invention

[0007] This invention permits an operator of a digital still camera to position a high resolution image of a subject, with respect to the image of a background scene, in the camera instead of a computer. A low resolution composite image of the subject and background scene permits the operator to physically position the subject with respect to the background scene, while observing the composite image on the camera view screen.

Brief Description of the Drawing

[0008] The Figure is a diagram showing the functions required for a digital camera to preview and generate the layers of a multilayer composite image.

Detailed Description ofthe Invention

[0009] The positioning of a subject's image is explained by the following example and the Figure. Assume an individual wishes to take a picture of a group of seven friends at some exotic location. He finds the cost of transportation to be prohibitive, and getting them all together at one time is not practical. His solution to this problem is to use a digital camera (4) to photograph each friend separately as each becomes available. Using prior art techniques, the images would

then be downloaded into a computer for compositing onto a background scene.

[0010] A multilayer composite image, however, requires all of the subject images to be properly positioned with respect to each other and to the background scene. If the subjects are positioned in the computer, an image editing program such as Adobe Photoshop®, or similar program is required. To avoid this complex operation, one can pre-position the subjects in a digital camera when the camera is equipped with a simple compositing program using, for example, a chroma-key technique. Creating such a simple compositing program for use in present generation digital cameras is well within the abilities of persons skilled in the art.

[0011] The first step is to photograph the exotic background scene, or transfer a suitable background scene from an external source such as a computer (1) to the camera.

[0012] The second step is to select a backing (2) to place behind the subject. While a plain wall of almost any color may be used, extracting a good matte is readily achieved if the wall color is not used in the subject's wardrobe and is compatible with the requirements of the compositing program used by the camera. A matte is a black silhouette of the subject against a clear field that permits the subject to be placed over the background scene without creating a double exposure.

[0013] To prevent cluttering the composite image being viewed on the camera view screen by lighting fixtures, doors, and furniture lying inside the image frame, a windows feature in the compositing program is used to mask-out or crop these objects. Each of the window's four edges can be positioned to avoid all unwanted objects in the viewed image, thus effectively extending

the backing to the edge of the image frame. This window function permits the use of a relatively small backing extending only a short distance beyond the subject. For example, modern digital cameras typically include a four-position switch with a select button. A window, that is a rectangular outline, can be moved up/down and left/right using the four-position switch. Once in the desired position, typically surrounding the subject, but excluding the rest of the image, a button can be pressed to fix the window at the desired location. If the window is too small or too large, the window can also be resized using the four-position switch so that the window encloses the foreground subject while excluding everything else.

[0014] The position and size of each window is recorded as part of the subject image data at the moment the foreground subject is captured. This window placement and size data defines the window when the subject layers are composited in computer 1. Having the window size and placement data and layer number of the pre-positioned subjects permits image compositing with little or no human intervention.

[0015] The third step is to photograph the first subject (3) . The compositing program in the camera (4) composites the image of the subject over the image of the background scene, thus forming a composite image on the camera view screen (5) .

[0016] The compositing program in the camera need not actually composite the high resolution images of the subject and background scene. The number of pixels comprising an image on a camera's view screen is a small fraction of the number of pixels comprising a high resolution digital image in the camera. What is being

composited during the subject positioning process are low resolution replicas extracted from high resolution images, at the resolution required by the camera view screen.

[0017] The relatively small number of pixels comprising the image on the view screen, and a simple compositing program, permits rapid replacement of the composite image to keep it current as the subject's position is being changed.

[0018] As the operator observes the camera view screen (5) , the background scene will be fixed on the camera view screen, but the image of the first subject is free to follow the camera's movements. Pointing the camera is the simplest way to position the subject up, down, left or right, with respect to a background image. The size of the subject is adjusted by physically moving the subject to a new position toward or away from the camera. Within its limits, a size change can also be made by adjusting the camera's zoom lens.

[0019] Having properly positioned the subject with respect to the background scene, the operator presses the "Take' button on the camera. At this instant, the composite image on the camera view screen is captured. The composite image on the view screen now contains the first subject positioned on the background scene. This image is stored since it will become the background scene when photographing the second subject.

[0020] Simultaneously, the pre-positioned high resolution image of the first subject against its original backing is captured and stored. It is coded as subject layer #1.

[0021] When the second subject is available, he is placed against the backing. The composite image of the first

subject and background is recalled from memory and becomes the background scene for the second subject. The camera operator positions the second subject to place his image alongside the image of the first subject, keeping in mind to leave space for the remaining five subjects.

[0022] Being satisfied that the second subject is properly placed, the ^λ Take" button is touched. At this moment the second subject also becomes part of the background scene, and is stored. It will become the background scene for the third subject. Simultaneously the pre- positioned high resolution image of the second subject against the backing is stored in camera memory, and coded as subject layer #2. The backing used behind the second and later subjects need not be the same backing used for subject #1. A backing is often chosen because its color does not match any of the colors in the subject's wardrobe, thus assuring the extraction of a good matte.

[0023] The above process is repeated for each subject until all seven subjects have been positioned and photographed. The seven high resolution pre-positioned subjects, each in front of a selected backing, are downloaded to a computer for compositing with the selected background scene. If the background scene originated in the camera, then the original, high resolution background scene is also downloaded to the computer. At this point the low resolution composite image of the seven subjects may be discarded.

[0024] An added advantage of positioning the layers of a multilayer composite image in the camera instead of a computer, is the uniform resolution of all layers with respect to each other. When sizing is done in the

computer, those images that are enlarged will have a lower resolution than those images not enlarged.

[0025] Many image compositing programs will composite one or more image layers with little or no operator intervention if the images have been pre-positioned. A compositing program such as the ULTIMATTE "KnockOut" (Pat. No.6, 134, 346) will produce a flawless high quality composite image at the resolution of the images provided. There are essentially no limits to the number of layers in a multilayer composite image. One may composite a single layer or a hundred or more layers, as needed.

[0026] Having positioned a subject in the camera, it is also possible to make a high resolution composite image in the camera as well. However, to composite a high resolution image over a background scene takes a substantial amount of time to perform the data processing with the limited processing power currently available for inclusion in a small digital camera. However, as the cost of memory decreases and processing power increases, performing a high resolution composite image in the camera is possible since the necessary technology already exists.

[0027] All elements of this invention, including digital image compositing, are well known to those skilled in generating visual special effects for motion picture and television productions, and graphic arts. US patent 6,134,346 is an example of a digital software image compositing program. There are at least a dozen manufacturers making image compositing devices, or their equivalent in software. Many of these programs will

extract a suitable matte if an appropriate backing has been selected, and uniformly illuminated. 8] A simple digital compositing program using, for example, a chroma-key technique, because of its minimal processing requirement, can be incorporated into the design of almost any sized digital camera including a cell phone, personal pocket sized digital camera and the larger professional digital cameras. A mode switch on the camera permits camera function buttons to be used as compositing function buttons when positioning a subject's image. The compositing program and camera control functions can occupy common digital data processing and image storage chips.