DISPLAY

Technical Field Of The Invention

In general, the present invention relates to systems and methods that are used to create three- dimensional images that are displayed on an

electronic display at a computer interface. More particularly, the present invention relates to systems, methods used to creating three-dimensional images that are based upon live camera footage obtained by photographic or video recording of live subjects or real objects.

Background Art

People interact with computers for a wide variety of reasons . As computer software becomes more sophisticated and processors become more

powerful, computers are being integrated into many parts of everyday life. In the past, people had to sit at a computer keyboard or engage a touch screen to interact with a computer. In today's environment, many people interact with computers merely by

talking to the computer. Various companies have programmed voice recognition interfaces . For example, Apple Inc., has developed Siri® to enable people to verbally interact with their iPhones®. Amazon Inc. has developed Alexa® to enable people to search the World Wide Web and order products through Amazon®.

Although interacting with a computer via a voice recognition interface is far more dynamic than a keypad or touch pad, it still has drawbacks . The artificial intelligence algorithms used for voice interaction interfaces are very limited. As such, voice interaction interfaces can only answer certain basic queries and those queries must contain

specific key words. If more complex matters need to be communicated, there is no current substitute for talking to a real knowledgeable person. It is for this reason that most people prefer speaking to a live person on a telephone rather than interacting with a question and answer computerized phone system. Likewise, when a person is provided with the option of talking a live person on a telephone or

interacting with a live person face to face, most people prefer the face to face option. When two humans communicate face to face, many of the

communication queues used in the conversation are visual in natural . The manner in which a person moves their eyes, tilts their head, or provides any facial expression or hand gesture, conveys

significant additional meaning to words that are being spoken. When communications are purely based on audio signals, such as during a phone call, much of the nuance communicated by human body language is lost. Likewise, when a computer communicates with a human through an audio interface, nuanced

information is lost.

It is not possible to provide a live person to stand around every computer in the world. However, it is possible to provide a three-dimensional image of a live person or real object to any computer or digital display that is enabled to present such an image. In the prior art, there are many systems that enable a person to video conference with a live person at a remote location. There are also systems that enable people to call-up an avatar. For instance, in U.S. Patent Application Publication No. 2006/0294465 to Ronene, an avatar system is provided for a smart phone . The avatar system provides a face that changes expression in the context of a

conversation. The avatar can be customized and personalized by a user. A similar system is found in U.S. Patent Application Publication No. 2006/0079325 to Trajkovic which shows an avatar system for smart phones . The avatar can be customized, where aspects of the avatar are selected from a database. An obvious problem with such prior art systems is that the video image of the person you are talking to or the avatar you are viewing is two- dimensional. Furthermore, if a smart phone is being used, the image being viewed is on a screen that may be less than two inches wide. Accordingly, much of the visual information being communicated can be difficult to see and easy to miss.

Little can be done to change the screen size on many devices such as smart phones. However, many of the disadvantages of a small two-dimensional avatar can be overcome or minimized by presenting an image that is three-dimensional. This is especially true if the three-dimensional effects designed into the image cause the image to appear to project out of the plane of the display. In this manner, the image will appear to stand atop the smart phone during a conversation .

In the prior art, there are many systems that exist for creating stereoscopic and auto- stereoscopic images that appear three-dimensional. However, most prior art systems create three- dimensional images that appear to exist behind or below the plane of the electronic screen. That is, the three-dimensional effect would cause an image to appear to extend down into the screen of a smart phone. The screen of the smart phone would appear as a window atop the underlying three-dimensional virtual environment. With a small screen, this limits the ability of the 3D image to provide visual communication queues .

A need therefore exists for creating a 3D image that can be used to provide live communications between a person at a first location and one or more people at remote locations, wherein the images used for communication appear three-dimensional and also appear to extend out from the electronic display from which it is shown. This need is met by the present invention as described and claimed below.

Disclosure Of The Invention

The present invention is a system and method for communicating between a first location and one or more remote locations . A production set is established at the first location. At the production set, a first person or object is imaged with stereoscopic video cameras to obtain stereoscopic footage. The stereoscopic footage is digitally enhancing with 3D effects to create a production video file. The production video file is transmitted to an electronic device at a second location through a data network. The electronic device has a display screen that can be viewed by a second person at the second location.

The production video file is played on the electronic device, wherein the production video fil creates images of the first person or object at the first location. On the display screen, the images appear three dimensional to the second person.

Furthermore, the images appear to extend in front or vertically above, the display screen of the electronic device .

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present invention, reference is made to the following description of exemplary embodiments thereof, considered in conjunction with the accompanyin drawings, in which:

FIG. 1 is a block diagram schematic

illustrating the components utilized in an exemplary embodiment of the present invention;

FIG. 2 is a diagram illustrating the present invention system being utilized in a first exemplary use; FIG. 3 is a diagram illustrating the present invention system being utilized in a second

exemplary use;

FIG. 4 is a diagram illustrating the present invention system being utilized in a third exemplary use; and

FIG. 5 is a diagram illustrating the present invention system being used for a conference call with multiple parties simultaneously.

DETAILED DESCRIPTION FOR CARRYING OUT THE INVENTION

Although the present invention system and method can be used to create and display interactive

3D images for a variety of purposes, the embodiments illustrated show only a few exemplary applications of the technology. Additionally, although the 3D images can be displayed on any type of electronic display, the illustrated embodiments show the 3D images displayed on the screen of a smart phone and on a screen of a stationary display. These

embodiments are also selected for the purposes of description and explanation only. The illustrated embodiments are merely exemplary and should not be considered a limitation when interpreting the scope of the appended claims .

In the present invention, a communications system is established between at least two locations. As part of the communications system, enhanced 3D images are transmitted in at least one direction.

The 3D images are created by imaging a live person, or some other object, at a live production set.

Referring to Fig. 1, an exemplary embodiment is provided that shows a two-way communication system 10. The communication system 10 transmits 3D images 12 in one direction. As will be explained, the communication system 10 enables communications between a first person who creates the 3D images 12 and a second person who views the 3D images 12. The 3D images 12 are viewed on the display screen 14 of an electronic device 16. The display screen 14 is accessible by a viewer 18, wherein the 3D image 12 appears to have enhanced three dimensional effects when viewed. The enhanced 3D effects cause the 3D images to appear to extend vertically above, or in front of, the display screen 14 of the electronic device 16 displaying the 3D images 12. If the

electronic device 16 has a traditional display screen 14, such as an LED or LCD, then the 3D images 12 must be viewed through some form of 3D glasses 19 to appear three dimensional to the viewer 18. If the electronic device 16 has an auto-stereoscopic display or similar specialized 3D display, then the 3D image 12 can appear three dimensional to the naked eye .

The real person or object appearing in the 3D images 12 is referred to as the camera subject 20. The camera subject 20 is imaged at a live production set 22. The live production set 22 has stereoscopic video cameras 24 and other hardware necessary to image the camera subject 20 and produce the 3D images 12. It should be understood that the live production set 22 need not be an existing place. Rather, the live production set 22 can be created wherever needed. For example, the live production set 22 can be temporarily erected in a college classroom to create a 3D image 12 from a professor teaching a class.

The 3D images 12 are based upon the imaging of the camera subject 20. Therefore, the 3D images 12 move as the camera subject 20 at the live production set 22 moves . The 3D images 12 also appear three dimensional, extending either vertically above or in front of the display screen 14 depending upon the orientation of the display screen 14. The

communication system 10 can be fully interactive. That is, not only can the camera subject 20 at the live production set 22 communicate with the viewer 18 through the 3D images 12, but the viewer 18 also can communicate with the camera subject 20. This two-way communication enables the viewer 18 of the 3D images 12 to ask a question and the camera subject 20 can react to the question and answer that question through the 3D images 12.

With continued reference to Fig. 1, the creation of the 3D images 12 is first explained. In the live production set 22, there are at least one set of stereoscopic cameras 24. Each set of

stereoscopic cameras 24 include a left view camera and a right view camera. The stereoscopic cameras 24 produce raw footage 26 that is processed by a production computer system 28. The production computer system 28 runs specialized 3D effects software 30 that alters the raw footage 26 and superimposes the raw footage 26 from the

stereoscopic cameras 24 to create a production video file 32 with integrated 3D effects. The 3D effects are engineered to cause the 3D image 12 to appear to extend vertically above, or in front of, the display screen 14 when viewed. The operation of the 3D effects software needed to produce the stated 3D effects in 3D image 12 is explained in co-pending U.S. Patent Application No. 15/665,423, entitled "System, Method And Software For Producing Live Video Containing Three-Dimensional Images That Appear To Project Forward Of Or Vertically Above A Display", the disclosure of which is herein incorporated by reference.

The production video file 32 is transmitted to the electronic device 16 of the viewer 18 via a dat network 34. The data network 34 can be a cellular network or a computer network, such as the World Wide Web. The production video file 32 can be recorded for later play by a viewer. For example, i the electronic display 14 where an information station at a museum display, a prerecorded

production video file 32 can be played each time an information button is pressed at the museum display However, since the production video file 32 is created from a live camera subject 20, the

communication system 10 is especially well suited for live streaming the production video file 32 to the electronic device 16 of the viewer 18. In this manner, the viewer 18 can also stream questions and comments back to the camera subject 20 in the live production set 22.

If the electronic device 16 being used by the viewer 18 has a microphone, then the viewer 18 can send audio messages back to the live production set 22. Likewise, if the electronic device 16 has a camera and a microphone, as do most smart phones and tablet computers, then the viewer 18 can send audio and video messages back to the live production set 22. Messages sent upstream from the viewer 18 to the live production set 22 are displayed on an off- camera display 36 that can be viewed and/or heard by the camera subject 20, if the camera subject is a person. Alternatively, the message can be fielded by an off-camera production worker. In either scenario, someone at the live production set 22 can hear and/or see questions and comments posted by the viewer 18. The camera subject 20, a production worker, or a software operated AI assistant can then react to those questions and comments. The viewer 18 receives reactions and answers from the camera subject 20 through the 3D images 12 extending vertically above, or in front of, the display screen 14 of the viewer's electronic device 16.

In addition to showing the 3D images 12 of the camera subject 20 to the viewer 18 in forward or vertically projecting 3D, the production video file 32 can be augmented to better communicate

information from the camera subject 20 to the viewer 18. In Fig. 1, it can be seen that the production computer system 28 has access to a 3D model database 38. In the 3D image database 38 contains virtual models 40 that have been previously created and engineered to contain enhanced 3D effects. The enhanced 3D effects in the virtual models 40 are also the type that cause the virtual models 40 to appear to project forward, or above, the display screen 14 of the electronic device 16 when viewed. The creation of virtual models 40 with such effects is disclosed in co-pending U.S. Patent Application No. 15/665,423, entitled "System, Method And

Software For Producing Live Video Containing Three- Dimensional Images That Appear To Project Forward Of Or Vertically Above A Display", the disclosure of which is herein incorporated by reference.

The selection of a virtual model 40 from the 3D image database 38 can be done by the camera subject 20 or someone else at the live production set 22 using an effects control unit 42. The effects control unit 42 can be as simple as a computer mouse or slide show controller. The camera subject 20 can select from a menu posted on an off-camera

production display 44. Alternatively, the image or video can be recalled from the 3D model database 38 by a production assistant who understands the choreography of the presentation. If an AI assistant is being used, then the selection of a virtual model 40 may be managed by an automated system programmed to respond according to a predetermined routine, or to respond based on input or requests from a viewer.

An example of this feature is shown in Fig. 2. Referring to Fig. 2 in conjunction with Fig. 1, it will be understood that the exemplary camera subject 20 is a professor 20A who is providing a live online lecture. The viewer 18 may be a medical student who has a question about some aspect of the heart . The viewer 18 communicates that question to the camera subject 20 using an electronic device 16 in the form of a smart phone 16A. Upon receipt of the question, the professor 20A can recall a 3D virtual model 40 of a heart 40A from the 3D model database 38. The professor 20A selects the image and

positions that virtual model 40 by viewing the off ^¬ screen production display 44. The virtual model 40 selected by the professor 20A is superimposed over the production video file 32 being create live. This produces an augmented production video file 32A that is sent to the viewer 18. The viewer 18 sees both the 3D images 12 of the professor 20A and the

virtual model 40 of the heart 40A in the same

virtual environment. By referencing the off-screen production display 44, the professor 20A can point to different aspects of the heart 40A as he/she answers the question. Such a communication system 10 is highly useful for customer support centers, wherein a customer support representative can

demonstrate to a customer how to connect a cable box, assembly modular furniture, or otherwise trouble shoot some problem.

Augmenting the production video file 32 with a virtual model 40 requires more than simply

overlaying the two files. Rather, in order to

maintain the optimal 3D effects, the virtual model 40 must be oriented to match the camera settings used in the creation of the production video file 32. In the live production set 22, the camera subject 20 is being imaged by stereoscopic cameras 24. The height of the stereoscopic cameras 24, the

orientation of the stereoscopic cameras 24, and the distance between the stereoscopic cameras 24 are all very important parameters when creating the enhanced 3D effects. These settings, among others, are read by the production computer system 28.

The virtual models 40 held in the 3D model database 38 are computer constructs . Accordingly, they can be viewed at any angle from any viewpoint in a virtual environment. The selected virtual model 40 is configured to be at viewed with the same camera settings as are being used in the live

production set 22 to view the camera subject 20. In this manner, when the virtual model 40 is

superimposed over the production video file 32, both images have continuity in viewpoint and camera settings. This makes the virtual model 40 and the 3D images 12 visually consistent in the same virtual environment. Being visually consistent, the

engineered 3D effects are also visually consistent and work together seamlessly when seen by the viewer 18.

In the previous embodiments, the viewer 18 is viewing the 3D image 12 or the augmented 3D image using his/her own electronic device 16. Assuming such an electronic device 16 is a smart phone, tablet, laptop computer or desktop computer, the electronic display 16 has a traditional display that is designed to display 2D images. With such displays, 3D glasses 19 are required to see the 3D effects.

Referring to Fig. 3, an example is provided where a viewer 18 can utilize the present system without their own electronic device and without specialized 3D glasses. In Fig. 3, an exemplary reception

station 42 is shown. The reception station 42 may be the concierge desk at a hotel, an information desk at a hospital, or a kiosk in an amusement park. Regardless, the reception station 42 contains a display 44, a camera 46, a microphone 48 and a user interface 50. The reception station 42 is connected to a remote live production set 54 that contains one or more live receptionists 52.

A viewer 14 approaches the reception station 42 and activates the display 44. The activation of the reception station may be achieved using passive sensors which detect an approaching user, or may be achieved via the user interface 50. The display 44 here is preferably an auto stereoscopic display based on any technology capable of showing 3D stereoscopic images without the need for specialized glasses . Once activated, the reception station 42 is linked to the live production set 54 and a 3D image 56 appears on the display 44. The live production set 54 can see and hear the viewer 18 via the camera 46 and microphone 48 at the reception station 42. The user interface 50 at the reception station 42 may also contain a credit card reader, ticket dispenser, or other such device or integrated software that may assist in conducting business . The viewer 18 can interact with the 3D image 56 in the same manner as if the receptionist 52 where

physically present. It has been previously stated that although a production video file is produced by imaging a camera subject in a live production set, the 3D images being viewed by a viewer need not look like or sound like the camera subject . Rather, the movements of the camera subject at the live

production set can be used to animate a virtual avatar. Referring to Fig. 4, it can be seen that markers 62 can be worn by a camera subject 60 in a live production set 64. The camera subject 60 is imaged in the manner previously described. The production computer system 66 runs modeling software 68 that generates a virtual model 70 and manipulates the virtual model 70 as directed by the movement of the markers 62 on the camera subject 60. The 3D effects that cause the virtual model 70 to appear to project forward or above the display screen when viewed are added. This produces a final live avatar 72 that appears in 3D. The live avatar 72 can look like anyone or anything. In the shown example, the virtual live avatar 72 looks like a cartoon

fairytale princess. Such a virtual live avatar 72 can be used at reception stations in an amusement park .

In the previous embodiments, two-way

communications are established. However, 3D images are only sent in one direction and are viewed by one party. This need not be the case. Referring to Fig. 5, it can be seen that multiple production sets 22A, 22B, 22C can be established. Each production set 22A, 22B, 22C has a camera subject 12A, 12B, 12C with the before-described equipment and software needed to produce enhanced 3D images 82 of each camera subject 12A, 12B, 12C. In addition, each production set 22A, 22B, 22C is provided with an electronic display 80 that can display the 3D images 82 from a different location. Each electronic display 80 is part of electronic equipment that runs a conference call management software application 84. The conference call management software application 84 enables any viewer at any of the live production sets to view the 3D image of any or all of the 3D images

partaking in the conference .

It will be understood that the embodiments of the present invention that are illustrated and described are merely exemplary and that a person skilled in the art can make many variations to those embodiments. All such embodiments are intended to be included within the scope of the present invention as defined by the claims .