The present invention relates to video conferencing systems and telepresence.

Background of the invention

Conventional videoconferencing systems comprise a number of end-points communicating real-time video, audio and/or data (often referred to as duo video) streams over and between various networks such as WAN, LAN and circuit switched networks.

A number of videoconference systems residing at different sites may participate in the same conference, most often, through one or more MCU 's (Multipoint Control Unit) performing i.a. switching and mixing functions to allow the audiovisual terminals to intercommunicate properly. However, representing moving pictures requires bulk information as digital video typically is described by representing each pixel in a picture with 8 bits (1 Byte). Such uncompressed video data results in large bit volumes, and can not be transferred over conventional communication networks and transmission lines in real time due to limited bandwidth. Thus, enabling real time video transmission requires a large extent of data compression which may compromise with picture quality. The compression of multimedia data to be transmitted, as well as the decompression of the multimedia data to be received, take place in a processor unit conventionally referred to as a codec. As videoconferencing involves various recourses and equipment simultaneously interoperating at different localizations and capabilities, there is also a need for the possibility to manage the resources involved both for scheduled and ad hoc videoconferences through a video conference manager tool.

Video conferencing systems presently provide communication between at least two locations for allowing a video conference among participants situated at each station. Conventionally, the video conferencing arrangements are provided with one or more cameras. The outputs of those cameras are transmitted along with audio signals to a corresponding plurality of displays at a second location such that the participants at the first location are perceived to be present or face-to-face with participants at the second location. Further, the images captured by the plurality of cameras must be arranged and displayed so that they generate a non-overlapping and/or contiguous field of view, so-called continuous presence. Continuous presence is a mixed picture created from far-end sites in an MCU. For example, in case of a videoconference of five participants, each site will receive a picture divided into four quadrants with the picture captured from each of the other sites inserted in the respective quadrants.

Continuous presence or several displays with only one camera prevents the feeling of eye-contact among participants in video conferencing systems. Typically, a camera is placed somewhere above the display at which a participant is observing a display of the participant from the remote station. Consequently, the camera captures the participant at an angle above and on the side of the participants viewing level or head. Thus, when an image of that participant is displayed at the remote station, it appears as if the participant is looking down or to the left or right. Previous solutions to this problem have required complex optical systems and methods using, for example, a plurality of lenses and mirrors. The solutions have usually been designed for use when the camera is capturing an image of a single participant, and they fall short when simultaneously capturing images of multiple participants. In addition to the lack of sufficient eye-contact, there are also other limitations in conventional videoconferencing limiting the feeling of being in the same room. Continuous presence and small displays also limits the size of the displayed participants. Low capturing and display resolution and highly compressed data also contribute to a reduction of the experience of presence. Some solutions have tried to improve this by introducing so-called telepresence systems requiring dedicated high bandwidth communication lines. However, these solutions are not well suited to be connected to a conventional LAN or WLAN, and are not interoperable with conventional videoconferencing systems.

US-20080246834 describes a communication system that solves these issues, wherein at least two telepresence terminals, each terminal having at least one transceiver, are connected by more than one separate point-to-point connection between the separate transceivers of each of the telepresence terminals. Which transceiver in which endpoints to connect to each other, are set up when a conference is established; e.g. on a request from a control unit or from a conference manager, a master site starts transmitting instructions to the other participating telepresence systems on how to set up connections between them. The instructions are sent to the master transceiver, which in turn relay the instructions to the other transceivers in question. The instructions are typically incorporated in an open field in the message flow specifying the control protocol of establishing video conference call, such as ITU H.241, H.242 or H.243. Previous instructions schemes have been unreliable, and as such there is a need in the art of an improved method and system of establishing telepresence conference calls.

Summary of the invention It is an object of the present invention to provide a system and a method avoiding the above described problems.

The features for solving said problems are defined in the enclosed independent claims.

In particular, in a first aspect, the present invention discloses a method of setting up communication sessions in a telepresence call between at least two telepresence systems (TPs), each TP having at least one transceiver, wherein information required for setting up the communication sessions is embedded in a control protocol message flow establishing a first communication session between a first transceiver of a first TP among the at least two TPs and a first transceiver of a second TP among the at least two TPs.

Brief description of the drawings

In order to make the invention more readily understandable, the discussion that follows will refer to the accompanying drawings, Figure 1 shows an example of an endpoint assembly according to the present invention;

Figure 2 shows another example of an endpoint assembly according to the present invention;

Figure 3 shows an exemplary message flow between a a first telepresence system and a second telepresence system according to the present invention, and

Figure 4 shows a state diagram for a method according to the present invention.

Detailed description the invention

In the following, the present invention will be discussed by describing a preferred embodiment, and by referring to the accompanying drawings. However, people skilled in the art will realize other applications and modifications within the scope of the invention as defined in the enclosed independent claims. The present invention introduces a novel system for providing realistic presence of participants, or telepresence, in a videoconference by means of standard based communication. An endpoint compliant to this system is disclosed in figure 1. It comprises at least one codec and one camera for each display. The displays are preferably large-scaled flat screens placed side by side with the cameras respectively placed on top of the displays.

There is also preferably an additional display e.g placed below the other displays being reserved for data input i.a. presentations and shared applications or for communicating with regular videoconferencing endpoints usually having only one camera and displaying several sites in the same display. Alternatively, the additional display is replaced by several table mounted displays as shown in fig 2.

There is preferably also a control unit installed in an endpoint according to the present invention. The control unit have a user interface e.g. with a pressure sensitive screen, making the user able to initiate calls in addition to all other functions and adjusting all settings that the system is capable of providing. The control unit could also be able to receive and execute commands from a conference manager tool, such as setting up scheduled conferences etc. The control unit is in turn connected to the other parts of the endpoint through a master device that represents the system as a whole. The master device could be a part of the control unit, or it could be one of the codecs. The master device would also be the only device registered with a conference manager.

Accordingly, telepresence in this context means a videoconference between videoconferencing endpoints, where at least one of the endpoints has at least two displays, and at least one codec and camera for each of the displays. A telepresence compliant endpoint as described above will from now on be described as a telepresence system. As readily understandable for a person skilled in the art, a telepresence compliant endpoint might be replaced by a telepresence compliant Multipoint Control Unit (MCU), also known as a telepresence server.

A conference between telepresence systems is arranged by setting up site-to-site connections between respective codecs of the endpoints, even if the conference is a multi-site conference. Which codecs in which endpoints to connect to each other, are selected to optimize the feeling of presence for the participants in the conference. As an example, when the codec associated with the right camera and the right screen of site A in a conference is directly connected in a video conference link to the codec associated with the left camera and the left screen of site B, the participants of site A will experience that the participants of site B turn to them when the participants of site B look at the left screen where the participants of site A are displayed. Examples on how this will effect the selection of connections paths in different conference constellations are discussed further below. According to one aspect of the present invention, when a conference is established, the one telepresence system initiating the conference is selected as the master site. The master site controls the other telepresence systems in the conference keeping track of the status, controlling the establishment of the conference and rearranging the communication links when a telepresence system joins or leaves during the conference.

When a conference is to be established, e.g. on a request from the control unit or from a conference manager, a first call is established from a first master device of a first telepresence system to a second master device of a second telepresence system. Within the first call the master device and the second master device exchange information, such as the addresses of the other devices, needed to setting up the site-to-site connections between respective codecs of the telepresence systems.

The information are preferably incorporated in an open field in the message flow specifying the control protocol of establishing video conference call, in particular the vendor specific messages of the ITU H.245.

An exemplary message flow according to the method of the present invention is shown in Fig. 1, wherein a first telepresence system (TPl) is initiating a telepresence call to a second telepresence system (TP2). The first step of the telepresence call (not shown) is to establish a first H.323 call between the master device of TPl and TP2. When the first H.323 call is established, a first H.245 channel is also established, either directly or proxied via at least one gatekeeper, between the respective master device of TPl and TP2. In step 301 the master devices performs a Master/Slave determination, and in step 302 the master devices exchanges the capability sets of the master devices. After steps 301 and 302 are completed, it is possible to transmit vendor specific messages using the H.245 protocol.

In step 303, the master device of TPl starts transmitting a first initiation message, or hello message, identifying itself as a telepresence system, to the master device of TP2. The master device of TP2, after receiving the first initiation message from TPl, transmits a second initiation message also identifying itself as a telepresence system to TP 1. In addition to the value indicating a telepresence system the initiation message also comprise a determination number and the version of the protocol being used. The determination number will be described further below with reference to step 304, master/slave negotiation. In addition, the initiation message can preferably also include the number of screens in the telepresence system.

The master device initiating the telepresence call, i.e. TPl, selects a determination number in a first range, the device receiving the initiation message then select a determination number in a second range, wherein the second range comprise numbers of lower values than the first range. Preferably the first range is <32768 — 66635>, and the second range is <0 - 32767>. The telepresence system with the highest value is determined to be the master, and accordingly the telepresence system with the lowest value is the slave. In step 304, TPl transmits a first acknowledgement (ACK) message to TP2 confirming that TPl is the master. After receiving the first ACK message, TP2 transmits a negative ACK message to TPl confirming that TP2 is not the master.

In step 305, the master device of TPl transmits a first Sitelnfo message to the master device of TP2. After receiving the first Sitelnfo message, the master device of TP2 transmits a second Sitelnfo message to the master device of TPl . The Sitelnfo message comprises the name of the system, the type of the system and the number of any additional devices, or telepresence sockets, which is to be connected.

In step 306, the master device of TPl transmits a first number of TPSocket- messages to the master device of TP2, the number of TPSocket-messages corresponding to the number of telepresence sockets in TPl. After receiving the first number of TPSocket-messages, the master device of TP2 transmits a second number of TPSocket-messages to the master device of TPl, the number of TPSocket-messages corresponding to the number of telepresence sockets in TP2. The TPSocket-message comprises the position of a socket relative to the position of the corresponding master device, and the number to be called to reach the socket. In step 307, the telepresence sockets (TPSocket) of TPl establishes communication sessions with the TPSockets of TP2, by using normal H.323 procedures.

In another exemplary embodiment according to the present invention, the message flow can also include messages to send or request modes of operations of the telepresence conference. Some exemplary modes of operation are described below. In yet another exemplary embodiment according to the present invention, the message flow can also include messages to invite more sites into the conference.

According to an exemplary embodiment of the present invention, the messages are sent using the following format: prefix, command: keyl='valuel ' ... keyn='valuen' prefix, e.g. TTPP, is preferably used to identify the protocol. However, as readily understandable by a person skilled in the art, prefix can be omitted if not required in a specific application without leaving the scope of the present invention. Exemplary commands, or message identifiers, and their associated keys and values are described below. The initiation message described above in relation with step 303 is identified with Experia. The Experia message indentifϊes the transmitting system as a telepresence system. The Experia message also contains at least two key values; determinationNumber and Version. The determinationNumber value is a number within a first or a second range of numbers, wherein the second range comprise numbers of lower values than the first range. The telepresence system initiating a call will assign itself as master by choosing a number from the first range.

Accordingly, the receiving telepresence system chooses a number from the second range when transmitting the Experia message. Preferably the first range is <32768 - 66635>, and the second range is <0 - 32767>. The Version value is the version of the protocol. Preferably the Experia message also contains a Screen-value that identifies the number of screens of the system transmitting the message. An exemplary Experia message is shown below:

TTPP, Experia: determinationNumber=66635 Version=l Screens=3

The acknowledgement message described in relation with step 304 is identified with ExperiaAck. The ExperiaAck message has one key value; iAmMaster. The iAmMaster value is a Boolean telling the status of the master/slave negotiation as described above. A system with a determination number in the first range transmits a "True" value, while a system with a determination number in the second range transmits a "False" value. An exemplary ExperiaAck message is shown below:

TTPP, ExperiaAck: iAmMaster=true The Sitelnfo message described in relation with step 305 is identified with Sitelnfo. The Sitelnfo message has three key values; Name, Type and TPSockets. The Name value is the name of the system transmitting the message, and can be any chosen name. The Type value identifies what type of system is transmitting the message. The Type values are predefined values for known telepresence compatible systems, e.g. TANDBERGExperiaT3 identifies a TANDBERG T3 telepresence endpoint and CodianTPS identifies a TANDBERG Codian Telepresence server. The TPSockets value gives the number of any additional devices, or telepresence sockets, to be connected for each telepresence system. An exemplary Sitelnfo message for a three screen TANDBERG T3 telepresence system is shown below: TTPP, Sitelnfo: Type=TANDBERGExperiaT3 name=A, TPSockets=2

The TP-Socket messages described above in relation with step 306 is identified with TPSocket. The TPSocket message has at least three key values; xPos, yPos and number. The number value is the number the master device will be calling to reach the telepresence socket. Preferably, the TPSocket message also includes a protocol value identifying the relevant communication session protocol, default value being H.323. The xPos and yPos values gives the respective horizontal and vertical position of the telepresence socket in relation to the centre socket, 0 is the centre position. Exemplary TPSocket messages for a three screen telepresence system where all screens are in the same horizontal plane, is shown below:

TTPP, TPSocket: xPos=-l yPos=0 number= ALeft@tandberg.com TTPP, TPSocket: xPos=l yPos=0 number= ARight@tandberg.com In another aspect according to the present invention, the telepresence systems select a mode of operation from a predefined set of camera presets, by a transmitting a Conference message. The Conference message is preferably transmitted as a part of the site info exchange in step 305. The Conference message is transmitted from the site that owns the conference. The conference owner can be different from the master of the conference. The conference owner is the site that is the master of a multisite conference, i.e. a telepresence server will own the conference if multiple telepresence systems is connected to the telepresence server. Similarly, a TANDBERG T3 telepresence system will own the conference if multiple endpoints is connected to the T3 system. When the call is set up, the other endpoints or telepresence systems can preferably also request the conference owner to change the camera preset by transmitting a CameraPresetRequest message to the conference owner. If the conference owner decides to follow the request, the conference owner replays by transmitting a Conference message. The Conference message and the CameraPresetRequest message both have one key value, namely cameraPreset. The cameraPreset values are predefined camera preset values, e.g. p2p2p (Point to point, two person zoom), ot4p (One table, four person zoom), ot2p (One table, two person zoom), p2p3p (Point to point, three person zoom) and otόp (One table, six person zoom). An exemplary Conference message is shown below:

TTPP, Conference: cameraPreset=ot4p In yet another aspect according to the present invention, the master device invites additional sites during an ongoing telepresence conference, by transmitting an Invite message. Invite message have two key values; number and protocol. The number value is the number the master device will be calling to reach the invited site to be invited. The protocol value is identifying the relevant communication session protocol, e.g. SIP or H.323, used for this call by the master device. The receiving master device will respond by transmitting an InviteResult message, the Invitemessage having one key value; status. If the invite is successful the status value is OK, on the other hand if the invite is not successful the status value is Error followed by an corresponding error message. An exemplary invite message exchange is show below:

TTPP, Invite: number=newsite@domain.com protocol=sip TTPP, InviteResult: status=Error errorMessage As described above, the first step of the method of the present invention is an initiation step, or a handshake state, 401. During the handshake state the master device of a first telepresence system transmits a first message, identifying the first telepresence system as a telepresence system, to a master device of a second telepresence system. On receiving the first message, the master device of the second telepresence system transmits a second message, the second message identifying the second telepresence system as a telepresence system. The first and second messages respectively also include additional information about the respective telepresence system, such as a determination number, the version of a protocol and the number of screens of the system. The master device initiating the telepresence call selects a determination number in a first range, the device receiving the initiation message then select a determination number in a second range, wherein the second range comprise numbers of lower values than the first range. Preferably the first range is <32768 - 66635>, and the second range is <0 - 32767>. In the Master/Slave Negotiation state, 402, the master devices of the telepresence systems determines which of the two telepresence systems that should be the master of the call between the two telepresence systems. The telepresence system with the highest value is determined to be the master, and accordingly the telepresence system with the lowest value is the slave. The first master device transmits a first acknowledgement message to the second master device confirming that the first telepresence system is the master of the call. After receiving the first acknowledgement message, the second master device transmits a negative acknowledgement message to the first master device confirming that the second telepresence system is not the master. In the Sitelnfo Exchange state, 403, each of the telepresence systems transmits a message to the other telepresence system comprising the name of the telepresence system, the type of telepresence system, and the number of additional devices to connect, or TPSockets, to setup the telepresence call. Preferably, during the Sitelnfo Exchange state the telepresence systems select a mode of operation from a predefined set of camera presets, by a transmitting a Conference message as described above.

In the TPSocket Exchange state, 404, each of the telepresence systems transmits a TPSocket message for each of the TPSockets available in the telepresence system. The TPSocket message comprises at least the horizontal and vertical position of the TPSocket in relation to the master device of the system, and the number to be called to connect to the TPSocket. Each telepresence systems checks that a TPSocket message is received for the number of TPSockets received in the Sitelnfo message, then the method proceeds to Initialise Call state. In the Initialise Call state, 405, the master telepresence system dials the TPSockets as specified in the TPSocket messages. The TPSockets receiving a call answer an expected call, i.e. a call from a socket identified in one of the TPSocket messages, and reject calls from any other callers. Then, a user of the system can preferably request a new camera preset by transmitting a camera preset request message as disclosed above.

After the Initialise Call state, 405, the method proceeds to the Monitor Call state, 406. The master device of the master telepresence system in the call monitors the different tpSockets in the call and keeps track of the status of the conference and the different connections, then being able to re-establish or rearrange the connections if necessary. When the call controlling the telepresence call, i.e. the initial call between the master devices of the two telepresence systems, is dropped, all other TPSockets calls will be dropped and the telepresence conference is ended, 407. On the other hand, when one of the other TPSocket calls is dropped, the master device re-establishes the call.