To provide a video conference system by which progress of a more natural and smoother video conference can be realized.

In this video conference system, video images of conference participants of other terminals are side by side displayed on a long sideways screen of a video output device 17, a plurality of video cameras 11a-11z are respectively placed above the video images of the respective participants displayed on the long sideways screen. Non-verbal interpretation processing is applied to the motion of each conference participant on the basis of a plurality of video information sets obtained by photographing a talker in different directions by using the video cameras 11a-11z so as to specify a person to which the talker speaks thereby attaining control such as automatic transfer of a right to speak from the talker. Thus, the progress of the video conference can be realized more naturally and smoothly.