To carry out the desired processing in an input mode that can eliminate the user burden to an information device in a multi-modal input integration system which contains a user interface to deal with the information device.

An input part 1 recognizes the contents which are inputted via an input means having plural modes and outputs the result of recognition. A planner part 2 receives the words of recognition result from the part 1 and converts the inputted words into a generalized command script. Then the command script sent from the part 2 is interpreted and executed at an agent part 4. Thus, the inputs of various types are integrated into a significant meaning and commanded to an information device.