To enable high-precision noise removal by providing a mechanism which finds information on ratios (SN ratio) of input power by frames and power of an estimated noise spectrum and adjusts the time constant of noise spectrum estimation according to this SN ratio information.

The speech recognition system consists of a sound analysis part 101, a CSS processing part 102, and a pattern recognition part 103. The sound analysis part 101 and CSS process part 102 correspond to the background noise removing device. The CSS process part 102 consists of an SN ratio calculation part 110, an SN ratio/time constant conversion part 111, a noise spectrum update part 112, and a noise spectrum removal part 113. Then the mechanism finds SN ratios are found at constant intervals of frames and adjusts the time constant of a continuous noise spectrum estimating process by an SCC method according to the found SN ratios or SN ratios smoothed in the temporal direction. Therefore, noise spectrum estimation from a frame having a large SN ratio or a frame which may be a speech with high possibility is equivalently canceled.