Sound enhancement in closed spaces

FIELD OF THE INVENTION

The invention relates to a system and method to enhance sound intelligibility. In a particular embodiment the invention relates to sound enhancement in a small closed space. An embodiment relates to a system and method to enhance sound intelligibility in a car.

BACKGROUND OF THE INVENTION

From US patent No. 4,965,833 a voice enhancement system is known that can be used to help a passenger in the back seat of a car to hear speech from a passenger in the front seat more clearly. A microphone in the front of the car picks up the speech and a loudspeaker in the back of the car outputs the speech. A plurality of microphones and/or loudspeakers may be used.

US 4,965,833 discusses signal processing operations that have to be applied to the microphone signal before outputting it at the loudspeaker in the back of the car. The described signal processing operations include frequency shifting, to prevent "howl around", notch filtering and low and high pass filtering.

SUMMARY OF THE INVENTION

Among others, it is an object to improve sound enhancement by a system and method for picking up sound in a closed space, processing the sound and outputting back the processed sound into that closed space.

A sound processing system according to claim 1 is provided. Herein feedback via the microphone is suppressed by subtracting a predicted feedback component from an input signal. A postprocessor effectively multiplies the result of subtraction by frequency factors. A controller selects the factors in proportion to a ratio of an estimated signal strength reduced and an estimated reverberation component strength at respective frequencies. The effective multiplication may be realized for example by means of a temporal response function of the post processor, wherein the Fourier transform coefficients of the response function are substantially in proportion to the factors.

It is noted that multiplication of a signal with factors selected to reduce reverberation effects is known per se from WO 2006011104. In the present case use of factors is combined with use of subtraction of a feedback component and using factors determined from a result of that subtraction, based on the discovery that this subtraction will result in reverberation like effects even in small spaces, such as a car cabin, where reverberation normally is not a perceptible effect.

In an embodiment the controller controls a gain applied to the signal dependent on an estimated background noise level. As used herein "gain" may correspond to attenuation or increasing the strength of the signal. Thus the reinforcement realized by the output signal can be made to increase in the presence of increasing background noise. This improves intelligibility of speech.

In an embodiment the controller computes a reverberation decay factor dependent on the adjusted gain, and to estimate the estimated reverberation component strength dependent on the computed reverberation decay factor. This is based on the insight that this gain is significantly determinative for artificial reverberation in confined spaces, such as in a car. The reverberation decay factor is used to model decaying contribution of previous reverberation strength on the following reverberation signal strength in the estimation of the reverberation.

In an embodiment the system has an external input for an external signal, such as a music signal or a telephone signal. The external signal is added to the output signal. A predicted microphone feedback component due to the external signal is subtracted from the input signal. The controller adapts the reverberation estimate to the external signal.

In an embodiment the system comprises a microphone and a loudspeaker mounted in a car.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects and advantageous aspects will become apparent from a description of exemplary embodiments, using the following figures.

Figure 1 shows a sound processing system Figure 2 schematically shows an application to a car

Figure 3 shows a further sound processing system

DETAILED DESCRIPTION OF THE EMBODIMENTS

Figure 1 shows a sound processing system. The system comprises a series connection of an input microphone 10, a processing unit 12 and a loudspeaker 14. The processing unit comprises series connection of a pre-filter 120, a subtractor 122, a post- processor 124, a first amplifier 126 and a second amplifier 127 coupled between microphone 10 and loudspeaker 14. Furthermore, the processing unit comprises a feedback component predictor 128 and a controller 129. The feedback component predictor 128 has an input coupled to an output of first amplifier 126 and an output coupled to a subtraction input of subtractor 122. Controller 129 has inputs coupled to receive signals Z, Y, R from the outputs of pre-filter 120, feedback component predictor 128 and subtractor 122 respectively. Controller 129 has outputs coupled to control inputs of post-processor 124, feedback component predictor 128 and first amplifier 126 respectively. Pre-filter 120 is a high pass filter with a pass-band starting from 200Hz for example, to remove strong noise components. It should be noted that amplifiers 126, 127 may in fact attenuate the signal. As used herein "amplifier" connotes any element that may change a signal strength by a factor, irrespective of whether this factor is larger or smaller than one.

Figure 2 schematically shows an application of the sound processing system to a car cabin 20, with front seats 22 and back seats 24, microphone 10 being located in front of front seats 22 and loudspeakers 14 being located next to back seats 24. As may be appreciated from figure 2, although one loudspeaker 14 is shown in figure 1, a plurality of loudspeakers may be used.

In an embodiment processing unit 12 is implemented partly or wholly by means of a programmable signal processor programmed to make the processor perform the functions of part or all of processing unit 12. In this case the system also comprises an input ADC (Analog to Digital Converter) and an output DAC (Digital to Analog Converter) coupled to microphone 10 and loudspeaker 14 respectively. In this embodiment pre-filter 120, subtractor 122, post-processor 124, first amplifier 126 and second amplifier 127 correspond to the signal processor in correspondence with respective program parts to perform the corresponding functions. In the application to a car a purpose of the system is to help passengers in back seats 24 understand speech from a passenger (including the driver) in front seats 22. For this purpose microphone 10 picks up sound at the front and loudspeakers 14 output sound in the back. Processing unit 12 processes the sound signal in between.

In an embodiment processing unit 12 automatically adapts the strength of the sound signal from loudspeakers to the background noise level. Thus, more sound reinforcement is provided when the noise level is high (e.g. under driving conditions and/or in the presence of busy traffic) and less sound reinforcement is provided when the noise level is low. For this purpose controller 129 receives the input signal Z output by pre-fϊlter 120, identifies speech free time intervals and estimates the background noise signal strength In from the signal Z in these time intervals. Subsequently controller 129 controls an adjustment of the gain of first amplifier 126 dependent on the estimated background noise signal strength In. This may be done repeatedly during use, so that the strength of reinforcement is adapted dynamically during use. The overall strength of reinforcement may be set by manually adjusting the gain of second amplifier 127. Thus the level set with this second amplifier is proportionally varied by automatic gain adjustment of first amplifier 126.

One problem of the reinforcement system is the existence of a feedback loop that arises because sound from loudspeakers 14 may reach microphone 10. A known method to reduce feedback problems in general is to subtract a predicted feedback component from an input signal. This method is applied in processing unit 12. Processing unit 12 uses feedback component predictor 128 to predict the signal component Y of the input signal from microphone 10 that is due to sound received from loudspeakers 14 (after filtering by pre-filter 120). The predicted signal component Y is formed from the output signal X that is fed towards loudspeaker 14, taken from the output of first amplifier 126. Subtractor 122 subtracts this predicted signal component Y from the output signal Z of pre-filter 120.

Feedback component predictor 128 has to account for variable sound transmission conditions in the car, which may be affected by the number and position of the passengers. For this purpose controller 129 adapts the transfer function of feedback component predictor 128, i.e. the way in which it uses the output signal X to form the predicted signal component Y, to observed conditions in the car. Methods for adapting the transfer function of a predictor to actual conditions are known per se and will therefore not be described here.

It may be noted that the transfer function of feedback component predictor 128 is not affected by the gain of first amplifier 126. This is because the feedback component predictor 128 has to simulate the signal transfer in the path from the output of the first amplifier 126 and the output of pre-filter 120, which does not contain first amplifier 126. Therefore, in principle, the gain of this first amplifier 126 can be automatically adapted to

background noise conditions without modifying the transfer function of feedback component predictor 128.

A known method to reduce feedback problems is to use a frequency shifter in the path between the microphone and the loudspeaker. In a further embodiment such a frequency shifter may be included in post-processing circuit 124.

In practice it has been discovered that the use of feedback suppression by means of feedback component predictor 128 and subtractor 122 may lead to an artificial perceptible reverberation effect. Although perceptible reverberation effects are of course known for large confined halls, one would not normally expect perceptible reverberation in a small confined space such as a car. Nevertheless it has been found that perceptible reverberation may occur especially when the gain (e.g. of second amplifier 127) is set to a relatively high value and/or when persons in the car move so quickly that the transfer function of feedback component predictor 128 cannot be adapted to track the effects of these movements. This has lead to the insight that the perceptible reverberant component is mainly introduced by errors in the predicted feedback signal, which lead to errors in the output signal of subtractor 122.

Post-processor 124 is configured to filter the signal by Wiener-type filtering, i.e. by reducing the amplitude of frequency components of the signal at frequencies where proportionally more of the signal strength is due to interference, relative to the amplitude of frequency components of the signal at frequencies where proportionally less of the signal strength is due to interference.

Use of Wiener type filtering to reduce reverberation effects is known per se from WO 2006011104. This publication describes a method of determining reverberation model parameters from a microphone signal and selection of frequency factors dependent on the estimated model parameters.

Controller 129 selects the filter function dynamically during use for use in Wiener type filtering. In an embodiment a frequency dependent filter factor is selected that is a ratio between a running average of the strength of the desired signal As (due to speech from a person in front seat 22) as a function of frequency and the average strength of the total signal Asn, including the desired signal and interference as a function of frequency.

F= As(f)/Asn(f)

Herein a reverberant signal component is held to contribute to the interference. Controller 129 estimates the average strength of the desired signal As by subtracting an estimated strength of the reverberation component Ir(f) as a function of frequency from Asn(f).

As(f)=Asn(f)-Ir(f)

In a further embodiment, the effect of phase errors in the predicted feedback signal may also be accounted for by replacing Asn(f) by the difference between Az(f), the strength of the input signal Z as a function of frequency and Ay(f), the strength of the predicted feedback signal Y as a function of frequency:

As(f)=Az(f)-Ay(f)-Ir(f)

Further interference components may be subtracted as well to realize a more accurate estimate of the signal strength As(f). These further interference components may include the estimated strength In of the background noise In(f) in the input signal Z as a function of frequency.

In an embodiment the strength of the reverberation component Ir(m,f) in a time segment labeled by "m" and for a frequency "f" is computed from

Ir (m,f) = (a-b)*Ir(m-l,f) +b* Az(m-l,f)

Herein Az(n-l,f) is the strength of the received signal Z during time segment "m-1" for the frequency f.

Controller 129 computes the factors "a" and "b". "a" is a damping factor that defines the reverberation time scale and "b" is factor that indicates the relative strength of direct signal components and reverberant signal components.

Based on the discovery that the reverberation effect in a small confined space like a car occurs mainly due to the feedback suppression and large gains it is possible to compute the factors "a" and "b" from the gain of the part of the processing unit between the input of post-processor 124 and the output of first amplifier 126. In an embodiment the factor "a" is set to an exponent with a power proportional to this gain, averaged over frequency

a = exp ( -fac*averaged gain)

As may be noted, the averaged gain depends on dynamic conditions, such as the background noise level that is used to set the gain of first amplifier 126. The factors "b" and "fac" may be set to a predetermined value, selected by minimizing the audible effect at design time.

Thus, by using the input signal Z from pre-filter 120 the output signal R from subtractor 122 and optionally the predicted feedback signal Y, as well as a dynamically modelled reverberation effect due to the gain in part of the processing unit 12, controller 129 computes a strength Asn of the signal with interference due to reverberation and a strength As from which the modelled effect of reverberation has been removed. From these strengths controller computes a frequency dependent factor that is to be applied by the postprocessor 124. In an embodiment controller 129 clips the frequency dependent factor when it exceeds a lower or higher limit (a clipping level CO). Application of the resulting factor may be realized by filtering the input signal of post-processor 124 using a filter (e.g. a FIR filter) with a selected transfer function. Controller 129 computes the transfer function from the frequency dependent factor. Methods of selecting transfer functions on the basis of a given frequency dependent factor are known per se and will not be described in detail. As used herein postprocessor 124 is said to effectively multiply its input signal with the selected factors in this case. Other ways of obtaining effective multiplication include Fourier transforming the signal, multiplying the resulting Fourier coefficients with the factors and inversely Fourier transforming the products, or band filtering the input signal of the post-processor in a plurality of frequency bands, multiplying respective band filtered signals with the respective factors and combining the products. However, the use of a computed FIR response is preferred in order to limit the filter delay.

The various strengths Asn, Az, Ay may be computed by taking the absolute value of the Fourier transform of the corresponding time dependent signals. Thus for example, controller 129 may compute the strength Asn as the absolute value abs(F[R]) of the Fourier transforms F[R] of the signal R during a time segment, as a function of frequency f. Similarly the strengths Az and Ay may be computed as the absolute values abs(F[Z]) and abs(F[Y]) of the Fourier transforms of the input signal Z and the predicted signal Y as a function of frequency respectively. However, it should be emphasized that this specific way of setting the transfer function of post-processor has been described only by way of example. Many variations are possible. For example, instead of computing the strength from ratio of

absolute values Asn, As as an absolute value of the Fourier transform, squares of the absolute value, others powers, a logarithms or other monotonically increasing functions may be used. What matters is that the factor is relatively smaller for frequencies where the interference is stronger relative to the desired signal. Furthermore, a different model may be used to estimate the strength of the interfering reverberation component. Reverberation models are known per se. For example, the reverberation strength Ir (m,f) may be estimated by weighing strength estimates Az(m- n,f) with weight factors bn for more than one time segment (n=l, 2 etc.), and/or frequency dependent weight factors a and b (bn) may be used. In this case preferably the weights are adapted dependent on the gain to scale the reverberation time scale in proportion to the gain. The model in terms of two factors is only one example of a reverberation model. What matters is that the effect of the adjustable gain in the feedback suppression path on reverberation damping is accounted for in the model, if the gain is adjustable. This is based on the insight that the artificial reverberation effect is due to feedback suppression in combination with large gain. As described this may be done by directly using the gain to adapt the reverberation model. In a further embodiment, however, parameters of the reverberation model, such as the weight factors may be estimated from the signal.

Figure 3 shows an embodiment of a sound processing system wherein an external sound signal input 30 and output 32 are provided for. Examples of external sound signal inputs include a car radio receiver, an audio player (e.g. MP3 player, CD player, tape player), a car telephone etc). Examples of external sound signal outputs include a car telephone.

In addition to the components shown in figure 1 , the sound processing system comprises an adder 34, a further feedback component predictor 36 and a further subtractor 38. The output of second amplifier 127 and loudspeaker 14 is coupled to a first input of adder 34. A second input of adder 34 is coupled to external input 30. The output of adder 34 is coupled to the loudspeaker 14 near back seats 24. In operation adder 34 adds an external output signal (i.e. not due to sound originating in the car) to the reinforcement signal obtained by processing the signal from microphone 10. In addition the external output signal may be output at further loudspeakers in the front of the car.

This results in additional signal component in the signal from microphone 10 due to the external signal. Further feedback component predictor 36 and further subtractor 38 are used to remove this component from the signal from microphone 10. Further subtractor 38 is coupled between pre-filter 120 and subtractor 122. Further feedback component

predictor 36 is coupled between external input 30 and a subtraction input of further subtractor 38. In operation further feedback component predictor 36 predicts the component in the microphone signal due to the external signal (as it appears behind pre-filter 120) and further subtractor 38 subtracts this prediction from the microphone signal. Controller 129 has an output coupled to a control input of further feedback component predictor 36 to control its transfer function.

In this embodiment controller 129 adapts the frequency dependent factor that is applied by postprocessor 124. In this embodiment controller 129 estimates the strength of the reverberant component from the strength of the signal Z2 at the output of further subtractor 38 (e.g. the absolute value of the Fourier transform of Z2), obtained after subtracting the estimated contribution of the external signal:

Ir (m,f) = (a-b)*Ir(m-l,f) +b* Az2(m-l,f)

The external signal does not initially contribute to reverberation. By avoiding that it directly contributes to the estimated reverberation a better estimate of reverberation strength is obtained. Alternatively, Az2 may be replaced here by the difference between the original input signal strength Az(f) and the strength of the predicted in the microphone signal due to the external signal feedback signal, obtained for example by taking the absolute value of the Fourier transform of the output signal of further feedback component predictor 36.

Furthermore, in an embodiment controller 129 estimates the strength As(f) of the desired signal using the signal Z2.

As(f)=Az2(f)-Ay(f)-Ir(f)

Here further noise estimates, such as an estimate of the strength of external noise In(f) may be subtracted.

When the system also has an external output 32 a further post-processor 39 may be provided coupled between subtractor 122 and the external output. In operation further post-processor 39 applies a further frequency dependent factor to the signal from subtractor 122. This may be done for example by Fourier transforming this signal, multiplying the frequency components of the Fourier transform with the factor and inversely Fourier transforming. Alternatively, a corresponding time domain filter may be used.

Controller 129 computes the required further frequency dependent factor dynamically from the estimated strengths of various signals on the basis of a ratio of the estimated desired signal strength Bs and the combined strength of signal and noise Asn. In an embodiment the desired signal strength Bs may be estimated according to

Bs(f)=Az(f)-Ay(f)-Ay2(f)-Ir(f)

Herein Ay2 is the amplitude of the Fourier transform of the further predicted feedback signal (produced by further feedback component predictor 36, in response to the external signal). As before, other estimated interfering signal strengths, such as background noise strength may be subtracted. In an embodiment the resulting further frequency dependent factor Bs(f)/Asn(f) is clipped, the lower clipping level Clow being selected equal to a factor times the clipping level CO of the frequency dependent factor As(f)/Asn(f):

Clow= CO * (Ay(f)+Ir(f))/(Ay(f)+I(f)+Ay2(f))

That is, the clipping level Clow is selected to be smaller than the clipping level CO by a factor that becomes smaller as the strength of the external input signal is bigger. The clipping level Clow is set to its highest level if feedback due to the external input signal is small compared to the interference. This signal level dependent clipping level has the advantage that external parties (calling by phone) will be prevented from hearing echoes of his or her own voice which were played via the car-loudspeakers.

When the system of figure 3 is implemented as a time discrete processing system, and its sample rate is not equal to that of the external signal, the external signal may first be downsampled to the sample rate at which the microphone signal is processed by the system. After the signal processing, the output signal may be upsampled, added to the external signal and played by the loudspeaker(s). Alternatively other methods of equalizing the sample rates may be used. Upward rate conversion of both signals and interpolation of one of the signals may be used to equalize the sample rates. While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

For example, it should be appreciated that, although the figures show various components as individual units, in practice several components may be implemented in

combination in a single component, for example by means of a programmable signal processor device with a program that causes that device to perform the functions of the components. Thus, the nodes between components shown in the figures need not correspond to actual circuit nodes. They may correspond to intermediate signals produced during processing, or they may merely represent a conceptual intermediate result used to explain the effect of a more complex signal processing operation. Preferably a time discrete digital signal processor is used. Alternatively, one or more analog signal processing circuits may be used for all or part of the components.

Furthermore, it should be appreciated that the components that are shown may be moved to different places in the circuit, without affecting the function, or that other components may be added without affecting the function. For example, first amplifier 126 could be moved in front of post-processor 124 without affecting the function, or it could be combined with second amplifier 127, provided that a corresponding amplifier is included in the feedback prediction path before or after feedback component predictor 128. Further components may be added in the signal paths, for example to perform signal scaling etc.

Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.