BACKGROUND

The present invention describes a system and method for locating and tracing multiple sound sources that can be stationary or moving in space and visualizing the resultant sound pressure fields in 3D space in real time.

Presently, there are no known systems and tools that enable one to visualize the sound pressure field produced by arbitrary (stationary/moving) sound sources in 3D space in real time. There are systems and tools available, however, to identify a sound source using the beamforming technology, and systems and tools to visualize a 3D sound field via nearfield acoustical holography (NAH) technology separately.

Typically, systems and tools based on beamforming technology require the use of a camera and an array of 30 - 60 microphones to measure the sound pressure, and then overlay the high sound pressure spots on the image of a test object captured by the camera to indicate the locations from which sounds are emitted.

The underlying principle behind beamforming is a delay and sum technique. By changing the time delays, namely, the phases of sound signals in the individual microphone channels and bringing all of them to be in phase so as to constructively reinforce each other, one can form a peak sound pressure, i.e., a beam that points in the direction of sound wave propagation in the space. This delay and sum process is equivalent to rotating the microphone array until it is in line with the incident sound wave. By using a camera and taking a picture of a test object that creates sound, one can overlay the high sound pressure on the image of the test object to indicate where sound is emitted. Note that since beamforming relies on a plane wave assumption, it can only reveal the direction of wave propagation but not the depth information, i.e., the distance of a sound source. The use of a camera compensates this shortcoming as a camera image is 2D, so the depth information is automatically suppressed.

In reality most source sources are 3D with complex geometry. Therefore, the acoustic information offered by beamforming is usually quite limited. Moreover, the sound pressure graph provided by beamforming is on a 2D measurement surface, but not on a 3D source surface. In particular, beamforming is effective for impulsive and broadband sound signals that contain high frequency components. In fact, the higher the frequency content and the broader the frequency bands are, the higher the spatial resolution of beamforming is. This is because the spatial resolution of beamforming is no better than one wavelength of a sound wave of interest, so it cannot discern two sources separated by a distance less than one wavelength. Hence beamforming is not suitable for low frequency cases. Also, the delay and sum technique is not applicable for locating sinusoidal, narrowband or tonal sound source. Finally, beamforming can not be used to monitor multiple sound sources in motion simultaneously.

NAH enables one to obtain 3D images of a sound field and very detailed and accurate information of the acoustic characteristics of a complex structure, including the source locations. However, NAH requires taking measurements of the acoustic pressures via an array of microphones positioned at a very close distance around the entire source. In particular, if a 3D image of a sound field is desired, measurements should include not only the source surface, but also the reflecting surfaces including floor, surrounding walls and ceiling, which is unfeasible in engineering applications. Finally, the state-of-the-art NAH does not allow for visualization of a 3D image of a sound field in real time. All visualization must be done in post processing.

SUMMARY

Described herein is a new way of tracking and tracing multiple noise sources that can be stationary or moving in space, and displaying the resultant sound field in 3D space in real time using four microphones only. This new technology includes two facets, the first being locating and monitoring multiple noise sources that can be either stationary or moving in space; the second being visualizing the resultant sound acoustic field produced by these sources in 3D space in real time.

In contrast to beamforming, this system does not need a priori knowledge of the location of a test object and uses four microphones only, so it is very simple to set up and easy to use, and hardware costs are significantly reduced. The underlying principle of this invention is a hybrid acoustic modeling and triangulation techniques to track and trace the positions of multiple noise sources in 3D space simultaneously in real time. In particular, this invention is applicable for broadband and narrowband sound sources over the entire audible frequency range from 20 ~ 20,000Hz. Another salient feature of this new technology is its capability to locate multiple objects that emit tonal sounds at low frequencies.

The disclosed system enables one to monitor multiple sources anywhere including behind microphones, unlike beamforming that requires that the microphone array be aimed at a test object and the distance of the test object be specified. However, both beamforming and this method require that the line of sight of each microphone to a test object be clear at all time.

Table 1 shows the comparison of features and functions of this method and beamforming. Clearly, the former can provide much more than the latter does, yet it uses much fewer microphones, runs much faster and costs much less than the former does.

Table 1. Comparison of features and functions of the present invention and state-of-the- art beamforming technology

After locating sound sources, the sound pressures generated by these sources are calculated and the resultant sound pressure field in 3D space including the source surfaces is visualized. This 3D soundscaping produces direct and easy to understand pictures of sound pressure distribution in 3D space and how they change with time. In contrast to NAH, this new method uses the propagating component of the sound pressure in the visualization process. This approximation is acceptable for the sound pressure in far field that is dominated by the propagating component. The near- field effects decay exponentially as the sound wave travels to the far field. This approximation greatly reduces complexities of numerical computation, increases the post processing and makes real-time 3D soundscaping possible. Since measurements are taken at a remote location, not at very close range to a target source as it does in NAH, the setup and operation of this new technology become very easy, simple and convenient.

The present invention requires four microphones that can be mounted at any position so long as they are not on the same plane. For example, one can place three microphones on one plane and the fourth microphone in another plane perpendicular to the first plane. A processor receives signals from these four microphones based on sound pressure signals received from a single or multiple sound sources in unknown locations, and the processor determines the locations of the sound sources based on the signals from the microphones.

The features of this new invention can be best understood from the following specifications and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 illustrates a system for carrying out a 3D soundscaping according to one embodiment of the present invention, positioned in front of noise sources.

Figure 2 illustrates sound pressure fields as determined by the system of Figure 1.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

A system 10 for carrying out a 3D soundscaping is shown in Figure 1. The system includes a computer 12, including a processor 14, digital signal processor (DSP) 15, memory 16 and mass storage 18. Computer software for performing the functions described herein and any implied or necessary attendant functions are in memory 16

(such as RAM, ROM, hard drive, or any magnetic, optical or electronic storage or other computer readable medium) and executed by processor 14. Results may be displayed on a display 20, output to another system or process, or printed by a printer.

The computer 12 receives noise signals from microphones 1 - 4 arranged on a stand 22 (connections are not shown for clarity). Microphones 2 - 4 are arranged in a common y, z plane. Microphone 1 is positioned along the jc-axis in front of the y, z plane. The sound sources, 51, S2, in this example an engine, may be positioned anywhere even inside the y, z plane. Note that the description of the method below is for determining the location of a sound source S, which can be performed separately for each of the noise sources Sl, S2, ..., SN, and for visualizing the resultant sound field produced by all these sources. Consider the case in which the microphones 1 - 4 are mounted at (x _\ , y _\ , Z ₁ ), (x ₂ , yi, Zi), Ot ₃ , J ₃ , z ₃ ), and Ot ₄ , _y ₄ , ζ ₄ ), respectively, and the unknown sound source is at Ot, y, z). Accordingly, the relative distances between the unknown source and the individual microphones can be expressed as

Jb-X ₂ Y +(y-y ₂ Y + (z-z ₂ Y =J(χ-χ ₁ Y ₊ (y- yj +iz-zj ₊ (cAt ₁₂ ), (1)

J(χ-χ ₃ Y ₊ (y- y ₃ Y ₊ (z-z ₃ Y =J(χ-χ _ι Y ₊ (y- y _ι Y ₊ (z-zJ ₊ (cAt _l3 ), (2)

J(χ-χ ₄ Y ₊ (y-yJ ₊ (z- Z ₄ Y = J(x-xJ ₊ (y- yj ₊ (z-zj ₊ (cAt _l4 ), (3) where c is the speed of sound in the air, Δt ₁₂ , Δt ₁₃ , and Δt ₁₄ represent the time delays between the 1 ^st and 2 ^nd , 1 ^st and 3 ^rd , and 1 ^st and 4 ^th microphones, respectively. These time delays can be obtained by taking cross correlations between the 1 ^st and 2 ^nd , 1 ^st and 3 ^rd , and 1 ^st and 4 ^th microphones, respectively, when the incident wave contains broadband frequencies. When the incident sound wave is narrowband or contains a single frequency, a different methodology should be used to determine the relative time delays. Once these time delays are obtained, the location of an unknown source can be determined by solving Eqs. (1) to (3) simultaneously. The general solution is very involved and long. As an example, we present a simplified version of the solution of Eqs. (1) to (3) with microphones 1 - 4 mounted on three mutually orthogonal axes at equal distance with respect to the origin of the coordinate system, namely, (do, 0, 0), (0, do, 0), (0, 0, do), and (0, 0, -do). Under this condition, Eqs. (1) to (3) reduce to 2 d ₀ {x- y)-{c At ₁₂ ) ² = 2c At ₁₂ -J{x -d ₀ f + y ² + z ² , (4)

2J ₀ (x+ y)- {cAt ₁₃ ) ² = 2cAt _nΛ l{x-dJ + y ² + . (5) where J ₀ is a given distance with respect to the origin of the coordinate system. Equations (4) and (5) can be combined so as to write y in terms of x as

y = a ₁ x + b ₁ , (7)

where

At ₁₃ + At ₁₂

C ² At ₁₂ At ₁₃ (At ₁₃ -At ₁₂ ) 2d ,[At ₁₃ +At ₁₂ )

Similarly, Eqs. (4) and (6) can be combined to express z in terms of x as

z = a ₂ x + b ₂ , (9)

where

Substituting Eqs. (7) and (9) into (6) yields a binomial equation for x

A ₁ X + B ₁ = -^A ₂ X ² - B ₂ x + C ₂ , (11) where

2cΔ? ₁₄

B ₂ = 2{d _Q -cφ _ι -a ₂ b ₂ ) , (12d)

Solution for JC is then given by

_ -(2A ₁ B ₁ + fl ₂ )± _A /(2Afl ₁ + £ ₂ ) ² -4(A ₁ ² -AJg ₁ ² - C ₂ )

The solution is calculated by the processor 14 or DSP 15. Once x is solved, the other coordinates y and z are given by Eqs. (7) and (9). Note that there are always two real roots in Eq. (13), which will lead to two different source locations. Apparently, this cannot happen. Therefore one of the roots must be wrong, but the correct root can be picked automatically.

The correct root is selected in this approach. Assume that Eq. (13) yields two roots that lead to two locations at r _si = x _si e _x + y _s] e _y + z _s] e _z and ■ Listed below are steps for picking the correct location. 1. Calculate the distances between r _sl and r _S2 with respect to each microphone, respectively, namely, R _{n t} = T ₃₁ - T ₁ and R _S2 , = T ₃₂ - T ₁ , where i = 1 to 4.

2. Find the minimum and maximum distances with respect to all microphones: min{R _{sl ι} ) , max\R _{sl ι} ), xmv\R _{S2 1} ) , and max\R _{S2 1} ) , where i = 1 to 4. 3. Identify the microphones with mm[R _{sl l} ) and max{R _{sl l} ), and πήn[R _{S2 ι} ) and max\R _{S2 1} ) , respectively.

4. If r _sl and r _S2 are on opposite sides of the coordinate system, perform Steps 5 and 6.

5. Calculate time delays in these microphones with respect to source locations ^ ₁ and r _S2 , and compare them with those of cross correlations.

6. The correct source location will have the time delay that closely matches the measured one obtained by cross correlations.

7. If ^ ₁ and r _S2 are on the same side of the coordinate system, perform Steps 8 through 11. 8. Assume that the source is located at r _sl with amplitude A. The amplitudes of the acoustic pressures at the microphone that corresponds to mm[R _{sl l} ) and

max\R _sl ) can be written, respectively, as /r ¹ ' [min(R _sl )J = -, ^ and

^{1^ '} ' ^min KJ / _m ! [ ^max (R _s i i )\ ⁼ _{7 Y} • Similarly, for the source located at r _S2 with an

™A ^R siJ amplitude A, the amplitudes of the acoustic pressures at the microphones that

correspond to min{R _{S2 ι} ) and max{R _{S2 ι} ) can be expressed, respectively, as

Pπ ^{ L [minte _d )1 = —rz — τ and p^ [max(tf _S2 )] = _j - — * .

9. Calculate the rates of decay of the acoustic pressures emitted from sources at r S _< l

Ap ^{2) P1 ^{ [MR _S2 J- P1 ^{ WΦ _S2 J A

Δr max(R ₅₂ J-min(R ₅₂ J InSx(R ₅₂ Jm-Ii(R ₅₂ J ^'

10. In practice, the amplitude A is unknown a priori. However, test results have shown that the value of A can be approximated by taking a spatial average of the

1 ⁴ measured acoustic pressures A = — ∑pl _{ns ι} , where p _m ² _{s ι} , i = 1 to 4, are the

^ ι=l root-mean-squared values of the acoustic pressures in each microphone.

11. The rms values of the acoustic pressure in the microphones corresponding to the minimum and maximum distances are known. Therefore, by comparing the rate of decay of the measured acoustic pressure at these two microphones with that of the calculated ones, we can identify the correct source location. The above procedure would be performed by the processor 14 or DSP 15. It is emphasized that the above procedures can be used to identify the source location (x, y, z) when microphones are placed at any position so long as they are not all on the same plane. This provides great flexibility in the set up of its microphone array. The present method can also track and trace a sound source that produces a single frequency (tonal sound), another feature that cannot be matched by any other technology such as beamforming.

The main difficulty in locating a source producing sinusoidal (tonal) sound is due to the fact that the signal is cyclic and continuous. Thus, there is no beginning or end in the time history of the incident sound wave. This makes it impossible to apply cross correlations to determine the time delays among individual microphones.

To circumvent this difficulty, the following procedures calculate the time delays among individual microphones.

1. Place two indices separated by a small time interval, say, 10Δ? _umt , where At _umt is the unit time step equal to the inverse of the sampling rate, on the time-domain signal in, say, microphone channel no. 1.

2. Search and put the 1 ^st index at the peak amplitude of the incident tonal sound wave in the microphone channel 1 and 2 ^nd index a small time interval next to the 1 ^st index on the time-domain signal. 3. Synchronize the indices in microphone channel 1 with those in microphones 2,

3, and 4, so they are all aligned at exactly the same time instances. 4. In order for this method to work, we require that the time delay Atu between the 1 ^st and 2 ^nd , 1 ^st and 3 ^rd , and 1 ^st and 4 ^th channels satisfy the inequalities

Δ; _h <A ₍ l _4a)

2c

Δt,. < -J(X ₁ -X ₁ T Hy ₁ - y,f + (Z ₁ -Z ₁ T (14b)

λ < 2 _y l(x ₁ -x _ι ) ² + (y ₁ - y _ι ) ² +(z ₁ - z _ι ) ² , (14c) where i = 2, 3, and 4, λ is the wavelength of the incident wave, and c is speed of sound in the air.

5. Check the slopes of the indices in all microphone channels. The slope in the 1 ^st microphone channel is always negative because the 1 ^st index is always at the peak amplitude of an incident sound wave and the 2 ^nd index is some time later.

The slopes in other channels, however, may be positive or negative. If the slope in the * channel is negative, then the incident sound wave reaches the 1 ^st microphone first, so the time delay between the 1 ^st microphone and i ^th microphone Atu is positive. If the slope in the * channel is positive, then the incident sound wave reaches the * microphone first, thus the time delay Atu between the 1 ^st microphone and the i ^th microphone is negative.

6. If the time delay Atu is positive, move indices in the * microphone channel in the increasing time direction until the 1 ^st index reaches the peak amplitude of the time-domain signal. The difference between the 1 ^st indices in the 1 ^st and * microphone channels is the time delay Atu-

7. If the time delay Atu is negative, move indices in the * microphone channel in the decreasing time direction until the 1 ^st index reaches the peak amplitude of the time-domain signal. The difference between the 1 ^st indices in the 1 ^st and * microphone channels is the time delay Atu.

These calculations are performed by the processor 14 or the DSP 15. Apparently, this approach has a limit on the highest frequency it can handle, which is determined by the microphone spacing. The larger the microphone spacing is, the lower the frequency of the incident tonal sound this device can handle. Hence, to increase the upper frequency limit the microphone spacing must be reduced. Note that this invention has no restriction whatsoever on the placement of the microphone and therefore, users can adjust the microphone spacing, if necessary, in tracking and tracing a tonal sound source. Once the locations of sound sources are identified, the resultant sound field in

3D space can be visualized by superimposing contributions from all the individual sources. To simplify the numerical computations and speed up post processing, we choose to consider the propagating component of the sound pressure in this process only. This approximation is acceptable, especially when our goal is to visualize the sound pressure field throughout the entire space. Assume that the strength of the m ^th source in a particular frequency band is

S _m = A _mn r _Sn , (15) where S _m represents the strength of the rn^ source, A _mn implies the spatial average of the rms value of the acoustic pressures measured at all four microphones for the n ^th frequency band and r _Sn stands for the radial distance of the dominant sound source in the n ^th frequency band. where p^ _{s t} represents the rms values of the acoustic pressures measured in all four microphones. The rms value of the acoustic pressure resulting from this dominant source at any location in the 11 ^th frequency band can then be approximated by

n( ⁿ ) ( \ _ 1 V n ^{(n) r} ' ^Sn H 7 _Ϊ

4 _I=1 r _n [x, y, z) where r _n (x, y, z) represents the radial distance from the n ^th frequency band source to anywhere in 3D space. Note that we choose to consider the dominant source within each frequency only in order to simplify the computations. As a result, the resultant sound pressure field is approximate. With this approximation, we can track and trace the motions of multiple incoherent sources in 3D space in real time.

The resultant sound pressure field in the entire frequency range is then given by and the overall sound pressure level is given by

L (x, y, z) = lOlog pL( ^χ > y> z) (19)

'ref

where p _re f = 2 x 10 ⁵ (Pa) is the reference acoustic pressure. This system 10 is very effective and accurate in identifying the location of an unknown sound source S or multiple incoherent sound sources simultaneously. Also, it works when sources are moving in space. This is possible because the calculations of source locations and visualization of the resultant overall sound pressure field can be done in real time, which is not possible using beamforming or other technologies. The accuracy and spatial resolution of this technology increases with the dimensions of the microphone array and signal to noise ratio. The larger the microphone spacing and signal to noise ratio are, the higher the accuracy and spatial resolution of the 3D soundscaping becomes. Test results have demonstrated that satisfactory results may be obtained using this system when sampling ratio per channel is 109 kHz or higher. Figure 2 illustrates an example 3D soundscape of sound pressure fields as determined by the system of Figure 1 (with a different example set of noise sources).

This invention can be used in a variety of fields such as homeland security or a battlefield where locations of snipers need to be identified; hearing loss prevention in a construction site, factory floor or manufacturing environment where background noise level is very high and workers are constantly subject to health hazardous noise exposure; and last but not the least, in identifying noise sources of sound producing products. In many manufacturing industries, engineers are concerned with locating unknown sound sources such as in quality control and troubleshooting buzz, squeak, and rattle noise problems of a car seat, inside a passenger vehicle or aircraft cabin. It enables one to get a quick "look" at sound sources accurately and cost-effectively.

In accordance with the provisions of the patent statutes and jurisprudence, exemplary configurations described above are considered to represent a preferred embodiment of the invention. However, it should be noted that the invention can be practiced otherwise than as specifically illustrated and described without departing from its spirit or scope. Alphanumeric identifiers on method steps in the claims are for convenient reference in dependent claims and do not signify a required sequence of performance unless otherwise indicated in the claims.