METHOD FOR LOCATING AND TRACKING ELASTIC SOURCES WITHIN THIN MEDIA DESCRIPTION

The present invention relates to a method for locating and tracking elastic sources (for example tactile interactions) within thin media, such as a panel.

It also relates to a method for locating and tracking the contact of an object such as a pencil, a finger, a fingernail, a brush or a screwdriver, on a panel such as a screen, a shop window, a board surface, or the casing of a household domestic appliance.

It further relates to a writing system utilizing a rod and a panel, using the aforesaid method.

Two known methods are commonly used to locate elastic sources within solid media. One is known as "location template matching" (LTM).

In this, the interaction point is found by comparing the measured value with the data bank containing responses corresponding to known positions on the panel, previously acquired during set-up. This method is independent of the panel properties (material, shape, etc.) and the elastic wave propagation model. It requires at least one receiving sensor. However the set-up requires considerable time to acquire all the responses, particularly for large-surface panels, and to achieve accurate solutions. Moreover this location technique is very sensitive to the environmental states (temperature, humidity, etc.). In

addition it cannot solve the problem of tracking continuous interactions.

Another is known as "time delay of arrival" (TDOA). This method assumes that the elastic wave propagation velocity within the panels is constant. Consequently, from an evaluation of this velocity and the arrival time of signals acquired from different sensors, the interaction position can be calculated by effecting different triangulation procedures. The most common is the hyperbola intersection point procedure. The limitation to this method is that it doles not consider the wavefront dispersion phenomena within the thin media. In this respect, within the thin media, the wave phase velocity varies with frequency, and the wavefront may not be recognizable, even of the wave has travelled only a short distance from the interaction point. The estimate of delay in the wave arrival time and the interaction point position may be inexact. In addition, even this method cannot solve the problem of tracking a continuous interaction. An object of the present invention is to provide a method for locating and tracking elastic sources (for example tactile interactions) within thin media, which is simple and accurate.

This and further objects are attained, according to the present invention, by a method for locating an elastic source on a panel, comprising at least four vibrational sensors, each of which receives a vibrational signal generated by said elastic source on said panel, and

each of which sends a corresponding electrical signal to a processor, comprising the steps of: determining the elastic properties of the panel; determining the corresponding dispersion curve; determining the relative distances (δdw) between said at least four vibrational sensors and the position (x _τ , VT) of said elastic source; said position (XT, YT) of said elastic source being determined using the Tarantola iterative technique for non-linear inverse problems. Further characteristics of the invention are described in the dependent claims. For evaluating the position of elastic sources on a board, the method proposes to use the dispersion phenomenon as a useful information source rather than a source of disturbance.

Up to the present time, this phenomenon was always seen as a problem which could only interfere with location methods. The method is effective on any arbitrary surface provided it is uniform and not excessively thick. The method is based on the joint analysis of the signals acquired by an assembly of 4 or more vibrational sensors. It is a passive method and hence does not require the injection of signals of any type into the panel. It is able to locate and correctly track a continuous elastic source. It is able to function on any panel (a shop window, a board surface, the casing of a household electrical appliance, a screen - LCD or CRT) provided this satisfies the following requirements.

* A material structure which is homogeneous or semi-homogeneous (homogeneous in layers).

* A material structure which is isotropic or semi-anisotropic (orientated fibres are present which cause the propagation velocity to have separate but measurable components).

^* A medium which is thin (relative to the wavelength concerned), hence for the illustrated example thicknesses of the order of a centimetre, generally depending on the wavelength.

In addition, the method: ^* Is able to operate on curved surfaces independently of their dimensions.

* Is robust against acoustic occlusions, in that it is able to dynamically reconfigure the effectiveness of the sensors present.

* Enables the interaction signature to be reconstructed at the point of contact, enabling it to be used for a robust recognition of the object used to induce the elastic stress (in the tactile case, a finger, a fingernail, a brush, a screwdriver, etc.).

There are many applications for such a method.

* Large-dimension touch screens. *. Shop windows which become a tactile interface for consulting catalogues or information on-line.

* Touch-sensitive projection boards, for advanced interactive didactic applications.

* Touch-sensitive household electrical appliances, offering the user a

reconfigurable tactile interface.

* Sensitive objects for developing innovative furnishing elements.

* Advanced monitoring and safety systems for recognizing individuals by their signature or their touch. * For locating intruders or dangers.

* Advanced domotic applications, with particular attention to disabled persons.

* Interactive publicity applications.

The characteristics and advantages of the present invention will be apparent from the ensuing detailed description of one embodiment thereof, illustrated by way of non-limiting example in the accompanying drawings, in which:

Figure 1 shows schematically an embodiment of a system for locating and tracking elastic sources within thin media, in accordance with the present invention;

Figure 2 shows schematically a system for estimating the elastic properties of the panel;

Figure 3 shows schematically an embodiment of a system for determining the elastic properties of the panel, in accordance with the present invention.

With reference to the accompanying figures, a system for locating and tracking elastic sources within thin media according to the present invention comprises a panel (or board) 10, and a series of vibrational sensors 11 , which are four in number in the example. The

sensors 11 are connected to an acquisition card 12, which possibly feeds power to the sensors 11. The acquisition card 12 is connected to a computer 13.

A screwdriver 14, schematically represented, traces on the panel 10 a line 15, which is displayed at 16 on the computer 13.

It will be assumed that the panel 10 is homogeneous or semi- homogeneous, isotropic or semi-anisotropic and is relatively thin. For example two common materials satisfying these requirements are plexiglass (PLX) panels and medium density fibre (MDF) panels. These latter are a composite wood product, made from machined wood residual fibres glued together with resin, heated and pressed. Both have a high attenuation coefficient in the high frequency range. In a test example, the panel dimensions are 150 x 100 x 0.5 cm. The sensors used to acquire the signals transmitted into the panel are preferably piezoelectric. For example those of type BlM 771 were used, these being accelerometers marketed by Knowles Acoustics.

These sensors have a large band width (10 kHz). In this example, the signal must preferably be filtered by a low pass filter, with a cut- off frequency for example between 5 and 8 kHz, to compensate the undesirable effects of non-linearity.

The acquisition card 12 receives the signals originating from the sensors 11 and powers them. It filters the signals and transfers them to an audio acquisition card of a computer 13.

At the frequencies of interest (those produced by the touch of a finger) and for small panel thicknesses, the only waves which can be excited and propagated within the medium are guided waves known as Lamb waves. Propagation follows the Viktorov theory. If t is the panel thickness (= 2d), kβ is the S-wave number (transverse waves), v _α and V _β are the P-wave (longitudinal wave) and S-wave (transverse wave) velocities, v _s is the Lamb wave symmetric mode phase velocity and v _a is the Lamb wave anti-symmetric mode phase velocity, the characteristic symmetric mode equation is:

and the anti-symmetric mode equation is:

where

Assuming that λ » t, λ « I and λ « w are always satisfied, there

will always be Lamb-guided wave propagation. To calculate the dispersion curves, the elastic properties of the medium must be known, for example the P-wave and S-wave velocities v _α and v _β . Other quantities can be used which, as in the case of compressional and transverse velocities, are uniquely representative of the panel elastic parameters, for example Young's modulus and the Poisson coefficient, or Lame parameters, etc. These properties are then estimated and the dispersion curves calculated for the main modes (of 0 order) S ₀ (symmetrical) and a ₀ (anti-symmetrical). In practice the calculation is done only for the a,, mode, as the sensors used, usually positioned on the panel surface, sense the. elastic displacements along the z axis, the elastic displacement component of the so mode normal to the panel being negligible at the frequencies concerned, and for these panel dimensions.

Various methods are possible for determining the elastic properties of the panel 10.

An active method will firstly be considered.

As can be seen from Figure 2, a transducer T converts the electrical energy into mechanical energy, which propagates through a thin panel by means of elastic waves. Two sensors R _x i and Rχ ₂ disposed at a known distance apart and at a known distance from the transducer T, are used to receive the signals associated with this wave, such as to calculate their phase difference and hence measure

their phase velocity. Different implementations of the phase velocity can then be used to evaluate the panel elastic properties (for example, v _α and Vβ ).

The transducer T generates a series of pulses at different frequencies, for example from 1500 Hz to 8000Hz in 250Hz steps. For each frequency the difference between the theoretical phase velocity v _{a C} ai and measured phase velocity v _a . _Ob s is measured. The optimum values for the P-wave and S-wave velocities can be

obtained by minimizing the residue R _d : The residue R _d can be minimized by an exhaustive search technique, seeing the low computation cost, or by any other known minimization technique. However the minimization problem can be poorly conditioned and require further constraints.

To facilitate minimization of the residue R _d , the Poisson coefficient value v can be fed in, if known.

Using the panel elastic properties and wave propagation theory the phase velocity v _a can be calculated at different frequencies.

The agreement between the observed and calculated curves confirms the solution to the optimization problem, which can be considered an inverse problem, solved using the exhaustive search technique or any other known minimization technique.

A passive method will now be described.

The passive method is based on subdividing the spectrum of acquired signals induced by an elastic source into bands. At the central frequency of each band a calculation is made of the phase constant of the signals acquired by sensors (at least two) situated in known positions. An evaluation of the panel elastic properties is then made by the procedure described for the active method. A further passive method will now be described for determining the elastic properties of the panel 10. This method is more exact and robust than those previously described as it does not involve the difficulty of parameter optimization. It is based on an exhaustive search of the best adaptation between the dispersion curves for the test material and an assembly of dispersion curves stored in a data bank and obtained by simulation. This procedure also gives an evaluation of the type of material under examination.

Knowing the physical variability intervals of the elastic parameters (for example v _α and v _β ) of the panel material, for each pair of these parameters the corresponding dispersion curves (i.e. the phase velocity as the frequency v _a (f) varies) are generated and stored in a data bank.

The equation used is that initially given for anti-symmetric modes. Four sensors 11 will be considered (the method requires at least two), fixed onto a panel to form a rectangle, together with an elastic

source (for example a tactile interaction) 20 in a known position, as in Figure 3.

The elastic source induces propagation of Lamb waves within the panel, the signal of which is acquired by the four sensors. The source is preferably positioned in line with two sensors at an equal distance from them. This can facilitate operations, by averaging the value of the signals received and hence increase the signal/noise ratio. A search is made to find which curve gives the best residue between a received reference signal and a calculated signal propagating with the phase velocities stored in the data bank. A pair of sensors is used, of which one is taken as the reference sensor, for example the sensors Rχ ₃ and R _X4 , of which this latter is the reference sensor. Specifically, the relative distance is determined between the sensors Rx ₃ and R _X4 , knowing the position T of the elastic source: the signal S ₃ propagates inversely from the sensor Rχ3 to the point T and then directly towards the sensor R _x4 , using one of the dispersion curves stored in the data bank, by means of the following equation, in which the frequency domain signals are considered:

where

S ₃ (f) is the Fourier transform of the signal acquired at the sensor

Rx3, δ043 = 04τ - d3τ , with 03τ and 04τ being respectively the

distances between the sensors Rχ ₃ and R _x4 from the position of the point T, v _a (f) is phase velocity of the Lamb anti-symmetric mode, of order 0, and SU(O ^{is tne} Fourier transform of the calculated signal. The residue between the signal S ₄ received at the sensor R _X 4 and the calculated signal s' ₄ is determined, for example by the following equation:

Rd (Va) = I _n (S _4n - S' ₄ n) ² where

S _4n is the discrete Fourier anti-transform of the signal SU(O- S ₄ n is the discrete signal received by the sensor R _x4 .

R _d is a parameter which depends on v _a.

The residue calculation can be likewise performed by representing the frequency domain signals (Parseval theorem).

These operations are carried out for all the stored curves, that dispersion curve which has the lowest residue being chosen. To this, there corresponds a pair of elastic parameters (for example v _α and v _β

) for the material of the panel under examination.

If more than two sensors are present, as commonly happens in the system for locating and tracking elastic sources within thin panels, in which at least four sensors are present, the setting procedure can be repeated several times, a different sensor pair being considered each time. The partial results can then be considered jointly (for example averaged) to increase the precision of the overall estimation of the elastic properties of the panel under examination.

We can now calculate the relative distances between the sensors 11 and the elastic source position.

With reference to a case for example with four sensors (of which the fourth, without loss of generality, is assumed to be the reference sensor) a Lamb wave source (for example a tactile interaction) will be considered at position (X _T , y-r) within a panel 11 , the signals acquired by the two sensors R _X 4 and R _x i being S ₄ and si, their distances from the elastic source being d ₄ τ and diτ respectively. The difference between the distance of the source from the sensor R _x4 (d ₄ τ ) and the distance of the source from the sensor R _x i (dn ) is known as the relative distance of the sensors R _x4 and R _x i, and is equal to:

We can now calculate S ₄ by the propagation of si through the distance δdv- Moreover an evaluation of δd4i by the propagation of si through different distances which increase until the distance equals S ₄ . The best can be obtained by an exhaustive search technique, although any other known minimization technique can be used. Specifically, a wave S ₁ is propagated inversely from the sensor R _x i to the point T and then directly to the sensor R _x4 .using that dispersion curve previously determined for the panel in use, by means of the following equation, in which the frequency domain signals are considered: S' ₄ (f) = Si(f) e-^VV ⁰

where

Si(f) is Ia Fourier transform of the signal acquired at the sensor R _x i, δ041 = d4τ - diτ , with diτ and dπ being respectively the distances between the sensors R _x i and R _x4 from the position of the point T ₁ v _a (f) is the phase velocity of the Lamb anti-symmetric mode, of order 0, and S' ₄ (f) is the Fourier transform of the calculated signal. The residue between the signal s.> received at the sensor R _X4 and the calculated signal s' ₄ is determined, for example by the following equation: R _d (δd ₄ i) = ∑n (S4n - S' ₄ n) ² where s ^# 4n is the discrete Fourier anti-transform of the signal SU(Q-

S4n is the discrete signal received by the sensor R _x4 .

Rd is a parameter which depends on δd4i. The residue can be likewise calculated by representing the frequency domain signals (Parseval theorem).

These operations are carried out for each distance assumable by

δd ₄ i, which can be defined on the basis of the panel dimensions, that

δd4i having the minimum value being chosen. For example, within those values which δd ₄₁ can assume, the aforesaid procedure is effected at 1 mm steps to achieve a precision of 1mm.

Another known method for calculating δd4i is for example that described in WO03005292.

Once the relative distances δd _4< , i = 1, ..., 3 have been calculated, the location problem can be solved using the Tarantola inversion theory.

The inverse problem can be formulated in the following manner. Let RXJ fa, yi), i = 1, ..., 4, be the sensor positions. Let m = [XT yτ] ^τ , be the unknown position of the elastic source (for example a tactile interaction) on the panel.

Let Ad*, i = 1,..., 3, be the relative distances of the sensors; in matrix terms, dobs = [δd ₄ i, δd ₄ 2, δd ₄ 3] ^τ . The distances drr are definind as:

d _n = yj(x _τ -X ₁ ) ² + {y _τ -y _λ f d _2T = yj(x _τ -X ₂ ) ² +(y _τ -y ₂ f d _3T = -j(x _τ -χ ₃ f +{γ _τ - y ₃ Y d _4T = τ](x _τ -X ₄ ) ² +{y _τ -y ₄ f

Consequently the relative distances are given by:

δrf •4,n1 = d _λ 4τ7- -- du _ir — d «27-

Ad ' _A 4-3> = ds 4FT — d hτ

The relationship between the model and the data leads to the following Jacobean matrix G ₁ linearized about a reference model m ₀ - [XTO YTO] ^τ , for example the panel centre.

yτo)

The linear system

can be resolved for the model m = [xτyτ] ^τ -

The direct model is not linear and the solution can be determined with good accuracy using an iterative inversion procedure such as the Tarantola technique for non-linear inverse problems. The following procedure was used to find the solution:

Where the a priori model has a mean rrtpr and the covariance matrix CM, dobs is the vector of the observed data, C _d is the covariance matrix of the measurement uncertainties and the modelling errors, GK is the Jacobean matrix, linearized at the iteration K, m* and mj< ₊₁ are the vectors of the model estimated at the iterations K and K+1. The iteration terminates when:

where M = 2 and 27 is the required solution accuracy. In the experiments carried out, the average number of iterations required to obtain an accurate solution is about four. With four sensors available, the location procedure can be repeated four times, considering a different reference sensor for each of them.

The partial results can then be considered jointly (for example, averaged) to increase the accuracy of the overall estimation of the source position.

Other known methods for determining the source position, knowing δd- _H , are for example those described in the article by A. Tobias, "Acoustic emission source location in two dimensions by an array of three transducers", Technical report, NDE, 1976; and in that by A. Mahajan and M.Walworth, "3-D position sensing using the differences in the time-of-flights from a wave source to various receivers," IEEE Trans, on Robotics and Automation, vol. 17 No. 1 , Feb. 2001.

With the present method a continuous elastic source on the panel (for example a continuous tactile interaction) can also be tracked by iteratively applying the previously described inversion technique. From an initial source position (evaluated by the described method or by another, such as the TDOA, as high accuracy is not required for estimating the initial source position) the immediately subsequent position of the source is evaluated by effecting non-linear inversion of the relative distances between the sensors (at least three). The procedure for tracking the continuous elastic source is the following:

1. Determination of the initial source position.

2. Inversion of the relative distances between the sensors, starting from the previous source position, using the iterative Tarantola

technique for non-linear inverse problems. Updating the real source position.

3. If the source is still moving, then point 2 is repeated, otherwise the procedure is terminated. Further improvements can be obtained by methods for regularizing the estimated trajectory (for example by Kalman filtering).