Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
STATISTICAL SHOOTER RANGE ESTIMATING METHOD USING MICROPHONE ARRAY
Document Type and Number:
WIPO Patent Application WO/2021/006825
Kind Code:
A1
Abstract:
A statistical shooter range estimating method using microphone array, using both deterministic and probabilistic approach comprising recording acoustic signals which come from the microphones to a database (1), classifying detected shock and muzzle candidates using machine learning techniques, recording classified shock and muzzles with certainty on shock and muzzle stack, selecting every combination of a shock and muzzle by algorithm among the shock and muzzle candidates from shock and muzzle stack, extracting shot features with probabilistic approach thereby machine learning for every shoot combination, recording shot features for every combination of shock and muzzles and storing on a shot feature stack, and estimating azimuth angle and elevation angle of shooter by using muzzle angle's time difference of arrival (TDOA) and estimating range by using probabilistic approach which utilizes machine learning on shot features and deterministic approach which has distance information of the shooter.

Inventors:
GEVREKCİ LÜTFI MURAT (TR)
ŞAHIN ÖMER (TR)
DEMİRÇİN MEHMET UMUT (TR)
Application Number:
TR2019/050538
Publication Date:
January 14, 2021
Filing Date:
July 08, 2019
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
ASELSAN ELEKTRONIK SANAYI VE TICARET ANONIM SIRKETI (TR)
International Classes:
G01S3/808; G01S5/22
Domestic Patent References:
WO2010077254A22010-07-08
Foreign References:
US5831936A1998-11-03
US0617814A1899-01-17
RU2007110535A2008-10-10
Other References:
BORZINO ANGELO M C R ET AL: "Gunshot signal enhancement for DOA estimation andweapon recognition", 2014 22ND EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO), EURASIP, 1 September 2014 (2014-09-01), pages 1985 - 1989, XP032681858
SANCHEZ-HEVIA HECTOR A ET AL: "Maximum Likelihood Decision Fusion for Weapon Classification in Wireless Acoustic Sensor Networks", IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, IEEE, USA, vol. 25, no. 6, 1 June 2017 (2017-06-01), pages 1172 - 1182, XP011649960, ISSN: 2329-9290, [retrieved on 20170523], DOI: 10.1109/TASLP.2017.2690579
Attorney, Agent or Firm:
DESTEK PATENT, INC. (TR)
Download PDF:
Claims:
CLAIMS

1. A statistical shooter range estimating method using microphone array, using both deterministic and probabilistic approach characterized by comprising; following steps in below;

• Recording acoustic signals which come from the microphones to a database (1 ),

• Detecting valid shocks and muzzles thereby using filtering techniques,

® Classifying detected shock and muzzle candidates using machine learning techniques,

• Recording classified shock and muzzles with certainty on shock and muzzle stack,

• Selecting every combination of a shock and muzzle by algorithm among the shock and muzzle candidates from shock and muzzle stack,

• Extracting shot features with probabilistic approach thereby machine learning for every shoot combination,

• Recording shot features for every combination of shock and muzzles and storing on a shot feature stack,

• Estimating azimuth angle and elevation angle of shooter by using muzzle angle’s time difference of arrival (TDOA) and estimating range by using probabilistic approach which utilizes machine learning on shot features and deterministic approach which has distance information of the shooter by searching for gun type and possible ranges jointly in by using speed-range look-up-tables thereby mixing with a gating network that has configurable rates, by taking into consideration microphone geometric parameters and resistor (2) which is imported because of the temperature has an effect on velocity of sound,

• Estimating absolute coordinates by using an IMU (3) and a GNSS (4) unit.

2. A statistical shooter range estimating method according to claim 1 , characterized by comprising; using time elapsed between shock and muzzle, time between positive and negative peaks of shock (Tsize), shock azimuth and elevation angle, muzzle azimuth and elevation angle, shock and muzzle normalized energy ratio, shock and muzzle classification probability, median cross correlation ratio (pearson correlation) of muzzles within shot parameters in extracting shot features with probabilistic approach thereby machine learning for every shoot combination step.

3. A statistical shooter range estimating method according to claim 1 , characterized by comprising; eliminating improbable shock-muzzle combinations using a rule based approach after selecting every combination of a shock and muzzle by algorithm among the shock and muzzle candidates from shock and muzzle stack step.

4. A statistical shooter range estimating method according to claim 3, characterized by comprising; using arriving muzzle after shock, being exceedingly larger amplitude of shock than muzzle, computing normalized cross correlation (NCC) score of each microphone with respect to a reference microphone and exceeding median value of normalized cross correlation (NCC) a certain value for eliminating echo for urban shooter localization to great extent as rule based approach.

5. A statistical shooter range estimating method according to claim 1 , characterized by comprising; using machine learning methods like decision trees, deep neural networks in extracting shot features with probabilistic approach thereby machine learning for every shoot combination step.

Description:
Statistical Shooter Range Estimating Method Using Microphone Array

Technical Field

The present disclosure is related a statistical shooter range estimating method using microphone array by means of computing time difference of arrival (TDOA) of a shock and muzzle signals. Deterministic and probabilistic range estimation have been used with machine learning techniques in mentioned disclosure. Prior Art

Existing methods do not utilize mixing deterministic and probabilistic range estimation, and they are not capable of making estimates beyond the gun-types and ranges that are hard coded by the look-up tables provided at system initialization. In existing time difference of arrival (TDOA) methods, range and gun-type is searched in an exhaustive manner jointly. However, range is limited by the look-up table size and gun-type effects the range estimate.

In known art, US617814B1 document refers an acoustic counter-sniper system. A low cost and highly accurate sniper detection and localization system uses observations of the shock wave from supersonic bullets to estimate the bullet trajectory, Mach number, and caliber. If available, muzzle blast observations from an unsilenced firearm is used to estimate the exact sniper location along the trajectory. The system may be fixed or portable and may be wearable on a user's body. The system utilizes a distributed array of acoustic sensors to detect the projectile's shock wave and the muzzle blast from a firearm. The detection of the shock wave and muzzle blast is used to measure the wave arrival times of each waveform type at the sensors. This time of arrival (TOA) information for the shock wave and blast wave are used to determine the projectile's trajectory and a line of bearing to the origin of the projectile. A very accurate model of the bullet ballistics and acoustic radiation is used which includes bullet deceleration. This allows the use of very flexible acoustic sensor types and placements, since the system can model the bullet's trajectory, and hence the acoustic observations, over a wide area very accurately. System sensor configurations can be as simple as two small three element tetrahedral microphone arrays on either side of the area to be protected or six omnidirectional microphones spread over the area to be monitored. Sensors may also be mounted onto to a helmet as used with the wearable system. Sensor nodes provide information to a command node via wireless network telemetry or hardwired cables for the command node comprising a computer to effect processing and display. However, there is no signal classification and statistical range estimating.

RU20071 10535 document can be used for determining the trajectory of a supersonic projectile. At least the initial part of signals is measured, containing information only on impact wave, using five or more acoustic sensors, spread out in space such that they form an antenna. From this measured initial part of signals, the difference in arrival time for a pair of sensors is determined. A genetic algorithm is applied to the initial chromosome, which contains initial estimated parameters of the projectile trajectory. For a given number of generations, projection errors are calculated for solutions, obtained from chromosomes from the genetic algorithm. The ratio of solution with the least values of projection errors to the ambiguous solution is calculated, and if this ratio is greater than a given value, the solution with the least value of calculated projection error is chosen as the correct trajectory of the projectile. However, mentioned invention is related with completely look up table method and does not involve statistical range estimating.

Proposed method extracts features from muzzle wave and shock wave, and uses these attributes to make a range estimation as well as azimuth and elevation angle estimation probabilistically using machine learning techniques. Feature engineering and the machine learning techniques proposed enabled both accurate and computationally efficient shooter localization framework.

Brief Description of the Invention

The main object of the invention is to provide a statistical shooter range estimation method using microphone array by means of computing time difference of arrival (TDOA) of a shock and muzzle signals.

Another object of the invention is to provide a statistical shooter range estimation method using microphone array thereby using deterministic and probabilistic range estimation with machine learning techniques.

Another object of the invention is to provide a statistical shooter range estimating method using microphone array which has low error rate by using deterministic and probabilistic range estimation. Another object of the invention is to obtain a statistical shooter range estimating method using microphone array which uses feature engineering and the machine learning techniques for high precision.

Another object of the invention is to obtain a statistical shooter range estimating method using microphone array which localize shock and muzzle signals using machine learning algorithms, even with low signal to noise ratios (SNR) coming from long ranges.

The invention is a statistical shooter range estimating method using microphone array, using both deterministic and probabilistic approach comprising; following steps in below;

• Recording acoustic signals which come from the microphones to a database,

• Detecting valid shocks and muzzles thereby using filtering techniques,

® Classifying detected shock and muzzle candidates using machine learning techniques,

• Recording classified shock and muzzles with certainty on shock and muzzle stack,

• Selecting every combination of a shock and muzzle by algorithm among the shock and muzzle candidates from shock and muzzle stack,

• Extracting shot features with probabilistic approach thereby machine learning for every shoot combination,

• Recording shot features for every combination of shock and muzzles and storing on a shot feature stack,

• Estimating azimuth angle and elevation angle of shooter by using muzzle angle’s time difference of arrival (TDOA) and estimating range by using probabilistic approach which utilizes machine learning on shot features and deterministic approach which has distance information of the shooter by searching for gun type and possible ranges jointly in an exhaustive manner by using speed-range look up-tables thereby mixing with a gating network that has configurable rates, by taking into consideration microphone geometric parameters and resistor which is imported because of the temperature has an effect on velocity of sound,

• Estimating absolute coordinates by using an IMU and a GNSS unit. In order to better understand the present invention, its exemplary embodiment is shown in the attached figures. The details of the present disclosure shall be evaluated by taking the whole description into consideration. These figures are as the following;

Brief Description of the Drawings

Figure 1 is a schematic view of how running statistical shooter range estimating method. Figure 2 is a schematic view of how running statistical range estimating.

Figure 3 is a schematic view of mixing of experts for range estimating.

Figure 4 shows view of time of arrival.

Figure 5 shows range estimating geometrically and sample shooting informations.

Reference Numbers

1. Database

2. Resistor

3. IMU

4. GNSS

Detailed Description of the Invention

Proposed system solution estimates the range of a shooter with high precision using a microphone array system. System localizes shooter using a super-sonic shock wave along a regular muzzle sound. System essentially computes time difference of arrival (TDOA) of a shock and muzzle signal. This information is sufficient to produce an azimuth and elevation angle for the shooter. However, range of the shooter can only be estimated if the gun type is known. One possible solution is to generate a cost function for every gun type and every possible range. This approach is a look-up table (LUT) approach where gun-type and range is searched exhaustively.

An innovative solution is proposed in this work to estimate the range using machine learning and feature engineering with this disclosure. This machine learning range estimator is weighted with LUT approach to generate a mixture-of-experts fashion to reduce the errors in range estimation. This technique blends a deterministic approach and a probabilistic approach under the same framework. LUT is a deterministic approach, in which every cost has a well defined coefficient for range estimation. Machine learning approach is probabilistic such that it provides generalization capability to the system for estimating ranges, and angles that are not seen before.

System has an IMU (3) and a GNSS (4) unites. The IMU (3) provides platform pitch, roll, while GNSS (4) unit provides true heading, latitude, longitude, high resolution timer information. This gives the capability to estimate shooter on earth coordinates even with high speed moving platforms, such as car.

In mentioned method, first acoustic signals are recorded which come from the microphones to a database (1 ). Then, putative shocks and muzzles are detected using filtering techniques (i.e. MR, FIR) These detections are treated as shock muzzle candidates and shock muzzle candidates are classified using machine learning techniques. Classified shock and muzzles with certainty are recorded on shock and muzzle stack. There can be multiple shock and muzzles for each microphone within a certain time, since shooting can be performed in burst mode. Shooters can be in different geographic locations and might shoot simultaneously.

After that, every combination of a shock and muzzle is selected by algorithm among the candidates from shock and muzzle stack. For every shoot combination following features are extracted stored for every shock and muzzle by machine learning. Shot features are recorded for every combination of shock and muzzles and stored on shot feature stack. Then, azimuth angle and elevation angle of shooter is estimated using muzzle angle using Time-difference-of-arrival (TDOA) and range is estimated by using probabilistic approach which utilizes machine learning on shot features and deterministic approach which has distance information of the shooter by searching for gun type and possible ranges jointly in an exhaustive manner by using speed-range look-up-tables thereby mixing with a gating network that has configurable rates, by taking into consideration microphone geometric parameters and resistor (2) which is imported because of the temperature has an effect on velocity of sound. Absolute coordinates are estimated by using an IMU (3) and a GNSS (4) unit. Azimuth and elevation angles are computed from the shooter to the sensor unit. These angles are computed using the microphone geometry of shown in Figure 4. In this disclosure, four microphones are utilized and this is the minimum number of microphones that can be utilized for angle estimation and can be increased for performance. At least one microphone should be positioned other then the plane to estimate both elevation in addition to azimuth. Azimuth and elevation angles are computed using Time-difference- of-arrival (TDOA) and microphone geometry. This procedure enables to find the direction relative to the microphone array where sound source is located.

As can seen in figure 1 , when probabilistic and deterministic mixture of expert range estimation networks operating for range estimation, microphone geometric parameters and resistor (2) which is imported because of the temperature has an effect on velocity of sound are taken consider too. Then, the IMU (3) and GNSS (4) units are used for absolute coordinate estimation.

As can seen in figure 2, probabilistic approach uses shot features, coming from both shock and muzzle to estimate range while extracted features for every shock and muzzle. This approach uses machine learning for extracting the continuous input/output relationship between shot features and range. Several machine learning methods can be utilized for machine learning, such as decision trees, deep neural networks. Feature engineering is the critical part for success. Feature engineering approach allows to represent raw data in a compact and meaningful manner. For an example system with a 51 Khz, with 4 microphones, a shot means 51200 * 4 samples, which is 0.2 million samples. With feature engineering a shot is represented by only 5 scalars with float precision. Shock and muzzle’s azimuth and elevation angle difference is taken by cosine distance. Time elapsed between shock and muzzle is fed by normalization with sampling rate. Shock and muzzle strength given as ratio. Tsize computation is described in following sections using first positive and first negative peak of shock signal.

Deep neural network in Figure 2 provides the optimal performance with hidden number of layers 8. Data is partitioned into training, validation, and testing with certain percentages for hyper-parameter tuning. Inputs are normalized to [0,1 ] range for optimal convergence in neural network. Sigmoids are used as activation function but they can be changed with other functions as well. As can seen in figure 3 for probabilistic and deterministic mixture of expert range estimation, deterministic range estimation network outputs the distance of the shooter by searching for gun type and possible ranges jointly in an exhaustive manner by using speed-range look-up-tables (dictionary). This approach is easy to debug and interpret possible errors. However, deterministic approach fails for unseen gun types and ranges that are out of the dictionary. Probabilistic approach is superior for second type of scenarios. We propose using both approaches in a mixture of experts framework to estimate range close to the ground truth. Gating network that has configurable rates, gives higher weight to probabilistic network for high range values.

In a preferred embodiment of the invention, time elapsed between shock and muzzle, time between positive and negative peaks of shock (Tsize), shock azimuth and elevation angle, muzzle azimuth and elevation angle, shock and muzzle normalized energy ratio, shock and muzzle classification probability, median cross correlation ratio (pearson correlation) of muzzles within shot parameters are used in extracting shot features with probabilistic approach thereby machine learning for every shoot combination step.

In a preferred embodiment of the invention, improbable shock-muzzle combinations are eliminated using by a rule based approach after selecting every combination of a shock and muzzle by algorithm among the shock and muzzle candidates from shock and muzzle stack step. Arriving muzzle after shock, being exceedingly larger amplitude of shock than muzzle, computing normalized cross correlation (NCC) score of each microphone with respect to a reference microphone and exceeding median value of normalized cross correlation (NCC) a certain value for eliminating echo for urban shooter localization to great extent are used as rule based approach.

In a preferred embodiment of the invention machine learning methods like decision trees, deep neural networks are used in extracting shot features with probabilistic approach thereby machine learning for every shoot combination step.