FIELD

The present invention relates to a method and/ or system for magnetic localization. BACKGROUND

Recent advancement in miniaturization and sensitivity in magnetic sensor technology has motivated emerging applications of magnetic tracking and localization techniques in robotics and mechatronics systems. Magnetic field-based positioning systems are compact, low-cost, robust, non-invasive, unobtrusive, energy-efficient, and safe. Magnetic localization involves capitalizing on sensitivity of modern solid-state sensors to detect minute changes in the measured vector magnetic flux density (MFD) due to relative positional changes between the magnetic source and sensors. By affixing the source (or the sensor) onto the moving target and referencing the known position of the sensor (or source), the instantaneous orientation and position of the target can be extracted through active measurement and monitoring of the sensor.

This determination, however, requires a field model, a function that relates position to MFD, and the most commonly used model is the magnetic dipole (MD) model.

One of the key advantages of magnetic field-based localization is that passive magnetic sources exist. This not only allows the localization target to be completely un-tethered but also eliminates the requirement of a power source to generate the field. This results in a very compact sensing system on the target which consists of only a permanent magnet. In addition to sensing across non-ferromagnetic mediums including air and the human body, magnetic fields are invariant to temperature, pressure, radiation and other environmental factors, which make them an excellent choice for tracking and localization applications in medical environments. It is employed in tracking of tongue motion during speech-language therapy, assessing the location of the probe during endovascular catheterisation, assisting in robotic capsule endoscopy and for anatomical mapping and motility studies of the digestive system. The non-contact nature of field-based localization is harnessed in industrial applications for high accuracy sensing as well as feedback control of actuators. In the realm of robotics, it is integrated in UAVs to locate power lines for perching and an integral component of the magneto-elastomeric force sensor in a running quadruped.

Magnetic localization involves tackling the inverse problem and since explicit expression for the inverse model is not available, the forward model (the field model) is used instead. This is achieved by using a nonlinear optimization algorithm to minimize the deviation between measured and modelled magnetic field. A significant proportion of literature on magnetic localization uses the MD model. The issue with the MD model is that it does not take into account the geometrical shape or magnetization of the physical magnet. Due to its formulation, while it is able to adequately characterize the magnetic field at large distances from the source, the model accuracy rapidly deteriorates as it approaches the surface of the magnet. The distributed multi-pole (DMP) model, which consists of a spatial array of dipoles, was conceived to address the shortcomings of the single dipole model. However, it requires a fairly involved and manual process to determine a suitable spatial dipole array.

Thus for many medical applications where the source and sensors are in close proximity, these two models are either not adequate or too tedious to perform high accuracy model-based tracking. While finite element analysis can provide a more accurate location, it is not feasible for real-time localization.

SUMMARY

In general terms the invention proposes using a neural network trained for a given magnet, to determine the location of the magnet using the field data from a series of magnetic sensors. The advantage to that accuracy may be improved when the source and sensors are in close proximity.

In a first specific expression of the invention there is provided a method of real time magnetic localization according to claim 1.

In a second specific expression of the invention there is provided a system for real time magnetic localization of an in vivo device or a medical instrument configured for insertion into a patient according to claim 11.

Embodiments may be implemented according to any of claims 2 to 10 or 12 to 15. BRIEF DESCRIPTION OF THE DRAWINGS

One or more example embodiments of the invention will now be described, with reference to the following figures, in which:

Figure 1 is a schematic diagram showing the coordinate system for axisymmetric object and magnetic dipole models; Figure 2a is a flow diagram if an ANN field modelling process according to an example embodiment;

Figure 2b is a representation of a GUI for automated ANN field modelling where the user inputs the magnet dimensions on the left and via field measurements, the coefficients of the ANN field models are generated on the right.

Figure 3 is a photo of an Experimental setup of SENIS MMS-1-R magnetic field mapping system during scanning;

Figure 4 is a graphed contour plot of averaged measured MFD in 'L' domain for both magnet geometries. (Units: mT). The white boxes with black border denote the physical boundary of the permanent magnet (PM);

Figure 5 is a graph of RMSE of ANN fit of Bz as a function of hidden nodes;

Figure 6 is a graph of absolute error for MD field model illustrated spatially for both magnet geometries. (Units: mT);

Figure 7 is a graph of absolute error for DMP field model illustrated spatially for both magnet geometries. (Units: mT);

Figure 8 is a graph of absolute error for ANN field model illustrated spatially for both magnet geometries. (Units: mT); and

Figure 9 is a graph of spatial tracking performance between ANN and MD field models for XY-plane figure '8' trajectory.

DETAILED DESCRIPTION

Artificial neural networks (ANNs) are mathematical models that are inspired by the functional structure of biological neural networks. Paired with supervised learning, back-propagation ANNs can be engaged to infer function from observations and are particularly useful where the underlying function is overly complex or unknown. Compared to other methods of function approximation, the order of the approximation is easily tuned through the architecture of the ANN. While the magnetic field of an object satisfies Maxwell's Equations, there are a multitude of parameters as well as physical imperfections that affect the magnetic field of a magnetic source..

What we are attempting to do is to relate (or map) an arbitrary set of inputs to another set of outputs (here we relate position to magnetic field measurements). This is also referred to as functional fitting. Some methods include straightforward Look-Up Table (LUT) methods, conventional least squares (LS) using basis functions of polynomials, sinusodials, etc and artificial neural networks (ANN). Both the LUT and LS approaches are more extensively used due to their simplicity, but we have identified that ANNs are more adaptable when mapping multiple inputs and outputs. In addition, adding hidden layers or nodes has minimal effect on computation time as only arithmetic operations are required during real-time operation of ANN.

Look up tables:

A LUT is a structured data where the location within this data structure contains pre-computed value of the desired output. These locations (known as lattice points) are discrete in nature and computation of the output for non-lattice points are obtained using interpolation methods. An issue with using LUT with interpolation is the significant increase in complexity in constructing the LUT and performing successive interpolation as the number of independent field measurements increases. For mappings requiring more than 3 axes of independent field measurement, this method becomes memory intensive due to the large array of pre-computed data and requirement of numerous interpolating operations.

Least Squares Models:

Another commonly used method in data fitting is least squares. The best fit in the least-squares sense minimizes the sum of squared residuals. A residual is defined as the difference between a desired observed value and the value provided by a model. A model can be linear or nonlinear and for the single input case, some of the well-known models are polynomial or sinusoidal (Fourier series). Like LUT, it is not easily scalable with increased inputs and higher order models. Moreover, these models are limited to a single output. For multiple outputs, independent multiple models must be used.

Artificial Neural Networks:

As mentioned an artificial neural network is a mathematical model that tries to mimic the structure and functional aspects of biological neural networks. Paired with supervised learning, back propagation ANNs can be trained to fit a desired set of inputs to a corresponding set of outputs by iteratively adjusting the weighting coefficients in the network. A commonly used cost function is the mean-squared error which tries to minimize the average squared error between the network's output desired target values over all data pairs. We have identified two approaches in obtaining the minimum: Gauss-Newton algorithm and the gradient descent method. However, the Levenberg-Marquardt algorithm, which interpolates between both methods, may be employed. Neural networks are scalable as the general training algorithm is not dependent on the number of inputs and outputs. The order of the network is easily controlled by the number of hidden layers and number of hidden nodes within each layer. I. MAGNETIC FIELD MODELLING

In the source-free and current-free space ( J=0) around a stationary magnetic object, the magnetic field (or flux density) B satisfies the two magnetostatic equations:

VxH = 0 (1 )

V- B = 0 (2) where H is the magnetic field intensity defined by the magnetic flux density (MFD), B. If the object has a magnetization of M, the relationship between H and B is expressed by

Η = Β / ₀ - Μ (3) where μ ₀ is the magnetic permeability of free space. The vector B at a point in space x=[x y z] ^T can be expressed as a gradient of the magnetic scalar potential Φ:

{x) = [B _x B _y β _ζ ] ^Τ = ^÷ - _ο ν (4)

The solution of the potential Φ is obtained from the Poisson equation, given and approximated by

where primes denote coordinates of the magnetic material and m = jMc/V is the magnetic moment. The combination of (4) and approximation in (5) forms the basis for the ubiquitous magnetic dipole (MD) model, which assumes M is well-behaved, localized and more importantly, x » x'.

A. Dipole Field Modelling

For an axisymmetric (about z-axis) magnetic object with its coinciding with its axis of symmetry z , as illustrated in Figure 1 , the magnetic field produced by this object can be approximated by a dipole (yellow square) at the origin O. Without loss of generality, the axisymmetric geometry is represented by an annular cylinder with length L and outer and inner radii of Ri and R ₂ respectively. The origin is set to coincide with the centroid of the annular cylinder. Hence, the magnetic flux density at point p due to this dipole with a dipole moment of m = mi at O IS

An alternative method of modelling magnetic fields is the distributed multi-pole (DMP) model, which involves multiple discrete source and sink poles. For the magnetic object in Figure 1 , the corresponding magnetic flux density at point p due to a discrete pair of source (+) and sink (-) poles (red circles) each with strength m and separated by the distance / is m m

B _p {DMP} = (7)

m P - ^m ₊ where m ₊ =[0 0 l/2] ^T and m-=[0 0 -//2] ^T are the spatial location of the source and sink, respectively. The pole distance is limited by 0 < / < L to prevent singularity at the magnet surface. To determine the unknowns of m and / of the MD and DMP models, computing the minimum of the following error function E is required:

£ =∑||B _m ^ {MD,D P} -B _e exp (8) where B _exp is the experimentally obtained field data. Again, it can be observed that the shape and size of the magnet are not factored into the dipole models.

Since the radiating field of an axisymmetric object, with magnetization coincident to the axis of symmetry, is also axisymmetric, describing B _p in cylindrical coordinates [B _p Β _θ BJ ^T is advantageous as Β _θ = 0. Therefore, the entire 3D magnetic vector field around the object can be completely characterized by only 2 variables (B _p and B _z ) in the singular radial slice demarcated by the dashed red-lines in Figure 1. B _p can be converted from cylindrical (CYL) coordinates to Cartesian (CAR) coordinates and vice versa:

where w is used to select the correct sign of B _p and is defined as:

[0 if sign(x)=sign(¾) or sign(y)=sign(^)

(10)

1 otherwise

For completeness, converting between cylindrical and Cartesian coordinates for p is

B. Artificial Neural Network (ANN) Field Modelling

By exploiting the axisymmetry of the magnetic field and operating in cylindrical coordinates, an artificial neural network (ANN) can be harnessed to model fields with high accuracy and low computational overheads. A flowchart outlining the ANN based magnetic field modelling process is illustrated in Figure 2. Here the spatial coordinate of interest p, expressed in Cartesian format, is transformed into cylindrical coordinates with (11 ). Θ and B _e are omitted because the field is axisymmetric. This reduces the required ANN mapping to a simple 2-input-2-output network architecture, in contrast to a potential 3-input-3-output ANN architecture when operating in Cartesian coordinates. After evaluating the ANN estimated B _p and B _z , B _p is transformed back into Cartesian format using (9). Back propagation ANNs (with architecture of j hidden layers, k hidden nodes per hidden layer) can be trained with Levenberg-Marquardt supervised learning algorithm to fit a desired set of inputs to a corresponding set of outputs by iteratively adjusting the weighting coefficients in the network to minimize the root mean squared error (RMSE) over N data pairs. In this case, the inputs are the two spatial cylindrical coordinates p and z, and the outputs are the two non-zero magnetic flux densities B _p and B _z . Mathematically, the neural network can be represented as

where g(°) is the activation function, ω is the weight function, x, is the - ^h input (p,z) and y _k is the output (Bp, B _z ). The number in parenthesis signifies the layer. The RMSE, which is used for ANN and dipole model evaluation, is expressed by

where v (1 < v≤ N) is an integer representing the training set index. In order to map spatial positions to magnetic flux densities, the selected 'L' shaped domain (the area in Figure 4 surrounding the PM) is discretized into a uniform spatial grid to generate N data sets. If the spatial spacing in the grid is d (d may be optimised for accuracy verses computation effort eg: 0.1 mm), N is related to the other spatial variables via the following expression

N = \ + (W + H)l d + (WH-R,L I 2) l d ² (14)

C. Model-based Localization

With a magnetic field model, the position of the magnetic sensor relative to the magnetic source can be determined by assigning a cost function C to be the difference between the observed/measured magnetic field and the model (DM, DMP, ANN) predicted field. For a sensor located at x _s , this can mathematically be expressed as follows:

By minimizing this cost function through an iterative nonlinear least-squares optimization, the relative position of the sensor can be estimated from the field measurement.

II. EXPERIMENTAL INVESTIGATION & DISCUSSION

Figure 3 shows a MMS-1 -R magnetic field mapping system (SENIS GmbH, Zurich, Switzerland) which was used to obtain experimental magnetic field data of cylindrical magnets for evaluation of the ANN and dipole based field models. These experimentally derived models were then utilized for positional tracking analysis. The MMS-1-R consists of an xyzQ Cartesian moving platform (resolution of 10 μιη) on which a 3-axis SENIS GmbH hall probe (± 1000 mT, > 0.1 % accuracy) is mounted. The magnetic flux density measured by the hall probe is processed by the SENIS 3-axis 03A02F magnetic field transducer. A computer is connected to an NI-6212 DAQ system (National Instruments, Austin, TX) that controls the SSMD1 stepper motors and receives data from the transducer. The zero gauss chamber is used for calibrating the hall probe.

Two axially magnetized N52 grade Neodymium permanent magnets (KJ Magnetics, Jamison, PA), a solid cylinder (D86-N52) and an annular cylinder (RC44-N52), were separately mounted on the MMS-1-R using non-ferromagnetic brackets and scanned. For a specified Θ slice, the probe position and B _p in Cartesian coordinates were recorded at 0.1 mm increments as the hall probe moved along p. Once the boundary of the scanning domain was reached, the z position of the probe changed by 0.1 mm, and the probe retraced its movement in p direction. The scanning range of the ' .' shaped domain was set to four times the characteristic lengths of the magnet, meaning W= Ri and H = 4(L/2), rounded to the nearest 0.1 mm. Due to physical dimensions of the probe, measurements taken closest to the magnet were 1.1 mm from the top surface of the magnet and 1.3 mm from the side. The scanning domain for the solid and annular magnet comprises of 44,475 and 44,268 data points respectively.

The magnet may be any shape. For most applications, especially clinical, the desired geometry is usually something that is axis-symmetric (cylindrical) so that it can fit into tubing/instruments. For strength, it varies from application to application but embodiments are able to take into consideration of this aspect.

The magnet may be fabricated by fusing of rare-earth magnets into one geometry. An alloy of neodymium, iron and boron to form the Nd ₂ Fe ₁₄ B tetragonal crystalline structure may be used.

To minimize experimental measurement errors and to verify symmetry of the magnetic fields, each magnet was scanned 8 times at random Θ (9=0°, 47°, 112°, 140°, 169°, 313°, 315°, 354°), and these 8 'L' domain slices were consolidated to create an average field (containing both B _p and B _z ) slice which is used to fit the dipole and ANN field models. From these 8 slices, the field variance, characterized by the standard deviation is computed for both magnets and tabulated in Table I. For both magnet shapes, the average variation among the 8 randomly obtained slices for B _p and B _z was around 2 mT, which is in the same order of magnitude with the accuracy of the hall probe. This provides experimental verification that the magnetic field of the cylindrical magnets is sufficiently axisymmetric and validates the approach to operate in cylindrical coordinates. The maximum deviation however does reach above 20 mT in certain isolated locations, which can be due to physical imperfections of the magnet. An observation to note is that the annular magnet exhibits a slightly higher maximum standard deviation in Bp and this can be attributed to the hollow region of the magnet. The experimental contour plots of the averaged '/.' domain for both magnets are shown in Figure 4. The plots in Figure 4 clearly demonstrate the significant difference between the magnetic field around a solid and annular cylindrical magnetic source, which is especially evident near the surface.

TABLE I. STATISTICAL ANALYSIS ACROSS 8 'L' DOMAIN SLICES

Using MATLAB's optimization toolbox (Mathworks, Natick, MA), the experimental average field slice was used to fit the dipole and multi-pole models based on Levenberg-Marquardt algorithm (LMA). The fitted parameters for the MD and DMP models, as well as the magnet parameters, are consolidated in Table II. For the ANN-based field model, the number of hidden layers was set to 1 (/=1) and the RMSE of the ANN fit was recorded for B _z of the solid magnet as the number of hidden nodes k varied. The results, illustrated in Figure 5, clearly show an asymptotic decline of RMSE once k=20 is reached. A similar feature is also present for B _p as well as for the annular magnet. Hence a single hidden layer ANN with 20 hidden nodes is selected for field mapping.

In order to calibrate or train the system for a given magnet, the first step is to obtain a magnetic field map of the target/unknown magnet. This can be quickly and easily done by placing the magnet onto a magnetic field camera (http://www.magcam.com/). Next, by extracting the axis-symmetric field slice, using automated symmetry analysis, this slice can be used to train/fit the desired ANN model. The number of hidden nodes is slowly increased and the residue error of the fit is actively monitored. Once it is detected that the increase in the number of hidden nodes only reduces the residue error less than the minimum threshold, the procedure stops and the ANN model is complete. At each iteration, the stoppage of ANN training is also governed by standard ANN training and stopping criteria. Hence all steps are automatic except the insertion of the magnet and entering the dimensions of the magnet. The output will be the ANN field model, with all the weights/coefficients of the model available for use.

TABLE II. MAGNET SPECIFICATIONS AND FITTED MODEL PARAMETERS

A. Comparison among Field Models

With the fitted parameters of the MD and DMP models in Table II, as well as the weighting coefficients of the ANN {k=20), the absolute field error between the field model prediction and actual experimental data can be computed at each spatial coordinate for both the solid and annular cylindrical magnet. The absolute field error for B _p and B _z illustrated using contour plots are shown in Figure 6, Figure 7and Figure 8 for the MD, DMP and ANN field models respectively. In Figure 6, it can be observed that the MD model adequately represents the field at locations far from the magnet surface (error is close to zero). However, at locations near the surface (1-2 mm) of the solid magnet, the error exceeds 200 mT and 350 mT for B _p and B _z respectively. Comparing this to the experimental field values in Figure 4, it represents a percentage error that exceeds 50%. For the annular magnet, the absolute errors are even higher (more than 250 mT for B _p and over 500 mT for S _z ) and more extensive as shown by the lighter areas that extend into space in the right column of Figure 6.

Direct spatial comparison between the contour plots of Figure 6 and Figure 7 suggest that the DMP model, with its additional parameter /, offers an improvement over the MD model in both magnets. For the solid magnet, the absolute errors are now restricted to 160 mT and 260 mT for Bp and B _z . For the annular magnet, while the peak of absolute errors of the DMP model is still comparable to the MD model, the areas of high absolute errors (lighter areas of the contour plot) have receded.

Finally Figure 8 depicts the spatial distribution of the absolute error of the ANN field model for both geometries. There are two important observations:

• The peak absolute errors of B _p and B _z for both geometries do not exceed 40 mT at all spatial points.

• The entire ' .' domains are almost completely dark (representing low absolute errors). Only isolated light spots are present.

To facilitate further comparisons, Table III summarizes the RMSE of the various field models for the two types of magnets. As expected, the MD performed the worst with an RMSE of 9.31 mT and 27.2 mT for the solid and annular shaped magnet respectively. With the ability to adjust the separation of the dipole, the DMP model fared slightly better. The ANN model however possess an impressive RMSE of just 0.84 mT for the solid magnet (about 1 order of magnitude less than DM model) and a 0.75 mT for the annular magnet (improvement by more than 1 order of magnitude from the DM model). From these observations, it is obvious that unlike the ANN, both the DMP and MD models are unable to adequately model the high degree of non-linearity of the magnetic field that occurs close to the magnet surface. While the DMP and MD models are highly sensitive to magnet geometry, the ANN model was unaffected.

TABLE III. COMPARISON OF RMSE ACROSS FIELD MODELS AND MAGNET GEOMETRIES

B. Trajectory Tracking Performance

To evaluate the tracking performance of each of the magnetic field models, the probe on the MMS-1-R was programmed to follow a pre-determined figure '8' trajectory (radius of 6 mm) in the XY and XZ planes centered about the axial axis of the stationary solid cylindrical magnet. The Cartesian coordinates and magnetic field measurements of the probe were taken at 360 points. These field measurements (S , B _y , B _z ) taken by the sensor on the probe were converted into cylindrical coordinates from Cartesian coordinates with (9). Utilizing the Levenberg-Marquardt algorithm to minimize the cost function in (15) , the probe position (p, z) was estimated based on the MD, DMP and ANN models. Thereafter, Equation (9) was also used to determine the angular data Θ needed to reconvert the calculated p and z back to Cartesian coordinates (x,y,z) for comparison purposes using (11 ). Figure 9 spatially illustrates and compares the tracking results. From visual comparison, the ANN outperforms the MD model as it is able to trace the figure '8' trajectory better. Statistical analysis of the tracking errors for both XY and XZ plane paths, which includes the DMP model, is summarized in Table IV.

As with the field model comparison, the ANN based trajectory tracking performed the best with average tracking errors of less than 1 mm for both paths. The MD model exhibited an average tracking error of 1.7 mm and surprisingly the DMP model faired poorer than the MD model with an average tracking error of over 2 mm. This anomaly can be attributed to the selected path which may have coincidently passed through isolated locations where the MD model outperforms the DMP model.

It is worth noting that the larger tracking errors correlate with the distance from the source as seen in Figure 9. This is to be expected because of the reduced signal-to-noise ratio (SNR) which has a detrimental effect on model-based localization. A network of sensors may be used to boost (SNR) and improve the nonlinear least-squares optimization algorithm performance. Hence as only 3 sensing axes were used in this experiment, the tracking performance can be further improved via additional sensors.

TABLE IV. COMPARISON OF TRACKING ERROR USING DIFFERENT FIELD MODELS FOR SOLID MAGNET

III. APPLICATIONS

One or more embodiments may be used in a magnetic tracking system to assist in nasogastric (NG) tube insertion. Currently this procedure is done blind and X-Ray is used as definitive confirmation into the stomach. This system will consist of the nasogastric tube that is essentially unmodified except for the inclusion of a permanent magnet that is hermetically sealed at the tip of the NG tube. Each NG tube and magnet will characterized at the factory and the parameters required by the neural network modeling is embedded in a bar code that is printed on the wrapper of the NG tube.

The patient will be located beside a magnetic sensing unit and connected to a processing unit. This processing unit will consist of circuitry to acquire the electrical signals from the sensors and convert them into digital signals. The localization algorithm will be run on an embedded computer for computational speed to achieve real-time localization.

After the parameters are entered (by scanning) into the processing unit, the tube can be inserted into the patient. Due to the close proximity of the NG tube (carrying magnet) and the sensors, motion and position of the NG tube can be localized. This localization information can be broadcast over current bed-side medical screens in existing bedside monitors (where you see blood-pressure, pulse, etc).

In a further example, the contents of Singapore Patent Application No. 2013087366, "Apparatus for Real-time Non-Invasive Magnetic Field-Based Localization of Nasogastric Tube." Filed 21 Nov 2013, are incorporated herein by reference.

Other applications include medical guidance, minimally invasive surgery, biomechanical analysis and kinesiologic studies, high precision electric motor/actuator control and robust displacement and force sensing systems.

Embodiments may incorporate specificity and imperfection of the source, which is not possible with the magnetic dipole model. This may result in a highly accurate field model that can be capitalized for precise localization. In experiments using a solid and annular magnets, the ANN based magnetic model performed 10 times better in representing magnetic field than the traditional single dipole based models, thus enabling position tracking accuracy of less than 1 mm with only 3 sensing axes.

While example embodiments of the invention have been described in detail, many variations are possible within the scope of the invention as claimed as will be clear to a skilled reader.