Background to the Invention In many signal processing applications, the sample signals provided by suitable detectors are mixtures of many unknown signals from unknown sources. The "separation of sources"problem is to extract at least one of the unknown signals from such a mixture.

Typically, the signal sources as well as the characteristics of the combined mixture signal are unknown. Such signal processing, without knowledge of the signal sources, other than the general statistical assumption of source independence, is known within the art as the"blind source separation. problem".

The blind source separation problem can arise and have utility in variety of contexts such as, for example the separation of radio or radar signals sent by an array of antennas, separation of odours in a mixture by a suitable sense array, the parsing of the environment into separate objects by our biological visual system and the separation of biomagnetic sources by a super conducting quantum interference device array in magnetoencephalography. The blind source separation problem finds particular application in, for example, sonar array signal processing and signal processing

within a telecommunications system and in particular within a cellular telecommunication system. An example of a context in which a blind source separation problem may arise is two people speaking simultaneously with each person being a different distance from two microphones.

Each microphone records a linear mixture of the two voices. The separation of those voices from the combined signal without knowing the characteristics of the source of the signals would represent such a blind source separation problem.

Blind signal processing and blind source separation are terms used within the art for the recovery of unknown source signals from a received mixture of a composite signal. US 5,706,402 discloses a"Blind signal processing system employing information maximisation to recover unknown signals through unsupervised minimisation of output redundancy". A neural network system and unsupervised learning process is disclosed for separating unknown source signals from their received mixtures by solving a blind source separation problem. The unsupervised learning procedure solves the general blind source signal processing problem by maximising joint output entropy through gradient ascent. The neural network system can separate a multiplicity of unknown source signals from measured mixture signals where the mixture characteristics and the original source signals are both unknown. However, the system disclosed in US 5,706,402 does not provide an analytical solution and accordingly, is a relatively slow process.

It is an object of the present invention at least to mitigate some of the problems of the prior art.

Summary of Invention Accordingly, a first aspect of the present invention provides a signal processing method (and system) for deriving a signal from a mixture signal comprising a combination of a plurality of independent signals, the method comprising the steps of storing a first data set (x) representing sampled digital data of the mixture signal ; defining a measure of the predictability of the first data set, the measure being a function of the first data set (x) and a first operand (Wi) ; processing the first data set to produce an output signal (yi) which results from a critical point of a variation of the measure of predictability with the first operand such that the measure of predictability of at least one of the source signals is equal to or greater than the measure of predictability of the sampled mixture signal Preferably, an embodiment provides a method wherein the definition of the measure of predictability is where yT is the value of the signal y at time T, y, is a short-term moving average of values of y ; Yr is a long- term moving average of values of y; U, the denominator, is a measure of the extent to which yT is predicted by the short-term moving average, Yr s of the past values of y ;

and V, the numerator, is a measure of the variability of y in terms of the extent to which yT is predicted by the long-term moving average, Yr r of the past values of y.

An embodiment provides a method wherein the short- term moving average of the values of y is given by ##=#S#(#-1)+(1-#S)y(#-1) where 0##S#1, where #S is arranged to have a first predetermined half-life, hs and in which the long-term moving-average of the values of y is given by ##=#L#(#-1)+(1-#L)y(#-1) where 0##L#1, where iL iS arranged to have a second predetermined half-life, hL. The relationship between a half-life, h, and the parameter A, is given by #=2-1/h.

Preferably, the second predetermined half-life is longer than the first predetermined half-life and, more preferably, the second predetermined half-life is at least 100 times longer than the first predetermined half- life.

A second aspect of the present. invention provides a method (and system) wherein the definition of the measure of predictability is where a scalar signal, Yir is formed from the application of a 1xM matrix, Wi, to a vector, x, representing potential or actual linear mixtures of the first data set, C is an MxM long-term covariance matrix of x, and C is a short-term covariance matrix of x.

Preferably, an embodiment provides a method in which the long-term covariance, Cy, between the ith and jth

mixtures of x are such that and Advantageously, the values (xi#-xi#) and (xi#-#i#) only needs to be calculated once for a given set of mixture signals.

The technique of gradient ascent can be used to determine the critical points (e. g. saddle points).

Suitably, an embodiment provides a method further comprising the step of maximising F with respect to Wi, 2Wi 2Wi whichgives #WiF=#-#.<BR> <P> V U Preferably, the determination of critical points is an interative process. Accordingly, an embodiment provides a method further comprising the steps of iteratively optimising F with respect to Wi, using Wi=Wi+##WiFr where 77 has a predeterminable value. A preferred value of Q is 0.001 Having determined the maxima, an embodiment further comprises the step of calculating yi=Wix from the value of Wi for which F was at a maxima.

Ultimately, an embodiment may output the recovered signal in a physical form. Accordingly, an embodiment provides a method further comprising the step of outputting a signal based on Yi- Preferably, an embodiment provides a method further comprising the step of calculating eigenvectors, Wi, for the critical points of F. The step of calculating the eigenvectors, Wi, comprises calculating the eigenvalues,

#=V/U, where #WiF=0, which gives Wi#=V/UWi#, which, in terms of the eigenvalues, i, gives WiC=AWiC.

Advantageously, embodiments of the present invention provide a solution to an eigenvalue problem, which if the number of eigenvalues is less then 5, is an analytical solution to the problem of blind source separation by utilising temporal predictability.

The result of voice mixtures described above exemplify three universal properties of linear mixtures of statistically independent source signals which are: (1) temporal predictability (conjecture): the temporal predictability of any signal mixture is less than (or equal to) that of its components or signals, (2) Gaussian probability density function: the central limit theorem ensures that the extent to which the probability density function (pdf) of any mixture approximates a Gaussian distribution is greater than or equal to any of its component source signals, and (3) statistical independence: the degree of statistical independence between any two signal mixtures is less than (or equal to) the degree of independence between any two source signals. Property 2 forms the basis of projection pursuit as disclosed in Freedman, JH. 1987, exploratory projection suit. J amer. Statistical association, 82 (397), 294-266. Properties 1 and 2 are assumptions underlying independent component analysis as disclosed in, for example, Jutten & Herault, 1998, independent component analysis-v-PCA, pages 643-646 of: Proc.

Eusipco. and Bell, AJ and Sejnowski, TJ, 1995, An information-maximisation approach to blind separation and blind deconvolution, Neural Computation, 7,1129-1159

the entire contents of which are incorporated herein for all purposes. All three properties are generic characteristics of signal mixtures. However, unlike properties 2 and 3, property 1, that is, temporal predictability forms the only basis of blind source separation for the embodiments of the present invention.

The embodiments of the present invention are based upon the above temporal predictability conjecture which is that the temporal predictability of any mixture is less than (or equal to) that of any of its component signals. Examples will be given below of the use of the temporal predictability conjecture for separation of physical signals, such as, for example, voices and music.

Problem definition and temporal predictability Before describing an embodiment of the present invention, the theory upon which the embodiments are based will now be described.

Consider a random vector s = (slls21 sK) t of K statistically independent source signals, where the ith row in s is a signal si measured at n time points and the superscript t denotes a transpose operator. In the theory presented hereafter it is assumed that the source signals are statistically independent unless otherwise indicated. A random vector x= (x1#x2#...#xM)t of M > K linear mixtures of signals in s can be formed with an M x K mixing matrix A such that x = As. If the rows of A are linearly independent then any source signal si can be recovered from x with a 1xM matrix, Wi, such that si=Wix.

The blind source separation problem to be addressed here

consists in finding an unmixing matrix W = (W1|W2|-|WK) t such that each row vector Wi recovers a different signal Yi, where yi is a scaled version and/or sign reversed version of a source signal si.

The embodiments of the present invention for recovering source signals are based on the following conjecture: the temporal predictability of a signal mixture xi is usually less than that of any of the source signals that contribute to xi. For example, the waveform obtained by adding two sine waves of different frequencies is more"complex"than either of the original sine waves. A measure of temporal predictability, F (Wi, x), is defined and used to estimate the relative predictability of a signal, yi, recovered by given a matrix, Wi, where yi=Wix. If source signals are more predictable than any linear mixture, Yi, of those signals,. then the value of Wi which maximises the predictability of an extracted signal, yi, should yield a source signal, that is, Yi = csi, where c is a constant. Information theoretic analysis of F shows that maximising the temporal predictability of a signal amounts to differentially maximising the power of the Fourier components with low (non-zero) frequencies see, for example, Stone J V, 1996a, a Canonical microfunction for learning perceptual invariances, Perception, 25 (2), 207- 220 the entire contents of which are incorporated herein for all purposes. In an embodiment, the definition of signal predictability, F, used herein is

where y# is the value of the signal, y at time r.

The term U measures the extent to which y# is predicted by a short-term moving average, y, of values in y. In contrast, the term V is a measure of the overall variability in y, as measured by the extent to which yz-is predicted by a long-term moving average, ##, of values in y. The predicted values yr and Yr are both exponentially weighted sums of signals values measured up to time, such that recent values have a larger weighting than those in the distant past. Therefore, it <BR> <BR> <BR> <BR> follows that ##=#s#(#-1)+(1-#S)y(#-1) where 0##S#1<BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> Yr =#L#(#-1)+(1-#L)y(#-1) where 0 ##L#1 The half-life hL Of ? L iS much longer (typically of the order of 100 times longer) than the corresponding half-life hs of Rs It should be noted that maximising only V would result in a high variance signal with no constraints on the temporal structure of the signal. In contrast, minimising only U would result in a DC signal.

It will be appreciated that in both cases, trivial solutions would be obtained for Wi because V can be maximised by setting the norm of Wi to be large, and U can be minimised by setting |W|= 0. In contrast, the ratio V/U can be maximised only if two constraints are both satisfied which are (1) y has a non-zero range, that is, a high variance, and (2) the values in y change relatively slowly over time. It should also be noted that the value of F is independent of the norm of Wi, so that only changes in the direction of Wi effect the value of F.

Extracting Signals by Maximising Signal Predictability Consider a scalar signal, Yi, formed from the application of a 1xM matrix, Wi, to a random vector, x.

Given that yi = Wix, equation (1) can be written as: Wi#Wit F=log @@@@ (3)<BR> <BR> <BR> WKiCWi' where C is an MxM long-term covariance matrix of signal mixtures and C is a corresponding matrix of short-term covariances. A long-term covariance C ij and a short-term covariance C ij between an ith and jth mixtures are defined as : It should be noted that the embodiments advantageously only need to compute C and C once for any given set of signal mixtures and that the terms (xi#-#i#) and (xi-xir), can be precomputed using fast convolution operations such as are disclosed in, for example, Eglen, S, Bray, A, and Stone, J V, 1997, Unsupervised discovery of invariances, Network, 8,441-442, the entire contents of which are incorporated herein for all purposes.

A gradient ascent on F with respect to Wi can be used to maximise F, thereby maximising the predictability of yi. The derivative of F with respect to Wi is : <BR> <BR> <BR> <BR> <BR> <BR> 2Wi 2Wi<BR> #wiF=#-#<BR> <BR> <BR> V U (5) An optimisation procedure is applied which consists of iteratively updating Wi until a maximum F is located such that W, =W, +77VWF, where il is a small constant that is, preferably, 0.01. It will be appreciated that the function F is a ratio of quadratic forms. Therefore, F has exactly one global maximum and exactly one global minimum, with all other critical points being saddle points. Therefore, the gradient ascent is guaranteed to find the global maximum in F.

Once a single source signal has been extracted, the repeated application of the above procedure to a single set of mixtures extracts the same (most predictable) source signal. Therefore, it will be appreciated that embodiments are required to ensure that different source signals are extracted. Preferably, the embodiments are arranged to achieve this aim using an unmixing matrix W obtained as the solutions to a generalised eigenproblem defined as follows: the gradient of F defined in equation 5 is known to be zero at an eigenvalue. Therefore, at an eigenvalue solution Wi#=V/UWi# (6)

WEC=AWiC (7) It can be appreciated that equations 6 and 7 have the form of a generalised eigenproblem, where the eigenvalues, X = V/U, provide the solution. Critical points in F correspond to values of Wi that satisfy equation 7 where Wi's are the corresponding eigenvectors.

The first such eigenvector defines a maximum in F and each of the remaining eigenvectors define saddle points in F. It should be noted that these eigenvectors are orthogonal in the metrics defined by C and C which implies that different Wi's are linearly independent.

Borga M 1998, Learning multidimensional signal processing, Linkoping University, which is incorporated herein for all purposes, discloses a review of generalised eigenproblems.

It will be appreciated by those skilled in the art that such problems typically have scaling characteristics of O (N3), where N is the number of signal mixtures, that is, the speed of the determination of solutions varies with the cube of the number of independent sources. Once the eigenvectors have been obtained, all K signals can be recovered using y = W x, where each row of y corresponds to exactly one extracted signal, yi, which, as indicated above, represents a scaled version of a corresponding independent source signal si.

Preferably, an embodiment provides, in circumstances where the number of mixtures, M, is greater than the number of sources, K, for the reduction of M using principal component analysis as is well known within the

art. Principal component analysis is used to reduce the dimensionality of the signal mixtures by discarding eigenvectors of x which have eigenvalues close to zero.

Brief Description of the Drawings Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which: figure 1 depicts three signal mixtures to be processed by an embodiment of the present invention; figure 2 shows three signals having different probability density functions that were used to synthesise the three signal mixtures shown in figure 1 ; figure 3 illustrates signals corresponding to two male and two female voices where the original source mixed signal is shown by continuous lines and corresponding recovered signals shown as dotted lines ; figure 4 shows a flow chart for implementing a method according to an embodiment of the present invention; figure 5 shows two signals comprising an original signal, Si, denoted by the solid line and a convolved signal, xl, denoted by the broken line, for 100 time steps processed according to the present invention; and figure 6 depicts the correlation between an original source signal, s2, and a deconvolved signal, c2.

Description of the Preferred Embodiments Referring to figure 1 there is shown three mixed signals 100,102 and 104 each comprising 3000 samples (although only the first 1000 samples of each mixture are

shown in figure 1). The three signals were derived from the signals 200 shown in Figure 2. The three source signals, s = {si, s, S3} , shown in figure 2 represent a super-Gaussian signal (such as the sound of a train- whistle) 202, a sub-Gaussian signal, such as, for example, a sine wave, 204 and a sorted Gaussian noise signal 206. Each of the signals 202 to 206 were used to synthesise the mixture signals 100,102 and 104 shown in figure 1 using a random matrix A applied to the set of three signal mixtures given by a x = As.

During the processing of the mixture signals 100 to 104, the K source signals, where K = 3 in the present embodiment, were used to generate M = K mixture signals.

Again M = 3, using a KxK mixing matrix (A) and the M mixture signals were used as input signals for an embodiment. Preferably, each mixture signal was normalised to have zero-mean and unit variance. Each mixing matrix was obtained using, for example, a random number generator function. The short-term and long-term half-lives defined in equation 2 above were set to hs = 1 and hL = 9000, respectively. Referring again to figure 2, the signals shown by dotted lines, that is, signals 208, 210 and 212 represent amplitude or dc shifted recovered signals which, as can be seen from figure 2, correlate strongly with the original source signals notwithstanding having been mixed by a random matrix A. Table 1 below shows the correlation between the source signals and the signals recovered from the mixtures of the source signals with different probability density functions.

Table 1 Source 1 Source 2 Source 3 Recovered 1 0. 0003 0. 0011 1. 0000 Recovered 2 1. 0000 0. 0000 0. 0000 Recovered 3 0. 0417 0. 9991 0. 0016

It can be appreciated that the three recovered signals each had a correlation of r>0.99 with only one of the original source signals whereas the other correlations were close to 0. Therefore, embodiments of the present invention can be used to separate mixtures of source signals that have different probability density functions.

An embodiment of the present invention was applied to the separation of sounds where 50, 000 data points were sampled at a rate of 44100 Hz, using a microphone to record different voices from a VHF radio using a computer. Two sets of 8 sounds were recorded. The two sets represented voice and music. The voices were a mix of male and female voices. The music was classical, with and without accompanying singing. An embodiment was tested on mixtures of normal speech. Referring to figure 3 there is shown a number of signals 300 which show as complete lines, the original source signals 302 to 308 and amplitude or dc shifted versions of the recovered signal 314 to 316 as dotted lines. Table 2 below shows the correlation between each of the four source signal (voices) and every signal recovered from the method. It can be appreciated that each source signal has a high correlation with only one recovered signal.

Table 2 Source 1 Source 2 Source 3 Source 4 Recovered 1 0.. 0973 0. 9938 0. 0281 0. 0488 Recovered 2 0. 9963 0. 0809 0. 0124 0. 0191 Recovered 3 0. 0015 0. 0419 0. 9946 0. 0953 Recovered 4 0. 0295 0. 0756 0. 1014 0. 9916

Table 3 below shows the correlation between each of eight source signals (voices) and every signal recovered using an embodiment. Again, it can be appreciated that each source signal has a high correlation with only one recovered signal. It can be appreciated that with correlations of this magnitude, it is not possible to discern audibly the difference between the original and the recovered speech signals. It can be seen from Table 3 that a correlation of 0.956 was found between source signal 8 and the recovered signal 3. This, in this example, represents a worse case performance of the method for all data processed according to the above- described embodiments of the present invention.

Table 3 Source Source Source Source Source Source Source Source 1 2 3 4 5 6 7 8 Recovered 1 0. 0014 0. 0076 0. 0037 0. 0026 0. 0278 0. 0456 0. 9881 0. 1430 Recovered 20. 99360. 01280. 00290. 00130. 00110. 01720. 01550. 1093 Recovered 3 0. 1790 0. 0008 0. 1612 0. 0110 0. 0366 0. 1020 0. 1338 0. 9560 Recovered 4 0. 0150 0. 0118 0. 0065 0. 9990 0. 0240 0. 0317 0. 0044 0. 0044 Recovered 5 0. 0035 0. 0204 0. 9934 0. 0002 0. 0207 0. 0082 0. 0074 0. 1088 Recovered 6 0. 0103 0. 0034 0. 0264 0. 0176 0. 0208 0. 9919 0. 0442 0. 112 Recovered 7 0. 0267 0. 9994 0. 0116 0. 0015 0. 0003 0. 0102 0. 0088 0. 0021 Recovered 8 0. 0151 0. 0027 0. 0269 0. 0215 0. 9979 0. 0197 0. 0208 0. 0431

The embodiments were also tested using mixtures of eight segments of music. The correlations between the source signals of the eight segments of music and the recovered signals for the eight segments of music are shown in Table 3. It can be appreciated, again, that the correlations are approximately R = 0. 99 and it is not possible to discern audibly the difference between the original and recovered music signals.

It has been found that the embodiments are largely insensitive to the values used for the short-term and long-term half-lives defined in equation 2 above providing the latter is much greater than the former.

It will be appreciated from the above that the method used in the embodiments is based on the assumption that different source signals are associated (via Wi) with distinct critical points in F. However, if any two source signals have the same degree of predictability, F, then two eigenvectors Wi and Wj will also have equal eigenvalues (and are associated with the same critical points in F). Therefore, any vector Wk which lies in the plane defined by Wi and Wj also maximises F, but Wk cannot, in general, be used to extract a source signal. This can be demonstrated by creating two mixtures, x = A s, from two signals s1 and s2, where si is a time reversed version of s2. Even though Si and s2 have different time courses, they share exactly the same degree of predictability F and cannot be extracted from the mixture x using this method. In practice, however, signals from different sources typically can be separated because each source signal has a unique degree of predictability.

Furthermore, every set of signals in which each signal is

from a physically distinct source has been successfully separated using embodiments of the present invention.

Although the above embodiments have been described with reference to processing single dimension data, the present invention is not limited thereto. The method can equally well be utilised to realise embodiments which process spatially distributed or N-dimensional data, that is, the method can be applied to realise embodiments that operate in the spatial domain. The definition of predictability can be generalised by replacing the exponentially weighted means y and Y, with general functions f and g respectively, where the values of f and g at time T are non-linear functions of y computed up to time. These functions may implement more accurate predictions of yX for specific signal types.

It will be appreciated that if all three properties listed above apply to any statistically independent source signals and their mixtures, an embodiment can be realised which relies on constraints from all of these properties to deal with a wide-range of signal types. It is widely acknowledged that ICA forces statistical independence on recovered signals, even if the underlying source signals are not independent. Similarly, the embodiments of the present invention may impose temporal predictability on recovered signals even where none exists in the underlying source signals. Therefore, an embodiment which incorporates constraints from all three properties should be relatively insensitive to violations of the assumptions upon which the embodiments are based.

A framework for incorporating experimentally relevant constraints based on physically realistic properties has

been formulated in the form of weak models, see, for example, Porrill, J, Stone, J V, Berwick, J, Mayhew, J, and Coffey, P, Analysis of Optical Imaging Data using Weak Models and ICA, in: Girolami M, (ED) Advances in Independent Component Analysis, Springer-Verlag, which is incorporated herein by reference for all purposes.

It will be appreciated that an application of the present invention may be in the analysis of medical images and EEG data. Alternatively, the present invention could be applied within a hearing aid context in which a selected signal, voice signal, having predeterminable characteristics can be separated from a mixture of voice signals.

Referring to figure 4, in broad terms, there is shown a flow chart 400 for implementing a method according to an embodiment of the present invention. A measure of predictability, F, is defined at step 410 in terms of covariance matrices and at least one unmixing matrix, Wi. Preferably, the definition relates to the definition of F above. Data samples representing a sampled mixture signal which, in turn, is a combination of a plurality of independent source signals is received at step 420 and, optionally, stored in a memory of a digital signal processor. The expression representing the predictability measurement is optimised to determine local critical points in terms of the unmixing matrix, Wi, and the given data samples at step 430. The eigenvalues and, in turn, eigenvectors, of the optimised predictability measurement are determined at step 440 using, preferably, an optimisation procedure. Step 450

calculates using the above equations an independent signal from at least one of the eigenvalues.

Although-the embodiments have been described above in relation to a method of processing digital data samples derived from or representing physical signal measurements, the present invention is not limited thereto. Embodiments of the present invention encompass a digital signal processing system or processor that is arranged to implement the above-described methods as well as computer program elements and computer program products for implementing such methods or comprising software for implementing such methods.

Although the above embodiments have been described with reference to recovering source signals and are based on the following conjecture: the temporal predictability of a signal mixture xi is usually less than that of any of the source signals that contribute to xi, the present invention is not limited thereto. Embodiments can be realised in which the continuity. predictability of a signal is used as the measure of predictability such that the continuity predictability of a signal mixture is less than that of any of the source signals that contribute to the mixture signal. Accordingly, the present invention is not limited to temporal continuity and encompasses spatial continuity and n-dimensional continuity.

Deconvolution The above embodiments have been described with reference to the separation of blind sources. However, the present invention is not limited thereto.

Embodiments can be realised in which deconvolution of signals can be performed.

As will be appreciated by one skilled in the art, a signal can be altered by the effects of the environment in which the signal propagates. For example, a room typically causes speech signals to be heard with attenuated high frequencies and with multiple echoes.

The room effectively acts as a filter on the signal.

Such effects can be reversed using appropriate deconvolution filters. Suitably, embodiments of the present invention are arranged to implement deconvolution filters using the above described temporal predictability.

Firstly, the mathematics underlying these embodiments of the present invention will now be described. Consider a filtered version, x, of a signal, s, such that: <BR> <BR> <BR> <BR> <BR> x=a*s (1) where a is a filter a = [al, az,...]. The convolution operator * is defined by : xt=α1st-1+α3st-2+... (2) where the subscript t denotes time. If a deconvolution filter, ß = [ß1, ß2,...], exists for a then y = s can be recovered from x using Y=) g*x (3)

However, in practice, the correct deconvolution filter cannot be obtained exactly, so that y # s.

Therefore, the embodiments of the present invention use the deconvolution strategies discussed below.

A strategy for Deconvolution The strategy used to estimate the deconvolution filter B will be described for a simple example.

Consider an IIR filter, a, which is a low-pass filter: =e-nlT(4) where T is the half-life of the exponential. The effects of such IIR filters can be reversed using a simple FIR deconvolution filter: [1,-e-1/T] A low-pass filter, such as a, effectively smooths s to yield x = α*s, that is the relatively high frequency components are removed from s. This smoothing operation inevitably makes x more predictable than s. In order to emphasise this, consider the limiting case as T##. In the limit, x has a constant value, and is therefore very predictable. It follows that a deconvolution filter which makes x less predictable may be able to approximately recover s.

Therefore, ? is defined as that filter which at least reduces, and preferably minimises, a measure of predictability of y =a*x. The differences between the

mathematics for deconvolution according to embodiments of the present invention and the above-described blind- source separation is as follows. Given a set of signal mixtures, a linear combination of those mixtures usually exists such that the original signals can be recovered.

It was shown above that such a linear combination could be obtained by maximising the predictability of extracted signals. In contrast, it is shown below that minimising predictability is sufficient to deconvolve a signal.

The convolution operations defined above need to be reformulated in terms of mathematically equivalent vector-matrix operations. Consider a deconvolution filter ß = = [Pl, P2], such that y = P*x. This can be re- written in vector-matrix notation y (3x by defining a vector variable x as: x={x#xz-1}t (6) where each row of x is a time-shifted version of x, and the superscript t is the transpose operator. The shift operator Z-k is defined by Xt-k = xtz-k (7) The convolution y=ß*x and the vector matrix multiplication y=ßx are exactly equivalent.

Measuring signal predictability The definition of signal predictability F used here is :

where yT = ßx# is the value of the signal y at time #, and xt is a vector of K signal mixture values of time T.

The term Ui reflects the extent to which y# is predicted by a short-term"moving average" y# of values in y. In contrast, the term Vi is a measure of the overall variability in y, as measured by the extent to which yX is predicted by a long-term"moving average"y, of values in y. The predicted values ## and y, of yX are both exponentially weighted sums of signal values measured up to time (#-1,), such that recent values have a larger weighting than those in the distant past: <BR> <BR> <BR> <BR> <BR> <BR> Yr = 2 s#(#-1)+(1-#S)y(#-1) : 0 # #S#1<BR> <BR> <BR> <BR> <BR> <BR> <BR> Yr=#L#(#-1)+(1-#L)y(#-1):0##L#1 (9) The half-life hL of I'L is much longer (typically 100 times longer) than the corresponding half-life hs of AS.

The relationship between a half-life h and the parameter A is defined as # = 2-1/h.

It should be noted that minimising only Vi would result in a low variance signal with no constraints on its temporal structure. In contrast, maximising only U would result in a signal with arbitrarily high amplitude.

In both cases, trivial solutions would be obtained for/3 because Vi can be minimised by setting the norm of ß to be

zero and U can be maximised by setting the norm of/3 to be large. In contrast, the ratio Vi/Ui can be minimised only if two constraints are both satisfied: 1) y has a small non-zero range (i. e. low variance), and 2) the values in y change"quickly"over time relative to hs. It should also be noted that the value of F is independent of the norm ouf 0, so that only changes in the direction of/3 affect the value of F. Advantageously, conventional iterative gradient descent on F shows that the length of varies only a little throughout the optimisation process. This is in contrast to the embodiment of the present invention.

Deconvolution Using Temporal Predictability Given that y=ßx, equation (8) can be re-written as: <BR> <BR> <BR> <BR> (10)<BR> F=log####, where C is an M x M matrix of long-term convariances between signal mixtures, and C is a corresponding matrix of short-term convariances. The long-term covariance and the short-term covariance C-between the ith and jth mixtures are defined as: It should be noted that that C and C need only be computed once for a given set of signal mixtures, and

that the terms (xi#-#i#) and (xi#-#i#), can be pre-computed using fast convolution operations.

Referring to figure 5 there is shown two signals comprising an original signal, sl, denoted by the solid line and a convolved signal, x1, denoted by the broken line, for 100 time steps.

Figure 6 depicts a further original signal, s ?, denoted by the solid line, and a deconvolved, c2, signal y=ßx, denoted by a broken line.

Gradient descent on F with respect to ß could be used to minimise F, thereby minimising the predictability of y. The derivative of F with respect to ß is: <BR> <BR> <BR> <BR> <BR> <BR> 2ß 2ß<BR> <BR> #ßF=#-#<BR> V U (12) In an embodiment, one optimising procedure would consist of iteratively updating 0 until a minimum of F is <BR> <BR> <BR> <BR> located: ß=ß-##ßF, where 77 is a small constant (typically, 77 = 0.001).

It should be noted that the function F is a ratio of quadratic forms. Therefore, F has exactly one global maximum and exactly one global minimum, with all other critical points being saddle points. This implies that gradient descent is guaranteed to find the global minimum in F.

Deconvolution as an Eigenproblem

The gradient of F is zero at a solution where, from Equation (12): <BR> <BR> <BR> <BR> <BR> )ß#=V/Uß# (13) Critical points of F correspond to values of ? that satisfy Equation (13), which has the form of a generalised eigenproblem (Borga 1998). Solutions for 41 can therefore be obtained as eigenvectors of the matrix (C-'C), which has corresponding eigenvalues y=V/U. As noted above, the first and last such eigenvector define a maximum and a minimum in F, respectively, and each of the remaining eigenvectors define saddle points.

It should be noted that eigenproblems have scaling characteristics of O (N3), where N is the number of signal mixtures. The vector P can be obtained using generalised eigenvalue routine.

Given an nth order FIR filter, W is an n x n matrix.

The deconvolved signal can then be recovered by using the smallest eigenvector ßK in W : y=ßKx. W : y=ßkx.

Deconvolution Results The embodiments of the present invention were demonstrated using 5000 samples of a signal, sl, which was the sound of a choir singing, sampled at 8192Hz. The signal, sl, was convolved with a low-pass filter with n = 32 coefficients α=e-n/T, where T = 8 samples. The resultant convolved signal xi=a*s was used as input to for the embodiments.

Figure 5 displays a short segments 500 of si and xl, and the correlation between si and xi is r = 0.430. After applying the embodiment described above, the correlation between the deconvolved segments 600, that is, signal y=ß*x and c2 is r = 0. 980 as can be appreciated from figure 6. It can be appreciated from figure 6 that there is good agreement between s and the recovered signal y.

It will be appreciated from the above that the flowchart shown in figure 4 is equally applicable to deconvolution with the exception that deconvolution, in contrast to the maximisation operations of the bss problem, attempts to minimise predictability.

Source Separation for Image Mixtures Although the above embodiments have been described with reference to processing single dimensional signals, embodiments of the present invention are not limited thereto. Embodiments of the present invention can be adapted for a spatial model, although it remains essentially the same. In the spatial model, an embodiment would maximise a function F=Iog (VIU) in which the long-term and short-term variances are spatially defined as where i, j are spatial indices spanning, for example, data representing an extracted image y. The quantities yi, j and Yi, j represent long-range and short-range

exponentially weighted spatial means of y, where the weighting is centred at position i, j.

As in the temporal case, V and U can be computed as: <BR> <BR> <BR> <BR> <BR> t<BR> F=log---, (15)<BR> <BR> <BR> wu and solution vectors W=(W1,...,WK) can be obtained using the same eigen decomposition as was used in the temporal case.

However, in the case of 2D images, the long-range covariance C,. and short-range covariance Cj between the ith and jth image mixtures are: The quantities Cj and ci, j are calculated by convolving each image mixture with exponentially decaying masks # and tt of different half lives: It should be noted that the entries in the masks are rotationally symmetric and decay exponentially as a function of distance from the mask centre at r, c. The rates of decay in and # are defined by the spatial short-range and long-range half-lives, hL and hs,

respectively. Preferred values for these are hL = 90000 pixels and hs= 1 pixels.

Therefore, the main difference between the spatial case and the temporal case is the source of contributions to y and y. Whereas the temporal version sums outputs of a single unit over time, the spatial version sums the outputs over a spatial array.

The extracted images il,..., Y,, are recovered from the set of K image mixtures are: y1=w11x1+w21x2+...+wK1xK (18) where (wl, wiz,..., wK) are elements of the ith row vector in W.

It will be appreciated that even though the above embodiment has been described with reference = to processing 2-D data representing images, the present invention is not limited thereto. The 2-D data could equally well represent data other than image data.

The above embodiments make reference to the term "critical points". It will be appreciated that the meaning of this term includes maxima, minima and saddle points, see, for example, Penguin Dictionary of Mathematics, J Daintith, R Nelson, 1989.

It will be appreciated from the above that the blind source separation problem requires maximising predictability which implies maximisation of F which, in turn, implies Wi=Wi+##WiF. In contrast, deconvolution

requires minimisation of predictability which implies minimisation of F which implies W =WjçVWF, that is, where F is a measure of predictability, it is maximised for BSS and minimised for deconvolution.

It will be appreciated by those skilled in the art that the above inventions and embodiments can be implemented using, for example, a conventional computer and appropriate digital signal processing software.

Alternatively or additionally, a specific hardware platform may be realised for implementing the inventions or embodiments.

The reader's attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.

All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly

stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of any foregoing embodiments. The invention extends to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed.