BACKGROUND OF THE INVENTION The present invention relates generally to dynamic systems and more particularly to a method of training a dynamic system commonly referred to as a state machine. The behavior of a state machine is controlled by the interaction of its variables through a system of state equations. The state equations are produced by an analysis of the system being simulated. The resulting state equations are functions of state variables and parameters in the state equations. For example, to write the state equations to simulate the planetary motion of the earth around the sun, the first step would be to define the state variables as the position and velocity of the earth along two perpendicular axes.

The fact that the derivative of the position along these axes is equal to the velocity can be used to define two of the four necessary state equations. The two remaining state equations would describe the rate of change of velocity along the two axes as a function of position. This system of equations could then be processed to predict the future position of the earth.

The normal way state machines are set up and used is by first defining their state equations as a system expressing the interaction of a number of state variables.

The initial conditions of the state machine are established by defining the initial values of the state variables and their associated parameters. The state machine then is allowed to run to perform its designated task. However, a state machine running in such an open- loop manner cannot adapt to its environment. Changing external conditions will eventually render the state machine inadequate for operation in its intended environment.

Manually adjusting a state machine in an attempt to compensate is time consuming,

expensive, tedious, and prone to error. Therefore there is a need for a technique to automate the training of a state machine.

SUMMARY OF THE INVENTION According to the invention, a state machine system is provided which is based on state equations where the number and relationship of the state equation are not specific to the desired behavior. The behavior of the system is compared with the desired model. This information is-then used to adjust the value of the parameters built into the system so that the behavior can be incrementally modified until the behavior of the system is acceptable. To increase the number of parameters used in the state equations, multivariable power series can be used. The use of multivariable power series permits very rapid increase in the number of parameters used in the state equations without restricting its structure.

In a specific embodiment, the first step in the training of a state machine system is to obtain derivatives of parameters used in the system with respect to the level of a variable whose behavior is to be controlled. To accomplish this task, the concept of derivative variables is introduced. A derivative variable is a variable whose level is representative of the instantaneous value of the derivative of a particular normal state variable with respect to a particular parameter. Each normal variable in the system will have a unique derivative variable for each parameter in the system. Once the information represented by the level of all the derivative variables for the normal variable whose behavior is to be controlled is available, the system can be trained using an LMS-like algorithm.

The technique used for training a state machine is then a two-step process : 1) development of a technique for developing and processing the derivative variables ; and 2) use of the derivative variables along with the error in the level of the output variable to adjust the parameters used in the state equations controlling the behavior of the system.

The invention will be better understood by reference to the following detailed description in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows the transformation of normal state equations to the derivative variable state equations of the present invention.

Fig. 2 is a simplified block diagram of the process for training a state machine.

Fig. 3 is a flow graph for processing and training a state machine.

Fig. 4 is a flow graph for processing and training a state machine using change of level.

Fig. 5 is a flow graph for processing and training a state machine using the matrix training algorithm.

Fig. 6 is a flow graph that is an expansion of block 196 of Fig. 5 showing a task related to updating the parameter when using either the reiterative algorithm or the first matrix technique.

Fig. 7 is a flow graph of an initialization sequence for the multi-point method for processing the state variables of a state machine.

Fig. 8 is a flow graph for the multi-point method used to processing the state machine.

Fig. 9 is a flow graph of a recursive technique for evaluating a multivariable power series.

Fig. 10 is a flow graph of a recursive technique for calculating the derivative of all parameters used in a multivariable power series with respect to its output.

Fig. 11 a is diagram of a z-transform circuit for calculating the derivative using two samples of the input signal.

Fig. 1 lb is a diagram of typical z-transform circuit for simulating an RC circuit.

Fig. I I c is a diagram of a z-transform circuit for calculating the integral using two parameters so the constant input gain and the time constant can be varied independently.

Fig. 12a is a diagram of a z-transform circuit for calculating the derivative using three samples of the input signal.

Fig. 12b is a diagram of a z-transform circuit developed from the three- point-method for calculating the integral.

Fig. 13 is an alternate design of the state machine that does not use integrators, etc.

DESCRIPTION OF SPECIFIC EMBODIMENTS The development of the technology for training of the state machine is dependent on two technologies that are used in conjunction with one another. They are : 1) calculating the derivative of the parameters being adjusted with respect to the state machine's output ; and 2) using these derivatives along with the error in the system's output to adjust the system's parameters.

Other problems that should also be discussed are : 1) techniques for processing the state machine to determine the gradual change of its variables to new states ; and 2) techniques for processing the state machine's initial conditions as parameters that can also be adjusted and learned.

I. DETERMINING STATE EQUATIONS FOR DERIVATIVE VARIABLES The technique for determining the derivatives of the parameters with respect to the output of the state machine involves the use of derivative variables. A derivative variable is defined as a variable whose value is used to predict the value of the derivative of a particular parameter with respect to a particular output of the state machine. For a given normal state variable, there is a unique derivative variable for each parameter in the system. For example, in a system having 5 normal state variables and 10 parameters, there are 50 derivative variables. Derivative variables define another set of state variables processed by their own set of state variable equations. The state equations used to process the derivative variables are based on the normal state equations used to process the normal state variables. To illustrate this process, the state equations used to process the derivative variables for a typical parameter aj will be developed.

The state equations used to control the behavior of the system will assumed to be : <BR> <BR> <BR> <BR> <BR> dv ;-<BR> <BR> <BR> dyj = fi (Y) (1)<BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> Y, dy, (2)<BR> <BR> <BR> dt (2)<BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> Ys = fs (Y) (3) This system of equations is capable of duplicating the behavior of that of a linear system including poles and zeros, or that of a non-linear system. Each variable in the system is assumed to have its value defined or controlled by one of these three equation types. Equation (1) is usually the only equation type required to define a state machine. Equation (2) is included to introduce zeros and is usually included in the definition of a state machine, although not explicitly stated. Equation (3) was added to my definition to remove any unnecessary restriction on its structure.

To make this discussion as general as possible, functionsfi and f were used in Equations (1) and (3). The use of y with a bar over it represents all y variables, while Yi represents a particular variable. The different subscripts used in these equations

are meant to prevent confusion, because a particular variable which has its value defined by Equation (1) cannot also have its value defined by Equation (2). In other words each variable in the system will fall into one of three groups and its value will be defined by either Equation (1), (2) or (3). The parameters whose values will be adjusted to train the system are hidden in the functions. Typically, Equation (1) means that the rate of change of variable y, is controlled by a function of a subset of the y variables.

A restriction-built into the definition of a state machine defined by Equations (1) through (3) is that the functions are not a function of time or, in other words, are time invariant. This does not restrict the results, but does simplify the mathematics used in their derivation.

The state equations for processing the derivative variables are produced by taking the derivatives of the normal state equations with respect to each parameter comprising the normal state equations. By taking the derivative of Equation (1) with respect to parameter aj, the results is : where N is the number of state variables in the function fi (y).

The term on the left side of Equation (4) can be rearranged to be a derivative of dyi/daj with respect to time. This permits dyi/daj to be defined as a new term, vy, referred to as the derivative variable V, j. Using this substitution, Equation (4) can be rewritten as : wheregij (y) and win (y) are functions of all state variables.

Equation (5) can be rewritten as : Equation (6) is a state equation used for processing the derivative variable v, and is equivalent to Equation (1), which is used for processing the normal state variable y,. The value of derivative variable vy is the derivative variable for real variable Yi and parameter ay, and is representative of the derivative of real variable y, with respect to parameter aj.

Following a similar procedure, the state equations for processing the derivative variables based on Equations (2) and (3), are : v dt and (7) =A (). (8) Equations (6), (7) and (8) are used to process the derivative variables through the network nodes defined by the normal state equations.

The foregoing discussion is summarized in Fig. 1. The box labeled Equation (1) shows the normal state equation 101 of state variablesyi, followed bv the derivation steps to arrive at the equivalent state equation 103 for the derivative variables v ; y. The box labeled Equation (2) shows the normal state equation 104, followed by the derivation steps to arrive at the equivalent state equation 106 for derivative variable Vry.

The box labeled Equation (3) shows the normal state equation 107, followed by the derivation steps to arrive at the equivalent state equation 110 for derivative variable Vsj..

II. DERIVATIVE VARIABLES AND INITIAL CONDITIONS When a number of state variable equations are used to simulate a system, a number of initial conditions must be established. These initial conditions are the initial values of the state variables when the simulation is started. The procedure is to treat these initial conditions as additional parameters whose value can be adjusted by the training process.

The discussion up to this point has assumed that the derivative variables included only those parameters used in the state equations. Derivative variables can also include parameters used to define the initial conditions of the normal state variables. In addition to the problem of the initial conditions of the actual variables of the state machine, we must also establish the initial conditions of the derivative variables themselves. The notation that will be used is : Yilo = Pi 5 (9) which means the state variable, evaluated at time zero, i. e., when the simulation is started, is equal to pi. To prevent confusion between the parameters used in the functions and the parameters used to definite initial conditions, the parameters used in the functions are a parameters while the parameters used to define initial conditions are p parameters.

By taking the derivative of Equation (9) with respect to pi, the result is :

The initial conditions of all other derivative variables will be zero, thus : when i xj ; and (11) for all i and j. (12) By increasing the number of derivative variables to include initial condition parameters, the initial conditions can also be learned as part of the training process.

III. UTILIZING THE DERIVATIVE VARIABLES TO TRAIN A SYSTEM Fig. 2 is a simplified block diagram representation of an adaptive system in accordance with the invention. A Desired Behavioral Model 10 produces a desired or reference output. A State Machine Block 100 contains the state equations of the system to be trained. As mentioned above, the state equations are functions of an array state variables y and an array of parameters a hidden inside the function controlling the systems behavior.

The goal of the Adjustment Block 105 is to adjust the parameters in the state equations in the State Machine Block 100 so that the behavior of the particular state variables yi matches the behavior of the corresponding variables w, supplied by the Desired Behavioral Model 10.

Signals flow from State Machine 100 and Desired Behavioral Model 10 to a summer 104 to produce error signals. Also flowing out of the State Machine 100 are derivative variables, dyj/day, of those state variables v which have contributed to the error signals. The error signals and the derivative variables are sent to an Adjustment Block 105 where they are processed in accordance with the algorithm described herein to calculate an array of change values (day) used to adjust the value of the parameters. These change values are then added to the parameters in the State Machine 100, thereby adjusting their values.

Fig. 3 illustrates the processing that takes place in the system, both to process the state variable system to calculate its new state given its existing state, and to collect and process data in the Adjustment Block 105 to calculate the array of change values Sa used to adjust the parameter values. The algorithm used in the Adjustment Block 105 uses the value of the derivative variables to allocate the error to the parameters. The error in this case is a variable defined as the difference between the desired signal level wi fed from the Behavior Model 10 and the actual signal level vs fed from the State Machine 100. The actual training

can only take place at instantaneous samples of the error and the derivative variables referred to as data points. In this discussion, a data point comprises the value of an instantaneous sample of the error and an array of values of instantaneous samples of associated derivative variables, all collected at the same instant. By learning enough data points along a desired curve or sequence of values, the behavior of the State Machine 100 can be trained and controlled.

The training-algorithm used by the Adjustment Block 105 is similar to the LMS algorithm. Since the specific algorithm used will be explained in another section, the LMS algorithm will be reviewed. The LMS algorithm updates the parameters immediately after processing each data point. The amount of change of each parameter's value will be proportional to the error at this data point and the derivative of this parameter with the output. The constant of proportionality is usually assumed to be a constant ; that is it has the same value for all data points. As a result then the change in the parameter's value can be calculated from : error 4VIi K da Where y, is the variable being trained. If it is desired to attempt to completely compensate for the error so that after the training the error no longer exist, the constant K in the above equation can be chosen so that the change in the output will equal the error. The change in the output can be calculated from : J change out = ai da) (14) wu dal By substituting the value of Say from Equation (13) into Equation (14) and setting change output to be equal to error, the results is : The calculated of the value of K that completely eliminates error is not normally used in the LMS algorithm, but will be used later in this discussion. In Equations (13) the values of error and dys/day are sampled values of the error and derivative variables produced by the I summer 104 and the State Machine 100 module respectively, in Fig. 2. The above equations are use to learn only one data point on the curve. In order to control the State Machine's behavior, many such data points must be learned.

The foregoing discussion relates to the equations used in blocks 128 in Fig.

3. In order to understand the source of the equations and techniques used in blocks 122 and 123 to process the state machine refer to the section"Technique for Processing a State Machine".

In Fig. 3 the normal processing sequence is through blocks 122, 123, 124, 125 and 126. During this sequence, the new values of the normal state variables and derivative variables are calculated.-Occasionally, the sequence will deviate through blocks 127 and 128, during which information is collected to be used to train and adjust the parameter values. To reduce the number of times the value of the parameters are actually changed, the information from a number of data points are collected and averaged in blocks 128 and 131. The averaging reduces the number of times the values are changed. The reason for the averaging strategy is that as a result of updating the parameter values, an error is introduced into the value of the derivative variables. To limit this effect from interfering in the training of the State Alachine 100, the state variables will only be updated through sequence 122, 123, 124, 125, 126 until the error in the level of the derivative variables have had a chance to correct themselves. The objective of the delay in block 126 is to control the rate of processing the blocks 122 and 123 so that the correct time delay occurs between each update.

IV. TRAINING USING CHANGE OF LEVEL In another embodiment of the invention, an alternative technique is disclosed for training a state machine based on changes in levels of the state variables themselves, represented by ty. This requires producing the derivative of the change in level with respect to each parameter aj. It can be shown that : d (o. v > da J interval change interval change This means that the derivative of the change in level with respect to a parameter ay, namely----, is equal to the change in the corresponding derivative variable da during that same interval.

This embodiment is illustrated in Fig. 4 for training the state machine based on changes of a state variables during a time interval. At the start of the training period, control block 142 and block 143 store the value of all derivative variables. At the end of the training period as determined by block 130, the difference or change in the value of all

derivative variables is calculated in block 150. This change in the value of the derivative variables is calculated using the following equation : d (Ay ;) dYs t dy ; | d da | da | (16) da. end da. end stnrt In block 151, the error in the change in level of the variable being trained is calculated using the following equation: ERR=#yi|desired=#yi|actual, (17) where the value of #yi|disired and #yi+actual is caluculated calculated using following equations equations #yi|desired=yi|end desired-yi|start desired #yi|actual=yi|end actual-yi+start actual In block 152, the change in the value of all derivative variables is used to calculate K using the following equation : In block 153, the error in the change of the variable is used along with the change in the derivative variable to calculate a change in the corresponding parameter using the following equation : In block 132, the change in the parameters is incorporated. As in Fig. 3, the normal state variables and the derivative variables are processed in blocks 122, 123, and 124.

The technique of obtaining the derivative variable for a change of level can be extended to obtaining the derivative variable for a change of the change of level, or Y). The technique for finding and using these new change of change derivative daj variables is the same as that used to obtain the above derivative variables, as illustrated below : da ; da X end da start (20) da da. da. -' Md-'tari' Av. =Ay-Ay Ay. Ay.-Ay ERR = t Yilde5ired A Yllactual) (21) Yi | desired Yi | end desired Yi | start desired A2 yi I actual 'y end acrua ! yt I srarr actual

To train an output of a state machine system, periodic samples of the output level and all level derivatives are taken. From this sequence of samples, the derivative variables for the change of level and the error in the change of level are calculated. While in principle it is possible to learn the signal from the level training only, due to the large coupling between the data points or similarity between the values of the derivative variables this is difficult to do. By being able to train change of level and change of change of level in addition to level itself, it is possible to reduce the task of training.

TECHNIQUE FOR EQUALIZING VALUES OF DERIVATIVE VARIABLES The technique used to calculate the change of a particular parameter from a single data point is proportional to the value of the particular parameter's derivative variable.

If the amplitudes of the various derivative variables are very unequal then it will be very hard for the system to adjust the parameter of the smaller derivative variable. As an example, if the ratio of peak amplitudes of two derivative variables is one to 105 (1 : 105) then the only parameter that will be adjusted is the parameter with the large derivative variable.

Even when using a matrix to learn the difference between two data points and the only change is the value of the smaller derivative variable, there is the possibility that the matrix technique will reject the data because it will not recognize that the two data points are linearly separable.

The value of the derivative variable is calculated using the following equation : This equation can be modified as follows :

Then when the change in parameters aj is calculated in the normal manner, its change is multiplied by the same constant before being added to the parameter, as shown by the following equation : aj+ = AjAaj The value of can be either larger or smaller than one, and its value is adjusted so the peak amplitudes of all derivative variables are comparable.

This technique works because the base equation has the form of : The form of this equation is modified to : As a result of this change, the value of the derivative variable has been changed from x11x21jx3kto Aijkx1ix2jx3k. This results in a larger derivative variable, which in turn results in a smaller change to a smaller aijk. However, it is an object to calculate the change necessary to the product aijk Aijk * V. TRAINING ALGORITHM The training algorithm can be broken down into three sections. The main training algorithm will be a reiterative algorithm that is based on a modification of the LMS algorithm used for training non-recursive filters. The LMS algorithm updates the parameters based on information collected from each data point immediately after the data point is processed. The main modification of this procedure is to reiterate of the data points to adjust the values in a change variable. The change variable is a prediction of the change in the state machine's output that will result from an update of the parameter values. The second technique is a technique that uses a matrix to compensate for the interaction of the adjustment of the parameter from each data point. The limitation of this technique is that it assumes that each data point can be learned exactly. This is not possible if the number of data points is large or if the data points are not linearly separable. As a result, before this technique can be used, it is necessary to evaluate the matrix's determinant. If the value of this determinant is or approaches zero, the matrix technique can not be used.

The normal processing sequence to use level training of a state machine is shown in Fig. 5. As in Fig. 4, the normal processing sequence is through blocks 122, 123,

124, 125, 126 and then back through 126. During this sequence no data points are collected. This sequence is used to process the state machine and continue the simulation.

When a data point is collected, control block 125 will direct the flow through block 195 where a data point is collected. During this collection the data points will be numbered and store sequentially. Until the end of the training interval is reached, control block 130 will direct the flow back to block 126. When the end of the training interval is reached, control block 130 will direct the flow to block 196 which has been expanded into Fig. 6.

In Fig. 6 the first test that is performed is to test the state of an internal flag that will tell the system if the first matrix technique is to be attempted. If the first matrix technique is to be attempted, the coefficients of the matrix will be calculated in block 203. Then in control block 204, the value of the determinant of the first matrix technique will be calculated and tested to determine if the value of the determinant approaches zero. If the value of the determinant approaches zero control block 204 will direct the system to abandon the use of the first matrix technique and instead use the reiterative technique. If the determinant of the first matrix technique does not approach zero, the matrix is solved in block 205 producing an array of parameter change values. Because of the existence of second order derivative variables, if the desired change is to large, not all the change can be incorporated. This decision is incorporated into block 208 which adds as much of the array of parameter change values as possible to the individual parameters. When the reiterative algorithm is used the rate of convergence can be increased by using change data point and change of change data points. In block 206 this new type of data points are calculated from the normal data points. In block 207, the reiterative algorithm will be used to determine an array of parameter change values. After this array of parameter change values have been determined in block 207, they are used in block 208 to actually change the parameter's values.

A. Reiterative Technique The objective of the reiterative technique is to adjust the parameters in a change variable so that the predicted change in the state machine's output is equal to the error variable. The error variable is defined by sequence of error values of the individual data points. The normal way the procedure will be used is that data points will be taken periodically over the training interval. The parameters that will be adjusted in the change variable will be the array of parameter change values. At each data point the change variable will be subtracted from the error variable to produce a value called the training error. Since the objective is to make the change variable to approximate the error

variable, the objective can be also be stated as an attempt to minimize the sum of square of the training error at all data points. The procedure will then be to reiterate over all data points making minor change to the parameters in the change variable to reduce the training error.

The change variable is in general a linear combination of the derivative variables ; however, if both the first and second order derivative variables used to calculate the value of the change variable, the results would be : Y, Y 1 Y ; change \0 g f-L. A +lf (A.) (25) y i I-i k-Y 2a In the second term of Equation (25) k can not be equal to j.

At each data point, the values of the parameter change values will be updated by : A Train Error 0 d (change) 0 (26) K Beta d (jazz Where : 'd (change) (27) Beta = ( j=0 The value of K is chosen as a compromise between a good rate of convergence and a value that will permit value of zlaj to approach a stable value when all data point can not be learned exactly and some training error remains.

To increase the rate of convergence in the adjustment of the array of parameter change values, change data points can be used. A change data point is defined as a data point created from two normal data points. In a change data point, the values of error and all derivative variables are the results of subtraction of the respective values of the two data points. The basic idea behind the use of change data points is that the change in the value of the derivative variables in any interval can be used to train the change in level of the output during the same interval.

This data point data collected in block 195 of Fig. 3 can be processed to create a change data point before the information is used to train the system. By taking data for one data point at the start of the period, and subtracting it from the data of a second data point taken at the end of the period, a change data point can be created.

d (t j) dYj dyj da da da end starr In the normal data point, the stored level variable is the error at the instant the data point was collected. In a change data point, the level variable stored is the change in error that took place during the interval as shown by the following equation : AERR zu ERR|end-ERR|star. (29) The change in the value of all derivative variables is used to calculate Beta using the following equation : 2 Beta = j d (Ayi) (30) i-o dai When processing a change data point to adjust the parameter change values it is necessary to calculate the change in the level of the change variable between these two data points. If the value of the change variable is calculated using only the first term of Equation (25), the value of the change in the level of the change variable can be calculated using : (ov > Ochange = 1'a (31) 7 , J In all cases, the value of the change of the change variable between two data points will have to be calculated using : Achange = change end-change Srart (32) Regardless of the technique used to calculate the change in the level of the change variable, Equation (31) or (32), the change in the training error between the two data points is calculated using : ATrain Error = DERR-Achange (33) As point, the change in the training error is allocated to the array of parameter change values using : Train Error d (Ayi) (34) K Beta da. J Where : , Beta = j=O daj j

The technique of obtaining the derivative variable for a change of level can be extended to obtaining the derivative variable for a change of the change of level, or Y). The technique for finding and using these new derivative variables is the same as daj that used to obtain the foregoing derivative variable, as illustrated below: A,) A,) (A) da-da da aa"aa' A=AF !-AF !, (36) chance = Y (7) j=o dai 0'TrainError = A'ERR-A'change (38) + = A 2Traii7 Error d (A2 yi) 0 a + _ ( 9) K Beta da Where : Beta = , j To review, to train an output of the state machine over an interval, the output is sampled periodically, storing both the error in the output level and the value of all derivative variables. By reiterating over these data points by first calculating the value of the change variable and then calculating the training error, and then allocating the training error to the parameter change values. The rate convergence of this reiterative process can be increased also by using change data points and change of change data points.

The class controlling the training process in a working C-program is TrainClass and is defined below : /* TrainClass. h */ struct DataPtClass double* pDoutDa ; double Error, Beta ; } ; class TrainClass { DataPtClass RawDataPt[40], DelDataPt [40], Del2DataPt [40] ; int DataPtNum, DataAvailable ; /* DataPtNum=number of RawDataPtr collected. */ double SystemError ; double DPTrainError [40] ; int CycleCount ;

/* Above used for display. */ double* pParChange ; /* stores the array of parameter change values. */ double K ; int ProcessAsReceived ;/* TRUE or FALSE */ /* used by CollectDataPt (). */ double AbsoluteValue (double D) ; public : -TrainClass (void) ; /* TrainClass2. cpp */ TrainClass (void) ; /* TrainClass2. cpp This procedure initalizes the class and allocates space for pointers, etc. */ void CollectDataPt (VarType& VarObj) ; /* TrainClass2. cpp This function is used if VarObj is an error derivative variable */ void CollectDataPt (VarType& VarObj, float Error) ; /* TrainClass2. cpp This procedure is calmed each time a point of training data is collected. This procedure will build the DelDataPt0j and Del2DataPtu, calculate Beta on all data points, and reiterate on the new data point as it is being collected if ProcessAsReceived is TRUE. This procedure also increments DataPtNum. */ int Reiterate (void) ; /* TrainClass2. cpp This procedure reiterates over data points collected to converge and uses IncorporateChange (pParChange) to update parameter values.

If DataAvailable is TRUE Incorporate () will do its task. Error codes are : 0=OK.

1=No Data Available. */ void IncorporateChange (void) ; /* Trainclass1. cpp This procedure uses the array of parameter change values pointed to by pParChange to the correct parameters. */ void Reset (void) ; /* TrainClass2. cpp */ int GetDataPtNum (void) {return (DataPtNum) ;} friend TrainDisplayClass ; friend DerivDisplayClass ; friend OutputDisplayClass ; friend ErrorDisplayClass ; friend ChooseDPNumClass ; friend AdjParDisplayClass ; friend FixParDisplayClass ; } ; The TrainClass contains a group of objects that are members of the DataPtClass. The definition of this class in also contained within this file. As you can see from the structure definition that this structure contains a float pointer, pDoutDa, that points to the array of derivative variables, and the floats Error and Beta. The float Error of course contains the value of error at this data point. If the structure is being used as a change data

d (ov.)<BR> <BR> point, then the derivative variables being pointed to would be values of instead of<BR> <BR> <BR> Jv values of--. The value of Beta stored is the value of Beta appropriate for this type of daj data point. This is the proportional constant that is used in the allocation of the Train Error to the parameter change values as shown in Equations (32), (40), and (45). In the normal use of this code, data points wilrtaken periodically over to training interval. At the time the level data point is collected, a change data point and change of change data point are calculated. In the code the structure servicing this function are named DelDataPtn and Del2DataPtn, respectively. The code used to calculate and store the values in these structure is in function CollectDataPt () and is in the following file. void TrainClass : : CollectDataPt (VarType& Var) { float Error=-Var. Value () ; CollectDataPt (Var, Error) ; return ; void TrainClass : : CollectDataPt (VarType& Var, float Error) { int i ; double* pTo ; double* pFrom1 ; double* pFrom2 ; double Beta, Temp ; if (DataPtNum< 40) { /* store level data. */ RawDataPt [DataPtN um]. Error= Error ; pTo=RawDataPt [DataPtNum]. pDoutDa ; Beta=0. 0 ; for (i=O ; i< ParMax ; i++) { * (pTo+i) =Temp= Var [i] ; Beta += Temp*Temp ; } RawDataPt [DataPtNum]. Beta=Beta ; /* generate del data point. */ if (DataPtNum> 0) Del DataPt [DataPtNum]. Error=RawDataPt [DataPtNum]. Error- RawDataPt [DataPtNum-1]. Error ; pFrom2=RawDataPt [DataPtNum]. pDoutDa ; pFrom1=RawDataPt[DataPtNum-1]. pDoutDa ; pTo=DelDataPt [DataPtNum]. pDoutDa ; Beta=0. 0 ; for (i=O ; i< ParMax ; i++) { * (pTo+i) =Temp= * (pFrom2+i)-* (pFrom1+i) ; Beta += Temp*Temp ;

DelDataPt [DataPtNum]. Beta=Beta ; if (DataPtNum> 1) { Del2DataPt [DataPtNum]. Error=DelDataPt [DataPtNum]. Error- DelDataPt [DataPtNum-1]. Error ; pFrom2=DelDataPt [DataPtNum]. pDoutDa ; pFrom1=DelDataPt[DataPtNUm-1]. pDoutDa ; pTo=Del2DataPt [DataPtNum]. pDoutDa ; Beta=0. 0 ; for (i=0; j< ParMax ; i++) { *(p To+i) =Temp= * (pFrom2+i)-* (pFrom1+i) ; Beta += Temp*Temp ; Del2DataPt [DataPtNum]. Beta=Beta ; } /* reiterate over data point. */ if (ProcessAsReceived==TRUE) { double CalChange, ActualChange, ErrorChange ; double DoutDa ; if (DataPtNum> 1) { CalChange=0. 0 ; pFrom1=Del2DataPt [DataPtNum]. pDoutDa ; for (i=0 ; i< ParMax ; i++) DoutDa=*(pFrom1+i) ; CalChange += DoutDa* (* (pParChange+i)) ; } ActualChange=Del2DataPt [DataPtNum]. Error ; ErrorChange=ActualChange-CaIChange ; Beta=Dal2 DataPt[DataPtNum]. Beta ; if (Beta> 0) for (i=0 ; i< ParMax ; i++) DoutDa= * (pFrom1 +i) ; (pParChange+i) +=K*DoutDa*ErrorChange/Beta ; } } } if (DataPtNum> 0) Cal;Change=0. 0 ; pFrom1=De ! DataPt [DataPtNum]. pDoutDa ; for (i=0 ; i< ParMax ; i++) DoutDa= * (pFrom1 +i) ; CalChange += DoutDa* (* (pParChange+i)) ; ActualChange=DeIDataPt [DataPtNum]. Error ; Error Change=ActualChange-CalChange ; Beta=DelDataPt[DataPtNum]. Beta ; if (Beta> 0) for (i=0 ; i< ParMax ; i++)

{ DoutDt=*(pFrom1+i) ; * (pParChange+i) += K*DoutDa*ErrorChange/Beta ; } CalChange=0. 0 ; pFrom1=RawDataPt [DataPtNum]. pDoutDa ; for (i=O ; i< ParMax ; i++) { DoutDa= * (pFrom1 +i) ; CalChange += DoutDa* (* (pParChange+i)) ; } ActualChange=RawDataPt [DataPtNum]. Error ; ErrorChange=ActualChange-CalChange ; Beta=RawDataPt [DataPtNum]. Beta ; if (Beta> 0) { for (i=O ; i< ParMax ; i++) DoutDa=*(pFrom1+i) ; * (pParChange+i) += K*DoutDa*ErrorChange/Beta ; <BR> <BR> }<BR> <BR> <BR> } DataPtNum++ ; return ; void TrainClass : : Reset (void) DataPtNum=0 ; return ; void TrainClass : : IncorporateChange (void) int i ; DataPtNum=0 ; float* pParameters=ParData.pParamenters ; for (i=0 ; i< ParMax ; i++) * (pParameters+i) += (* (pParChange+i)) ; return ; } The code used to reiterate over the stored data points to determine the array of parameter change values is function Reiterate () and is shown in the following file. int TrainClass : : Reiterate (void) { double* pFrom1 ; double Beta, Change, Error, TrainError, Sum, Temp, Ktemp ; double DoutDa, ChangeSystemError, PreiousSystemError ; double KLevel, KDel, KDel2 ; int DPNum, i ; Level=4. 0 ; KDel=10. 0 ; KDel2=20. 0 ; if (DataPtNum<= 0) return (1) ; Sum=0. 0 ;

for (i=O ; i< ParMax ; i++) * (pParChange+i) =0. 0 ; for (DPNum=0 ; DPNum< DataPtNum ; DPNum++) { Change=0. 0 ; pFrom1=RawDataPt [DPNum]. pDoutDa ; for (i=0 ; i< ParMax ; i++) { DoutDa= * (pFrom1 +i) ; Change += DoutDa* (* (pParChange+i)) ; Error=RawDataPt[DPNum]. Error ; Temp=DPTrainError [DPNum] =Error-Change ; Sum += Temp*Temp ; SystemError=sqrt (Sum/double (DataPtNum)) ; ChangeSstemError=SystemError ; CycleCount=0 ; while ((SystemErro> 0) && (ChangeSystemError> Limit) && (CycleCount < 40)) { <BR> <BR> CycleCount++;<BR> <BR> <BR> for (DPNUm=0 ; DPNum< DataPtNum ; DPNum++) Beta=RawDataPt [DPNum]. Beta ; if (Beta> 0) { pFrom1=RawDataPt [DPNum]. pDoutDa ; Ktemp=KLevel*Beta ; for (i=0 ; i< ParMax ; i++) DoutDa=*(pFrom1+i) ; * (pParChange+i) += DoutDa*DPTrainError [DPNum]/Ktemp ; } if (DPNum>=1) { Change=0. 0 ; pFrom =DelDataPt [DPNum]. pDoutDa ; for (i=O ; i< ParMax ; i++) DoutDa= * (pFrom1+i) ; Change += DoutDa* (* (pParChange+i)) ; } Error=DelDataPt [DPNum]. Error ; TrainError=Error-Change ; Beta=DelDataPt [DPNum]. Beta ; if (Beta> 0) Ktemp=KDel*Beta ; for (i=0 ; i< ParMax ; i++) DoutDt=*(pFrom1+i) ; * (pParChange+i) += DoutDa*TrainError/Ktemp ; } }

if (DPNum>= 2) Change=0. 0 ; pFrom1=Del2DataPt [DPNum]. pDoutDa ; for (i=0 ; i< ParMax ; i++) DoutDa= * (pFrom1+i) ; Change += DoutDa* (* (pParChange+i)) ; } Error=Del2DataPt [DPNum]. Error ; Train Error= Error-Change ; Beta=Del2DataPt [DPNum]. Beta ; if (Beta> 0) { Ktemp=KDel2*Beta ; for (i=0 ; i< ParMax ; i++) { DoutDa= * (pFrom1+i) ; * (pParChange+i) += DoutDa*TrainError/Ktemp ; } } Sum=0. 0 ; for (DPNum=0 ; DPNum< DataPtNum ; DPNum++) Change=0. 0 ; pFrom1=RawDataPt [DPNum]. pDoutDa ; for (i=0 ; i< ParMax ; i++) DoutDa= * (pFrom 1 +i) ; Change += DoutDa* (* (pParChange+i)) ; Error=RawDataPt [DPNum]. Error ; Temp=DPTrainError [DPNum] =Error-Change ; Sum += Temp*Temp ; } PreviousSystemError=SystemError ; SystemError=sqrt (Sum/double (DataPtNum)) ; ChangeSystemError=SystemError-PreviousSystemError ; return (0) ; The code is designed to reiterate over the data points as long as the sum of the squares of the training error is above a limit and the amount of improvement on each reiteration is above a limit. The proportional constant used to limit the correction at each data point is different for level data points and del (change) data points and del2 (change of change) data points.

B. First Matrix Technique The first matrix technique will work under some circumstances when the number of data points is limited. The ability to use this procedure in a given situation

must first be tested by evaluating the matrix's determinant. This procedure is covered in the following section.

Because the matrix technique is easiest to understand when training a power series where the time component has been removed, the matrix technique will first be developed for use on a power series and then modified for use on a system with sampled value of derivative variables etc. In the first matrix technique, a separate equation is constructed for each data point. The training of one data point will to some degree undo the training at another data point. This interaction of the training of different data points can be analyzed mathematically. During training, changing the power series'parameters to learn one data point t changes the value of output of the power series at another data point s by an amount : change /\ out =)'Aa,-- (40) da , _o The notation It used in Equation (40) is defined to mean"evaluated at data point t" ; likewise s means"evaluated at data point s."If the objective is to adjust the parameters so as to completely eliminate the error at a data point, the change of parameters would be calculated from : ERR dout (41) Beta da j Where : 2 'dottt Beta dan (42 Zu Thus, the change in the value of the output at data point s that results from adjusting parameters at data point t is : change a'0 E. dof) out dout| ERR t l Fdout j=o dai Beta, da,.

This result is obtained by substituting the result from Equation (41) into Equation (40). The error in Equation (43) is denoted by ERRt, or the applied error, to distinguish it from the value of the real error. By substituting the value of Beta, as defined by Equation (42), the results is :

change i dout dout daf cla. 5 RR t out S =,, (44) 'doux zu t where ERR, is the applied error used during parameter updating for training data point t. The ratio of the two sums is called the coupling coefficient Gsl from data point t to data point s : ozst 1 oZt I-I- Gs,_, dout y The total change in output at s will be the sum of the product of the coupling coefficients and the applied error at all data points. Since the change in output should be equal to actual error, the total actual error at s should be equal to the sum of the product of coupling coefficients and the applied errors at all data points. To illustrate the meaning of this statement, the technique will first be applied to learn two data points. The applied errors (ERR, and ERR,) that should be applied in the two cases to exactly eliminate the actual errors (errorl/and errorl,) at the two data points can be determined from the solution of the following matrix : 0 G, ERR error , G,, 1 ERR, error (46) The above equation can be generalized to an arbitrary number of data points as follows : 1 Gl 2 GI N-ERR, error l, 2. 1- 2'2 G21 1. G, N EPR, = error 12 (47) , (47) GN1 GN2 1 ', y 2YYDYIN This is a linear equation wherein the actual errors on the right are the product of the coupling coefficients and the applied errors on the left.

In a preferred embodiment of the present invention, the Adjustment Block 105 of Fig. 2 achieves overall convergence at a dramatically accelerated rate by the foregoing method. The Adjustment Block in Fig. 2, does not average the correction in parameters from a number of data points. Rather, it uses the above matrix technique to

calculate an array of applied error values (EAR,) from an array of actual error values (errorlt). To accomplish this, the Adjustment Block includes a coupling calculator that calculates the coefficients used in the matrix in Equation (47).

A person skilled in the art would know that an array of ERR values can be selected to satisfy Equation (47) using standard linear algebra techniques such as Gaussian elimination. After the array of ERR values has been calculated, it is used in the following equation to calculate the change in parameters. A a j I- = N _ da Beta Where : 2 Beta"_ da j=o n In the above discussion we where training a power series. The equations must be modified to use derivative variable etc. Also, the data points may now be obtained from different outputs. The equations that must be modified are Equations (45), (48) and (49). The structure of Equation (45) can be changed to : where o and are functions of s and t respectively.

In the matrix shown in Equation (47), each equation refers to a different data point. As a result then the error on the right side of the equation and the derivative variable used on the left must both have been collected from the same data point. A data point refers to an output's error and all its derivative variables collected from one output, all at the same instant.

As a review, after the matrix has been solved and the array of ERR values determined, the following equations are used to determine the change in the parameters.

Where :

o is a function of 7l.

Which of the two training techniques that should be used in a particular situation, should be determined by the number of data points, etc. A fact that should be remembered is that the reiterative technique can be used under all conditions.

C. Evaluating the Determinant Before using a matrix technique it is useful to first evaluate the matrix's determinant to determine if the matrix technique can be used. If the value of the determinant is or approaches zero, the matrix technique can not be used. If a determinant is defined by : aoo aoi aoir D (5')) ano anl 1 ann The value of the determinant can be determined from : n value = ß aOjC0j, j=o where Co is the cofactor of term CL. In the above equation, the value of the determinant is determined by expanding the determinant by the first row. The indices of the coordinates of the terms in the determinant have been changed to correspond to an index notation commonly used in computer programs, namely that the index numbering begins with zero, such as in the C programming language.

In this equation the cofactor of a term is defined as the determinant that remains after the row and column of the term have been removed. Each one of these cofactors can in turn be evaluated used the same procedure used in Equation (54). Each time this reiterative procedure is used, the size of the determinant that must be delt with is reduced by one. This technique can continue until the size of the determinant is only a single term. When this occurs the value of the cofactor is simply the value of the remaining term.

The technique that has been choosen for passing the determinant of coefficients to the procedure is a float pointer. The size of this array of float will be the product of the number of row by the number of collumns, and for the matrix shown in Equation (43) this number will be (n+1) 2. The order in which these terms appear in the array is arbitrary but order that was choosen was to list all items in the first row. then all the items in the second row, etc. Then within each row the terms are listed in increasing column order. The code which can be used to evaluate a square determinant follows :

1 float Train Class::EvaluateDeterminant(float*pLeft, int Size) 2 { 3 float* pLeft1 ; 4 float fAnswer, fTerm ; 5 int iCol, iRow, iCol ; 6 if (Size <= 1) 7 return (*pLeft) ; 8 fAnswer=0. 0 ; 9 pLeft1=(float*)malloc(Size-1) * (Size-1) *sizeof (float)) ; 10 for (iCol1=0 ; iCol1< Size ; iCol1++) 11 { 12 for (iRow=1 ; iRow< Size ; iRow++) 13 { 14 for (iCol=0 ; iCol< (Size-1) ; iCol++) 15 { 16 if (iCol<iCol1) 17 * (pLeft1 + (iRow-1) * (Size-1) +iCol) = 18 * (pLeft+iRow*Size+iCol) ; 19 else 20 * (pLeft1 + (iRow-1) * (Size-1) +iCol) = 21 * (pLeft+iRow*Size+iCol+1) ; 22} 23} 24 fTerm= ' (pLeft+iCol1) ; 25 if (Odd (iCol1)) 26 fAnswer +=-fTerm*EvaluateDeterminant(pLeft1, Size-1); 27 else 28 fAnswer += + fTerm*EvaluateDeterminant (pLeft1, Size-1) ; 29} 30 return (fAnswer) ; 31} 32 int TrainClass : : Odd (int input) 33 { 34 if ((input% 2) ==1) 35 return (TRUE) ; 36 else 37 return (FALSE) ; 38} A major part of this procedure is the construction of the determinant of the cofactor. This is done in the nestedfor loops between lines 12 and 23. The first for loop on line 12 steps through the rows of the determinant being constructed, while the second for loop on line 14 steps through the columns of the determinant being constructed. The location of the term of the cofactor determinant being constructed is in row zero and at the column position determined by integer variable iColl. This variable is used on line 16 and 24, and has its value determined by the for loop on line 10. Note the similarity of the code

on lines 26 and 28 to Equation (54). The choice of which of the two equations are used is determined by the odd properties of the column of the term. This decision is made by the code on line 25. The value of the term is determined by line 24. The value of the float fAnswer is changed by lines 26 or 28 on each pass through the for loop controlled by line 10.

This recursive process will continue reducing the size of the determinant by one on each recursion. However when the size of the determinant is one the value of the variable Size will be one and the code on lines 6 and 7 will return the value of the remaining variable in the determinant.

A problem with the training procedures discussed is the existence of second order derivative variables. Their existence limits the ability of the change variable to predict the actual change in the state machine's output. Such a statement is most applicable to the reiterative technique but is also applicable to the matrix technique.

If the change in the parameters is very large, the change in the state machine's output will differ from that predicted. The main problem is then to limit the change in parameters that will result from any learning experience.

VI. TECHNIQUE FOR PROCESSING A STATE MACHINE For the sake of completeness of explanation, the following discussion is provided. The discussion includes an explanation of the foregoing equations and techniques as shown in blocks 162 and 163 of Figs. 3, 4, and 5, as well as the techniques shown in Figs.

7 and 8. It is believed that the subject matter of this discussion may be found in other published sources.

The purpose of this discussion is to develop and explain techniques for predicting the next state of a state machine given its present state and its controlling state equations as defined by Equations (1), (2) and (3). Equation (1) is not in the form that is used to simulate a state variable system. The form shown in Equation (1) is the differential form while the form that is actually used during the simulation is the integral form. The integral form of Equation (1) is : The system of state equations will be processed so that the present state is used to predict where it will be a small time interval from now. This small time interval is At, sometimes referred to as h. As a result of this processing sequence, the variables can be treated as if their values are only defined at the discrete values of h. By using these discrete

values to define a polynomial that passes through these data points, this polynomial can be integrated or differentiated and the resulting values can be used as approximations to the value obtained by integrating and differentiating the real function. This technique is a key feature of the multipoint method. The implementation of this technique can look very different depending on the number of data points used to generate the polynomial. The multipoint method that will be developed in this discussion will use three points to determine the polynomial. To develop-this polynomial an L function will be used. The L function is defined so that Lu (tu equals one when i =j, while Lj equals zero when i xj. To start this process the functionf (y) can be approximated as the following weighted sum of L functions. f (y)=L1(t)f1+L2(t)f2+L3(t)f3.

The value offj, 7, andt3 are constants and are the values of the functionf (y) at intervals t/, t"and t3 respectively. The way the technique is normally used is that, ils the present value of f(y) and f2 and f3 are previous values separated by a common time interval.

When the system of equations are processed, the following iteration is made : , f2=f2, f1=f(#).

The size of the interval between ti and t"and t2 and t3 will be h. The L functions are polynomials of t that are designed so that L, (t,) equals one and that Ll (t,) and LI both equal zero. This can be done by defining the L functions as follows : <BR> <BR> <BR> <BR> (t-t2)(t-t3) (t-t1) (t-t3) (t-t1) (t-t2)<BR> L1(t)=#, L2(t)=#, and L3(t)=#.<BR> <BR> <BR> <BR> <BR> <P> (t1-t2)(t1-t3) (t2-t1)(t2-t3) (t3-t1)(t3-t2) This equation can be worked with more easily by changing variables. In this discussion z is used and define to be zero at tl. The locations of relative to t, is minus h, and the location of t3 relative to t is minus h. Using this information the L function can be defined as functions of z : 1/2h2(z+h)(z+2H), L2(z)=-1/h2z(z+2h), and L3(z)=-1/2h2z(z+h).

Recall that the original problem is to integrate the function f(y) over the interval ti to tl+h. This is equivalent to integrating z from O (zero) to h : The value of this integral can be approximated by :

These integrals can be evaluated in a straightforward manner leading to the following equation : To prevent confusion over notation, some other technique must be used to represent a time index. This is because the subscript to thef function is being used to define whichf function is being used. As a result Equation (58) will be rewritten as : To rewrite Equation (59) to represent an array of equations : The same procedure can be used to calculate the derivative of a variable whose value is defined at set intervals. By taking the derivative of both sides of Equation (56) with respect to z, the result is : <BR> <BR> <BR> <BR> <BR> <BR> d(f(#)) d(f(#)) dL1 dL2 dL3<BR> <BR> <BR> <BR> #=#=f1#+f2#+f3<BR> dt dz dz dz dz The objective of this exercise is to apply the results of Equation (61) to the evaluation of the derivative used in Equation (2), repeated here for convenience as Equation (62).

Y,-dYn dt (62) The value of this derivative is required for z =0. Using the notation shown in Equation (61), the result is : Yr =1/2h[-3yn|1+4yn|2-yn|3] (63) Before the sequence of equations can be reiterated to determine the new state of the state machine, the previous value of the functions and variables must be determined.

In both Equations (59) and (63), assume the existence of previous values of the function or variable. When the simulation is first started this information will not be available. To start the simulation it is suggested that a rectangular approximation integration procedure be used at a much smaller step size or timing interval. In Figs. 3, 4, and 5, the multi-point method is used to reiterate the sequence of equations to determine its new state. To start the sequence Fig. 7 is used. In Fig. 7 the data required to use the multi-point method is calculated while

the state equations are iterated using the rectangular integration procedure at a smaller step size than that used in Figs. 3, 4 or 5. In the rectangular integration procedure, the function is assumed to have a constant value over the duration of the integral. Graphically the area under the curve added to the value of the variable looks like a rectangle-hence the name, rectangular integration procedure.

VII. USING MULTIVARIABLE POWER SERIES FOR THE FUNCTIONS IN EQUATIONS CD AND (3) According to the present invention, a technique that can be used to construct the functions used in Equations (1) and (3) is a multivariable power series. A power series is defined as a weighted sum of product terms. The highest power of a variable in the power series is defined as the order of that variable. As an example : I. J. K i j k outpzct = a, k x x (64) i, y, xi-=o The parameter lazzis the weight of the term XX7X3k. In this sum, the highest power of the variable x, is I. I is said to be the order of the variable x.

The purpose of this discussion is to outline the C++ code that can be used to calculate : 1) the value of a multivariable power series ; 2) the derivative of output of the power series with respect to all of the parameters used in its construction ; and 3) the derivative of the output of the power series with respect to the variables used in its construction.

A. Evaluating the Value of Multivariable Power Series A technique that can be used to build a power series of one variable is to repeat the sequence of adding a parameter then multiplying the result by the variable.

This procedure is shown by the following equation : output = ((((ase + a,) x + a3) x + a,) x + a,) x + ao.

To illustrate how this technique can be used to build a multivariable power series, replace each parameter in the above equation with a power series of another variable.

This procedure is used to build a power series of three variables, each variable raised to the first power. output = ((a", x, + al0) x2 + (a101x1+a100))x3 (65) + (a011x1+a010)x2+(a001x1+a000) The number of multiplications required to evaluate Equation (65) is seven and the number of additions is seven. This is the minimum amount.

There are several techniques that can be used to make these substitutions.

The normal C programming structure would be nestedfor loops. The problem with this structure is that a different program must be written for each different control structure.

Another control structure that can be used is recursive calls to the same procedure. There are a number of techniques that can be used to define this procedure. The technique I have chosen is that each nesting reduces the value of a number until the value of zero is reached.

When the zero value is reached the-procedure no longer calls itself but instead builds the base polynomial using actual parameters. A flow graph outlining this recursive procedure is shown in Fig. 9.

Before the procedure shown in Fig. 9 can be started, the array of variables used to calculate the power series'output must be defined. This array is defined in an array of floats pointed to by the pointer xStr. The number of variables contained in this array is stored in an integer niiin var. The parameters used to calculate the power series'output are contained _pur. The order of the resulting polynomial with each input variable is defined by an array of integers order. The function cal out (n) has an entry point in block 262. The block between this entry point and return blocks 270 and 277 are used to define the functions. In block 272 the value returned by call to function cal_out (n-l) is answer,_, and is assigned to the value of answer". In block 275, the value returned by the call to cal out fz-1) is answer,_, and is added to the value of answer,. The function is first called in block 261 with an integer equal to one less than the number of variables. The result of this call will be to have the value of the multivariable power series put in the float value. All dashed lines to the function's entry point block 262 represent calls to the function. The value in the parenthesis of the functional call becomes the value of n in the function. As an example when the function is called from block 261 the value of num_var-l becomes the value of n in this functional call. Also when the function is called from blocks 272 and 275, the value of n-l becomes the value of n on this functional call.

The functional call ends when a return statement is reach at which time you return to the statement originating the call. Each recursive call to the function reduces the value of the input integer by one. This process continues until in control block 263, n has the value of zero. Then in blocks 264 through block 269, the actual parameters are used to build a power series. In blocks 271 through block 276, the power series is built not with parameters but the values returned from the cal out (n-1) in place of the parameters. Each call to the function cal-out (n-1) return the value of a power series built with the same variables only constructed with different parameters. In block 271 through block 276 the value of these calls are used

to build another power series with variable n using power series of variables whose indexes are n-I through zero as building blocks. Control block 263 is used to decide if the power series being built will be built with actual parameters or with other power series. The parameter used in the construction of the power series is determined by an array of integers, it7. The function location () uses this array of integers, iL1, to calculate the index to an array of parameters. Each pass through the loop of block 273 through block 276 or block 266 through block 269 changes value of an integer ilnJ, and hence will cause the call to the function cal out (n-1) to build a power series using different parameters. Both the blocks, 273 through 276, and blocks 266 through 269, build the power series by first starting with parameter or power series and then successively multiplies the results by a variable and then adds a parameter or power series.

The procedure shown in Fig. 9 is the same procedure used to construct Equation (65). The equation is developed left to right. This means that the parameters will be required in descending order. The mathematical procedure used to determine the index of the parameter array can be simplified using this information.

The code used to implement the flow graph of Fig. 9 will now be shown : long sp_type : : location (void) 2 { 3 int degree ; 4 long answer ; 5 answer=i [var_num-1] ; 6for (degree=varnum-1 ; degree> 0 ; degree--) 7 { 8 answer *= order [degree] +1 ; 9 answer += i [degree-1] ; 10} 11 return (answer) ; 12} 13 float sp_type : : cal_out (intn) 14 { 15 long offset ; 16 int temp ; 17 float answer ; 18 if (n > 0) 19 { 20 i [n] = order [n] ; 21 answer=cal out (n-1) ; 22 for (i [n] = order [n]-1 ; i [n] >= 0 ; i [n]--) 23 { 24 answer *= * (x_ptr+n) ; 25 answer += ca ! out (n-1) ; 26} 27} 28 else 29 { 30 i [0] = order [0] ; 31 offset=location () ;

32 answer= * (a_ptr+offset) ; 33 for (i [0] = order [0]-1 ; i [0] >= 0 ; i [0]--) 34 { 35 answer*=*x_ptr ; 36 offset=location () ; 37 answer += * (a_ptr+offset) ; <BR> <BR> 38}<BR> <BR> <BR> <BR> <BR> <BR> <BR> <BR> 39} 40 return (answer) ; 41} Note in lines 1 through 12 where the value of the function location () is calculated. Where the loop starts is determined by the value of nus var which indicates how many variables are used. The control block 263 of Fig. 9 has been replaced by the code on line 18. The code between lines 20 through 26 replaces blocks 271 through 276, while the code between lines 30 through 38 replaces blocks 264 through 269.

B. Calculatina Derivative of Output with Parameters There are two sources of the derivative of a function's output with respect to a parameter. The reasons for this are : (1) the function actually contains the parameter in question ; and (2) the input variables are derivatives of this parameter. The discussion in this section will only be concerned with the derivative that results, because the function actually contains the parameter in question. The objective of determining these derivatives will be so they can be used to train the state machine. The parameters that will control the behavior of the state machine will be in the parameters of the power series defining the functions in the set of equations used to define the state machine.

Fig. 10 is a flow graph describing the technique used to calculate the derivative of the output of a power series or function with respect to the parameters used to calculate its output. The value of the derivatives being calculated will be in an array pointed to by pointer dout_daStr. For a particular parameter, the index to this array will correspond to the index to pointer aStr, which stores the value of all parameters. As in Fig. 9 the function used to make this calculation has an entry point. This entry point is block 282 and all dashed lines to this block represent calls to this function. The function cal_dout_daCn, index, value) contains three variables, n, index, and value. The first two n and index are integers while value is a float. The value of these variables are assigned at the time the function is called.

The first call to this function is from block 281. At this time the values of the variables n, index and value are assigned the values of num_var-l, zero and 1. 0, respectively.

The structure of the procedure used in Fig. 10 is very similar to the procedure used in Fig. 9.

The dashed line between blocks 296 and 282 denotes a recursive call in which the value of 77 is reduced by one on each recursion. The values shown in the function cal_dout_da(n- l, index+jn, value) in block 296 become the values in cal_dout_da(n, index, value) in block 282. Note it is only in blocks 285 and 288 that the values are assigned to the derivatives * (dout dnptr+index+jo). In the loop of block 295 through 298 only the index and n are change on each recursive call of the function cal_dout_da().

The following code-can be used to implement the flow graph of Fig. 10 : void sp_type : : cal_dout_da (int n, int index, float value) 2 { 3 int j ; 4 if (n> 0) 5 { 6 index *= order [n] +1 ; 7 cal_dout_da(n-1, index, value) ; 8 for (j=1 ; j<= order [n] ; j++) 9 { 10 value *= * (x_ptr+n) ; 11 cal_dout_da (n-1, index+j, value) ; 12}<BR> <BR> <BR> 13}<BR> <BR> <BR> 14 else 15 { 16 index *= order [0] +1 ; 17 * (dout_da_ptr+index) =value ; 18 for (j=1 ; j<= order [0] ; j++) 19 { 20 value*=*x_ptr ; 21 *(dout_da_ptr_index+j)=value ; 22} 23} 24} The code on line 4 corresponds to the control block 283 in Fig. 10. The code between lines 6 and 12 implement the blocks 292 through 298, while code between lines 16 and 22 implement the blocks 284 through 290.

C. Calculating the Derivative of a Power Series'Output with Its Input The key to determining the derivative of the output of a multivariable power series with respect to a particular input is to rearrange Equation (59) to be : J I K output = y 1 1 ajkXlX3 (66) j=0 i, k=O By taking the derivative of Equation (61) with respect to x, the results is :

The code that can calculate this derivative can be broken into two parts. One part will calculate the value of the inner summation which is called cal_deriv(num_var-2), and the other which calculates the value of the outer summation called cal_dout_dy(v_num).

The code used for calculating the value of cal_dout_dy(v_num) follows : 1 float sp_type : : cal_dout_dy (int v_num) 2 { 3 int j, jj ; 4 long offset ; 5 float temp, value, y1, answer ; 6 for (jj=0 ; jj< numvar ; jj++) 7 { 8if (jj==vnum) 9 { 10/* do nothing */ 11} 12 else 13 { 14 num [j] =jj ; 15 j++ ; <BR> <BR> 16}<BR> <BR> <BR> 17} 18 y1=*(x_prt+v_num) ; 19 value=1.0; 20 answer=0 ; 21 if(num_var>=2) 22 { 23for (j=1 ; j<= order [v_num] ; j++) 24 { 25 i [v_num] j ; 26 temp=cal_deriv(num_var-2) ; 27 answer += float (j) *value*temp ; 28 value *= y1 ; 29} 30} 31 else 32 { 33 for (j=1 ; j<= order [v_num] ; j++) 34 { 35 i [v_num] =j ; 36 offset=location () ; 37 temp=*(a_ptr+offset) ; 38 answer += float (j) *value*temp ; 39 value *= y1 ; <BR> <BR> 40}<BR> <BR> <BR> 41} 42 return (answer) ; 43} One of the first thing cal_dout_dy(v_num) function does is to build the array num[]. This array of integer will contain all number from zero to var_num-l except the number passed to the function as v_num. The function cal out () uses the variables in a particular order. The value of the integer v_num correspond to the order the variables are used in in the normal use of the function cal_out(). One of the first task of

cal_bout_dy(v_num) is to construct an array of number in which v ntini has been eliminated.

This array of number is used by cal_deriv(var_num-2) to calculate the value of the inner summation in Equation (65). The code between lines 23 and 29 evaluate the output sum.

The if statement on line 21 determines if the function cal-derive will be used. If the power series has only one variable, var_num_wil have a value of one and as a result cal-derive would be called with an integer whose value is less than zero. To properly calculate the derivative of the output with-respect to a single variable input, the code between lines 33 and 40 must be used.

The code for the function cal_deriv(var_num-2) follows : 1 float sp type : : cal_deriv (int n) 2 { 3 long offset ; 4 int temp ; 5 float answer ; 6 temp=num[n] ; 7 i [temp] = order [temp] ; 8 if (n > 0) 9 { 10 answer=cal deriv (n-1) ; 11 for (i [temp] = order [temp]-1 ; i [temp] >= 0 ; i [temp]--) 12 { 13 answer *= * (x_ptr+temp) ; 14 answer += cal-derv (n-1) ; 15} 16} 17 else 18 { 19 offset=location () ; 20 anwser=*(a_prt+offset) ; 21 for (i [temp] = order [temp]-1 ; i [temp] >= 0 ; i [temp]--) 22 { 23 answer *=(x_ptr+temp) ; 24 offset=location () ; 25 answer+= * (a_ptr+offset) ; 26} 27} 28 return (answer) ; 29} The code for cal_deriv(var_num-2) is very similar to the code for cal_out().

The primary difference is in lines 6 and 7 where the array nus (J is used to control the order the variables and parameters are used in the construction of the power series. Both functions are recursive, use the function locationo to determine which parameter to use, and have a control statement to test the value of the integer the function is called with.

VIII. CONVERTING STATE MACHINE SYSTEM TO POWER SERIES AND Z-TRANSFORMS There are circuits within z-transform whose behavior approximates that of integrator and differentiator. The circuit capable of differentiating is shown in Figure I la.

This circuit will differentiate if the values of parameters b1 and b2 are assigned the following values : b, = + and b, where h is the delay caused by The normal z-transform whose behavior most closely approximates that of an RC time constant circuit is shown in Figure lib. The problem with this circuit is that the constant level input gain is very dependent on the value of a,. The constant level input gain is determined by : DC Gain = 1-a, What is desired is a circuit in which the time constant and the constant input gain can be varied independently. The circuit shown in Figure lie satisfies these requirements. This circuit contains two parameters a and b. Parameter b controls the circuit's constant input gain, while parameter a controls the circuits time constant. It is now possible by combining elements from Figure 11 a and 11 c with the blocks representing multivariable power series to construct a model of the state machine used in the Controller AzIodule 110.

A state machine system can then be simulated by a system of z-transforms and multivariable power series. A summary of the conclusions are : 1) that a delayed sample feedback to a summer is functionally equivalent to an integrator, and 2) that a difference between the present value of a function and its delayed sample is equivalent to differentiating.

The z-transform technique just discussed results in a rectangular approximation to integration. This technique can be generalized to become a graph representation of the multipoint integration and differentiation. Using the three point method to determine the present value of a derivative, the following equation is used : dx[n]#1[-3x[n]+4x[n-1]-x[n-2]], (68) dt 2h where h is the sample interval.

Equation (63) can be reduced to the z-transform flow graph shown in Figure <BR> <BR> 12awith b, =-, b2 =e ~ : and b3 =-.<BR> <P> '2h'-h'2Ji Similarly, the three point method of integration as shown by Equation (58), and repeated here for your convenience : can be reduced to the z-transform flow graph shown in Figure 12b with <BR> <BR> <BR> <BR> 23h 4h 5h<BR> <BR> <BR> @=+#; c2=-#; and c3=+#.<BR> <BR> <P> 12 3 12 Using the two z-transforms just developed, the system of state equations can be reduced to a system of non-linear multivariable power series and z-transforms.

However, these are not ordinary z-transforms, but a restricted set of z-transforms combined with the power series in a very orderly manner.

IX. ALTERNATE DESIGN OF POWER SERIES STATE MACHINE The design of the state machine previously discussed was based on the use of state equations and how to modify these state equations. The new set of state equations is used for processing the derivative variables, where the derivative variables were used to train the system. A technique exists according to the invention to simulate the structure using multivariable power series and z-transforms. Since this system of multivariable power series and z-transforms can be trained, it follows that any system built with these same components will also be trainable. An alternate design exists for construction of a power series state machine. This design is to take the output of a power series and put it through a clock delay and feed it back to the input of the power series.

This design is as simple as that shown in Fig. 13. Another way of looking at the use of derivative variables is that the reason the concept works is that it couples the linear concepts of integration and differentiation with the concept a complete derivative. The concept of a complete derivative refers to the following : If yj = f (y), then In other words, the complete derivative of a function with a parameter is defined as the derivative of the function with the parameter plus the sum for all variables of the product of the derivative of the function with a variable inputted, and the derivative of this

variable with the parameter. By coupling this concept with the linear operation of integration, you have state equations for processing derivative variables.

In Fig. 13 with the multivariable power series block 301, the inputs come in the top and the outputs of the many power series it contains are fed out the bottom.

Along with the level signals, the level derivative variables are also passed around through the loop of the multivariable power series and the clock delay block 302. The inputs to the system are supplied to clock delay block 302 and consist of only level signals. The output from the system comes from the clock delay block and comprises both level signals and level derivative variables. Because of the design of the system, the behavior of the signals can be discontinuous at the clock edge. This means the system is capable of doing binary logic. The output from the system could be designed not to do anything unless they are above a high threshold or below a low threshold. Otherwise, the signal can be considered unchanged from their previous state. As with any state machine the output derivative variables can be used to train the system using the same technique as used to train the state machine containing integrators, etc.

X. PRACTICAL APPLICATIONS The practical applications of a trainable state machine are wider than that of a non-trainable state machine. A state machine could be used in any situation in which the object being controlled must vary with time. This could be the positioning of a robotic arm to perform a task. Power series etc. could be used to map the variables that should be feed to a robotic arm to put it in a certain position, but getting it to that position in a reasonable time period is a dynamic process and could be easily be controlled by a state machine. This state machine could be trained by receiving positional error information on where it should be and when it should be there after the differential equation in the state machine has been initiated.

When an individual has spinal injury, the machinery of the legs are still intact but the mechanism for their control is not. I have recently heard of using pacemaker technology to control muscle in the limbs. If, given the ability to control enough muscles and enough sensor information on leg position etc., a state machine could be trained to control the leg to simulate normal walking, etc. By controlling this state machine a crippled person could in essences control his legs in the same way he now controls a wheelchair. The main advantage to this system would be that the programming would be limited to making the processor simulate a state machine and use the feedback on position errors etc. to correct its behavior by adjustment of parameters. Another part

of the system could contain a model of the desired behavior and supply the required desired positional information required for training. When occasionally the operator would fall over, the system would also learn to maintain the required balance.

The invention has been explained with reference to specific embodiments.

Other embodiments will be apparent to those of ordinary skill in the art. It is therefore not intended that this invention be limited, except as indicated by the appended claims.