The present invention relates to methods and systems for synchronization taking into account delay asymmetry. It is particularly, but not exclusively, related to methods and systems which make use of the delay asymmetry to improve the accuracy of the estimation algorithm used to estimate the phase offset in a synchronization algorithm (for example under IEEE 1588 PTP).

Background of the Invention

Timing synchronization using a protocol such as IEEE 1588 PTP and a well-designed slave clock recovery mechanism can provide time synchronization in the sub-microsecond level and lower. Time synchronization requires an accurate measurement of the communication path delay between the time server (master) and the client (slave) in order to compute the time offset between them.

The PTP message exchange process (i.e., the PTP Delay Request/Delay Response flow) between a master and a slave is illustrated in Figure 1 and described below.

IEEE 1588 PTP allows for two different types of timestamping methods, either one-step or two-step. One-step clocks update time information within event messages (Sync and Delay_Req) on-the-fly, while two-step clocks convey the precise timestamps of packets in general messages (Follow_Up and Delay_Resp). A Sync message is transmitted by a master 1 to its slaves 3 and either contains the exact time of its transmission or is followed by a Follow_Up message containing this time. In a two-step ordinary or boundary clock, the Follow_Up message communicates the value of the departure timestamp for a particular Sync message.

Figure 1 illustrates the basic pattern of synchronization message exchanges for the two-step clocks. The master 1 sends a Sync message to the slave 3 over the packet network 2 and notes the time T ₃ , at which it was sent. The slave receives the Sync message and notes the time of reception τ ₂ . The master 1 conveys to the slave 3 the timestamp T ₃ , by one of two ways: 1 ) Embedding the timestamp r, in the Sync message. This requires some sort of hardware processing (i.e., hardware timestamping) for highest accuracy and precision. 2) Embedding the timestamp τ ₁ a Follow_Up message which is the sent to the slave (as shown in Figure 1 ). Next, the slave 3 sends a Delay_Req message to the master 1 and notes the time T ₃ at which it was sent. The master receives the Delay_Req message and notes the time of reception T ₄ . The master 1 conveys to the slave 3 the timestamp r, by embedding it in a Delay_Resp message.

At the end of this PTP message exchange, the slave 3 possesses all four timestamps {τ _ι , T ₂ , T ₃ , T ₄ }. These timestamps may be used to compute the offset of the slave's clock 5 with respect to the master clock 4 and the mean propagation time of messages between the two clocks. The computation of offset and propagation time assumes that the master-to-slave and slave-to-master propagation times are equal, i.e. that there is a symmetrical

communication path. Clock frequencies change over time, so periodic message exchanges are required. Because these clock variations change slowly, the period between message exchanges is typically on the order of milliseconds to seconds.

In most existing approaches, the path delay estimation is based on the important assumption that the time delay from master to slave is equal to that from slave to master. However, in reality, the communication paths are not perfectly symmetric, mainly due to dissimilar forward and reverse physical link delays and queuing delays. Even in cases where the physical link delays are known and properly compensated for during clock synchronization, queuing delays, which are variable, can still exist when timing messages go through the packet network and are queued for forwarding. In particular, the processing and buffering of packets in network devices (switches, routers, etc.) introduce variations in the time latency of packets traversing the packet network. This mainly happens when timing transfer is done in an end- to-end manner without any form of timing assistance from the network to help mitigate the effects of the variable queuing delays.

Variable queueing delays and any asymmetry in the forward and reverse queuing delays directly lead to increased errors in the phase offset estimation between a master and slave. US 9,112,628B describes a technique which enables the forward queueing delays to be approximately tracked sample-by-sample within a window but with an unknown fixed offset for the duration of the window. Similarly, the reverse queueing delays can be approximately tracked sample-by-sample but with a different unknown offset. In US 9,112.628B these forward and backward queue delay (and offset) approximations are used in an additional step after the Kalman filter estimation of phase offset to modify and improve the estimate.

US 9,112,628B discusses how the error in the phase offset estimation error, which is caused by queueing delay asymmetry, can be used to produce a phase offset estimate which more accurately represents the true phase offset between master and slave. This delay asymmetry compensation process is applied after the Kalman filter estimation of the phase offset as illustrated by equation 4 and the subsequent discussion in that document. Equation 25 of that document also suggests that the measurement noise should be coloured by the difference between the change in the forward delay between consecutive samples and the change in the reverse delay between consecutive samples, i.e., by Δqf(n)— Δq _r (n).

US 9,184,861 B uses the variance of the difference between the forward and reverse message packet delay variations ("PDVs") as the measurement noise term for the Kalman filter realisation. This measurement noise term is used directly by the Kalman filter estimation algorithm. The approach in US 9,184,861 B2 is to accept that there is noise introduced by the queueing delays and then try to represent this noise faithfully.

An object of the present invention is to provide an improved way of accounting for the delay asymmetry when estimating the phase offset of a slave clock.

A further object of the present invention is to provide a way of maintaining the accuracy of estimations during periods of high load on one of the forward and reverse channels in a timing network.

Summary of the Invention

At their broadest, aspects of the present invention provide for methods and systems which estimate the phase offset of a slave clock by taking into account an estimated measure of the asymmetry in the transmission paths between the time server and the time client.

A first aspect of the present invention provides a method of estimating a phase offset of a local clock in a time client compared with a master clock in a time server, connected to the time client over a network, the method including the steps of: sending first timing messages in a forward direction from the time server to the time client and second timing messages in a reverse direction from the time client to the time server, and recording timestamps which are the times of sending and times of receipt of the timing messages; estimating, for each message in a window of predetermined length, the transmission delay experienced by the message; determining, from said estimates, a minimum estimated delay experienced by timing messages in the forward direction in that window, and a minimum estimated delay experienced by timing messages in the reverse direction in that window; determining, for each message in that window, the difference between the estimated transmission delay for that message and the minimum estimated delay for messages travelling in the same direction as that message; determining, from said differences, a queuing asymmetry measure for each exchange of first and second timing messages in that window; estimating the phase offset of the local clock at a time in said window using said timestamps and said queuing asymmetry measure as inputs to an estimation algorithm.

The network may be a packet network (using technologies such Ethernet, IP, MPLS, etc.).

The timing messages are preferably timing messages under the IEEE 1588 Precision Time Protocol (PTP).

By using a method according to the present aspect, the noise introduced by the queueing delays can be reduced by using the estimations of the queueing delays.

Thus the method according to this aspect may significantly reduce the errors in phase offset estimation which result from variable and asymmetrical queueing delays.

In particular, the method of the present aspect preferably uses a queuing asymmetry measure (preferably based on good estimations of the difference in the forward and reverse delays) directly in an estimation algorithm (such as a Kalman filter algorithm) as an integral part of the estimation process, rather than as a modifier to be applied after the operation of the estimation algorithm. This can lead to efficiencies in that the estimate algorithm is able to converge on a solution much faster and arrive at a more accurate solution in the time available. In certain embodiments, the queuing asymmetry measure may be determined as: wherein: is an estimate of the queuing delay

experienced by the n-th first timing message; is an estimated of the queuing delay

experienced by the n-th second timing message; is the determined minimum value of

in said window; and is the determined minimum value of in said window. This particular queuing asymmetry measure has been found to be particularly close to the true queuing asymmetry in the delay for a particular pair of timing messages, whilst being calculated from values which are readily determined from the message exchange and estimations obtained in the method of the present aspect.

In certain embodiments the step of determining the queuing asymmetry measure includes: determining, from said timestamps, a forward displacement factor which is the difference between the interval between the times of receipt of successive first timing messages by the client and the interval between the times of sending of said successive first messages by the server; determining, from said timestamps, a reverse displacement factor which is the difference between the interval between the times of receipt of successive second timing messages by the server and the interval between the times of sending of said successive second timing messages by the client; maintaining a running sum of said forward displacement factors over a predetermined time period; maintaining a running sum of reverse displacement factors over a predetermined time period; and calculating the queuing asymmetry measure from said sums. In preferred embodiments, the estimation algorithm is a Kalman filer, which may, for example, use the measurement equation:

is the timestamp recorded by the server on sending of an n-th

first timing message from the server to the client is the time of receipt as recorded by the client on receipt of the n-th first timing message is the timestamp recorded by the

client on sending of an n-th second timing message from the client to the server; is the time of receipt as recorded by the server on receipt of the n-th second timing message; Q(n) is the queuing asymmetry measure; v _n is the measurement noise; and is an estimate

of the local clock offset.

By inputting the queuing asymmetry measure directly into the calculations of the Kalman filter, the Kalman filter may converge much more rapidly on its solution.

The method may further include determining the load conditions on the forward and reverse transmission paths. This may be of particular benefit in allowing stored values for minimum delays or minimum displacements from times when the load conditions are determined to be low to generate better estimates of minimum delays or minimum displacements when one of the transmission paths has high load conditions.

The load conditions on a particular path can cause significant bias and/or error in estimating minimum values and this can lead to errors or inefficiency in estimating phase offset. By storing minima during periods of low load conditions, these minima can be used later to estimate the "true" minima when a path is under high load and may be more accurate than the minimum determined from the timing messages sent under those conditions.

In particular embodiments, the method may further include the steps of: determining, from said timestamps, a forward displacement factor which is the difference between the interval between the times of receipt of successive first timing messages by the client and the interval between the times of sending of said successive first messages by the server; determining, from said timestamps, a reverse displacement factor which is the difference between the interval between the times of receipt of successive second timing messages by the server and the interval between the times of sending of said successive second timing messages by the client; maintaining a running sum of said forward displacement factors over a

predetermined time period; maintaining a running sum of reverse displacement factors over a predetermined time period; determining a noise condition of the forward transmission path and a noise condition of the reverse transmission path over said predetermined time period; if it is determined that both the forward and reverse transmission paths have a low noise condition during said predetermined time period, determining a minimum of the sum of forward displacement factors and a minimum of the sum of reverse displacement factors over that period of time and storing the determined minima as reference minimum values; and if it is determined during a future time period that one of the forward or reverse transmission paths has a high noise condition, whilst the other path has a low noise condition, using said reference minimum values and the minimum of the sum of

displacement factors on the path with a low noise condition to estimate a minimum value for the sum of displacement factors on the path with a high noise condition.

The step of determining a noise condition on a transmission path may involve: calculating, based on the timestamps, a variance of a queuing delay on that path; applying a first moving average having a first time constant to the variance; applying a second moving average having a second time constant greater than the first time constant to the variance; calculating a difference between the first and second moving averages; and comparing the calculated difference to a threshold to determine a noise condition on the path. For example, the moving averages may be exponentially-weighted moving averages (EWMAs).

Using a pair of moving averages allows determination of whether the current noise conditions (as represented by the moving average with the shorter time constant) have changed from an earlier steady-state period (as represented by the moving average with the longer time constant). The threshold can be set a-priori if the conditions of the detection environment are known in advance or the threshold can be determined experimentally during operation (or a combination of both may be employed).

An alternative way of determining the noise conditions is to use the error variance-covariance matrix from a Kalman filter which is used in said estimation algorithm. This matrix is normally referred to as P and is a square matrix with the form

where P _n is the error variance of the phase offset, P ₂₂ is the error variance of the skew (fractional frequency offset), and P ₁₂ and P ₂₁ represent the error covariance between the phase offset and the skew.

Thus in some embodiments of the invention, the determination of a noise condition on a transmission path involves: determining the error variance-covariance matrix

from a Kalman filter which is used in said estimation algorithm,

where P _{l t} is the error variance of the phase offset, P ₂₂ is the error variance of the skew (fractional frequency offset), and P ₁₂ and P ₂₁ represent the error covariance between the phase offset and the skew.

As the value P ₂₂ depends on the oscillator's stability, its variance, i.e., the variance of the P ₂₂ term itself, is extremely low (almost non-existent). P ₂₂ changes significantly only due to the external noise (and the primary external noise input is typically caused by forward and/or backward traffic load changes).

When external noise changes occur (e.g. the traffic load changes), P ₂₂ reflects this by converging to a new stable level of error variance. Due to the filtering properties of the Kalman filter, this new level is, again, very stable (i.e., there is almost no variance). However, the magnitude of the new level will be very distinct from the previous state, provided there was a significant enough change in the noise conditions. Therefore, P ₂₂ can be employed as input provide reliable indications of changing load conditions.

The method of the present aspect may include any combination of some, all or none of the above described preferred and optional features. The method of the above aspect is preferably implemented by a time client according to the second aspect of this invention or a networked time system according to the third aspect of this invention, as described below, but need not be.

Further aspects of the present invention include computer programs for running on computer systems which carry out the method of the above aspect, including some, all or none of the preferred and optional features of that aspect.

A second aspect of the present invention provides a time client having a local clock and a processor, and connected over a network to a time server having a master clock, wherein the time client is arranged to: receive first timing messages sent in a forward direction from the time server to the time client; and send second timing messages in a reverse direction from the time client to the time server, record timestamps which are the times of sending and times of receipt of the timing messages by the time client; and receive timestamps which are the times of sending and the times of receipt of the timing messages by the time server, wherein the processor is arranged to: estimate, for each message in a window of predetermined length, the transmission delay experienced by the message; determine, from said estimates, a minimum estimated delay experienced by timing messages in the forward direction in that window, and a minimum estimated delay experienced by timing messages in the reverse direction in that window; determine, for each message in that window, the difference between the estimated transmission delay for that message and the minimum estimated delay for messages travelling in the same direction as that message; determine, from said differences, a queuing asymmetry measure for each exchange of first and second timing messages in that window; estimate the phase offset of the local clock at a time in said window using said timestamps and said queuing asymmetry measure as inputs to an estimation algorithm. The network may be a packet network (using technologies such Ethernet, IP, MPLS, etc.).

The timing messages are preferably timing messages under the IEEE 1588 Precision Time Protocol (PTP).

In the time client according to the present aspect, the noise introduced by the queueing delays and the corresponding effects on offset estimation can be reduced by using the estimations of the queueing delays. Thus the time client of the present aspect may significantly reduce the errors in phase offset estimation which result from variable and asymmetrical queueing delays.

In particular, the processor of the time client of the present aspect preferably uses a queuing asymmetry measure (preferably based on good estimations of the difference in the forward and reverse delays) directly in an estimation algorithm (such as a Kalman filter algorithm) as an integral part of the estimation process, rather than as a modifier to be applied after the operation of the estimation algorithm. This can lead to efficiencies in that the estimate algorithm is able to converge on a solution much faster and arrive at a more accurate solution in the time available. In certain embodiments, the processor determines the queuing asymmetry measure as:

is an estimate of the queuing

delay experienced by the n-th first timing message; q _r is an estimated of the queuing delay experienced by the n-th second timing message; „ is the determined minimum

value of qf (n) in said window; and q _rm in ^]S the determined minimum value of ) in said window.

This particular queuing asymmetry measure has been found to be particularly close to the true asymmetry in the delay for a particular pair of timing messages, whilst being calculated from values which are readily determined from the message exchange and estimations obtained in the method of the present aspect. In certain embodiments the processor is arranged to determine the queuing asymmetry measure by: determining, from said timestamps, a forward displacement factor which is the difference between the interval between the times of receipt of successive first timing messages by the client and the interval between the times of sending of said successive first messages by the server; determining , from said timestamps, a reverse displacement factor which is the difference between the interval between the times of receipt of successive second timing messages by the server and the interval between the times of sending of said successive second timing messages by the client; maintaining a running sum of said forward displacement factors over a predetermined time period; maintaining a running sum of reverse displacement factors over a predetermined time period; and calculating the queuing asymmetry measure from said sums.

In preferred embodiments, the estimation algorithm is a Kalman filer, which may, for example, use the measurement equation

wherein: ) is the timestamp recorded by the server on sending of an n-th

first timing message from the server to the client; is the time of receipt as recorded by

the client on receipt of the n-th first timing message; is the timestamp recorded by the

client on sending of an n-th second timing message from the client to the server ) is the time of receipt as recorded by the server on receipt of the n-th second timing message;

is the queuing asymmetry measure; v _n is the measurement noise; and is an estimate of the local clock offset.

By inputting the queuing asymmetry measure directly into the calculations of the Kalman filter, the Kalman filter may converge much more rapidly on its solution. The processor may be further arranged to determine the load conditions on the forward and reverse transmission paths. This may be of particular benefit in allowing stored values for minimum delays or minimum displacements from times when the load conditions are determined to be low to generate better estimates of minimum delays or minimum

displacements when one of the transmission paths has high load conditions. The load conditions on a particular path can cause significant bias and/or error in estimating minimum values and this can lead to errors or inefficiency in estimating phase offset. By storing minima during periods of low load conditions, these minima can be used to estimate the "true" minima when a path is under high load and may be more accurate than the minimum determined from the timing messages sent under those conditions. In particular embodiments, the processor is further arranged to: determine, from said timestamps, a forward displacement factor which is the difference between the interval between the times of receipt of successive first timing messages by the client and the interval between the times of sending of said successive first messages by the server; determine, from said timestamps, a reverse displacement factor which is the difference between the interval between the times of receipt of successive second timing messages by the server and the interval between the times of sending of said successive second timing messages by the client; maintain a running sum of said forward displacement factors over a predetermined time period; maintain a running sum of reverse displacement factors over a predetermined time period; determine a noise condition of the forward transmission path and a noise condition of the reverse transmission path over said predetermined time period; if it is determined that both the forward and reverse transmission paths have a low noise condition during said predetermined time period, determine a minimum of the sum of forward displacement factors and a minimum of the sum of reverse displacement factors over that period of time and store the determined minima as reference minimum values; and if it is determined during a future time period that one of the forward or reverse transmission paths has a high noise condition, whilst the other path has a low noise condition, use said reference minimum values and the minimum of the sum of displacement factors on the path with a low noise condition to estimate a minimum value for the sum of displacement factors on the path with a high noise condition.

The processor may be arranged to determine a noise condition on a transmission path by: calculating, based on the timestamps, a variance of a queuing delay on that path; applying a first moving average having a first time constant to the variance; applying a second moving average having a second time constant greater than the first time constant to the variance; calculating a difference between the first and second moving averages; and comparing the calculated difference to a threshold to determine a noise condition on the path.

For example, the moving averages may be exponentially-weighted moving averages (EWMAs).

Using a pair of moving averages allows determination of whether the current noise conditions (as represented by the moving average with the shorter time constant) have changed from an earlier steady-state period (as represented by the moving average with the longer time constant). The threshold can be set a-priori if the conditions of the detection environment are known in advance or the threshold can be determined experimentally during operation (or a combination of both may be employed).

An alternative way of determining the noise conditions is to use the error variance-covariance matrix from a Kalman filter which is used in said estimation algorithm. This matrix is normally referred to as P and is a square matrix with the form

where P _±1 is the error variance of the phase offset, P ₂₂ is the error variance of the skew (fractional frequency offset), and P ₁₂ and P ₂₁ represent the error covariance between the phase offset and the skew. Thus in some embodiments of the invention, the processor is arranged to determine a noise condition on a transmission path by: determining the error variance-covariance matrix

from a Kalman filter which is used in said estimation algorithm,

where P ₁₁ is the error variance of the phase offset, P ₂₂ is the error variance of the skew (fractional frequency offset), and P ₁₂ and P ₂₁ represent the error covariance between the phase offset and the skew.

As the value P ₂₂ depends on the oscillator's stability, its variance, i.e. , the variance of the P ₂₂ term itself, is extremely low (almost non-existent). P ₂₂ changes significantly only due to the external noise (and the primary external noise input is typically caused by forward and/or backward traffic load changes).

When external noise changes occur (e.g. the traffic load changes), P ₂₂ reflects this by converging to a new stable level of error variance. Due to the filtering properties of the Kalman filter, this new level is, again, very stable (i.e. , there is almost no variance). However, the magnitude of the new level will be very distinct from the previous state, provided there was a significant enough change in the noise conditions. Therefore, P ₂₂ can be employed as input provide reliable indications of changing load conditions.

The time client of the present aspect may include any combination of some, all or none of the above described preferred and optional features.

The processor of the time client of the above aspect preferably operates by performing a method according to the first aspect of this invention, as described above, but need not do so.

A further embodiment of the present invention provides a networked time system having: a time server having a master clock; a time client having a local clock and a processor, and a network connecting the time server and the time client, wherein the time server is arranged to: send first timing messages in a forward direction to the time client, and receive second timing messages sent in a reverse direction from the time client, record timestamps which are the times of sending and times of receipt of the timing messages by the time server, and send said timestamps to the time client, wherein the time client is arranged to: receive said first timing messages; and send said second timing messages to the time server, record timestamps which are the times of sending and times of receipt of the timing messages by the time client; and receive timestamps which are the times of sending and the times of receipt of the timing messages by the time server, wherein the processor is arranged to: estimate, for each message in a window of predetermined length, the transmission delay experienced by the message; determine, from said estimates, a minimum estimated delay experienced by timing messages in the forward direction in that window, and a minimum estimated delay experienced by timing messages in the reverse direction in that window; determine, for each message in that window, the difference between the estimated transmission delay for that message and the minimum estimated delay for messages travelling in the same direction as that message; determine, from said differences, a queuing asymmetry measure for each exchange of first and second timing messages in that window; estimate the phase offset of the local clock at a time in said window using said timestamps and said queuing asymmetry measure as inputs to an estimation algorithm.

The network may be a packet network (using technologies such Ethernet, IP, MPLS, etc.).

The timing messages are preferably timing messages under the IEEE 1588 Precision Time Protocol (PTP).

In the time client according to the present aspect, the noise introduced by the queueing delays and the corresponding effects on offset estimation can be reduced by using the estimations of the queueing delays.

Thus the time client of the present aspect may significantly reduce the errors in phase offset estimation which result from variable and asymmetrical queueing delays.

In particular, the processor of the time client of the present aspect preferably uses a queuing asymmetry measure (preferably based on good estimations of the difference in the forward and reverse delays) directly in an estimation algorithm (such as a Kaiman filter algorithm) as an integral part of the estimation process, rather than as a modifier to be applied after the operation of the estimation algorithm. This can lead to efficiencies in that the estimate algorithm is able to converge on a solution much faster and arrive at a more accurate solution in the time available.

In certain embodiments, the processor determines the queuing asymmetry measure as:

wherein: is an estimate of the queuing

delay experienced by the n-th first timing message; is an estimated of the queuing

delay experienced by the n-th second timing message is the determined minimum value of q/(n) in said window; and is the determined minimum value of in said

window. This particular queuing asymmetry measure has been found to be particularly close to the true asymmetry in the delay for a particular pair of timing messages, whilst being calculated from values which are readily determined from the message exchange and estimations obtained in the method of the present aspect. In certain embodiments the processor is arranged to determine the queuing asymmetry measure by: determining, from said timestamps, a forward displacement factor which is the difference between the interval between the times of receipt of successive first timing messages by the client and the interval between the times of sending of said successive first messages by the server; determining, from said timestamps, a reverse displacement factor which is the difference between the interval between the times of receipt of successive second timing messages by the server and the interval between the times of sending of said successive second timing messages by the client; maintaining a running sum of said forward displacement factors over a predetermined time period; maintaining a running sum of reverse displacement factors over a predetermined time period; and calculating the queuing asymmetry measure from said sums.

In preferred embodiments, the estimation algorithm is a Kalman filer, which may, for example, use the measurement equation

is the timestamp recorded by the server on sending of an n-th first timing message from the server to the client; is the time of receipt as recorded by

the client on receipt of the n-th first timing message is the timestamp recorded by the

client on sending of an n-th second timing message from the client to the server; is the time of receipt as recorded by the server on receipt of the n-th second timing message; Q(n) is the queuing asymmetry measure; v _n is the measurement noise; and is an estimate of the local clock offset. By inputting the queuing asymmetry measure directly into the calculations of the Kalman filter, the Kalman filter may converge much more rapidly on its solution.

The processor may be further arranged to determine the load conditions on the forward and reverse transmission paths. This may be of particular benefit in allowing stored values for minimum delays or minimum displacements from times when the load conditions are determined to be low to generate better estimates of minimum delays or minimum displacements when one of the transmission paths has high load conditions. The load conditions on a particular path can cause significant bias and/or error in estimating minimum values and this can lead to errors or inefficiency in estimating phase offset. By storing minima during periods of low load conditions, these minima can be used to estimate the "true" minima when a path is under high load and may be more accurate than the minimum determined from the timing messages sent under those conditions.

In particular embodiments, the processor is further arranged to: determine, from said timestamps, a forward displacement factor which is the difference between the interval between the times of receipt of successive first timing messages by the client and the interval between the times of sending of said successive first messages by the server; determine, from said timestamps, a reverse displacement factor which is the difference between the interval between the times of receipt of successive second timing messages by the server and the interval between the times of sending of said successive second timing messages by the client; maintain a running sum of said forward displacement factors over a predetermined time period; maintain a running sum of reverse displacement factors over a predetermined time period; determine a noise condition of the forward transmission path and a noise condition of the reverse transmission path over said predetermined time period; if it is determined that both the forward and reverse transmission paths have a low noise condition during said predetermined time period, determine a minimum of the sum of forward displacement factors and a minimum of the sum of reverse displacement factors over that period of time and store the determined minima as reference minimum values; and if it is determined during a future time period that one of the forward or reverse transmission paths has a high noise condition, whilst the other path has a low noise condition, use said reference minimum values and the minimum of the sum of displacement factors on the path with a low noise condition to estimate a minimum value for the sum of displacement factors on the path with a high noise condition.

The processor may be arranged to determine a noise condition on a transmission path by: calculating, based on the timestamps, a variance of a queuing delay on that path; applying a first moving average having a first time constant to the variance; applying a second moving average having a second time constant greater than the first time constant to the variance; calculating a difference between the first and second moving averages; and comparing the calculated difference to a threshold to determine a noise condition on the path.

For example, the moving averages may be exponentially-weighted moving averages (EWMAs). Using a pair of moving averages allows determination of whether the current noise conditions (as represented by the moving average with the shorter time constant) have changed from an earlier steady-state period (as represented by the moving average with the longer time constant).

The threshold can be set a-priori if the conditions of the detection environment are known in advance or the threshold can be determined experimentally during operation (or a combination of both may be employed).

An alternative way of determining the noise conditions is to use the error variance-covariance matrix from a Kalman filter which is used in said estimation algorithm. This matrix is normally referred to as P and is a square matrix with the form

where P _n is the error variance of the phase offset, P ₂₂ is the error variance of the skew (fractional frequency offset), and P ₁₂ and P ₂₁ represent the error covariance between the phase offset and the skew. Thus in some embodiments of the invention, the processor is arranged to determine a noise condition on a transmission path by: determining the error variance-covariance matrix

from a Kalman filter which is used in said estimation algorithm,

where is the error variance of the phase offset, P ₂₂ is the error variance of the

skew (fractional frequency offset), and P ₁₂ and P ₂₁ represent the error covariance between the phase offset and the skew.

As the value P ₂₂ depends on the oscillator's stability, its variance, i.e., the variance of the P ₂₂ term itself, is extremely low (almost non-existent). P ₂₂ changes significantly only due to the external noise (and the primary external noise input is typically caused by forward and/or backward traffic load changes).

When external noise changes occur (e.g. the traffic load changes), P ₂₂ reflects this by converging to a new stable level of error variance. Due to the filtering properties of the Kalman filter, this new level is, again, very stable (i.e., there is almost no variance). However, the magnitude of the new level will be very distinct from the previous state, provided there was a significant enough change in the noise conditions. Therefore, P ₂₂ can be employed as input provide reliable indications of changing load conditions.

The system of the present aspect may include any combination of some, all or none of the above described preferred and optional features. The processor of the time client of the above aspect preferably operates by performing a method according to the first aspect of this invention, as described above, but need not do so.

Brief Description of the Drawings

Embodiments of the invention will now be described by way of example with reference to the accompanying drawings in which:

Figure 1 shows the sequence of messages in a two-step clock synchronisation method under IEEE 1588 PTP and has already been described;

Figure 2 shows the relationship between master and slave clocks in situations with and without offset; Figure 3 illustrates the path delays that may exist on communications between a master and slave;

Figure 4 shows, in schematic form, the effect of packet delay variation on message inter- arrival times; and

Figure 5 shows, in schematic form, the operation of a method according to an embodiment of the present invention.

Detailed Description

Basic Clock Model

In order to consider the synchronization problem, we can define a generalized clock offset and skew equation for this problem. It is assumed that, at any particular time instant, the instantaneous view of the relationship between the master (server) clock 4 with timeline S(t) and the slave (client) clock 5 with timeline C(t) , can be described by the well-know simple skew clock model depicted in Figure 2, and described by the equation,

where the skew CC is a very small quantity in the order of parts-per-million. This snapshot is an instantaneous view of how well the two clocks are (mis)aligned. Figure 2 illustrates the influence of θ ₀ and OC on the alignment. The above equation can be extended to account for the case where the master clock 4 and slave clock 5 exchange messages over a communication link with delay. First, assume that the nth Sync message travels from a master to a slave experiences a fixed physical link (or propagation) delay of d _f plus variable cumulative queuing (or stochastic) delay of (see Figure 3). Similarly, assume that the nth Delay_Req message sent from the slave 3 to the master 1 experiences a fixed delay of d ,. plus variable cumulative queuing delay of q _r ( _n ) . An asymmetric path exists when the fixed delay components and/or queuing delay components in both directions are unequal. We assume that the physical link asymmetry is manually calibrated and compensated for but the queue-induced asymmetry can be compensated for using a technique as described in the embodiments below. Equations 1 and 2 below are the two standard equations that describe the relationship between the timestamps exchanged between a master and a slave device in a time distribution network.

Where Ti, Tz, T3 and T ₄ are the timestamps from the standard one- or two-step clock message exchange under IEEE 1588.8 PTP ( Ti is the time of sending of a Sync message from the master to the slave according to the master clock; T ₂ is the time of receipt of the Sync message according to the slave clock; T ₃ is the time of sending of a Delay_Req message from the slave to the master according to the slave clock; and T ₄ is the time of receipt of the Delay_Req message according to the master clock), qf [n] and q _r [n] are the forward and reverse queueing delays and θ ₀ is the initial phase offset between the master and slave clocks; and a[n] is the slave clock skew with respect to master clock.

The fixed forward and reverse path delays are assumed to be known. The compensation of their impact on the final timing estimates at the slave is described in more detail in US Patent 9, 1 12,6286, the contents of which are hereby incorporated by reference. From this it can be shown that:

where equation (3) is derived as the difference between equation (1 ) for time samples n and (n-1 ) and equation (4) is derived as the difference between equation (2) for time samples n and (n-1 ). It is assumed for this subtraction that

As in US 9,1 12,628B, Df _SUm (n) and D _rsum (n) are defined as the summations of the forward and reverse displacements at time sample, n, respectively.

From equation 9 of US 9,1 12,628:

Where D _e {n) is the difference in the forward displacement at time sample n. Using the skew corrected displacement values, as discussed in the above patent, equation (3) above can be used, thus:

If it is assumed that the predetermined point at which the summation, defined by equation (5) above, commences is n = 0, equation (5) can be expanded by setting

Thus by observation it can be seen tha tracks the forward delay at time instant n,

but with an offset equal to the forward delay at the predetermined point chosen to commence the summation, i.e. as the above example started at instant n = 0 the offset is

This procedure can be repeated for the reverse direction, in which case tracks the reverse delay at time instant n, but with an offset of

Note that, in the above, the summations are continuous and all estimates of the forward and reverse delays are relative to the initial ) respectively.

Now the minimum values o can be found over a window of, for

example, N samples.

It is known tha and thus it is desired to find the minimum value of over all samples in the window from n=1 to n=N.

As are both is a fixed value over the entire window, the

minimum value of , which we will call occurs when (n) is a minimum,

which will be referred to as

Hence

Now the minimum value, for that window is subtracted from each value of

within the window:

As are derived by summing the estimated delay variations between

subsequent samples, in reality are only estimates of the true values. The

standard notation will be used to represent these estimates. Thus the estimated forward delay becomes and the estimated reverse delay becomes From simulations by

the inventors it has been found that track the true forward and reverse

delays accurately.

Thus the above equation can be rewritten using to represent estimates:

Hence for each window a term which is approximately equal to the forward delay, minus an offset equal to the minimum value of the estimated forward delay, can be tracked in that window.

A similar procedure for the reverse direction means that an approximation to the reverse delay, minus an offset equal to the minimum of the estimated reverse delay, can also be tracked in that window,

The following equations relate the master and slave timestamps and the instantaneous value of the phase difference. Again it will be assumed that the fixed forward and reverse delays are addressed separately.

Adding 6 and 7 gives:

where θ _τ is the true phase offset which could be calculated if the value of the instantaneous delay asymmetry, was known. In practice of course q _f (n) - q _r (n) is unknown

and hence the instantaneous phase offset is measured as:

By observation of equations 8 and 9, the difference between the measured phase offset, , and the true phase offset

Most current implementations assume that, on average, there is no delay asymmetry. Thus when the phase offse is measured, the measurements at each time instant are

offset by half the value of the delay asymmetry at that time instant, i.e., by

The variations in occur every sample and can be quite large. These frequent

and sometimes large changes make the task of estimating the phase offset difficult for any system. Ideally the value would be removed before generating an estimate

of, what would then be, the true phase estimate for each sample.

Unfortunately that cannot be done as only is known, and qf(n) cannot be separated from in a real-life implementation. Similarly known and

cannot be separated from q _rmin . However, equation (8) can be manipulated such that

the true phase offset estimate is impacted significantly less than if the delay asymmetry was not addressed at all.

It was shown above that it is possible to derive and for any

window of samples of interest. Thus the following can also be derived by subtracting these two terms:

This term can be subtracted from the r. h. s. of equation (8) and a new approximation to the true phase estimate, θ _τ (η), that results can be redefined. Thus:

Rearranging gives:

Again in practice is unknown and hence true phase estimate,

becomes and is estimated as:

By comparing equations 1 1 and 8, it can be seen that the resultant phase difference between due to the substituted asymmetry term, is:

Rearranging this becomes:

As already mentioned, it has been determined from simulation results that is

a very good estimate of Hence the difference in the phase estimate is

dominated by This term is only updated at a frequency equal to the window

length. The value o ) will, also almost always, be smaller than

q _r (n)). Given the less frequent, and in general smaller changes which result from adopting the formation in equation (1 1 ), the task of any estimation algorithm is much less challenging. Therefore equation (1 1 ) can be used as the basis for a phase estimation process, for example as outlined below.

Note that this approach can be used with any timing estimation algorithm. The ability to substitute for the delay asymmetry by a term that varies much less in absolute terms and also less frequently than the raw delay asymmetry will significantly increase the accuracy of estimation of any algorithm. As an example, the above delay asymmetry correction can be used in any of the algorithms presented in [3]. For example, equations (3), (6), (18), (19) in the above reference could be updated with the queuing delay estimates presented in the present application in order to achieve better conversion, precision and accuracy. Therefore the algorithm described in detail in the embodiment below is only an example of such an algorithm.

Phase Estimation Process

The phase estimation process according to an embodiment of the present invention employs equation (10) above to define the estimation process and produce the θ _Ε (η) estimates. The process is illustrated schematically in Figure 5 and employs a Kalman filter acting on the timestamps, the queuing asymmetry measurement and a noise measurement to produce an estimated skew and an estimated phase offset for the slave clock.

The time stamps are measured at the master and slave as per well-established existing practice. The values of ^are

then generated as discussed above.

The Kalman filter is a particular example of an estimation algorithm that can take advantage of equation (1 1 ), although other estimation techniques could equally be employed. The conventional Kalman filter measurement equation is modified to accommodate equation (1 1 ). By rearranging equation (1 1 ), the measurement equation now becomes:

where the left-hand side of the above equation is the term Y _n , v _n is the measurement noise, and the term is made up of the measurement matrix D _n = [2 0] and the state vector All the other terms are as defined above. The Kalman filter state equation is unchanged as is a known M-by-M state transition matrix and

w _n is an M-dimensional process noise vector which is assumed to be drawn from a zero mean multivariate normal distribution with covariance The prediction and update phases operate as has been previously described, for example in US 9,1 12,628B. Some additional challenges are faced in the phase estimation process:

1. When there is very high background traffic on one of the paths (forward or reverse), the minimum reference found within a window for the highly loaded path is significantly biased and will be much higher than the "true" minimum. Note that the "true" minimum refers to the minimum that would be discovered if there was no background load on that particular path. The solution discussed above is of course also impacted by the background load and hence the "true" minimum is not found even for lightly loaded situations. However, under light background load conditions, it is assumed that the forward and reverse minimum queuing delays are almost equal, hence the system can be considered symmetrical. For a system that is close to being symmetrical there is thus a negligible impact on the accuracy of the phase estimation. The specific challenge referred to here is the situation where there is very high background traffic on one of the paths.

As the estimated skew facto , from equations (1 ) and (2) has only limited precision, an error term which represents the error between the true skew and

the estimated skew, can be explicitly represented in these equations. The distribution of values o is a zero mean, approximately, Gaussian distribution. As D _fsum (n)

and are cumulative summations (equation (5)) which are derived from

equation (3) (and the corresponding summation derived from equation (4) for the reverse direction), bot manifest random walk behaviour

(Wiener process-like). If the happen to be separated

significantly in time this Wiener 'drift' will introduce an error to the value of

This in turn will increase the error in the phase offset

estimation (which is impacted directly by It is therefore not possible to find the values, and during good conditions and

simply use those values as the minimum value estimates in windows which occur much later. To address this Weiner drift a further novel technique is introduced as discussed below.

Rations which define from equations (3) and (4) above, are reproduced

but now explicitly include the skew estimation error,

Note that, as the skew error has opposite signs in (12) and (13), the cumulative summation

) derived from equation (12) will drift in the opposite direction to the cumulative

summation from equation (13). As the skew, a(n) is estimated using a Kalman

filter, and the Kalman filter is effectively a whitening filter, the estimation error will be

very close to zero mean Gaussian white noise. The cumulative summations

generated from equation (12) and ) produced from the corresponding reverse

direction, equation (13), thus both suffer from random drift.

To address both of these problems the following approach is adopted:

1. When the conditions are favourable on both paths (see the next section on Noise Estimation), i.e., both the forward and reverse paths are lightly loaded, and

are found as described above and these two values stored as reference points for future use. These will be called for the

forward and reverse directions respectively.

2. Under lightly loaded situations the _m values continue to be

found in every window. If w is used to represent the index of the windows, these values can be designated as for window w. The

computations of and and

by the same absolute value, but in opposite directions due to the

Weiner 'drift'. Thus, if the Weiner drift in the forward direction is defined as WDf, then:

And, if the Weiner drift in the reverse direction is defined as WD _r , then:

And, thus as explained above:

3. Now when heavy loaded conditions are encountered on one path (see the next

section on Noise Estimation), say the reverse path, equations (14)-(16) can be employed to estimate ) using the discovered minimum values

and the two reference values . Equally if

the heavily loaded path is the forward path can similarly be estimated.

This can be represented as follows:

If the reverse path is less heavily loaded:

If the forward path is less heavily loaded: where are the re-estimated minimum values using the stored minimum reference values, the measured minimum on the less heavily loaded path and the relationship between them as defined by equations (14)-(16).

Noise condition estimation To gain an understanding of current noise conditions, a number of statistical properties of such as, mean, median, variance (and other statistical moments) can be

tracked and compared. As the performance of the Kalman filter algorithm depends on noise variance, and for synchronization the majority of the noise comes from the variability of forward and reverse queueing delays, it is most appropriate to focus on the variance

and var{q _r }.

From probability theory it follows that variance of the first difference of a random variable is related to the variance of the random variable as follows:

Now using equations (3) and (19) it follows that one may compute var{qf} and var{q _r ] as follows:

Since the contribution of the term are both at least several orders of magnitude less than the term

respectively, the above equations can be re-written as:

To monitor and compare noise conditions on the forward and reverse paths, only the changes in over time need to be tracked and this can be achieved

using, for example, an Exponentially Weighted Moving Average (EWMA) filter as disclosed in US 9,184,861 , the contents of which are hereby incorporated by reference.

To categorize the noise conditions on the forward and reverse directions, the average value of the noise variance on the forward path and the average value in the reverse direction both need to be found during steady state periods (i.e., when no change is occurring). The noise condition is deemed to have changed from the current category to a new category if the noise variance has changed sufficiently from the steady state average.

A change detector can be implemented using two EWMA filters with different time constants, i.e., an EWMA filter with a long time constant which approximates the average value of a variable and one with a short time constant which is sensitive to changes in the variable. An EWMA filter can be defined as:

where μ is the time constant (filtering factor) defined as follows:

and x(n) is the input variable which is being tracked (the outputs from (22) or (23) are both used in this case). Typically, T _Q and τ ₅ can be set as 60s and 600s respectively. Now the detection is obtained by comparing the outputs of the two filters as follows:

The absolute value o is compared to

the same threshold. threshold, a change is considered to have been detected. The threshold can be set a-priori if the conditions of the detection environment are known in advance or the threshold can be determined experimentally during operation (or a combination of both).

Note that the sign of e (n) from (27) defines whether the conditions have improved, or have deteriorated. A positive sign indicates worse conditions, i.e., higher noise. Thus if the value |e(n) | exceeds the threshold, calculated with as input and e(n) is positive, then a

load increase on the forward direction is indicated.

Part of the error variance-covariance matrix, P, from a Kalman filter (for example as descried earlier), can be used to provide the input variable for equation (27), instead of or var{q _r }.

Matrix P is a square matrix defined as:

where P _1± is error variance of the phase offset, P ₂₂ is the error variance of the skew

(fractional frequency offset), and P ₁₂ and P ₂₁ represent the error covariance between the phase offset and the skew. As the value P ₂₂ depends on the oscillator's stability (OCXO in the case of telecom applications), its variance, i.e., the variance of the term itself, is

extremely low (almost non-existent). P ₂₂ changes significantly only due to the external noise (and the primary external noise input is caused by forward and/or backward traffic load changes). This has been verified in realistic conditions using simulation.

When external noise changes occur (i.e. the traffic load changes), P ₂₂ reflects this by converging to a new stable level of error variance. Due to the filtering properties of the Kalman filter, this new level is, again, very stable (i.e., there is almost no variance). However, the magnitude of the new level will be very distinct from the previous state, provided there was a significant enough change in the noise conditions. Therefore, P ₂₂ can be employed as input to (27) to provide reliable indications of changing load conditions. To determine whether the load on the forward or reverse direction has changed, the values

from equations (22) and (23), are examined to see which one has changed significantly from the average.

Using Sync-E to improve the accuracy of the packet delay estimation

All packet delay estimation algorithms, including the algorithm presented above, are highly dependent on the accuracy of the skew estimate. Where Sync-E technology is available it provides very accurate frequency estimates. These Sync-E based frequency estimates would be much more precise than frequency estimations from other approaches, such as the Kalman filter.

The impact of Sync-E on estimation accuracy is obvious from equations (12) and (1 3), where the a _e term would be exceedingly small for Sync-E. If, is small, the Wiener drift would also

be exceedingly small and hence the packet delay estimation would be much more precise. The improved frequency estimation accuracy provided by Synch-E would also improve the performance of other parts of the synchronization system, further improving the overall accuracy of the master/slave phase offset estimation. The systems and methods of the above embodiments may be implemented in a computer system (in particular in computer hardware or in computer software) in addition to the structural components and user interactions described.

The term "computer system" includes the hardware, software and data storage devices for embodying a system or carrying out a method according to the above described

embodiments. For example, a computer system may comprise a central processing unit (CPU), input means, output means and data storage. Preferably the computer system has a monitor to provide a visual output display. The data storage may comprise RAM, disk drives or other computer readable media. The computer system may include a plurality of computing devices connected by a network and able to communicate with each other over that network.

The methods of the above embodiments may be provided as computer programs or as computer program products or computer readable media carrying a computer program which is arranged, when run on a computer, to perform the method(s) described above. The term "computer readable media" includes, without limitation, any non-transitory medium or media which can be read and accessed directly by a computer or computer system. The media can include, but are not limited to, magnetic storage media such as floppy discs, hard disc storage media and magnetic tape; optical storage media such as optical discs or CD- ROMs; electrical storage media such as memory, including RAM, ROM and flash memory; and hybrids and combinations of the above such as magnetic/optical storage media.

While the invention has been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the exemplary embodiments of the invention set forth above are considered to be illustrative and not limiting. Various changes to the described embodiments may be made without departing from the spirit and scope of the invention.

In particular, although the methods of the above embodiments have been described as being implemented on the systems of the embodiments described, the methods and systems of the present invention need not be implemented in conjunction with each other, but can be implemented on alternative systems or using alternative methods respectively.

References 1. US 9,184,861 Aweya et al. - Method and devices for compensating for path asymmetry.

2. US 9,112,628 Aweya et al. - Method and devices for synchronization.

3. A. Bletsas: "Evaluation of Kalman filtering for network time keeping," IEEE Trans, on Ultrason., Ferroel., and Freq. Control, 09/2005.

All references referred to above are hereby incorporated by reference.