RECEIVER FOR DIGITAL DATA

The present invention relates to methods according to the preamble parts of claims 1 , 2 and 5 and to an equalizer for performing such methods. This invention relates to equalizing received digital data by a maximum likelihood sequence estimator (MLSE) and more specifically to obtaining branch metrics for a maximum likelihood sequence estimator.

The MLSE using the Viterbi-Algorithm (VA) bases its symbol decisions on probabilistic decision variables, i. e. branch and path metrics differences, that are ultimately related to conditional probabilities of observing a given received signal when a given symbol (or symbol sequence) has been sent.

The basis for computing the relevant sequence decision variables are the so-called branch metrics, which in turn are based on a probabilistic model of the channel defined by a set of amplitude probability density functions (PDFs) or probability mass functions (PMFs), one for each channel state, i.e. for each sent bit pattern of a certain length.

In essence, for the detection to best approximate a true maximum likelihood detector, the metrics should represent the log-likelihoods for the events to observe specific quantized amplitudes when given symbol sequences have been sent, i.e. when the channel was in given channel states. In a practical system, the probabilistic channel model needs to be estimated in real-time and without channel-specific a-priori information. Moreover it needs to be updated in real-time in order to follow changing channel conditions e.g. due to drifts or due to dynamic effects such as polarization mode dispersion (PMD). This implies that the channel estimator needs to be blind and adaptive. To learn or acquire the channel model at the beginning of operation, the channel estimator is initialized with a crude channel model, resulting in a high initial error rate. New channel conditions are then estimated and used in next estimation period. Convergence of this channel model acquisition is not guaranteed, but in practice it is very robust. Channel model estimation methods may be parametric or nonparametric (cf. H. F. Haunstein, W. Sauer-Greff, A. Dittrich, K. Sticht, and R. Urbansky, "Principles for electronic equalization of polarization-mode dispersion" J. Lightwave Technol., vol. 22, pp. 1 169-1 182, April 2004, and cf. Langenbach, S.; Bosco, G.; Poggiolini, P.; and Kupfer, T., "Parametric versus Non-Parametric Branch Metrics for MLSE-based Receivers with ADC and Clock Recovery," Optical Fiber communication/National Fiber Optic Engineers Conference, 2008. OFC/NFOEC 2008, Conference on, Paper JThA60, 2008). When a parameterized functional form of the PDF is assumed, a parametric method estimates the PDF parameters and uses the functional form to compute the metrics. On the other hand, non-parametric methods do not assume knowledge of the PDF (cf S. Langenbach and N. Stojanovic, "Channel estimation and sequence estimation for the reception of optical signal", EP1 494 413 A1 , Jan. 5, 2005 (later referred to as COEP4); A. Farbert, S. Langenbach, N. Stojanovic, C. Dorschky, T. Kupfer, C. Schulien, J. -P. Elbers, H. Wernz, H. Griesser, C. Glingener, "Performance of a 10.7 Gb/s receiver with digital equaliser using maximum likelihood sequence estimation, " in Proc. ECOC, Stockholm, 2004, Th.4.1 .5; J.-P. Elbers, H. Wernz, H. Griesser, C. Glingener, A. Faerbert, S . Langenbach, N. Stojanovic, C. Dorschky, T. Kupfer, C. Schulien, "Measurement of the dispersion tolerance of optical duobinary with an MLSE-receiver at 10.7 Gb/s, " in Proc. OFC, Washington, 2005, OthJ4). COEP4 is incorporated herein by reference and cites further references.

Fig. 8 shows an optical receiver 10 which is essentially known from COEP4 and receives an analog input signal r(t) from an optical fiber 4. The receiver 10 comprises a physical interface (PI) 1 1 , a AGC or variable gain amplifier (VGA) 12, an ADC 13, a clock recovery (CR) subsystem 14, a sampling phase adjustment (SPA) circuit 15, an MLSE 17, a FEC decoder 18, a channel model unit 19 and a receiver control node 9.

The physical interface 1 1 performs an optical-to-electrical (O/E) conversion. The physical interface (PI) uses either a pin diode or an avalanche photo diode to convert the incident optical power to an electrical current. A transimpedance amplifier (TIA) is used to amplify and convert the photo-current to a voltage.

The analog serial signal data at the output of physical interface 1 1 is amplified by a high- gain high-dynamic, low-noise automatic gain control (AGC) or variable gain amplifier (VGA) circuit 12. The output signal of AGC 12 is designated r (t) .

The ADC 13 digitizes the analog signal r (t) and outputs quantized data y, _,s . Index t refers to a time slot and index s refers to different sampling phases. Index s may assume the values 1 to S for S-fold oversampling. S may be be 2. The ADC 13 receives a sampling clock from SPA circuit 15 which in turn receives a sampling clock from clock recovery subsystem14. The SPA circuit 15 operates as an adjustable delay in order to optimize the phase of the clock which is to say to optimize the sampling times of ADC 13. The quantized data y _ts are input into MLSE 17. MLSE 17 may implement a Viterbi algorithm (VA) and outputs the most likely sequence designated detected data u, to FEC decoder 18. In a typical optical receiver, with a powerful FEC code used, the bit error rate at the output of MLSE 17 ranges e.g. from 10 ^~2 to about 10 ^~4 . The subsequent FEC decoder 18 further reduces bit error rate to a range between 10 ^~9 and 10 ^~16 which is required for data transmission. FEC decoder 18 outputs decoded data x, for further processing. MLSE 17 and/or FEC 18 may obtain BER estimates and provide same to control node 9. Actually, the serial data output by the ADC are, in reality, de-multiplexed in the digital domain. Blocks 17, 18, 19, 9 all operate at lower speed.

Control node 9 receives a loss-of-signal (LOS) signal from physical interface 1 1 and may receive counter values or event frequency information from channel model unit 19 in order to obtain pre-processed statistics data for controlling the AGC/VGA circuit 12,

CR 14 and SPA circuit 15. Counter values may also be referred to as bin values.

Important for this invention is that the channel model unit 19 receives quantized data y _ts .

The channel model unit 19 further receives the present channel state b, and calculates and outputs branch metrics to the MLSE 17.

Returning to channel model estimation methods, we focus our interest on the non- parametric method, which uses empirical histograms being synonymous to empirical PMFs to obtain the branch metrics. This is generally called the histogram method (cf. O. E. Agazzi, M. R. Hueda, H. S. Carrer, and D. E. Crivelli, "Maximum-likelihood sequence estimation in dispersive optical channels", " J. Lightwave Techno/., vol. 23, pp. 749-763, Feb. 2005). More specifically, when the measured histogram bin values representing relative frequencies are directly converted to metrics values, without further postprocessing, we call it the canonical histogram method, as described in more detail in the following. Canonical Histogram Method

The total number of channel state (i.e. bit pattern) conditioned histograms depends on the so-called channel length M, which is often called the channel memory and, in a practical implementation, may e.g. have a value of 3, 4, or 5. The total number of channel states is 2 if an interference of /W-1 bits with the current bit indexed by t is to be allowed for, i.e. when channels are projected to be covered which require a sequence of /W bits to reasonably represent a channel state.

The received signal is quantized to bits; therefore, each histogram consists of 2 ^K bins or counter values and is an empirical estimate for the amplitude PMF of the quantized output values. When the received signal is oversampled the number of histograms is proportional to oversampling factor S. In contemplated embodiments S=2.

Each histogram is uniquely associated with a channel state b _t , and hence with a branch in the trellis of the Viterbi detector.

Let us denote the quantized sample values by y _{t s} , 1 < y _{fi S} ≤2 ^K , s = 1,2,..., S . The counter values Ci _j,s constitute event counts when i=y _ts and the channel is in state j, j = 1,2,...,2 ^M during a collection time T. The counter values c _ijiS may be grouped to histograms hj _s , which are uniquely associated to channel states j and the sampling phases s. When the number of samples collected is large enough, the normalized histogram is an estimate of conditional probability P(i = y _{t s} I j _s ) :

P(i = yt,s / h _{L s} ) = ' ^{u s} , i = \2,...,2 ^K , / = 1,2,...,2 ^Μ , s = 1,2,..., S (1 )

/=1

One immediately notes that the conditional probabilities P[i = y _{t s} I hj _S ) are normalized and may be considered as normalized histograms:

₂ K

∑P{i = y _t ,s l hj _{i S} ) = 1 , / = 1,2,...,2 ^Μ , s = 1,2,..., S (2)

/=1

This conditional probability P[i = y _{t s} l hj _S ) is used in the trellis for the best path calculation. To avoid multiplication in the searching of the best trellis path, the conditional probability is replaced by the absolute value of the logarithm of the conditional probability, and addition is used instead of multiplication. Further details of the metric calculation are comprised in COEP4. In practical systems, metrics are quantized to L bits. It means that log probability can take value from 0 to 2 ^L -1 . Additionally, receiver designers have to specify the minimum probability that should be quantized, P _min . We may define:

Λ .y, s := log(P(/ = y _s l h _{u s} )) , Λ,- _y - _s ≤ 0 (3) A _min := log(P _min ) . (4) When Λ _δ := A _min /(2 ^L - l) the metrics quantization rule is defined by

The branch metric bmi _j,s still depends on the sampling phase index s. How this dependency may be handled is described in COEP4. Disadvantages of the Non-parametric Channel Model Estimation

There are two problems that are more or less specific to the canonical histogram method: error propagation by PMF tail shape corruption, and metrics indifference. Both can cause performance degradation against perfect channel training which means that the sent sequence is known to the receiver. Error Propagation

The real-time channel estimation is decision-directed and as such suffers from the errors at the detector output. This results in histograms that do not faithfully represent the true channel conditions. Due to this circular dependency between metrics and channel estimations, decision errors cause wrong estimations and these, via the derived metrics, can cause further errors in future decisions. This problem is called error propagation. In principle, error propagation also occurs in parametric methods. However, the influence of decision errors on parameter estimates such as histogram mean values may be expected to be weaker, since, even under estimation error, at least the shape of the PDF is maintained. There are several situations in which error propagation can be detrimental. The general pattern here is that a temporarily high error rate might lead to a self-stabilized error propagation loop, with a residual error rate higher than that achieved with a trained channel model, i.e. a channel model without decision errors:

1 . In low noise situations (high optical signal-to-noise ratio (OSNR)), low-probability random decision errors create events in normally un-occupied histogram bins corresponding to the PMF tails. This can lead to large metrics changes, which influence the decisions in the next interval.

2. At the beginning of a channel model acquisition from crude starting channel models, the initial error rate is high, which enhances the chances of meta-stable error propagation. The MLSE may then converge to a wrong channel model.

3. Transient or modal disturbances caused for example by channel changes, temperature variations, power supply instability and jitter may result in histograms that still lead to a high error rate after the disturbance has disappeared. For example, short-time unpredictable processes can produce histograms that degrade BER performance even long time after such processes have disappeared.

Metrics Indifference

Metric indifference refers to a situation, where events with significantly different conditional probabilities are assigned the same or similar metrics. The branch metrics differences used by the VA do not represent the true log-likelihood ratio anymore. The resulting path metrics error misguides the decisions of the VA and degrades MLSE performance.

1 . One source of metric indifference is the lack of observation data for low probability events especially in the tails of the PDFs: In practice, the channel model must be updated after a finite channel estimation period. At high OSNR some histogram bins in the PDF tail region will remain empty, because the probability of observing the relevant events is too small. The canonical histogram method assigns the same metrics to such unobserved bins even if the true metrics difference should be large. This may cause an error floor due to metric indifference. 2. Another source of metric indifference is the finite log-likelihood quantization range and resolution in a practical metrics computer, which is described by the parameters K, L and P _min . A practical implementation must choose a fixed metrics quantization range, P _min . However, in low noise situations, the histogram bins with probabilities less than P _min will often be empty and will therefore obtain the same metrics, again causing metric indifference and resulting in an error floor. Note that this problem cannot be solved by choosing a longer channel estimation period, because even if one was able to correctly estimate very low probabilities, bins with observed relative frequencies smaller than P _min would have got the same quantized branch metrics.

Prior solutions

The above-identified disadvantages have not yet been discussed in the literature, and prior art solutions are unknown to the inventors.

The technical literature on channel estimation for MLSE covers mainly parametric methods.

With regard to non-parametric methods, only the canonical histogram method is usually discussed. Drawbacks of the canonical method other than the problem of longer measurement duration for the same statistical significance have not been discussed.

The performance of MLSE equalizers with non-parametric channel model estimation at low bit error rates is not accessible to simulations and is therefore neglected in the simulation-based literature.

Nevertheless reference should be made to N. Stojanovic "Tail Extrapolator and Method" EP application No. 07102182.8 and PCT application PCT/EP2008/051684, publication projected in August 2008. This reference discloses "tail extrapolation", which is some kind of post-processing of canonical histograms in order to set low probability bins or metrics to reasonable values. EP07102182.8 is incorporated herein by reference.

It is the object of this invention to provide improved methods for obtaining a set of path metrics and an equalizer implementing such methods.

This object is achieved by the subject matter of the independent claims.

Preferred embodiments of the invention are the subject matter of the dependent claims. In the following preferred embodiments of this invention are described referring to the accompanying drawings. In the drawings:

Fig. 1 illustrates the LER processing.

Fig. 2 illustrates two metrics vectors processed by LER.

Fig. 3 illustrates the CTE processing.

Fig. 4 illustrates two metrics vectors processed by CTE.

. 5 illustrates the DDMS processing.

Fig. 6 illustrates two metrics vectors processed by DDMS.

. 7 illustrates an inventive channel model unit.

. 8 shows a conventional receiver.

Abbreviations

ADC: Analog-to-Digital Converter 30 MLSE: Maximum Likelihood Sequence

AGC: Automatic Gain Controller Estimator

BER: Bit Error Rate NRZ: Non-Return-to-Zero

BM: Branch Metrics OFE optical front-end

BMC: Branch Metrics Computation OSNR: Optical Signal-to-Noise Ratio

CR: Clock Recovery 35 PDF: Probability Density Function

CTE: Convex Tail Enforcement PI: Physical Interface

DDMS: Distance Dependent Metrics PMD: Polarization Mode Dispersion Slope PMF: Probability Mass Function

DGD: Differential Group Delay (Histogram)

FEC: Forward Error Correction 40 RD: Residual Dispersion

HCNTACC: Histogram Counter SPA Sampling Phase Adjustment Accumulation TIA transimpedance amplifier HMODE: Histogram Mode Value VA: Viterbi Algorithm

HNORM: Histogram Normalization VGA Variable gain amplifier

LER: Local Extremum Removal 45

MMVA: Minimized Method Viterbi

Algorithm

Mathematical Symbols amd(n) allowed metric delta o index, 1 <o<2 ^K bm(i) branch metrics mst metric smoothing threshold b _t channel state mmdt maximum metric delta template

Ci _j s counter value 20 mtms minimum tail metric slope hj histograms parameter

/ ^' index, ^≤i≤2 ^κ pmd progressive metric delta

parameter

j channel state index, 1 <y-≤2 ^M

K resolution of ADC in bits Pmin minimum probability

k index, 1≤i≤2 ^K 25 P{y _j / hj) conditional probability

L metric resolution in bits S oversampling factor

/ index, 0</<2 ^L -1 s phase index, 1 <s<S

A _j ,s metric T collection time

Λ _δ quantization step for metric t time slot index

M channel length / channel memory 30 t _c time slot counter index m minimum metric index y _{t s} quantized symbol values n index, 1 <n<2 ^K

While the present invention is described with reference to the embodiments as illustrated in the following detailed description as well as in the drawings, it should be understood that the following detailed description as well as the drawings are not intended to limit the present invention to the particular illustrative embodiments disclosed, but rather the described illustrative embodiments merely exemplify the various aspects of the present invention, the scope of which is defined by the appended claims.

The metrics resulting from the canonical histogram method are post-processed. The post-processing efficiently copes both with metrics artifacts in the PMF tail regions, which are a result of error propagation, and with metrics indifference situations.

In using (rough) a priori knowledge of PDF (or PMF) tail shapes (such as logarithmic PMF being monotonically decreasing, convex, or not arbitrarily steep), our method can be interpreted as a pragmatic hybrid between truly non-parametric and parametric methods.

It is clear that in addition to those mentioned in this report, other known "features" of PDF (or PMF) shapes can be used to replace canonical metrics derived from unreliable histogram tails by "extrapolated" metrics. Unlike with many parametric methods, computationally intensive, numerically demanding methods such as PDF evaluations or PDF integrations (for PMF evaluation) are not required.

We have invented three computationally simple methods for branch metrics postprocessing:

Clearing unreliable bins that might be caused by error propagation.

Ensuring that the metrics slope related to the tails of a logarithmic histogram is always non-decreasing which is characteristic for most relevant PMF families such as Gaussian or chi-square.

^■ Ensuring maximum metric slope in order to avoid metric indifference far from the histogram center .

Extrapolation of cleared unreliable bins and metric slope control are based on a priori knowledge of rough, qualitative metrics vector shape.

The post-processing methods may operate on the quantized log probabilities, i.e. on branch metrics. From now on, we refer to a set of branch metrics being calculated from a single histogram as a metrics vector, which consequently consists of 2 ^K branch metrics bm(i) = birii _s , i = 1 ,2,... ,2 ^K that can take one of 2 ^L values from 0 to 2 ^L - 1 . bm(i) are still indexed by j and s, but this is irrelevant for the invention post-processing methods and will not be explicitly mentioned anymore. The highest probability corresponds to branch metric equal to 0. Each histogram and the corresponding metrics vector is characterized by the location of the bin with the minimum metric corresponding to the highest probability. This location is denoted by index m, m = 1 ,2,...,2 ^K and also referred to as histogram or metrics vector center. When more than one minimum metric location is found the one with the lowest index m is declared as the minimum location and metrics vector center.

Local Extremum Removal - LER

The purpose of local extremum removal (LER) is to clear local extrema and the entire tail. The tail is considered unreliable and will later be reconstructed by the distance- dependent-metric-slope method (DDMS), which will be described below. A metric smoothing threshold mst is used to ensure that only tails with sufficiently low probability are handled. When m = 3 or m = 2 - 2 the method maintains an extremum in the first or last bin if existent, respectively, that is likely caused by high tail probabilities. In the following, MATLAB or Scilab 4.0 code will be printed in Courier, a non-proportional font without serifs. LER code for the tail right of metrics vector center may read:

5 if m < 2 ^Λ Κ-2

for k = (m + 2 ) : 2 ^Λ Κ

if bm(k) <= bm(k-l) & bm(k) >= mst

bm(k:2 ^A K) = 2 ^A L-1; break;

end

10 end

end

LER code for the tail left of metrics vector center may read: if m > 3

15 for k = (m - 2 ) : -1 : 1

if bm(k) <= bm(k+l) & bm(k) >= mst

bm(l:k) = 2 ^A L-1; break;

end

end

20 end

The LER code is illustrated by figure 1 . Step 21 marks in the starting point and step 39 marks the end point of the LER code. The directions left and right refer to diagrams in which the index of the bins or corresponding metrics is plotted on the abscissa. In step

25 22 it is checked as to whether there are enough bins right of the metrics vector center m.

If this is not the case, a maximum in the last bin indexed 2 ^K is to be maintained. If m<2 ^K - 2, the loop variable k is initialized with m+2 in step 23. This loop consists of steps 24 to 28. While k<=2 ^K , the loop is continued in step 24. Otherwise the loop is aborted in step 24. Only if a metric bm(k) exceeds or is equal to a metric smoothing threshold mst,

30 which is checked in step 25, the bin k is further examined in step 27. Otherwise the loop variable k is incremented in step 26 and the loop proceeds to the next metric, if available. In step 27 local minima are detected. If two neighboring metrics in the tail have the same value, this is also interpreted as a local minimum. If it is determined in step 27 that the metric k is not bigger than the previous metric k-1 , all metrics from index k to the last index 2 ^K are set to the maximum metric value 2 ^L -1 in step 28 and the loop is aborted.

Steps 32 to 38 illustrate LER processing for the left-hand tail, which is mirror-like to the LER processing of the right-hand tail illustrated by steps 22 to 28. The reference numbers of like steps differ by 10.

Step 32 maintains a minimum in the first bin, if the metrics vector center is close to the first bin 1 . The loop variable k is initialized with m-2 in step 33. The loop is aborted in step 34, when the first metric has been processed. Step 35 makes sure that only metrics exceeding or being equal to the metric smoothing threshold mst are processed. If a metric bm(k) is not bigger than the previous metric bm(k+1 ), which is checked in step 37, all metrics from k to 1 are set to maximum metric value 2 ^L -1 in step 28 and the loop is aborted. If the loop is not aborted, the loop variable k is decremented in step 36. The following two examples are shown in figure 2.

Table 2: example for K=3, L=6 and mst=5

Convex Tail Enforcement - CTE

Convex Tail Enforcement (CTE) ensures that the metrics slope on the tails of a metrics vector is always increasing to ensure strict convexity. Using a progressive metric delta parameter pmd, a slope increase towards the tails can be forced. Using a minimum tail metric slope parameter (mtms) it can be ensured that the tail region begins with a given minimum slope. Using the metric smoothing threshold parameter mst it can be ensured that changes are only applied to low probably bins in the tail regions.

CTE code for the tail right of metrics vector center may read: if m < 2 ^Λ Κ-2

for i = max(4, (m + 2)) : 2 ^Λ Κ if bm(i) >= mst

bm(i) = min(2 ^A L-l, max(bm(i), bm(i-l) + max(bm(i-l)- bm ( i-2 ) , mtms ) + pmd) );

end

end

end

CTE code for the tail left of metrics vector center may read: if m > 3

for i = min(2 ^A K-3, (m - 2)) :-l:l

if bm(i) >= mst

bm(i) = min(2 ^A L-l, max(bm(i), bm(i+l) + max ( bm(i+l) - bm ( i+2 ) , mtms ) + pmd) );

end

end

end

The figure 3 illustrates CTE processing, which basically consists of two loops. The steps 43 to 47 form the loop for processing the right-hand tail and the steps from 53 to 57 form the loop for processing the left-hand tail. The reference numbers of like steps differ by 10.

The step 41 marks the beginning of CTE processing and the step 58 marks the end of CTE processing. The steps 42 and 52 check as to whether the metrics vector center m is not close to the last or first bin, respectively. If the metrics vector center m is close to the first or last bin, a left-hand or right-hand tail, respectively, does not exist and is consequently not processed.

The loop variable i is initialized in step 43 by the maximum out of 4 and m+2. If the last bin 2 ^K has been reached, which is examined in step 44, the loop for the right-hand tail is aborted. Step 45 makes sure that only metrics exceeding or being equal to the metric smoothing threshold mst are processed. The step 46 does several things: It makes sure that the metric difference between neighboring metrics bm(i) and bm(i-1 ) and neighboring metrics bm(i-1 ) and bm(i-2) increases by a progressive metric delta parameter pmd towards the last bin 2 ^K . Further it makes sure that the tail starts with a minimum tail metric slope parameter mtms. Finally the step 46 makes sure that the metric bm(i) does not exceed the maximum metric value 2 ^L -1 . The function min selects the minimum out of its arguments separated by commas. The function max selects the maximum out of its arguments. The loop variable i is incremented in step 47 after each loop cycle. For processing of the left-hand tail, the loop variable i is initialized with the minimum out of 2 ^K -3 and m-2. After the first metric has been processed, the loop is exited in step 54. Step 45 makes sure that only metrics exceeding or being equal to the metric smoothing threshold mst are processed. The step 56 makes sure that the metric difference increases by a progressive metric delta parameter pmd towards the first bin 1 , the tail starts with a minimum tail metric slope parameter mtms and the metric bm(i) does not exceed the maximum metric value 2 ^L -1 . The loop variable i is decremented in step 57 after each loop cycle. The following two examples are shown in figure 4.

Table 4: another example for K=3, L=6, mst=5, pmd=1 , and mtms=6

Distance Dependent Metric Slope - DDMS

The Distance Dependent Metric Slope (DDMS) criterion ensures a maximum metric slope, which may depend on the distance to the metrics vector center m, in order to avoid metric indifference far from the metrics vector center m. However, in the examples presented, the maximum metric slope does not depend on the distance to the metrics vector center m. A maximum metric delta template mmdt is a vector of 2 ^K elements that defines a maximum metric difference between neighboring bins. In general, the maximum metric difference template elements may have different values. However, in the examples presented, the same value is assigned to all elements of the maximum metric difference template mmdt. DDMS code: for n = 1:2 ^Λ Κ // for each amplitude

amd(n) = mmdt (abs (m-n) +1 ) ; // allowed metric delta at distance for o = 2:2 ^Λ Κ

bm(o) = min(bm(o), bm(o-l) + amd(o));

end for o = 2 ^Λ Κ-1:-1:1

bm(o) = min(bm(o), bm(o+l) + amd(o));

end

The DDMS processing is illustrated in figure 5. Steps 61 and 86 mark the starting and end point, respectively, of the DDMS processing. In a first loop, which comprises the steps 62 to 65, an allowed metric delta vector amd is initialized symmetrically to the metrics vector center m from the maximum metric delta template mmdt. The function abs in step 64 returns the absolute value of its argument. The steps 62, 63 and 65 manage this loop. The loop variable n is initialized in step 62 with 1 . The loop is aborted in step 63 if the loop variable n exceeds 2 ^K . The loop variable is incremented in step 65.

Instead of initializing all entries of the maximum metric delta template mmdt with the same value, all elements of the allowed metric delta vector amd may be initialized with this value thereby bypassing steps 62 to 65.

The loop which comprises steps 72 to 75 makes sure that the metric bm(o) does not exceed its left neighbor bm(o-1 ) by more than the allowed metric delta amd(o). The latter is specifically done in step 74. The steps 72 and 75 initialize and increment the loop variable o. The step 73 ensures proper loop exit.

The loop which comprises steps 82 to 85 makes sure that the metric bm(o) does not exceed its right neighbor bm(o+1 ) by more than the allowed metric delta amd(o). The latter is specifically done in step 84. The steps 82 and 85 probably initialize and decrement the loop variable o. The step 83 ensures proper loop exit. The following two examples are shown in figure 6. Output bm 19 10 1 0 3 12 21 30

Table 5: example for K=3, L=6, and mmdt(:)=9.

Input bm 63 0 44 63 63 63 63 63

Output bm 9 0 9 18 27 36 45 54

Table 6: another example for K=3, L=6, and mmdt(:)=9.

Implementation Overview

Figure 7 shows a data flow block diagram of an inventive channel model unit 19 together with MLSE 17. Within channel model unit 19, the metrics post-processing box 100 encompasses the inventive elements LER 20, CTE 40 and DDMS 60. MLSE 17 comprises e.g. a modified Minimized Method Viterbi Algorithm (MMVA) 101 for parallel Viterbi decoding and branch metrics storage element 102. The MMVA, as for example described in H. Dawid, G. Fettweis, H. Meyr: A CMOS IC for Gb/s Viterbi Decoding: System Design and VLSI Implementation, IEEE Transactions on VLSI Systems, Vol. 4, No. 1 , pp.17-31 , 10 March 1996, is essentially modified to operate on two samples per bit. This requires another branch metric table for the second sampling phase, such that the MMVA comprises a branch metric table for each sampling phase, and a modified branch metrics computation where the independent metrics for two samples per bit are added to form the overall metrics for each transition.

The MMVA 101 receives blocks of 96 sample duads. Such blocks comprise quantized symbol values y, _,s from ADC 13. The MMVA 101 further receives the clock CLK and the branch metrics from storage element 102. The MMVA 101 mainly outputs blocks of 96 detected bits also referred to as u, and further provides the associated channel states 103 also referred to as b, to the Histogram Counter Accumulation (HCNTACC) process 108 for performing event counting as explained in connection with equation (1 ).

The channel data accumulation period for the HCNTACC process 108 shall be controlled by the parameter t _c 109, which denotes the number of bits collected into a counter-based channel model, i. e. the sum of all counter values of the channel model is equivalent to S-t _c . To simplify Histogram Normalization (HNORM) 1 12, the data accumulation period synonymous to collection time and observation time T is given by t _c ^* (bit period). Bit period, symbol period and unit interval may be used synonymously. t _c may be a power-of-two multiple of some minimum value of t _c . The minimum value of t _c is for example larger than 2 ¹⁵ or 2 ¹² . The maximum value of t _c is for example not smaller than 2 ³² . The intention of this range is to allow fast acquisition and tracking, software- based processing at about 100 ...1000 Hz. In another embodiment, the maximum value of t _c may not be smaller than 2 ²² . Simulations suggest that 4096 = 2 ¹² bits would be 5 sufficient for fast acquisition i. e. 128 clock cycles at 64 bit block size with subsampling factor 2. This relaxed requirement is balanced against implementation restriction i. e. reduced power dissipation.

The process steps following HCNTACC 108 within channel model unit 19 are activated sequentially in a data-driven manner, i.e. when the preceding step has produced new i o and complete output data. The speed of the entire metrics update loop is therefore gated by the configured channel observation period of length t _c . Depending on configuration parameter t _c , the update speed of the operational metrics 102 can vary by orders of magnitude from "as fast as possible" (Ι Ομε-Ι ΟΟμε) range to "very slow" updates (10-1000s). Note that, in a practical system, software can stop processing at

15 each stage and can read and write the channel model memories 1 10, 1 13, 1 15.

This is to support start-up of the long term channel observation based metrics computation policy; in steady state, the updates will be slow, but during start-up a gradual increase is desired.

The dynamic range of frequency variables C and H shall be large enough to represent 20 the maximum number of collected bits of 10 ¹⁵ with a resolution of 10 ^~6 (or 2 ^"20 ). This means that observations of an interval of about 100 με length can be accurately represented in counters. A dynamic range of 2 ⁵⁰ > 10 ¹⁵ has been selected for frequency variables C and H.

The channel estimation process shall by default start after update of operational branch 25 metrics. There may be a short hold-off period to avoid using data from metrics transition in channel estimation.

There shall be an option for channel estimation restart without counter initialization. Normally, counters are reset to zero at the beginning of a channel observation. The incremental restart allows software to incrementally update operational metrics during 30 starting-up a periodical long-term channel estimation.

HNORM 1 12 implements equation (1 ), wherein counter histograms C(i, j, s), i =1 ,...,2 ^K for fixed j and s are normalized as relative frequency histograms and where the result is provided as H(i, j, s) = P(i = y _{t s} i ' hj _iS )- Histogram normalization of non-empty counter histograms implies 1 =∑H _{k s} for all j and s. Note that the occurrence of empty

/c=1

counter histograms normally is a defect condition that can be handled in application dependent ways. The Branch Metrics Computation (BMC) 1 14 essentially implements equations (3) to (5). In one embodiment, A _min may be chosen for each sampling phase s separately. Then exponentially spaced thresholds HT(l,s) for the relative frequencies H(i, j, s) are being pre-calculated:

HT(1 ,s) = 1 0 <=-4; s=1 , 2, ... S; (6) The other HT(l,s) are calculated iteratively:

HT(l,s)=HT(l-1 ,s) ^* HT(1 ,s); 1=2, 3 ... 2 ^L -1 , s=1 , 2, ... S; (7)

Taking of the logarithm is actually done by a look up process which may read in pseudocode: function [bm] = bmc (h, ht)

bm(l: 2 ^Λ Κ) = 2 L-1; // initialize with zero frequency metrics

for k = 1:2 ^Λ Κ

for i = 1: 2 ^A L-1

j = (64 - i) ;

if h(k) > ht(j)

bm ( k ) = j - 1 ;

end

end

end

endfunction function bma = BMC ( H, HT)

for s = 1:S

for j = 1:2 ^Λ Μ bma ( : , j , s ) bmc( H (1 : 2 ^Λ Κ, j , s) , HT(:,s) );

end

end

endfunction

The results of this look up process are the "canonical" branch metrics which are stored in an active branch metrics bank bm _a (1 :2 ^K , 1 :2 ^M , 1 :S) in metrics scratchpad 1 15. The metrics scratchpad 1 15 actually comprises a second set of passive branch metrics bm _p (1 :2 ^K , 1 :2 ^M , 1 :S), which are not shown in figure 7. The active branch metrics bm _a may be inventively post-processed by optional postprocessor-blocks LER 20, CTE 40 and DDMS 60 encompassed by the metrics postprocessing box 100 as described in more detail above. A simple programmable logic allows to configure executing these post-processing algorithms selectively and/or in any sequence. The preferred sequence is LER, CTE, DDMS. Figure 7 also shows parameters 120, 140 and 160, respectively. Multiplexer 1 17 selects the output of the active post-processor for returning the process branch metrics to metric scratchpad 1 15.

As an input, LER 20, CTE 40 and DDMS 60, all need the index of the maximum of the histogram. Index of the maximum of the histogram, mode of the histogram and the above-mentioned minimum metric index m are synonyms. The maxima of the histograms are searched in a process called Histogram Mode Value (HMODE) 1 18 and stored in m(1 :2K,1 :S) 1 19. The implementation of the HMODE process is fairly simple and implements a complete search in each histogram: function hmode = hmode(h)

hmode = 0 , hmax = 0 ; if h(i) > hmax

hmode = i, hmax = h(i) ;

end

end

endfunction function m = HMODE (H)

for s = 1:S

for j = 1 : 2 ^Λ Μ m(s, 1) hmode ( H (1 : 2 ^Λ Κ, j , s) ) ;

end

end

endfunction

Equivalently (not shown) it is possible to extract the required mode values from the metrics scratchpad, by searching for the locations of the best branch metric. The advantage of computing HMODE from histograms is that it can be done in parallel with BMC 1 14; moreover HMODE 1 18 can be refined e.g. by computing the histogram mean (HM, cf. equation (8)) and using this histogram mean suitably rounded as histogram center index m in LER 20, CTE 40 and DDMS 60. This histogram mean might be useful either for a semi-parametric metrics computation method (e.g. in software) or would allow a DDMS variant with resolution finer than an integral bin.

HM _S =∑k H, (8) k=l BMUPDATE 1 16 is a simple data transfer process that atomically updates the operational metrics 102 from the active metrics scratchpad 1 15, after the metrics computation (either canonical or canonical with post-processing) is finished.

Simulation Results

We demonstrate the strength of the proposed post-processing methods by presenting some simulation results.

The NRZ transmission of about half a million bits for undispersed optical channel (RD = Ops/nm) has been simulated. The 16-state MLSE with two samples per bit was used with parameters:

K=3, Z-=6, P _min =10 ^"12 , mst=4, mtms=5, pmd=1 , and dt(:)=2\ . First simulation was run at OSNR of 1 1 dB over one unit interval. In this simulation, the methods tend to suppress the effect of errors on building histograms. Main job was done by the LER and CTE.

Another simulation was done at OSNR of 14 dB. The histograms 1 1 1 10 and 1 1 1 1 1 had a "knee" at both sampling phases (1 and 2) as shown below. The MLSE generated 24 errors with a knee, and after post-processing the number of errors dropped to 4. In this case, the main job was done by the CTE. Similar results are observed in real measurements with 4-state MLSE.

Table 7 As explained above, the metrics indifference results in an error floor that location depends on MLSE construction. For example, the commercial (CoreOptics) oversampled 4-state MLSE using K=3 and L=4 shows an error floor at BER of 10 ^~10 . When the DDMS is used with all elements of the maximum metric delta template mmdt being set to 5 this error floor is eliminated. Further modifications and variations of the present invention will be apparent to those skilled in the art in view of this description. Accordingly, this description is to be construed as illustrative only and is for the purpose of teaching those skilled in the art the general manner of carrying out the present invention. It is to be understood that the forms of the invention shown and described herein are to be taken as the presently preferred embodiments.

reference list

4 optical channel

9 receiver control node

10 receiver

5 1 1 physical interface 12 AGC or VGA

13 ADC

14 clock recovery subsystem

15 sampling phase adjustment circuit

17 MLSE

10 18 FEC decoder

19 channel model unit

20 LER processing

21 -39 steps

40 CTE processing

15 41 -58 steps

60 DDMS processing

61 -86 steps

100 metrics post-processing box

101 MMVA

20 102 branch metrics storage element

103 channel state

104 detected bits

108 HCNTACC

109 number of unit intervals to be accumulated 25 1 10 accumulated counter channel model

1 1 1 bin number

1 12 HNORM

1 13 relative frequency channel model

1 14 BMC

30 1 15 metric scratchpad

1 16 BMUPDATE

1 17 multiplexer

1 18 HMODE

1 19 modes of histograms

35 120, 140, 160 parameters