The invention solves problems with synchronizing data flows by counteracting delays between signals so that they can be reproduced simultaneously.

That data streams from two or more transmitters are synchronized can be of great importance to different types of applications. A delay in synchronization is noted, for example, in case of analogue reproduction of a digitally processed signal. For example, when reproducing an audio and video recording, it is important that data rate for the audio and image is synchronized. Other applications where

synchronization is important is when several radar stations or radio systems work together. Other systems where synchronization is important are those that include adjacent antennas receiving the same signal, for example, in signal homing. Further examples are stereo cameras with high-speed shooting or simultaneous shooting with different techniques such as IR camera and regular camera, etc. In the case of errors in sample level, the errors will not be possible to detect or be difficult to detect, but the result of a data processing will be incorrect, which may be dangerous if the data being processed requires accuracy. Synchronization solutions often use multiple memories that are played in parallel, where each data stream / channel uses its own memory. If the order between the memories is wrong, it will be a delay in the synchronization. The invention described in the application is applicable, for example, to these above delays.

Although components such as Analog-to-Digital Converter (ADC) and Field

Programmable Gate Array (FGPA) are mentioned in the description examples, these can be replaced by other devices that have problems with delays. ADC is

interchangeable to any clocked data stream from eg. a camera or a video. FGPA is interchangeable with any other device such as a digital signal processor (DSP), a Graphics Processing Device (GPU), a Central Processing Device (CPU), an

Application Specific Integrated Circuit (Asic), or an analogue system, etc. In the following application the word "receiver" is used, which means an ADC or equivalent and a receiving device, such as an antenna. If input is digital, the receiver may not have a separate receiving device. The term "second device" is used for the device corresponding to FGPA, etc. Problems with delays in backgroud art

Causes of data stream delays may be due to clocking problems when an analog signal is to be converted to a digital signal. Two data sources send analog data on a channel each. The analog signals are received by each receiver such as an antenna and converted into a digital representation in, for example an ADC, i.e. each signal receiver has at least one ADC or equivalent. During the conversion, an analog signal is sampled and a clock signal is used as the trigger for the sampling. After the sampling, quantification occurs in the receiver where each measured sampling value is given a rounded off digital value in binary code. A sampled value in binary representation is sent to a second device separat from the receiver. Input data in the receiver can also be initially digital, and then the digital / binary value is transmitted to the second device from the receiver. The clock signal acts as an on / off signal for reading the sampled data. A flank up / on or down / off of the clock signal causes data to pass the output of the receiver and is to be written to a second device's input.

All receivers operate according to an often common clock signal that may be sent from, for example, a central clock signal generating device to the respective receiver, alternatively, the receivers operate according to a clock signal each. The central clock signal generating device is not shown in the figures. After processing in each receiver ml , a clock signal k1 and a data stream d1 are transmitted from each receiver. That is, each receiver ml , m2, ... mn send each a stream of data and a clock signal. Data stream d1 , d2 ... dn and clock signal k1 , k2 ... kn out of each receiver has a relationship, the data stream is clocked according to the clock rate. The clock signal output from each receiver is consequently referred to as data clock, k1 , k2 ... kn, to distinguish it from the initial clock signal the receiver operates according to.

Normally in digital conversion in an ACD, a sample is recorded on a clock pulse, at an alternate data transfer rate such as double data rate (DDR) two samples are recorded on a clock pulse. Increase of sampling rate increases data transmission per clock pulse.

A signal delay can occur on the way from the receiver to the second device. The receiver is connected to the second device via, for example, via a cord or via a circuit on a circuit board. The physical path between different receivers and the second device may be different long. The different receivers need not even be connected to the second device via the same circuit board.

The second device receives the data streams from each receiver and the data streams are to be synchronized with the second device's internal clock signal i_k. The internal clock signal i_k indicates the second device's work rate or clock rate. The inputs of the receiver signals on the second device include programmed modules that handle data from each receiver. The programmed modules receive the internal clock signal i_k for the second device. Receivers' data clocks k1 , k2 ... kn may be delayed / out of step with the second device's internal clock signal i_k.

The second device's task is to signal process data streams e.g. to get good reproduction at a later digital-to-analog (DAC) conversion. The second device also has the purpose of synchronizing data streams from multiple receivers ml , m2 ... mn. The second device reads data stream d1 from receiver ml , which is made with the data clock’s rate k1 , this happens for each data stream d1 , d2 ... dn. The second device’s data inputs are made up of pins. For a data stream sent from the receiver, for example, a port is used on the second device with 12 input pins, each pin receiving a bit of the binary code and an additional input pin for the data clock k1 from the receiver. Several receivers may be connected to the same second device.

There may be different delays inside the second device for the different data streams. A delay may occur within the second device because each input on the second device for the different receivers does not always have the same wiring on the circuit for the different receivers. The circuit is the computing device of the data stream in the second device, and on it a delay may occur on the path from the physical pin until the data stream reaches the programmed module.

An additional type of delay, offset delays, may occur in start-up mode when power is turned on, due to the different hardware components having different start-up times. The relationship between the second device's internal clock signal i_k and the data clocks k1 , k2 ... kn, used as clocks for read-in, affects the input of data from the input of the second device to a memory, buffer or register in the second device. The same clock relation affects if there is a delay for a data stream compared with another data stream. At a delay, a data stream’s value may be written to a buffer b1 while a second data stream’s value is not entered because the relationship between the different clocks is wrong, see Figure 1. Local heat generated within the second device may affect start-up-up condition so that any circuit is getting started faster than others which may cause additional offset delays. Startup in cold or hot conditions can give different delays.

The internal delays in the second device may be as small as in the order of nanoseconds, which is much less than other delays, such as those arising from different lengths between the receiver's connection to the second device. The internal delays in the second device are completely out of control, when choosing a hardware for a system, one chooses to use existing hardware such as FGPAs and thus have no control over their wiring.

How data streams from two or more receivers lie in relation to each other in memory is essential for the synchronization of the data. The delays mean that when the data stream is clocked to a common clock domain / the second device's internal clock signal rate i_k, it is not possible to know which sample is clocked over first and added first in memory / buffer / fifo queue. Each data stream / channel may have its own memory and multiple memories can be played in parallel, but if the order between the memories is wrong, it will be a delay in the synchronization. Data streams from two receivers will lie oblique to each other, which means they are not synchronized.

Figure 1 shows an example of the problem of a delay between two data streams.

Two data streams, d1 and d2 are clocked according to two individual data clocks k1 , k2 in each receiver and when they are to be reproduced according to a common clock domain / the second device's internal clock signal rate i_k, after processing in a second device, data for the two data streams is out of step in relation to the common clock domain / the second device's internal clock signal rate i_k. Sample n for the first data stream d1 is played simultaneously as sample n-1 for the second data stream d2. The invention

Figure 1 Illustrates the problem with data path delay

Figure 2 Illustrates a possible hardware design

Figure 3 Illustrates synchronization of trigger signal t in the flip-flop

Figure 4 Illustrates a possible internal FPGA design with DDR interface

Figure 5 Illustrates delays before the second device and how they are corrected by sorting samples with active synchronized trigger in the second device The invention solves the problems with the delays between signals intended for parallel / simultaneous reproduction, between a certain point until the signals have been sampled and have got a common clock domain. The delays are caused, for example, by different wiring lengths, by heat generation or due to the design of internal circuits. In the invention, two data streams or more are processed because synchronization errors between two or more data streams are to be corrected. Data streams can be signals that have been digitally converted to data streams, or alternatively, two digital data streams can be found at each receiver. Since especially synchronization errors in high-speed data structures are of interest to solve, it is the DDR that is used, but the invention also works at simple clock rate or other transmission methods.

The solution consists of a system, a method and a computer program where a flip- flop for each receiver has been added between each receiver and a second device. Briefly, the invention can be described by the output of flip-flop is used to mark each data stream d1 , d2 sent from a receiver ml , m2 to a second device, so that they are stored in the correct order in a buffer for each data stream in the second device. Upon receipt of the data streams of the second device a clock transition to the second device's clock domain occurs in a first buffer b1 per receiver. To solve the problem of different data streams being delayed the marking made by the flip- flops is used. Data samples that are not properly marked will be thrown while marked data samples are added to a second buffer b2 per receiver. When all of the data streams second buffers b2 contain information, they are read in parallel and the signals are synchronized. For more details, see the independent claims. The dependent claims describe alternative embodiments. A prerequisite of the invention to be able to control the synchronization of multiple data streams, for the invention to function, is that there is a time when the data streams are synchronized. If the receivers for example are synchronized with a common clock signal, sent from for example, a central clock signal generating device to the respective receiver, and if the receivers have the same wiring lengths from the antenna / source of the analog signals to the ADC then data and clocks are in rate. A further prerequisite for this inventive is that the infrastructure between the two receivers of a signal is controllable.

Figure 2 shows an example of the system with two receivers ml , m2, which have received and converted a signal each to a digital data stream d1 , d2. Alternatively, you can assume two initial digital data streams d1 , d2 found at each receiver. Data streams out from the at least two different receivers will have one data clock k1 , k2 each which is synchronized with one transmitted data stream d1 , d2 each. A second device receives data streams from the receivers. A flip-flop per receiver v1 , v2 is placed between the receivers and the second device. Each flip-flop v1 , v2 receives as input a data clock k1 , k2 from each receiver and a trigger signal t from the second device. Output from the flip-flop is a synchronized trigger signal s_t that is sent to the second device and used to mark samples from the receiver, this is described in detail later. In the invention, differences in propagation times are compensated, which is the difference in time it takes for a signal to reach a device compared to the time it takes for another signal to reach the device. An example of achieving the same propagation time is to use equally long wiring, alternatively, delay circuits can be added to compensate for delays. The compensation for differences in propagation time can be made, for example, by the wiring length of for example the data stream from the receiver to the second device and the wire length of the synchronized trigger signal s_t correspond to such an extent that the data stream d1 and the synchronized trigger signal s_t1 belong to the same receiver clock pulse when they arrive at the second device. A receiver clock pulse corresponds to a period of the data clock. The physical wire length may be the path of a signal on a circuit board or a separate wire. A prerequisite of the invention is that the propagation time of the trigger signal t from the second device to all the flip-flops is equal and that the synchronized trigger signal s_t and the data stream arrive substantially

simultaneously to the second device. This is possible, for example, by the propagation time of the data signal d1 between a receiver ml and the second device as compared to the propagation time of the synchronized trigger s_t1 from the flip- flop to the second device is substantially equal which means that the signals are substantially in rate, within the same receiver clock pulse, no more difference than 25% in clock pulse isrecommended.

Each flip-flop v1 , v2 ... vn is clocked with the corresponding receiver’s ml , m2 ... mn data clock’s rate k1 , k2 ... kn by each flip-flop, e.g. v1 , receives a data clock k1 from its respective receiver ml . A prerequisite for the invention is the propagation time of the data clock from the receiver ml to the flip-flop v1 is equal for all combinations of receivers and flip-flops ml , m2 ... mn and v1 , v2 ... vn.

Some delay occurs when processing the data clock in the flip-flop and when the data clock shall reach the flip-flop from the receiver. To compensate for delays it is not sufficient to match length for s_t1 with the length of d1 and k1 , because when the signal s_t1 reaches the second device also depends on the propagation time of k1 to reach the flip-flop and a small internal delay within the flip-flop v1 before s_t1 is on the flip-flop output.

The propagation time of k1 directly from the receiver to the second device should be as equal as possible with the combined propagation time, which includes: the time of k1 to v1 plus, the propagation time from v1 input to v1 output (i.e. the time of processing in the flip-flop), plus the propagation time from v1 to the second device.

The difference in propagation times is compensated for the signal to reach the second device so that the data stream d1 , d2 and data clock k1 , k2 from a receiver ml , m2 and the synchronized trigger signal s_t1 , s_t2, belonging to the respective receiver ml , m2, arrive at the second device within the same period of the pulse of data clock k1 , k2 from the receiver.

Matching the wire length is not always enough to compensate for all differences in propagation time, even professional component design and programming is required. Differences in wire lengths between the receivers ml , m2 ... may interfere with the size of buffers b2. The buffers b2, for example one per receiver, are in the second device. Buffers b2 store each receiver’s data and are filled up with different amounts to compensate for delays due to wiring differences and other delays that may occur. The difference in how the different receivers’ buffers b2 are filled up correspond to the difference in wire length and other possible delay factors. In one embodiment, the invention works on errors at sample level since the difference in wire lengths is not so great, but the invention also works on major differences in wire lengths. In the latter case, the buffers b2 in the second device are filled up the more before the data is read synchronized. The size of buffers b2 is suitably adapted to handle the amount of data.

Return to the previously introduced trigger signal t. The trigger signal t that the second device sends as input to the flip-flop / flip-flops consists of, for example, a positive flank. A prerequisite of the invention is that the propagation time of the trigger signal t from the second device to all the flip-flops is equal. Additional input to the flip-flop is the data clock from the receiver. Once the data in; the data clock k1 has a positive flank and the trigger signal t is a one, the output becomes a continuous one which is the synchronized trigger signal s_t1. The trigger signal t from the second device which initially may be asynchronous as compared to the rate of the receiver’s data stream d1 and data clock k1 , will after the clocking in the flip- flop be synchronized with the data stream from the receiver s_tl Asynchronous means that the trigger signal t is not synchronized with the data stream ^' s clock pulse. For example, when the positive flank of the data clock occurs simultaneously as the asynchronous trigger signal t is one, the flip-flop gives a one as output, this is an asynchronous trigger signal t which has become a synchronized trigger signal s_t which is synchronized with the data stream’s data clock. Positive flank / one is an example but it might as well be a negative flank / zero that triggers an output signal. In the above-mentioned embodiments thus, the flip-flop functions as a D-flip-flop. The flip-flop is replaceable with clocked flip-flops well-known in the area, such as SR or JK flip-flops. The trigger signal t does not have to be asynchronous initially, the flip-flop still works. A one corresponds in this example with that the trigger signal t is in active mode, while a zero corresponds to the trigger signal t being in passive mode, but it could as well be the opposite.

Figure 3 shows the input to the flip-flop; the receiver's data clock k1 and the asynchronous trigger signal t. The vertical dashed line on the left shows that the trigger signal t initially is not in rate with the data clock k1. The line does not coincide with the positive flank of the data clock k1. The asynchronous trigger signal t has passive mode before the left line and active mode after the left line. After processing the trigger signal t is a synchronized trigger signal s_t1 , which is in phase with the clock signal k1 , which is shown with the dashed line on the right. The active mode of the synchronized trigger signal coincides with the positive edge of the data clock. In this way, a new synchronized trigger signal s_t1 is created which is synchronized with one data clock k1 each and the corresponding data stream d1.

The second device receives for each signal the data stream d1 , the receiver's data clock k1 and a synchronized trigger signal s_tl For each receiver ml , m2 ... mn processing a signal (analogue or digital) the corresponding data stream, data clock and synchronized trigger signal are sent to the second device. In Figure 4 one sees the data in to a second device which consists of: the data clock k1 , the data stream d1 and the synchronized trigger signal s_t1 from a receiver ml , (the receiver is not visible in the figure). The data streams from the different receivers are clocked to a common internal clock rate in the second device. This is done by the second device for each received data stream received from the receiver performing a clock transition to the second device's clock rate i_k. i_k is a general clock for the second device, which does not go directly as a signal but is used by the second device as a work phase / general data clock. The clock transition consists of the data stream’s values from a receiver being written to a buffer b1 in the second device according to the rate of data clock k1 from the receiver and that the next memory reading of buffer b1 occurs at the rate of the second device's internal clock signal i_k.

The trigger signal t has the second device sent out and it has just passed the flip- flop before it is received again as the synchronized trigger signal s_t. In the exemplary embodiments, an additional pin is used on the second device for receiving the synchronized trigger signal. The synchronized trigger signal s_t out of each flip-flop is treated as belonging to its own data stream. It adds a data bit to the data transfer, for example if 12 bit data is used for the data stream, which is a series of data samples, from the receiver the second device processes the data as 13 bits. The number of bits is interchangeable for a person skilled in the art depending on the data transfer chosen. In the invention’s embodiments, a pin on the second device is used to send out the trigger signal t to all the flip-flops.

The data stream’s bits d1 are read at the input of the second device as a high or low voltage which corresponds to a digital one or a zero. The read values for the data d1 is written to a memory location in the first buffer b1 found in the second device. A flank up / on / start-up of the k1 signal causes the data to be written to b1. In an exemplary embodiment, the second device is an FGPA. In the embodiment shown in Figure 4, in a known manner, components BUFR, Idelay and IDDR are included in the FGPA. The ADC data clock k1 is enhanced in BUFR. The data is delayed in Idelay. IDDR is a flip-flop that reads the data on both descending and ascending flanks, and does it again so that we get two data in parallel on only ascending flank, after IDDR the system works only on the ascending flank. IDDR is clocked with the BUFR data clock kl The buffer b1 receives a data clock k1 from the receiver, a data stream d1 from the receiver and works first according to the receiver's data clock’s rate k1 , k2 ... kn, that is, write to buffer b1 takes place according to the receiver's clock rate. Then after the clock transition, b1 operates according to the clock rate of the second device's internal clock signal i_k. As output at the next memory reading, b1 provides a data stream which is in rate with the second device's internal clock signal i_k.

That is, in the next clock cycle of the clock signal i_k the memory location in b1 is read. The continued data processing in the second device is conducted according to the clock rate i_k. In the invention, only the value from the memory locations having an active trigger is read, the synchronized trigger signal s_t in active mode. The synchronized trigger signal s_t1 follows the data stream d1 and is treated as a part of d1 in b1. The data stream parts that do not have an active synchronized trigger s_t1 are deleted and are not processed further, this is shown in Figure 4 by the first cloud p1. In p1 all data stream parts / samples where s_t1 is not a one are thrown away.

The read values are written to a second buffer b2 found in the second device. For example, if the synchronized trigger signal s_t1 and the internal clock signal i_k have / get a high flank active trigger, it means that the data stream value d1 is written in the second buffer b2.

When all second buffers b2 for each data stream contain information, the content of the second buffers b2 is forwarded for parallel synchronized reading. This means that the invention includes a process that scans all of the second data streams’ buffers b2, waiting for data to be present in all. This is illustrated in Figure 4 by the second cloud p2. Figure 5 illustrates input to the second device from two receivers. Receiver 1 sends data clock k1 and data stream d1 to the second device. Receiver 2 sends data clock k2 and data stream d2. The data clock k1 and data stream d1 are synchronized and the data clock k2 and data stream d2 are synchronized. Data streams are now clocked / sampled according to the clock rate. The synchronized trigger signal s_t1 of the first receiver and the synchronized trigger signal s_t2 of the second receiver enters the second device within the same receiver clock pulse as corresponding receiver’s data stream d1 and d2, respectively. A delay in which data streams d1 and d2 are not in phase is illustrated in Figure 5 by Ab. A delay in which the rates for data clock k1 and k2 are not in phase is illustrated Ac. When the received data streams are not in phase due to different delays, a transition must be made so that data streams from receivers "1", "2" ... "n" become synchronized with an internal clock of the second device i_k. That read values are written to a second buffer b2 is illustrated in Figure 5 by data samples 1 and 2 from data stream d1 are thrown, marked with "s", while data samples 3 and 4 are added to the upper second buffer included in the FGPA, marked with "b2". Also the data stream d2 is processed in the same manner so that two different buffers b2 for each data stream, illustrated with an upper buffer b2 and a lower buffer b2 in Figure 5, are filled up with the data samples having a simultaneous active synchronized trigger signal. When there are data in both buffers b2 in Figure 5, data is played, which now is synchronized, in parallel.

In one embodiment, each data stream d1 , d2 ... dn has its own buffer b1 , i.e. there are at least two buffers bl This corresponds to the second device having a buffer (b1) per receiver. In one embodiment, each data stream d1 , d2 ... dn, has its own second buffer b2, i.e. there are at least two buffers b2. For example, b1 and b2 may consist of a buffer, a memory, a register or a first-in-first-out (fifo) queue. However, it is possible to use different buffer assemblies, such as sharing buffer. Depending on the type of buffer and the number of buffers used different adjustments known to a person skilled in the art, such as memory pointer programming, are required.

As a consequence of the invention, there are no longer any requirements to have equal physical wires between the various receivers and the second device or within the second device to achieve synchronization.