TECHNICAL FIELD

The present invention relates to a controller for a real-time control system (RTCS) for generating setpoints. The invention also relates to an agreement protocol or a method for generating setpoints for actuators in an RTCS. The present invention also relates to a controller system and to an RTCS.

BACKGROUND OF THE INVENTION

RTCSs are computing systems, which control an operation of a process or system in real time. The RTCSs typically use controllers for generating setpoints for actuators, which control the operation of another system or process, such as an electrical grid or a manufacturing process. The controllers receive labelled (time- stamped or logically sequenced) measurements from sensors for generating the setpoints. The controllers use the measurements to compute and issue setpoints to the actuators to complete the control cycle. The measurements and setpoints are sent over non-ideal networks, and might thus experience delays and/or losses. RTCSs tolerate delay and crash faults by replicating the controller. Each controller replica computes and sends the setpoints to the actuators over a network, which may drop or delay messages. Thus, the actuators may receive an inconsistent set of setpoints. Such inconsistency is avoided either by having a single primary replica compute and issue setpoints (in passive replication) or by a consensus or agreement algorithm, which selects one sending replica (in active replication). However, due to the impossibility of a perfect failure-detector, passive replication schemes may have multiple primaries operating simultaneously, causing inconsistency, especially in the presence of intermittent delay faults. Furthermore, the impossibility of bounded- latency consensus causes also the known active replication solutions to have poor real-time performance.

Passive replication is commonly used in RTCSs because it avoids agreement by having one primary compute and issue setpoints. If the primary is detected as faulty, the standbys elect a new primary by consensus. However, in non- synchronous settings, failure detection is imperfect. Consequently, the standbys could incorrectly detect the primary as faulty and elect a new primary, while the original primary is not notified (eg due to a lost notification), resulting in two primaries, leading to potential inconsistencies. However, these inconsistencies are rare under a crash- only fault model. Most passive-replication research considers the crash-only fault model, under which passive replication improves availability at a negligible cost of inconsistency. For RTCSs, however, a stronger fault model, which includes delay faults, is relevant. The intermittent nature of delay faults causes more false positives by failure detectors, hence more inconsistencies.

Alternatively, active replication schemes generally guarantee consistency by using consensus, for deciding the setpoint(s) to be sent or the sender of the setpoint. Consensus consists of four properties: agreement, termination, validity, and integrity. Validity and integrity address issues that arise due to Byzantine faults, and are not under consideration in the present invention. These properties are trivial to guarantee under the fault model considered in the present invention (which does not include arbitrary Byzantine faults). The agreement and termination are impossible to guarantee together within a bounded delay in a non-synchronous setting. Since RTCSs require a low bounded latency, this results in low availability. SUMMARY OF THE INVENTION

It is an object of the present invention to overcome at least some of the problems identified above relating to agreement, controllers and/or RTCSs.

According to a first aspect of the invention, there is provided a method of generating one or more setpoints for one or more actuators of a real-time control system as recited in claim 1 .

According to the proposed new solution, setpoints are generated by a controller based on digests received from other controllers. A digest is a message that contains a list of measurement indices and a controller state label present at a controller. In other words, contrary to existing solutions, the present invention bases the computation of setpoints on digests. The most appropriate digest can be chosen for setpoint computations and the same digest can then be used by all the controllers. Therefore, the controller votes on and chooses an appropriate digest (to guarantee consistency), which in turn means that it is choosing a set of measurements to use for the subsequent setpoint computation. In this manner guaranteed consistency, increased availability, bounded latency-overhead can be achieved. Performing agreement at the controller input (agreement on sensor measurements), rather than over the setpoints or the issuing controllers, affords a phase of collection prior to agreement. This increases the chances of a successful vote, thereby improving availability as explained later. In other words, to enhance the chances of a successful vote, voting is preceded by a phase of measurement and optionally state exchange(s) that enable(s) the controllers to have the same measurements and state. The proposed solution is also designed to have bounded latency-overhead thanks to quick agreement on the measurements.

According to a second aspect of the invention, there is provided a controller for a real-time control system as recited in claim 13.

According to a third aspect of the invention, there is provided a controller system for a real-time control system as recited in claim 14.

According to a fourth aspect of the invention, there is provided a real-time control system as recited in claim 15.

As all the controllers may send their setpoints (which are identical) to the actuators, there is no time lost in deciding which controller should send their setpoints to the actuators. The proposed real-time control system is consistent and has bounded latency-overhead.

Other aspects of the invention are recited in the dependent claims attached hereto.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features and advantages of the invention will become apparent from the following description of a non-limiting example embodiment, with reference to the appended drawings, in which:

• Figure 1 is a block diagram schematically illustrating an RTCS according to an example of the present invention;

• Figure 2 is a flow chart summarising a method of generating setpoints by a controller of the RTCS of Figure 1 according to an example of the present invention;

Figure 3 is a flow chart summarising a method of collecting measurements and states according to an example of the present invention; and

Figure 4 is a flow chart summarising a method of voting according to an example of the present invention. DETAILED DESCRIPTION OF AN EMBODIMENT OF THE INVENTION

An embodiment of the present invention will now be described in detail with reference to the attached figures. Identical or corresponding functional and structural elements that appear in the different drawings are assigned the same reference numerals.

The block diagram of Figure 1 schematically illustrates the architecture of the RTCS 1 where the teachings of the present invention can be applied. In this example, the RTCS 1 comprises m sensors 3, which are arranged to sense or detect events or changes in the operation of a unit, system or process 5 to be controlled by taking measurements from the controlled process 5. The sensors 3 may all measure the same or different properties of the controlled process 5. The sensors 3 are further arranged to send the measurements over a first communication network 7 to a set of controllers 9, also known as replicas. The number of controllers 9 in the RTCS 1 may be between 2 and 10, or more specifically between 2 and 5 or 2 and 4 or 2 and 3. In this example, all the controllers 9 in the set are substantially identical, ie they are configured to operate in the same manner when dealing with the same input. The sensors 3 are arranged to mark or label each measurement with a logical sequence r, referred to as a label or measurement label r. The measurement labels are global and are shared by all the sensors 3 in the RTCS 1. Such numbering can be obtained for instance by time-synchronisation or through logical clocks or by any other suitable means. The controllers 9 are arranged to compute and issue setpoints or control parameters or commands. In the example below, h setpoints will be generated. The controllers 9 are also arranged to send the generated setpoints over a second communication network 1 1 to actuators 13. In this example there are h actuators 13. Furthermore, according to this example, each controller 9 is arranged to generate one setpoint per actuator 13. As there are h actuators 13, this means that each of the controllers is arranged to generate h setpoints, which may or may not be all different. However, the controllers 9 need not issue setpoints/commands for all actuators or could issue several setpoints/commands for some or all of them. The teachings of the present invention apply as long as all the controllers behave substantially identically. The measurements and setpoints are sent over the networks 7, 1 1 , which are not ideal. This may thus lead to delayed or lost messages (ie the measurements or setpoints). In case of no loss, the propagation delay is bounded by 5 _n . The message handling or processing time of non-faulty controllers 9 is negligible with respect to <5 _n . This network model is called probabilistic synchronous. The RTCS 1 according to the present invention is expected to hold three properties as explained below.

Property 1 . The state used by the controllers 9 for computing the setpoints can be known exactly. This property is satisfied by a wide range of controllers including controllers using Kalman filters where the state is the gain matrix, or more

sophisticated controllers, where the state is the time of the most recent voltage violation. As the controllers 9 are not perfect, ie they may comprise commercial-off- the-shelf (COTS) components, they are susceptible to faults. In addition to crash faults, considered within the fault models of many classic agreement solutions, the present invention considers delay faults. Any controller 9 is marked delay faulty between two consecutive issuings of setpoints if its delay in issuing the previous setpoint is greater than a pre-defined value. A crash fault is a special case of delay fault with the controller's delay being infinite. The intermittent nature of delay faults makes real-time agreement more challenging. As a controller can be delay faulty for an arbitrarily long time, solutions that rely on failure detection will incur a high latency-overhead for each setpoint. In order to tackle the challenges of such intermittent faults, the controllers 9 are expected to satisfy the following property. Property 2. The controllers 9 can compute correct setpoints in the presence of intermittent measurements.

This property holds for controllers that receive measurements and send setpoints via a non-ideal network, eg wide area networks. Specifically, this applies to RTCSs that use a Kalman filter, which accounts for intermittent measurements.

Controllers of RTCSs that exhibit this property can compute and issue setpoints despite intermittent faults, either in the network (losses of measurements) or in the controller (delay faults). On the contrary, an RTCS that does not exhibit Property 2 will eventually permanently fail to compute and issue setpoints, even in the absence of crash faults. This is because a controller that fails to compute setpoints for a label r cannot compute setpoints for all labels greater than r.

From this property, it can be understood that a controller might compute setpoints for label k < r - 1 , be delay faulty for some time and then compute setpoints for label r, by using a controller state corresponding to label k, without the knowledge of the intermediate states. It is to be noted that the resulting setpoints would be correct, but might be sub-optimal when compared to those computed using the state corresponding to label r - 1 (ie more recent state).

Lastly, as the present invention intends to use active replication, multiple controllers 9 will issue setpoints of the same label to the actuators 13. Therefore, the following property is expected to hold.

Property 3. Actuators are able to handle duplicate setpoints, ie at least two setpoints (which are advantageously identical).

This property is generally exhibited in RTCSs that use absolute, rather than differential, setpoints. An example of absolute setpoints is an electrical grid controller instructing a battery agent that is injecting for instance 8kW to 'set the injected power to 10kW, rather than to 'increase the injected power by 2kW.

Receiving at least two identical setpoints has the same effect as receiving a single setpoint. For RTCSs with differential setpoints, Property 3 can also be achieved by deduplication using additional labels for the setpoints. The proposed method of generating setpoints is next explained in more detail with reference to the flow charts of Figures 2 to 4. It is to be noted that the method described in the flow charts of Figures 2 to 4 may be carried in parallel by all the controllers 9 of a set of controllers. Referring to the flow chart of Figure 2, in step 21 , the following parameters are set to zero: r, r, Z and H, where r is the highest label of measurements received by the controller at a given time instant, r is the highest label of measurements used by the controller for computation of a setpoint at a given time instant, also referred to as a controller state label, Z is a vector of measurements with label r, and H is a controller state after computing setpoints with label r. Thus, r can also be understood to be a tag of the controller state H. In step 23, the controller 9 receives the measurements from the sensors 3. Prior to that, the sensors 3 have previously taken measurements of the process 5. It is to be noted that some of the measurements may be delayed and/or lost in the first communication network 7 or delayed by the controllers 9. Accordingly, not all the controllers 9 receive or deal with the same set of measurements from the sensors 3. From the received measurements, in step 25, the controller 9 generates or updates the vector of measurements, Z, corresponding to label r. In step 27, it is determined whether or not r is larger than r. In other words, it is determined whether or not the received measurement(s) has (have) a higher label than any previously received measurement used for computation. If the response is no, then the process continues in step 23. If the response to the question in step 27 is in the affirmative, in step 29, a collection function is called. It is to be noted that even if a positive response has now been received in step 27, steps 23, 25 and 27 are repeated continuously allowing measurements to be received and aggregated in the vector Z. In this manner the vector Z is formed, and, when new measurements with label greater than r are received, r is overwritten and simultaneously also all the entries in the vector. This may be carried out in parallel with step 29. It is to be also noted that different measurements from the sensors 3 may be received at slightly different time instants due to non-optimal operation (in terms of delay) of the first communication network 7 and/or the sensors 3. Thus, immediately after having overwritten r, the vector Z may comprise only one measurement entry and the other entries would be zero before new measurements are received for this label. An example of such an operation is the time-alignment function in phasor data concentrators upon receiving time-stamped measurements from phasor measurement units (PMUs). The time-alignment function in phasor data concentrators takes care of aggregating a set of measurements with a given label/timestamp that are received at slightly different time instants. Thus, this function would provide the controller with all the measurements from PMUs that it received within a small time duration (typically the jitter in the network).

The collection function is outlined in the flow chart of Figure 3. In step 31 , the collector 9 forms a vector S, which in this example comprises a set of identities (IDs) of the sensors 3 from which it has received measurements with label r. Also in step 31 , the controller 9 sends a query to all the other controllers 9 asking for the missing measurements (entries of S). In this example, the query includes the label r to indicate to which label the missing measurements are requested. In step 33, the collector 9 informs the other controllers 9 about its r. In other words, the controller 9 sends an advertisement containing its state label r ^" to the other controllers 9. As mentioned before, r ^" is the label of the measurements involved in the setpoint computation leading up to that state, also referred to as the state label. In step 35, a timer is started. In this example, a timer of 2<5 _n is started. In step 37 it is determined whether or not the time measured by the timer has elapsed. In the affirmative, the process continues in step 39. In this step the controller 9 returns its set of

measurements, ie the vector Z, its current state H and its current state label r ^" to the calling function.

If in step 37 it was determined that the timer has not yet ended, then in step 41 it is determined whether or not the controller 9 has received a query asking for measurements the other controllers 9 have not received. If a query has been received, then in step 43 it is determined whether or not the received query contains the label r. If the received query contains the label r, then in step 45 the controller 9 sends a response to all the other controllers 9 with answers to the query. The label r is included in the response. In other words, for each query received, the controller 9 sends a response containing the queried measurements it has. It is however possible that the controller 9 does not have all the answers to the query. In that case, only the answers that the controller 9 has are sent. After step 45, the process continues in step 47. The process also continues in this step if in step 41 or step 43 the response is negative. In step 47, it is determined whether or not the controller 9 has received a response to the query it sent in step 31 . If a response has been received, then in step 49, it is determined whether or not the response contains the label r. In the affirmative, in step 51 , the controller 9 updates its set of measurements (vector Z) to include the measurements received in the response. From step 51 the process continues in step 53. In step 53, it is determined whether or not the controller 9 has received an advertisement. In the affirmative the process continues in step 55, where it is determined whether or not the received advertisement contains a state label I (I is a variable) that is smaller than r . If I < r, then in step 57, the controller 9 sends an update to all the other controllers 9. The update comprises the current state of the controller 9 and its state label r so that the other controllers 9 can synchronise their state accordingly. From step 57 the process continues in step 59. Also, if in step 55 it was determined that the condition I < r is not satisfied, then the process also continues in step 59. In step 59, it is determined whether or not the controller 9 has received an update message from any other controller 9. If an update has been received, then in step 61 it is determined whether or not the received update comprises a state label I which is larger than r ^" , ie whether or not I > r ^" . If this is the case, then in step 63, the controller 9 updates the current state of the controller 9 to the received state and the current state label to I as received in the update. From step 63 the process continues in step 37 to determine whether or not the timer has ended. If in steps 59 or 61 the response is negative, then the process also continues in step 37. It is to be noted at least some of the above message exchanges include the label r to ensure that stale (not most recent) measurements, caused by delay-faulty controllers 9 sending queries or responses, are ignored. The operations carried out in in the flow chart of Figure 3 are comprised in a measurement and state collection phase, collectively referred to as a collection phase or simply collection. In this example, the measurement and state collection phases are run substantially simultaneously. According to the described example, the collection phase lasts for at most 2<5 _n at each collector 9 (the time required for queries and responses to be delivered), resulting in a bounded latency of 2<5 _n for the collection phase. As explained above, the collection phase comprises the controllers 9 exchanging measurements and state. The collection, designed with delay faults in mind, is in this example terminated after 2<5 _n . Thus, non-delay-faulty controllers can proceed with a next step (a voting phase) without waiting for delay-faulty ones.

The collection phase ends with each controller 9, in step 64 (Figure 2), generating and sending its digest (a measurement summary) to all the other controllers 9. In this example, the digest with a label I consists of (1 ) an indicator set S of measurements (IDs of the sensors from which it has received measurements) with label I (in this case the label I is a label of the digest) that this controller 9 has, and (2) the state label r at this controller 9. The process continues in step 65 (Figure 2), where the controller 9 calls a voting function as outlined in the flow chart of Figure 4. The controllers 9 comprise a software unit, referred to as a voter, enabling the controllers 9 to vote on the digests as explained below. For the following voting phase called by the voting function in step 65, the present invention uses a total order among digests. Consequently, a function, referred to as a max, returns the largest digest, based on the total order. For each label, there exists a largest possible digest (full_digest) that does not contain any missing measurements and that has r = 1-1 . A possible implementation of the digest is a concatenation of the state label with the bitmask of the measurements. For instance, computing setpoints with label I = 25: if r is 24 and a controller 9 has received measurements from sensors 1 , 2, and 5 (m = 5), then its digest would be "24.1 1001 ". The lexicographic ordering of the digest string gives the total order, with "24.1 1 1 1 1 " being the full_digest. The present invention may also use a so-called composite voter that, in cases of no single majority, selects the output of one of the controllers 9 based on a predetermined order. Unlike some known solutions, where the order may be replica-based, the proposed solution uses an ordering based on the number of measurements, a scheme more suited to RTCSs. Also, the proposed solution may use plurality voting (someone received more votes than the others), instead of majority voting (someone received more than 50% of the votes), as it has higher availability.

In this example, each controller 9 maintains a list or vector v of digests received with label r from the other controllers 9, with at most one entry in the vector v per controller. From the vector v, the controllers 9 attempt to vote and choose a digest, such that all the controllers 9 choosing a digest with label r, choose the same digest. Each voter also maintains two other lists, namely S _mc and S _sec and three integers, namely f _mc , f _sec and f ₀ , that are updated when a new digest is received and stored in the vector v. As the same digest can be received from several controllers 9, the number of occurrences of each unique digest in the vector v is counted. The digests with the highest frequency are stored in S _mc , and their count in f _mc . Similarly, the digests with the second highest frequency are stored in S _se c, and their count in f _sec . Finally, the number of empty elements in the vector v, that is the number of controllers from which the voter has not received digests, is stored in f ₀ . Referring now to the flow chart of Figure 4, the process now continues in step 67, where the controller 9 initialises the vector of digests v, ie the entries of vector v are set to zero. In step 69, a timer is started. In this example, a timer of 3<5 _n is started. In step 71 it is determined whether or not the time counted by the timer has expired. In the affirmative, the process continues in step 73, where the process returns unsuccessfully to the calling function. On the other hand, if in step 73 it is determined that the time has not yet expired, then in step 75, it is determined whether or not a digest has been received. If no digest has been received, then the process continues in step 71 . If a digest has been received, then in step 77, it is determined whether or not the received digest contains the label r. If the received digest does not contain the label r, then the process continues in step 71 . If on the other the received digest contains the label r, then in step 79 the received digest is added to the vector v. In step 81 , it is determined whether or not a digest has been received from all the controllers 9. If all the digests have been received, then in step 83, the digest from the vector v is selected such that the selected digest has the majority. If more than one exists in a majority set, the largest digest is chosen.

According to this example embodiment, there are four cases in which the voter can choose a digest. In all other cases, the voter has to wait for more digests.

1 ) The number of empty cells in the vector v is zero (case of step 83). In this case, the voter has to choose one of the digests that is most common. If there is only one, it is chosen, otherwise, it chooses the largest among them. 2) There is only one most common digest, and it will remain so no matter what the controllers 9 that have yet to send digests send. It is chosen (as the obvious majority). This is the case of step 85.

3) There is only one most common digest, and there is at least another digest

(second most common), and no matter what the controllers 9 that have yet to send digests send, any second most common digest can be at most as frequent as the most common one. In this case, if the most common digest is larger than all the second most common digests, the voter chooses it. This is the case of step 87. 4) The full_digest is the only most common one and no other digest can become more common. Since it is the largest by definition, it is chosen. This is the case of step 89.

The above approach enables the collectors 9 to successfully vote, before receiving the digests from all the controllers 9, while guaranteeing consistency, thus accounting for delay faults. Specifically, item 4 enables a two-controller RTCS to be available when one of its controllers is faulty, if the other controller receives all the measurements and is up-to-date on the state (full_digest). In this example, the voter returns unsuccessfully after 3<5 _n , which is enough time to allow the non-faulty controllers to send their digests. It is to be noted that the voting phase assumes an upper bound on the number of controllers 9 (faulty or non-faulty). This can be achieved either by statically configuring the number of controllers every time a new controller is added or removed, or by using a group membership algorithm. As addition of controllers is not done on the real-time path, the introduction of a new controller can wait until the termination of the group membership algorithm. During this time, the RTCS 1 continues to provide consistency but with reduced availability due to a conservative count of the number of controllers.

Once the voting phase has been completed, the process continues in step 91 (Figure 2), where it is determined whether or not the voting was successful. If the voting was not successful, then the process continues in step 23. On the other hand, if the voting was successful, then in step 93, the controller computes the setpoints by using the measurements identified by the measurements IDs in the selected (voted) digest and the controller state. In other words, the computation of setpoints X corresponding to label r depends on the measurements of the label r (Z), state (H), and correction factor (r - r). When sufficient measurements of label r are received or a timeout occurs, the controller can begin computing setpoints. The computations are performed in a strictly increasing order in r. Also, each computation results in a new controller state. It is to be noted that all the controllers 9 select the same digest to form the basis for the computation of the setpoints. In step 95, the controller 9 sends the setpoints to the actuators 13. The above method of computing the setpoints can be summarised by

Algorithms 1 to 5 explained below. Algorithm 1 gives an overview of the whole method. Some of the steps of Algorithm 1 are explained later in more detail with reference to Algorithms 2 to 4.

Algorithm 1 : 1 : r <- 0; // r is the highest label of measurements received;

2: r <- 0; // r ^" is the highest label of measurements used for computation;

3: Z <- 0; // Z is a vector of measurements with label r;

4: H <- 0; // H is a controller state after computing setpoints with label r ^~

5: repeat // Thread 1 : Receive and aggregate measurements.

6: Z, r <- aggregate_received_measurements(r);

7: forever;

8: repeat // Thread 2: Compute and issue setpoints.

9: if r > r then

10: success, Z, H, r <- collect_and_vote(Z, H, r, r);

1 1 : end

12: decision <- ready_to_compute(Z, H, r);

13: if success and decision then

14: X <- compute_setpoints(Z, H, r - r);

15: H <- update_state(Z, H; r - r);

16: issue(X, r);

17: r <- r;

18: end

19: forever;

Algorithm 2 below gives an overview of the collection and voting phases. In Algorithm 2, the subscript "coll" refers to collection and gives a corresponding parameter value after the collection.

Algorithm 2: collect_and_vote(Z, H, r, r )

1 : // Part 1 : Collection 2: Zcoii <- collect_missing_measurements(Z, r);

3: Hcoii, r ^~ _oM <- collect_missing_state(H, r);

4: S <- entries in Z _con ;

5: send digest(S, r ^~ ₀ n, r) to all voters;

6: // Part 2: Voting

7: success, S _ch osen, Chosen < ^~ vote(r);

8: if success and r ^~ _hosen = r ^~ _oN and S _ch osen £ S then

9: return True, Z _C oii[S _C hosen], H _coN , r ^~ _oN ;

10: else

1 1 : return False, Z _CO ii[Schosen], Hcoii, r ^~ _oN ;

12: end

Algorithm 3 gives a detailed overview of the measurement collection phase. In Algorithm 3, Q refers to indices of Z received in a query, while P is a vector of measurements received in a response. Algorithm 3: collect_missing_measurements(Z, r)

1 : S <- entries in Z;

2: send Query<S, r> to all replicas; // Ask for missing entries

3: repeat

4: if Query<Q, l> received and I = r then

5: // Received query, send response

6: send Response< Z[Q n S], r > to all replicas;

7: else if Response <P, r> received and I = r then

8: // Received response, update set of measurements

9: Update Z to include P;

10: Add received entries to S;

1 1 : end

12: until timer 2S _n expires;

13: return Z; // Return set of measurements after collection

Algorithm 4 gives a detailed overview of the state collection phase. In Algorithm 4, H' refers to the state received in an update message.

Algorithm 4: collect_missing_state(H, r)

1 : send Advertisements > to all replicas; // Advertise state label

2: repeat 3: if Advertisement<l> received and I < r then

4: // Received advertisement with smaller label, send my state

5: send Update<H, r ^~ > to all replicas;

6: else if Update<H', l> received and I > r then

7: // Received state with higher label, update my state

8: H <- H';

9: r <- I;

10: end

1 1 : until timer 2<5 _n expires;

12: return H, r ^" ; // Return state and label after collection

Algorithm 5 gives a detailed overview of the voting phase.

Algorithm 5: vote(r)

1 : v <- []; // vector of digests from each controller

2: S _mc <- set of most common digests in v;

3: Ssec «- set of second most common digests in v;

4: f _mc <- count of each element of S _mc in v;

5: f _sec «- count of each element of S _sec in v;

6: fo <- number of empty cells in v;

7: repeat

8: // Receive collection message

9: if digest (S, r, I) received from replica j and I = r then

10: vD] «- (S, r);

1 1 : update S _mc , S _se c, fmc, fsec, fo using v;

12: // Attempt vote

13: if f ₀ = 0 then

14: // All digests received, pick max of S _mc

15: return True, max(S _mc );

16: else if |S _mc | = 1 and f _mc > f _sec + fo then

17: // Only one most common digest, and clearly the majority 18: return True, S _mc [0];

19: else if |S _mc | = 1 and f _mc = f _sec + fo and f _sec ≠ 0 then

20: // 2nd most common digests could have equal count

21 : if Smc[0] > max(Ssec) then

22: // Most common digest is the largest 23: return True, S _mc [0];

24: end

25: else if |S _mc | = 1 and f _mc = f _sec + fo then

26: // Other digests could have equal count

27: if S _mc [0] = full_digest then

28: // Most common digest is the largest

29: return True, S _mc [0];

30: end

31 : end

32: end

33: until timer 3<5 _n expires;

34: return False, NULL; // Return false because vote was unsuccessful

One aspect of the invention can be summarised as follows. It is proposed a method of agreeing on measurements before computing one or more setpoints for one or more actuators 13 of a real-time control system 1 . The method comprises:

• a first controller 9 receiving a first set of measurements from sensors 3, the measurements being individually labelled with a sequence number, referred to hereinafter as a measurement label;

• the first controller 9 receiving a second set of measurements from a second controller 9, the received measurements of the second set of measurements being individually labelled with the same measurement label as the measurement label of the measurements of the first set of measurements;

• the first controller 9 generating a first digest from at least the first and second sets of measurements, the first digest comprising a first data set for determining measurements the first controller has received, and a first controller state label;

• the first controller 9 receiving at least a second digest from at least the second controller 9, the second digest comprising a second data set for determining measurements the second controller has received, and a second controller state label; and

• the first controller 9 selecting one digest from a set of digests comprising the first and second digests for generating one or more setpoints. The first and second measurement sets may be different. The first controller 9 then generates the one or more setpoints by using at least the

measurements and a controller state derivable from the selected digest. The first and second data sets may comprise indices of the measurements and/or the (entire) measurements the first and second controllers 9 have received, respectively.

It was described above a method of agreement on the controller input for generating setpoints for actuators of a real-time control system. Three properties of RTCSs were identified above that enable active-replication schemes to agree on the measurements before computing, instead of using traditional consensus. Since in the present invention all computing controllers do so with the same state, the resulting setpoints are guaranteed to be consistent. The proposed active replication solution uses this approach to guarantee consistency and bounded latency-overhead. The present invention performs agreement on the measurements Z and the state H before computation, as opposed to agreement on setpoints done by existing solutions. By agreement on H for label r, controllers explicitly agree on the correction factor, r - r ^" . The measurement set used for the computation may be a subset of the whole measurement set comprising measurements from all the sensors. Furthermore, in the above explanations it was assumed that the measurements are not corrupted. The proposed solution stems from the observation that performing agreement on the input allows for an optimisation that increases the probability of a successful agreement. This optimisation was referred to as the collection phase, and involves the controllers exchanging measurements and state so as to minimise the missing information in each controller. The proposed solution provides higher availability than existing solutions, and the availability improvement is up to 10x with two controllers. While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive, the invention being not limited to the disclosed embodiment. Other embodiments and variants are understood, and can be achieved by those skilled in the art when carrying out the claimed invention, based on a study of the drawings, the disclosure and the appended claims. It is to be noted that some of the steps present in the flow charts of Figures 2 to 4 are optional. Furthermore, the order of the steps may also be interchanged.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used.