Field of Invention

The present invention relates to a method of monitoring a computational system.

Background

Computer systems may be configured in a highly complex manner to comprise many dynamically-interacting components (hardware or software) so as to allow a Service Provider (SP) to deliver a service to a user. For example, a computer system may comprise servers, routers, switches, gateways, firewalls, processors, controllers, power systems, orchestrators and databases, each of which interoperate in order to facilitate a cloud computing service to a user.

The SP may agree with a user a standard of service that is to be provided by the computer system, which typically commits the SP to provide a service that performs within certain performance bounds, and may include, for example: a maximum period of downtime in a given time period; a minimum level of security; a minimum bandwidth speed; a maximum query response time for accessing a database; and a maximum latency. The standard of service assured by the SP is typically codified in a so-called Service Level Agreement (SLA).

The ability of the SP to meet the SLA is dependent on the performance of the components of the computer system. However, beyond a point, the complexity of a computer system may overwhelm the ability for a SP (in particular, experienced human operators) reliably to predict how the many components may interact. This can result in unpredictable and undesirable behaviours of the computer system, which may in turn lead to the computer system underperforming the SLA, which may manifest as outages of the computer system, and in turn may lead to financial penalties, as well as reputational damage for the SP.

It is an aim of the present invention to alleviate at least some of the aforementioned problems.

Statements of Invention

According to a first aspect of the present invention, there is provided a computer- implemented method of monitoring a computational system, said computational system comprising a plurality of interoperating computing components for performing computational operations thereby to provide a computational service to a user, wherein said computational system comprises a plurality of performance requirements based on at least one performance attribute, the method comprising the steps of: selecting a performance attribute associated with a performance requirement of the computational system; identifying a set of computing components from the plurality of computing components, wherein each of the computing components in the set perform operations affecting the selected performance attribute; determining a mapping of relationships for the computing components in the identified set in relation to the selected performance attribute; retrieving a performance status of a computing component in the identified set; and in dependence on ( e.g . as a function of) said determined mapping and said retrieved performance status, calculating a probability of the identified set complying with the performance requirement for the selected performance attribute.

Preferably, said performance requirement is predefined. Optionally, the performance requirement is a requirement for a maximum and/or minimum value for at least one performance attribute. Preferably, the performance requirement is assessed over a predefined period of time. Preferably, the set is populated only by computing components that perform operations affecting the selected performance attribute. Preferably, said performance requirement is provided by a Service Level Agreement associated with the computational system. Preferably, said set is pre-defined or empirically populated. Optionally, the performance attribute is: bandwidth; latency; record processing time; number of security breaches; and service availability. Optionally, the performance status is: bandwidth; latency; record processing time; error rate; and/or processing load. Optionally, the computational system is in the form of: a data processing service; a telecommunications service; a file transfer service; a network security service; and/or a control system. Optionally, the computing components are in the form at least one of a: server; router; switch; firewall; processor; controller; power system; orchestrator; and/or database. Optionally, the computing components of the computing system form a distributed computing system. Preferably, the performance status is retrieved from a given computing component by means of a monitoring unit that forms part of said computing component. Preferably, the performance status relates to: a given computing component; an input directly received by said given computing component from another computing component; and/or an output generated by said given computing component. Optionally, the computational system is a computer simulation of a computational system.

Preferably, the probability is calculated in dependence on: a first distribution for an expected number of times that the identified set does not comply with the performance requirement within a predetermined time period; and a second distribution for an expected time required for the identified set to recover to a state that complies with the performance requirement having failed to comply with the performance requirement. Preferably, said predetermined time period is prescribed by the performance requirements. Preferably, the probability is calculated in dependence on individual first and/or second distributions for each computing component of the identified set. Preferably, the probability is calculated in dependence on an expected total time in which the identified set does not comply with the performance requirement, and wherein said expected total time is an output from the second distribution based on an input of a sample value for an expected number of times that the identified set does not comply with the performance requirement from the first distribution.

Preferably, probability is calculated in dependence on a count of the number of times over a plurality of sample values from the first distribution when the expected total time exceeds an upper limit for a total time when the identified set does not comply with the performance requirement.

Preferably, the probability is calculated in dependence on the retrieved performance status by: comparing the retrieved performance status to a threshold performance status value; determining that the performance status is below the threshold performance status value and therefore subsequently designating the computing component as having no effect on the selected performance attribute; and wherein the probability is calculated as a conditional probability of the identified set complying with the performance requirement when said computing component is designated to have no effect on the selected performance attribute.

Preferably, the performance status is retrieved from each computing component of the identified set, and wherein said probability is calculated in dependence on the performance status from each computing component.

Preferably, determining the mapping of relationships within the set of computing components comprises determining a sequence in which operations are performed by the computing components, and more preferably, includes determining a direction of a relationship.

Preferably, the method further comprises the steps of: comparing the determined probability to a threshold probability value; outputting a determination that the computational system is likely to comply with the performance requirement when the determined probability exceeds the threshold probability value; and outputting a determination that the computational system is unlikely to comply with the performance requirement when the determined probability does not exceed the threshold probability value.

Preferably, the computational system is determined to be likely to comply with the performance requirement having designated the computing component to have no current effect on the selected performance attribute. Alternatively, the determination that the computational system is unlikely to comply with the performance requirement is output despite not designating any computing component within the identified set to have no current effect on the selected performance attribute.

Preferably, the method further comprises the step of reconfiguring at least one of the computing components within the identified set in response to outputting a determination that the computational system is unlikely to comply with the performance requirement.

Preferably, the method further comprises the step of reconfiguring at least one of the computing components within the identified set so as to decrease the probability of the identified set complying with the performance requirement. Preferably, said reconfiguring is performed only up to an extent in which the decrease in the probability exceeds a threshold probability for complying with the performance requirement. Alternatively, the step of reconfiguring at least one of the computing components within the identified set may be performed so as to increase the probability of the identified set complying with the performance requirement. Preferably, the method further comprises the step of determining a relational weight value for each determined mapping of relationships, wherein the probability is calculated in dependence on said each relational weight value. Preferably, the probability is calculated from a product of one less than each said relational weight value. Preferably, the relational weight value is calculated for a relationship between a first computing component and a second computing component, and wherein said relational weight value is derived from a probability that the second computing component is capable of ensuring compliance with the performance requirement in the event that the performance status of the first computing component is below a or the threshold performance status value. Preferably, the second computing component receives as a direct input an output from the first computing component. Preferably, the threshold performance status value level is defined by the performance requirement and/or is a state of inoperability of a computing component. Preferably, the method further comprises the steps of: deriving an importance value for a computing component within the identified set, wherein said importance value is one less than a product of the relational weight values associated with that computing component; and calculating the probability in dependence on the importance value. Preferably, an importance value is calculated for each of the computing components within the identified set. Preferably, the step of reconfiguring the at least one of the computing components is performed by selecting a computing component to reconfigure in dependence on its importance value. Preferably, said selecting is first performed for a computing component having the highest or lowest importance value.

Preferably, the method further comprises the step of repeating a method as described above for a further performance attribute and/or for a further performance requirement of the computational system.

According to another aspect of the invention, there is provided a computer-readable storage medium comprising instructions that, when executed by a processor associated with a computational system, causes the telecommunication network to perform a method as described above.

The invention includes any novel aspects described and/or illustrated herein. The invention also extends to methods and/or apparatus substantially as herein described and/or as illustrated with reference to the accompanying drawings. The invention is also provided as a computer program and/or a computer program product for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein, and a computer-readable medium storing thereon a program for carrying out any of the methods and/or for embodying any of the apparatus features described herein. Features described as being implemented in hardware may alternatively be implemented in software, and vice versa.

The invention also provides a method of transmitting a signal, and a computer product having an operating system that supports a computer program for performing any of the methods described herein and/or for embodying any of the apparatus features described herein.

Any apparatus feature may also be provided as a corresponding step of a method, and vice versa. As used herein, means plus function features may alternatively be expressed in terms of their corresponding structure, for example as a suitably-programmed processor.

Any feature in one aspect of the invention may be applied, in any appropriate combination, to other aspects of the invention. Any, some and/or all features in one aspect can be applied to any, some and/or all features in any other aspect, in any appropriate combination. Particular combinations of the various features described and defined in any aspects of the invention can be implemented and/or supplied and/or used independently.

As used throughout, the word 'or' can be interpreted in the exclusive and/or inclusive sense, unless otherwise specified.

The invention extends to a method as described herein and/or substantially as illustrated with reference to the accompanying drawings. The present invention is now described, purely by way of example, with reference to the accompanying diagrammatic drawings, in which:

Figure 1 shows an example of a computational system;

Figure 2 shows a detailed view of an aspect of the computational system;

Figure 3 is a process for monitoring the computational system; and

Figure 4 is a plot showing relationships between components of the computational system. Specific Description

Figure 1 shows an overview of a computational system 100 that is operated by a Service Provider (SP) 110 for the purpose of rendering a computational service to a user 120. In one example, the computational service is: a data processing service; a telecommunications service; a file transfer service; a network security service; and/or a control system.

The computational system 100 comprises a Computing Unit (CU) 130, which comprises at least one Computing Component (CC), for performing computational operations so as to provide the required computational service to the user 120.

The CU 130 is in communication with the SP, by means of a network connection 140-1 , so as to permit configuration of the CU by the SP. The CU 130 is also in communication with the user, by means of a further network connection 140-2, so as to deliver the computational service (i.e. the output from the CU 130) to the user 120.

Upon initiation of the computational system 100, the SP configures the CU 130 to perform and to output the computational service that is sought by the user 120; this is performed, for example, by loading a configuration file to the CU 130.

The CU 130 is associated with a Monitoring System (MS) 150 that is configured to retrieve and to process performance information regarding the performance of the CU with which it is associated. Performance information is available to include computational and networking metrics ( e.g . processing load, error rate, latency, bandwidth, etc.). The MS 150 is in communication with the SP 110 so as to report performance information and analysis of performance information to the SP. Figure 2 is a detailed view of the CU 130. As best shown in Figure 2, the CU 130 comprises a plurality of Computing Components (CCs) 210, each of which is configured to perform a computational operation so that the plurality of CCs deliver, in aggregate, the computational service to the user 120. This is achieved by way of a sequence of operations that are performed by the CCs.

In one example, the CCs are in the form of: a server; a router; a switch; a database; a processor; a computer program, software or virtualised computer hardware; a firewall; an orchestrator; a sensor; and/or a controller. For conciseness, Figure 2 shows the CU 130 as having only seven CCs. Flowever, the CU 130 is available to have any number of CCs, and it is typical for a CU to have hundreds, if not thousands, of CCs.

In turn, each CC 210 comprises a Monitoring Unit (MU) 220 for retrieving performance information from its associated CC. In turn, each MU reports the retrieved performance information (upon request or periodically) to the Monitoring System (MS) 150, which aggregates the performance information from each CC, and in turn performs analysis of this aggregated performance information.

Under a predefined set of rules, referred to as a Service Level Agreement (SLA), the SP 110 agrees to provide to the user 120 a computational service that meets predefined performance requirements relating to a plurality of performance attributes. Failure to provide the requested service to the user within the performance requirements of the SLA may constitute a breach of the SLA by the SP.

For example, a given performance attribute, A _n , is available to be: bandwidth; latency; record processing time; number of security breaches; and service availability. In turn, the corresponding performance requirements for such performance attributes are available to be, for example: a minimum bandwidth of 20Mb; a maximum latency of 2s; a mean record processing time of 1s; a maximum number of security breaches; and a service availability of at least 90%.

Figure 2 shows, for a given first performance attribute, A ₁ , the relationships between the CCs 210, shown by way of solid lines 215 in which arrows indicate the direction of the sequence in which the CCs operate (when performing operations that relate to A ₁ ).

The operations performed by the CCs are available to be consequential to a plurality of performance attributes (i.e. not just A ₁ ). For example, relationships for another performance attributes (i.e. not A ₁ ) are alternatively shown via broken lines 217.

The relationships between the CCs are, for a given performance attribute, available to be one-way or two-way (as indicated by a single- or a double-arrowed line in Figure 2, respectively). For simplicity, for performance attribute A ₁ , comprises only one-way relationships.

In more detail, for A ₁ , and as shown in Figure 2, an output of a first CC 210-1 is provided as an input for a second 210-2, a third 210-3, a fourth 210-4 and a fifth 210-5 CC. In turn, the output of the fourth CC 210-4 is also provided as an input for the fifth CC 210-5. The output of the fifth CC 210-5 is provided as an input for a sixth 210-6 and a seventh 210- 7 CC. The output of the seventh CC 210-7 is available to form part of the computational service delivered to the user 120 or to form an input of a computational operation in relation to another performance attribute. The MS is provided so as to assess a likelihood that the CU will comply with each performance requirement, and therefore with the SLA.

Figure 3 is a diagram showing a process 300 by which the MS determines a probability that the computing system will comply with the SLA.

In a first step 310, the MS selects a performance attribute that characterises a performance requirement of the SLA pertaining to the computer system 100, such as performance attribute A ₁ .

At a subsequent step 320, the MS 150 identifies the CCs that perform operations that are consequential to the selected performance attribute. At a next step 330, the MS identifies the dependencies between the identified CCs for the selected performance attribute. In this way, there is derived a mapping of relationships between the identified CCs such as that shown in, and described with reference to, Figure 2.

The information identified in steps 320 and 330 is available to be derived from information provided by the SP 110 and/or from historical information of performance information from each CC by each MU 220 from which the MS is configured to infer or deduce relationships amongst the CCs.

At a subsequent step 340, the MUs 220 report to the MS the current performance information of the CCs identified in step 330.

With the knowledge of the identified relationships between the CCs for the performance attribute selected in step 310 and the current performance information retrieved at step 320, the MS 150 is configured to perform processing so as to help determine a probability of an upcoming SLA breach. To do so, the MS performs probabilistic analysis, the underlying principles of which are described in more detail further below. At a next step 350, an assessment is made as to whether preceding steps 310 to 340 have been performed for all performance attributes that define each of the performance requirements of the SLA. If so, the process 300 proceeds to step 370. If not, then the process 300 proceeds to select a new performance attribute at step 360, and the process reiterates to step 320.

Once the probability of an SLA breach has been determined for all constituent performance attributes of the SLA, an assessment is made as to whether any of the determined probabilities exceed a pre-determined threshold. If so, the process 300 outputs a result that there is a significant risk of an SLA breach 380-1 , and if not, the process 300 outputs a result that there is an insignificant risk of an SLA breach 380-2; the consequences of these outputs are also discussed in more detail further below.

The analysis that is performed by the MS 150 so as to determine the probability of an SLA breach, as performed at step 340, is now described.

For simplicity, we first take a SLA that consists of a single performance requirement that is in turn based on only a single performance attribute, A ₁ , that is governed only by a single CC. As a result, system-wide 100 compliance with the SLA is determined solely by the performance of this single CC in respect of A ₁ . Accordingly, there is a probability, P, of the CU complying with the SLA; as represented by the following notation:

P(pass; SLA(A ₁ )).

P(pass; SLA(A ₁ )) is taken to be a function of:

1 . a number of times that the SLA is expected to fail in relation to A ₁ , for example as calculated by considering the Mean Time To Failure (MTTF) within a time period of interest, t _F , over which compliance with the SLA is assessed; and

2. a time required to recover to an SLA-compliant state following a breach of the SLA in relation to A ₁ , herein referred to as a Mean Time To Recover (MTTR).

Typically, constant adherence with a performance requirement is not required in order to meet the SLA. Instead, overall adherence with a performance requirement is assessed over the time period of interest, t _F . As a result, instantaneous (relative to t _F ) underperformance of a performance requirement is tolerable for SLA compliance; it is for this reason that the MTTR forms a component of the function P(pass; SLA(A ₁ )).

The time period of interest, t _F , is defined within the SLA and is available to be of the order of minutes, hours, days, months or years. The time period of interest is available to vary for each performance attribute of the SLA.

In one example, the MTTF is modelled to a Poissonian distribution, which is appropriate given that the MTTF is a discrete distribution (i.e. counting the number of occurrences of an event, i.e. SLA breaches). Q(k; λ _F , t _F ) is the probability of observing k number of SLA breaches (i.e. the single CC of this example failing to comply with a performance requirement associated with A ₁ ) within the time period of interest, t _F , and the average expected number of SLA breaches is provided by r _F = 1/λ _F , where λ _F is a rate parameter of SLA breaches. Accordingly,

Q(k;λ _F , t _F )~ Poisson(λ _F , t _F ), and so,

From the distribution Q(k; λ _F , t _F ), a sample integer number of SLA breaches is taken, and this sample is denoted K.

In this example, the MTTR is modelled according to an exponential distribution given that the duration of events occur around a mean duration. Accordingly, the expected duration taken to recover from an SLA breach, G(t _F ; 1, λ _B ), follows: G(t _F ; 1,λ _B )~Г(1,λ _B ), which is a gamma distribution, Г, with a shape parameter of 1 and a scale parameter, λ _B , of 1/MTTR. Accordingly, For the time period of interest, t _F , the expected total duration of SLA breaches, U, is given by a compound function of the MTTF and MTTR probability distributions, such that:

U = P(t _F ; G\Q).

A specific eventuality is taken in which there are K instances of SLA breaches; in this circumstance, the expected total duration of SLA breaches, U, may therefore be taken to be: in which j is an index number of an instance of an individual SLA breach. That is, the total duration of SLA breaches, U, is a summation of the time required to recover to an SLA-compliant state following a total of K instances of SLA breach. This equation derives a value for the expected total duration of an SLA breach only in the single specific situation of there having been exactly K number of failures. By iteratively calculating U for a sufficiently large number of values for K, m, a long-term average for the expected total duration of SLA breaches is available to be derived. In this example, the system-wide 100 probability of complying with the SLA, P(pass; SLA) (which, in this example, is equivalent to P(pass; SLA(A ₁ )), given that the SLA consists of a performance requirement based only on A ₁ ) is calculated as one less than a count of the number of occasions of SLA breaches within t _F , [U _l ; SLA breach], divided by the total number of sets of samples, m: where l is an index number for each value of K, and where m > 1.

With reference to Figure 2, all (seven) of the CCs 210 perform operations that are consequential to performance attribute A ₁ . As a result, all of the CCs are responsible for helping ensure compliance with the SLA in respect of performance attribute A ₁ ; this set of CCs is denoted as Z, such that:

Z= {z _1, z ₂ ... z _n }, where z ₁ is the first CC 210-1 , and so on, and where n = 7. A distribution for P(pass; SLA(A ₁ )) is available to be generated at step 340 of process 300, which is determined by means of the MS 150 for the CUs 130 comprised within set Z; this is determined using Equation 1 above across the set Z, and combining the calculated probabilities (i.e. determining a product of the probabilities for CCs in set Z, given that they are conditionally dependent).

In the example of Figure 2, P(pass; SLA(A ₁ )) is the product of the values of Equation 1 for each CC constituent of set Z.

In this way, there is calculated a probability of the SLA being met for a performance requirement that is dependent on performance attribute A ₁ based on performance information from each CC; this overall probability accounts for conditions where individual CCs themselves may fail to operate as required (as determined based on retrieve performance information, as per step 320 of process 300), but where the overall SLA is still met. That is, there is provided by the MS an analytical system that is capable of calculating the probability of SLA compliance upon condition of failure of any of the CCs.

As per step 360 of process 300, an overall probability of compliance with the SLA as a whole (i.e. for all performance requirements for all performance attributes) is calculated by performing the probabilistic analysis described above for each performance attribute of the SLA.

The examples described assume an equal weighting of relationships between the CCs 130. However, typically, each CC that depends on another, earlier, CC may not necessarily have the same level of reliance on that earlier CC; this level of reliance is referred to as a relational weight, W. Accordingly, a relational weight value is determined and assigned to each relationship (in each direction, as applicable) between CCs.

For a given performance attribute, there is provided an overall system relational weight, W _system,z , which is the probability that the CU fails to comply with the SLA (for that performance attribute) given that CC z fails to perform to a predetermined threshold ( e.g . as determined by a performance requirement of the SLA or as otherwise prescribed).

In addition, there is also provided an relational weight amongst dependent CCs, W _y,z , which is the probability that CC y fails to comply with the SLA (for that performance attribute) given that CC z fails to perform to the predetermined threshold.

With this, the probability of the CU complying with the SLA for a given performance attribute, in dependence on non-homogenous relational weights, and given that CC z fails to perform to the predetermined threshold, is denoted as: where P (z ƒ ail) is the probability that CC z fails to perform to the predetermined threshold, and where y = {1, ..., l} is the set of all CCs that depend from CC z (therefore y ≠ z) ·

Accordingly, by considering relational weights, there is provided a model that more accurately reflects the non-homogeneity of the relationships between the CCs for delivering the computational service. Therefore, this model may allow more accurately to determine the probability of an upcoming SLA breach based on the current performance of CCs. Initial values for relational weight for each CC are available to be prescribed and then iteratively adjusted by the MS.

Based on the relational weight, an importance value is derived to indicate the influence of any given CC as to its effect on a given performance attribute, and therefore the influence in ensuring compliance with the SLA for that performance attribute. As such, the importance value, /, represents how impactful a given CC is for passing the SLA for a given performance attribute. More formally, the importance value, for a given performance attribute, may be defined as the probability that the overall system fails to meet the SLA because of the performance of that CC. The importance value for CC z is:

/ _z = 1 — P(pass; (SLA) │ z ƒ ail).

The importance of the first CC 210-1 , I ₁ , represents both the importance of its own processing to help pass the SLA, but also how important the output of that processing is to ensure that subsequent CCs which depend from it also help pass the system SLA.

Figure 5 is a plot that visualises the importance and relational weight values for the CCs of Figure 2. Specifically, radial distance represents the relative importance of a CC to the first CC 210-1 , and line weights are proportional to values for relational weights between connected CCs.

With reference to Figure 4, a CC that directly depends on the first CC 210-1 sits at a radius denoted by DL ₁ , which represents a highest level of importance and a direct and immediate relationship with the first CC (or a “first order” relationship). As a result, the second 210-2, third 210-3, fourth 210-4 and fifth 210-5 CCs lie at DL ₁ . Subsequently, the sixth and seventh CCs 210-6 and 210-7 sit at a radius denoted by DL ₂ , which represents an indirect relationship via on intermediate CC (i.e. the fifth 210-5 CC), or a “second order” relationship, with the first CC 210-1 , and therefore the sixth and seventh CCs 210- 6 and 210-7 are said to be less important than the remaining CCs.

The MS is therefore configured to ascertain importance and relational weight. In this way, the MS is capable of augmenting the information it receives regarding existence of a relationship between CCs, with quantitative information as to how important these relationships actually are. In turn, the MS is capable of better calculating the probability of compliance with the SLA given current performance information. Furthermore, the CU may be configured with importance values in mind, such as to ensure redundancy and resilience of CCs that have the highest importance values.

In one example, the computational system is available to be reconfigured (as initiated by the Service Provider 110, CU 130 or MS 150) so as to modify the operation of the CCs in dependence on the output of process 300.

For example, upon a determination that there is a significant risk of an SLA breach (see step 380-1 of process 300), the CCs are available to be reconfigured so as to help reduce the risk of an SLA breach. An example of such a reconfiguration includes increasing processing resources that are available to a computing component. Selection of CCs for reconfiguration is performed in a prioritised manner based on the importance and relational weight of the computing components.

In another example, upon a determination that there is an insignificant risk of an SLA breach (see step 380-2 of process 300), the CCs are available to be reconfigured so as to increase the risk of an SLA breach, but where such risk is still within the bounds of insignificancy. This is available to be performed, for example, so as to improve the efficiency of the allocation of processing resources in the CU. In effect, the computational system is provided with a process to reconfigure itself to an extent based on the risk of SLA breaches (or SLA-based orchestration), as determined using probabilistic analysis that is cognisant of the functional relationships between CCs and their importance and/or relational weight, but without needing to have knowledge of the actual function/s that is/are performed by any given CC.

Alternatives and Modifications

Whilst a particular quantitative methodology has been described above for determining the probability of SLA breaches, it will be appreciated that alternative methods exist that may be used within the scope of the invention.

In one example the network connections 140-1 , 140-2 are in the form of: a wireless (wide or local area network); a fixed-line network; and/or a satellite network.

In one example, the Computing Unit 130 is comprised of a distributed arrangement of networked computing components, and in particular in the form of a cloud computing system and/or a telecommunications system.

Each feature disclosed herein, and (where appropriate) as part of the claims and drawings may be provided independently or in any appropriate combination.

Any reference numerals appearing in the claims are for illustration only and shall not limit the scope of the claims.