The present invention relates to operator performance evalua- tion when at least a part of a communication network is shared between operators, and in particular to defining a key performance indicator for the operator performance evalua ^¬ tion. An example of communication network sharing is related to radio access technologies in general, in which concept radio access networks are shared between network operators (also known as RAN sharing) . However, communication network sharing is not limited to RAN sharing, and any type of sharing may be adopted, for instance core network sharing, site sharing, etc .

In today's typical radio systems, network operators are not always allowed to apply communication network sharing. This is mainly due to political reasons, where one of the main priorities was to create a set of parallel networks for each radio system to stimulate competition on tariffs for use of mobile telephony. For these networks (at least in most coun ^¬ tries) it was not allowed to apply roaming between operators in the host country - this also avoids operators to save money during the network roll-out phase.

Currently, there are two general trends in terms of communi ^¬ cation network sharing. The first trend is that operators are considering ramping down use of legacy systems like GSM

(global system for mobile communications) , and do spectrum harvesting (or spectrum refarming) for new and more bandwidth efficient systems. To allow for support of legacy mobile phones, the starting point here is to introduce spectrum sharing between operators, i.e. basically creating a shared nation wide system, where all operators can carry legacy traffic. The second trend is that new radio access technolo ^¬ gies like 3GPP (third generation partnership project) long term evolution (LTE) will start being deployed after national licensing auctions. Due to the heavy cost of rolling out a nation wide network for this system for each operator, considerations have been made on removing or relaxing the re ^¬ quirement for each operator having a full-blown nation wide network . There are different degrees of communication network sharing, from sharing a site location, but still having different hardware, to fully sharing all equipment and radio spectrum.

Here it is focussed on the cases where the bottleneck of the system is shared by the operators, at least in some part of time, for instance hardware capacity, radio capacity, soft ^¬ ware capacity, or user- and control-plane resources.

There may be different agreements or frameworks on how commu- nication network sharing can be done. Some examples are:

- Multiple operators share spectrum and hardware with a cer ^¬ tain agreed split in throughput and/or resources. Instantane ^¬ ously the throughput split can be different, for instance when one operator has no traffic the other operator may use all resources/throughput.

- A virtual operator is using spectrum and hardware of a "normal" operator. In this case a minimum guaranteed through ^¬ put and a maximum throughput may be agreed. Basically the normal operator provides a "bitpipe" with a maximum and mini- mum capacity to the virtual operator.

- An agreement between operators which includes a minimum guaranteed throughput, a maximum throughput and a split in resources if traffic is between the minimum and maximum throughputs . The present invention aims at providing a mechanisms for ap ^¬ propriately evaluating whether each operator got what he should get according to the agreement, i.e. answering the question whether the agreement is fulfilled or has been vio- lated.

In particular, the invention aims at defining an appropriate key performance indicator (KPI) for operator performance which monitors utilization and performance of each operator. This KPI may be used for documenting delivered service e.g. for billing purposes, and e.g. as input to an RRM (radio re ^¬ source management) entity of a cellular communication network system or other traffic management algorithms. This is achieved by the method and apparatus as defined in the appended claims. The invention may also be implemented by a computer program product.

In the following the invention will be described by way of embodiments thereof taking into account the accompanying drawings, in which:

Fig. 1 shows a flow chart illustrating an evaluation method according to an embodiment of the invention.

Fig. 2 shows a diagram illustrating resource usage between operators over time according to an implementation example of the present invention. Fig. 3 shows a diagram illustrating delays over time according to the implementation example.

Fig. 4 shows a schematic block diagram illustrating a structure of a control unit which may be used for implementing an evaluation method according to an embodiment of the invention. Defining a metric which can be used to evaluate whether an agreement or framework between operators using communication network sharing has been kept or violated is not straightfor- ward. This problem gets increasingly difficult when it is al ^¬ lowed that one operator can use the share of the other opera ^¬ tor when that operator is not using it. It is to be noted that the communication network may comprise a cellular communication network system and the framework may be used on a cell basis but also for larger areas of multiple cells.

For example, two operators have agreed on a 50%-50% share of a system which may comprise at least a part of a communica ^¬ tion network, and during the night one of the operators has no traffic. This may result in the other operator using all resources of the system during the night, provided there is enough traffic for that operator. Now just looking at the throughput or resource usage it seems that the agreement has not been met.

In the above example it is likely unacceptable to block dur ^¬ ing daytime the following day the operator that happened to utilize most resources during the night-time. So there is a temporal issue involved in the measurement of performance.

The KPI should provide long term cumulated measurements of for instance throughput or resource utilization, as well as short term measurements possibly averaged or filtered over short time intervals. A basic metric being cumulated and av- eraged is likely but not necessarily the same in both cases.

Long term measurements may be useful for bookkeeping, bill ^¬ ing, and documentation. Short term measurements may be useful for comparing present performance among operators e.g. to be used as input to a scheduler controlling operator sharing. The present invention proposes a metric for evaluating opera ^¬ tor performance, which can be used by an operator participat ^¬ ing in communication network sharing to evaluate whether a communication network sharing agreement/framework or service level agreement was kept or violated.

According to an embodiment of the invention:

- For every time period j it is evaluated whether the commu ^¬ nication network sharing framework has been violated or not. This is stored in a metric m(j) .

- A final metric is the sum of metrics m(j) over a time of evaluation: M=sum (m ( j ) ) .

The communication network sharing framework during time pe- riod j is violated for operator k when:

- Operator k is getting less than an agreed minimum number of resources or less than an agreed minimum throughput or is getting less than an agreed share of resources or less than an agreed share of throughput, A D

- Operator k is having enough traffic to be served to fill the agreed minimum number of resources or the agreed minimum throughput, or the agreed share of resources or the agreed share of throughput. Fig. 1 shows a flowchart illustrating a method for evaluating operator performance according to an embodiment of the inven ^¬ tion, which can be used by an operator participating in communication network sharing to evaluate whether a communication network sharing framework was kept or violated.

In step S10, when the evaluation method is started, an index j of a time period is set to 1.

In step Sll, for time period j, it is detected whether an op- erator k of at least a part of a communication network gets less than an agreed minimum number of resources of the part of the communication network or less than an agreed minimum throughput in the part of the communication network, or gets less than an agreed share of resources of the part of the communication network or less than an agreed share of

throughput in the part of the communication network. The agreed minimum number of resources or the agreed minimum throughput, or the agreed share of resources or the agreed share of throughput is defined in the sharing framework.

There may be at least two operators sharing the part of the communication network. Hence, k may be an integer equal to or greater than 2.

In step S12, in case it is detected that the operator k does not get less than the agreed minimum number of resources or less than the agreed minimum throughput, or less than the agreed share of resources or less than the agreed share of throughput, the process advances to step S13 where it is de ^¬ termined that the sharing framework is not violated and this result is stored in a metric m(j) as m(j)=0. Then the process advances to step S14 where the metric m(j) may be summed to a final metric M=sum (m ( j ) ) .

In step S15 the index j is incremented by 1, and in step S16 it is checked whether the time period j exceeds a time of evaluation J. If not, the process returns to step Sll. If the time period j exceeds the time of evaluation J in step S16, the process ends.

In step S12, in case it is detected that the operator k gets less than the agreed minimum number of resources or less than the agreed minimum throughput, or less than the agreed share of resources or less than the agreed share of throughput, the process advances to step S17 in which it is detected whether the operator k has enough traffic to be served to fill the agreed minimum number of resources or the agreed minimum throughput, or the agreed share of resources or the agreed share of throughput.

In step S18, in case it is detected that the operator k has enough traffic to be served, the process advances to step S19 where it is determined that the sharing framework is violated and this result is stored in the metric m(j) as m(j)=l. Then the process advances to step S14 where the metric m(j) may be summed to the final metric M=sum (m ( j ) ) .

In step S15 the index j is incremented by 1, and in step S16 it is checked whether the time period j exceeds the time of evaluation J. If not, the process returns to step Sll. If the time period j exceeds the time of evaluation J in step S16, the process ends.

In step S18, in case it is detected that the operator k has not enough traffic to fill its share of the resources, the process advances to step S20 where it is determined that the sharing framework is not violated and this result is stored in the metric m(j) as m(j)=0. Then the process advances to step S14 where the metric m(j) may be summed to the final metric M=sum (m ( j ) ) . In step S15 the index is incremented by 1, and m step S16 it is checked whether the time period j exceeds the time of evaluation J. If not, the process returns to step Sll. If the time period j exceeds the time of evaluation J in step S16, the process ends.

Measurements on which the above evaluation method is based should be averaged to avoid acting on spurious fluctuations. The term "averaging period" is used generically to describe how far back in time samples have a significant influence, e.g. a filter length when applying a finite (FIR) filter, or a number of time constants when applying an infinite (IIR) filter .

For example, packet scheduling in a communication network is based on measurements with an averaging period in the order of fractions of a second to achieve fulfilment of QoS (qual ^¬ ity of service) requirements for every user of the communica ^¬ tion network, and proper sharing among users includes sharing among operators on relatively short term basis. The averaging period for the metric m(j) (sharing performance metric), i.e. the measurements on which the metric m(j) is based (underly ^¬ ing resource/throughput measurements), should be greater than that of packet scheduler metrics in order to allow for short term deviations from requirements that are controlled by a packet scheduler performing the packet scheduling.

For identifying longer term deviance from targets, and in particular when used for control purposes, such as adaptation of parameters of the evaluation method, measurements for the sharing performance metric should be further averaged to achieve an averaging period in the order of for example 10 times the averaging period of the packet scheduler metrics, which is in the order of 100 ms (or more) . For example, two operators, which are called operator 1 and operator 2, have agreed on sharing spectrum and equipment (resources) of a communication network on a 50%-50% basis, meaning each has right to 50% of the resources when they have enough traffic to be served to fill the resources.

An actual resource usage over time is shown in Fig. 2. The graph depicted as solid line illustrates the actual resource usage of operator 1, and the graph depicted as dashed line illustrates the actual resource usage of operator 2. At a time tl there is too much traffic to serve at once in the part of the communication network shared by the operators 1 and 2. Thus, the operators are limited to their share. At a time t2, operator 2 has less traffic than its share can carry. Thus, operator 1 can use more resources.

Fig. 3 shows an average delay trend (delays) over time ac ^¬ cording to the above example. The graph depicted as solid line illustrates the delay for operator 1, and the graph de ^¬ picted as dashed line illustrates the delay for operator 2.

At the time tl, as the operators are limited to their share, delays start to build. At the time t2, as operator 2 has less traffic and operator 1 can use more resources, also the delay for operator 1 starts decreasing.

As long as the delay shown in Fig. 3 is below a certain minimum, no operator can complain, since traffic is carried fast enough. When the delays start exceeding the minimum threshold at time tl, it is interesting to monitor whether the sharing framework is violated. This can be done by monitoring from tl on the metric m(j), where: m(j)=0 when an operator gets its share of resources, i.e.

50%, AND the operator has traffic to fill the re ^¬ sources; and

m(j)=l otherwise. This results in values of 0 for all time periods j in the above example, since the communication network sharing agree ^¬ ment/framework is kept.

To summarize whether the sharing framework is kept all the time, the values of m(j) can be summed over the evaluation time J (a day, month, year, etc.) or continuously monitored. Dependent on the agreement, the condition for m(j) = 0 may vary. For example, the proportion (50% as mentioned above) may vary, and the way in which it is determined what share an operator has achieved may vary.

In the following, example measurements are described that may be used to quantify achieved shares between operators. It is assumed that c(oper,t) is a measurement of an achieved cumulated performance for users that are subscribers of op ^¬ erator "oper" in a transmission time interval (TTI) at time t. The achieved cumulated performance may be represented in terms of allocated resources or in terms of an amount of transferred data (bits) , dependent on the way a service level agreement is specified.

An averaged measure of achieved performance may be obtained by a first order recursive filter:

C(oper,t) = (1 - 1/N) C(oper,t-l) + (1/N) c(oper,t).

Obtained "instantaneous" shares may be defined as:

Share (operl) = C(operl, t) / (sum over all operators of

C (oper, t ) ) .

The parameter N determines the averaging period, as discussed above. The above formula assumes without loss of generality that the time, t, is an integer counter of TTIs. In other words, referring to Fig. 1, for determining whether the operator k gets less than the agreed minimum number of resources or less than the agreed minimum throughput, or less than the agreed share of resources or less than the agreed share of throughput, an averaged measure of a use of re- sources or throughput may be calculated for the operator k for the time period j by measuring the use at several con- secutive times including a time t corresponding to the time period j, thereby obtaining several consecutive measurements, and a weighted average of the several consecutive measure ^¬ ments may be formed, thereby obtaining the averaged measure, wherein the number of the several consecutive measurements included in the averaged measure corresponds to an averaging period .

Moreover, for determining whether the operator k gets less than the agreed share of resources or less than the agreed share of throughput, the averaged measure for each operator k of the part of the communication network may be obtained, thereby obtaining averaged measures, and the averaged meas ^¬ ures may be summed, and a share of the operator k may be cal- culated by dividing the averaged measure of the operator k by the summed averaged measures.

With this measure of shares the above rule for the metric m(j) = 1 reads as follows: m ( j ) =1 when

- Operator gets less than 50-margin % of resources AND

- Operator has enough traffic in buffers to be served to fill at least 50% of the resources.

Otherwise m(j)=0.

The variable "margin" can be used to take some variance into account .

There are several likely scenarios of sharing among opera ^¬ tors. In the following example scenarios and roles are de ^¬ scribed in relation to the above performance measurement, without limiting the performance measurement to these cases. According to a first example, an operator owns the communica ^¬ tion network and acts as a service provider towards one or more virtual operators that are purchasing access to re ^¬ sources and services provided by the service provider. A ser- vice level agreement may specify availability of a certain amount of recourses (e.g. throughput) to the virtual opera ^¬ tor, thus providing a "bitpipe" of certain volume.

The service level agreement is fulfilled towards the virtual operator when the certain amount of resources is available at times when the virtual operator has data amount ready for transmission that is sufficient to "fill the bitpipe". In case there is not sufficient data available for transmission, the service level agreement is fulfilled by default.

According to a second example 2, two or more operators are sharing a communication network on a fair sharing basis, such that there is an agreed division of resources in possibly un ^¬ even proportions among the operators. The resources must be provided to the operators in accordance with the specified proportions, or, in terms of throughput, whatever capacity can be achieved must be divided to the operators in accor ^¬ dance with the specified proportions.

The service level agreement is fulfilled when resource utili ^¬ zations or achieved operator capacities divide according to the specified proportions.

Fig. 4 shows a schematic block diagram illustrating a structure of a control 10 unit which may be used for implementing an evaluation method according to an embodiment of the invention.

The control unit 10 includes processing resources 11, memory resources 12 that may store a program, and interfaces 13 which may include (hardwire) links and/or a suitable radio frequency transceiver coupled to one or more antennas for bidirectional wireless communications over one or more wire ^¬ less links. The processing resources 11, memory resources 12 and interfaces 13 may be coupled by a bus 14. The terms "connected, " "coupled, " or any variant thereof, mean any connection or coupling, either direct or indirect, between two or more elements, and may encompass the presence of one or more intermediate elements between two elements that are "connected" or "coupled" together. The coupling or connection between the elements can be physical, logical, or a combination thereof. As employed herein two elements may be considered to be "connected" or "coupled" together by the use of one or more wires, cables and printed electrical connec ^¬ tions, as well as by the use of electromagnetic energy, such as electromagnetic energy having wavelengths in the radio frequency region, the microwave region and the optical (both visible and invisible) region, as non-limiting examples.

Programs stored by the memory resources 12 are assumed to in- elude program instructions that, when executed by the proc ^¬ essing resources 11, enable the control unit 10 to operate in accordance with the exemplary embodiments and aspects of this invention. Inherent in the processing resources 11 is a clock to enable synchronism among the various apparatus for trans- missions and receptions within the appropriate time intervals and slots required, as the scheduling grants and the granted resources/subframes are time dependent. The transceivers of the interfaces 13 include both transmitter and receiver, and inherent in each is a modulator/demodulator commonly known as a modem. The interfaces 13 may also include a modem to fa ^¬ cilitate communication over (hardwire) links.

In general, the exemplary embodiments of this invention may be implemented by computer software stored in the memory re- sources 12 and executable by the processing resources 11 of the control unit 10, or by hardware, or by a combination of software and/or firmware and hardware in the control unit 10.

The memory resources 12 may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable mem ^¬ ory. The processing resources 11 may be of any type suitable to the local technical environment, and may include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and proces ^¬ sors based on a multi-core processor architecture, as

non-limiting examples.

The control unit 10 may function as part of network elements of operator k. For example, the control unit 10 with its processing resources 11, memory resources 12 and interfaces 13 may be used to implement the evaluation method shown in Fig. 1. In other words, the processing resources 11 may exe ^¬ cute the evaluation method, using the memory resources 12 e.g. for reading out processing instructions corresponding to the method steps, caching and storing processing results, and the interfaces 13 e.g. for receiving/transmitting performance measurements and/or processing results from/to the outside of the control unit 10, e.g. control units of other operators of the part of the communication network. Alternatively or in addition, the method steps may be implemented by hardware in the processing resources 11, as mentioned above.

According to an aspect of the invention, an apparatus comprises first detecting means for, for every time period j of a time of evaluation J, wherein j is an integer equal to or greater than 1, detecting whether an operator k of at least a part of a communication network gets less than an agreed minimum number of resources of the part of the communication network or less than an agreed minimum throughput in the part of the communication network, or gets less than an agreed share of resources of the part of the communication network or less than an agreed share of throughput in the part of the communication network, wherein the agreed minimum number of resources or the agreed minimum throughput, or the agreed share of resources or the agreed share of throughput is de ^¬ fined in a sharing framework, wherein k is an integer equal to or greater than 2. In case it is detected that the opera- tor k does not get less than the agreed minimum number of re ^¬ sources or less than the agreed minimum throughput, or less than the agreed share of resources or less than the agreed share of throughput, first determining means of the apparatus determine that the sharing framework is not violated and store this result in a metric m(j) . In case it is detected that the operator k gets less than the agreed minimum number of resources or less than the agreed minimum throughput, or less than the agreed share of resources or less than the agreed share of throughput, second detecting means of the ap- paratus detect whether the operator k has enough traffic to be served to fill the agreed minimum number of resources or the agreed minimum throughput, or the agreed share of re ^¬ sources or the agreed share of throughput. In case it is de ^¬ tected that the operator k has enough traffic to be served, second determining means of the apparatus determine that the sharing framework is violated and store this result in the metric m(j) . In case it is detected that the operator k has not enough traffic to fill its share of the resources, third determining means of the apparatus determine that the sharing framework is not violated and store this result in the metric m ( j ) .

The apparatus may be a part of network elements of operator k. For example, the apparatus may comprise processing re- sources, memory resources and interfaces of the control unit 10 shown in Fig. 4. Summing means of the apparatus may sum each metric m(j) over the time of evaluation J to obtain a final metric

M=sum (m ( j ) ) .

For determining whether the operator k gets less than the agreed minimum number of resources or less than the agreed minimum throughput, or less than the agreed share of re ^¬ sources or less than the agreed share of throughput, first calculating means of the apparatus may calculate an averaged measure of a use of resources or throughput for the operator k for the time period j by measuring the use at several con ^¬ secutive times including a time t corresponding to the time period j, thereby obtaining several consecutive measurements, and forming a weighted average of the several consecutive measurements, thereby obtaining the averaged measure, wherein the number of the several consecutive measurements included in the averaged measure corresponds to an averaging period.

For determining whether the operator k gets less than the agreed share of resources or less than the agreed share of throughput, obtaining means of the apparatus may obtain the averaged measure for each operator k of the part of the com ^¬ munication network, thereby obtaining averaged measures, and sum the averaged measures. Calculating means of the apparatus may calculate a share of the operator k by dividing the aver ^¬ aged measure of the operator k by the summed averaged meas ^¬ ures .

The averaging period may be set greater than that of packet scheduler metrics and/or 10 times greater than that of the packet scheduler metrics.

The several consecutive times may correspond to an integer counter of time transmission intervals. According to an embodiment of the invention, in an evaluation method it is detected whether an operator k of at least a part of a communication network gets less than an agreed minimum number of resources/throughput or less than an agreed share of resources/throughput according to a sharing frame ^¬ work. In case it is detected that the operator k does not get less than the agreed minimum number/share of re ^¬ sources/throughput, it is determined that the sharing frame ^¬ work is not violated and this result is stored in a metric m(j) . In case it is detected that the operator k gets less than the agreed minimum number/share of resources/throughput, it is detected whether the operator k has enough traffic to be served to fill the agreed minimum number/share of re ^¬ sources/throughput. In case it is detected that the operator k has enough traffic to be served, it is determined that the sharing framework is violated and this result is stored in the metric m(j) . In case it is detected that the operator k has not enough traffic to fill its share of the resources, it is determined that the sharing framework is not violated and this result is stored in the metric m(j) .

It is to be understood that the above description is illus ^¬ trative of the invention and is not to be construed as limit ^¬ ing the invention. Various modifications and applications may occur to those skilled in the art without departing from the true spirit and scope of the invention as defined by the ap ^¬ pended claims.