VARIABLE PRIORITY OF NETWORK CONNECTIONS FOR PREEMPTIVE PROTECTION

Description

Background of the Invention

[01] The present invention is related to techniques of protecting traffic on electronic and optical networks.

[02] When a link of an electrical or optical network is broken, the traffic capacity between the two nodes of the broken link is necessarily reduced until the link is restored. Various protection techniques are used to ensure that the network connections using the link are nonetheless maintained. This is done by rerouting the signals around the broken network link, which can be performed by various protection techniques. The protection techniques rank the network connections using the link by a priority.

[03] To the user of a network connection, the level of protection afforded to a network connection is typically defined by a Service Level Agreement (SLA) with a network Service Provider (SP), essentially the network administrator. The SLA guarantees that a network connection will have no more than a stated amount of downtime, time during which network connection is lost. The SP uses the protection priority to guarantee the desired level of protection to the user of a connection. However, the resulting service protection techniques only support one kind of guaranteed protection level: full protection with almost no downtime (termed 99.999% availability). This is true for connection-oriented technologies, such as SONET/SDH (Synchronous Optical NETwork/Synchronous Digital Hierarchy), and as well as packet flow technologies, such as MPLS (Multiprotocol Label Switching), RPR (Resilient Packet Ring), etc. Hence it should be understood that network connections as used herein includes not only circuit connections but also packet flows.

[04] The various protection techniques theoretically allow different levels of priority so that network users with higher priority preempt users with lower priority to ensure their network connections are maintained. However, the priority levels are fixed and network connections with higher priorities simply maintain their preemption of network connections with lower priorities to ensure a "pecking order" of connectivity . The downtime of a network connection depends only on the other connections that compete with it and not on the allowed downtime of its SLA.

These techniques do not necessarily ensure that the connection's downtime does not exceed its allowed downtime if the priority of the connection is not at the highest level. [05] Thus current protection techniques do not guarantee a network connection a level of protection below full protection. There is no way to reduce the amount of protection bandwidth and at the same time provide meaningful protection assurance for all of the connections impacted by a network failure based on each connection's SLA.

Summary of the Invention

[06] To address this problem, the present invention provides for a method of operating a communications network which has a plurality of links and carries a plurality of network connections. The method includes the steps of: determining a priority of at least some network connections responsive to each link failure in the communications network; detecting a failure in a link of the communications network; and protecting a plurality of network connections from the failure by a priority of each of the plurality of network connections, a network connection with a higher priority preempting a connection with a lower priority, so that the actual downtime from the failure is shared among the plurality of network connections. Where the at least some of network connections have amounts of permitted downtime for a predetermined period, the priorities of the at least some of network connections are determined further responsive to the amounts of permitted downtime so that a number of the plurality of network connections in excess of permitted downtime is minimized. The determining step can be performed by the nodes of the network, the network management station, or the network billing system. [07] The present invention also provides for a communications network which has a plurality of links for connecting a plurality of nodes, each link carrying a plurality of network connections; and at least one unit for determining priorities of at least some of the network connections responsive to a link failure of the network connections. The plurality of nodes detects failures of the plurality of network connections and the plurality of nodes protects the plurality of network connections from the failures by the priority of each network connection so that a connection with a higher priority preempts a connection with a lower priority. Where the at least some of network connections have amounts of permitted downtime for a predetermined period, the priorities of the at least some of network connections are determined further responsive to the amounts of permitted downtime so that a number of the plurality of network

connections in excess of permitted downtime is minimized. The at least one priority determining unit determines the priorities of network connections carried by a failed link after failure of the link and further determines the priorities of network connections which already occupy alternate routes for network connections carried by the failed link. The at least one priority determining unit may include a node connected to the failed link, a network management station, or a network billing system.

[08] Furthermore, in a communications network having a plurality of nodes connected by a plurality of links, each link carrying a plurality of network connections, the present invention provides for at least one unit comprising at least one integrated circuit for determining a priority of at least some network connections responsive to a link failure of the network connections. The actual downtime is shared among the plurality of network connections. Where the at least some network connections have amounts of permitted downtime for a predetermined period, the at least one integrated circuit further determines a priority of each of the at least some network connections further responsive to the amount of permitted downtime so that a number of network connections in excess of permitted downtime is minimized. The at least one unit can comprise the nodes of the communications network, a network management station or a network billing system, each comprising at least one integrated circuit.

Brief Description of the Drawings

[09] Fig. IA illustrates a simplified communications network with only two nodes; Fig. IB is a table of connections of the Fig. IA network with SLA requirements.

[10] Fig. 2 is a flow chart of mesh network operations according to one embodiment of the present invention.

[11] Fig. 3 is a flow chart of ring network operations according to another embodiment of the present invention;

[12] Fig. 4 illustrates a communications network with a network management station and a network billing system, and

[13] Fig. 5 represents one organization of a network node, a network management station or a network billing system, according to an embodiment of the present invention.

Detailed Description of the Invention

[14] An example illustrates the problem of the present use of priority in re-establishing network connections in communications networks. In an exemplary and simplified two-node network 10 shown in Fig. IA, there are two links 11 and 12 between nodes 13 and 14. The network has network connections with traffic classes A, B, C and D which define the allowed downtime, the permitted amount of outage, set forth in the network connection SLAs. The downtime of each class is shown in the table in Fig. IB

[15] Assuming that the actual downtime from failures on both links 12 and 13 amount to 3 hours a year and that no protection bandwidth exists in the network 10, traffic protection for network connections of one class must preempt the traffic of the other network connections of other classes. Under existing preemption designs, the traffic of connection classes A and B have higher priority than that of connection classes C and D and preempt that traffic. As a result, the class A and B connections have zero downtime, while class C and D connections are down for 3 hours. This is true no matter how the traffic is routed. As a result, the SLAs for class C connections are violated.

[16] Rather than being fixed, the priorities of different network connections are determined in response to link failures and resulting downtimes of the network connections in accordance with the present invention. The varying priorities can ensure that the SLA of each connection is not violated or at least is violated in a minimal fashion. With respect to the exemplary network 10 of Figs. IA and IB, one embodiment of the present invention preempts the traffic of the class C connections for 2 hours. Once downtime approaches the allowed downtime (defined by the SLA of each connection), the priority of the class C connections rises above that of the class B connections. Therefore, the class B traffic is preempted for the remaining hour, and the SLAs of all the connections are satisfied.

[17] It should be noted that the present invention is independent of the various protection protocols used in communications networks. Thus the present invention is not particularly concerned how a particular network preempts circuit connections or packet flows, such as RSVP (Resource Reservation Protocol) requests or SONET APS (Automatic Protection Switching), or IP packet preemption. Rather, the present invention is concerned with how priorities are determined for the circuit or packet preemptions.

[18] The present invention determines the priority of a network connection so that the priority can be responsive to the connection's actual downtime, allowed downtime, and even its expected downtime. The actual downtime is the actual amount of outage which has occurred during the predetermined period of time up to a time T. The allowed downtime is the permitted amount of outage of the connection over a predetermined period of time, typically a year. This is set forth in the SLA (Service Level Agreement) between the connection user and the SP (Service Provider) of the connection. The expected downtime is the amount of outage expected from the time T to the end of the predetermined period of time. Priority can be determined in many ways and the following exemplary descriptions illustrate how priority can be varied. The varying priorities of the connections are then used for re-establishing the connections upon a network failure.

[19] Mesh Network Operation

[20] According to one embodiment of the present invention, a method of network operation is illustrated by the flow chart of Fig. 2. During the operation of the network as indicated by the dotted arrow from the top of the drawing, step 30 detects the failure of a link in the network. Such detection is dependent upon the particular communications network protocol and is well established and known to network designers. For example, the long-established SONET/SDH protocol provides link failure detection in many optical networks.

[21] Given the failure event, step 31 determines the set of network connections F which are directly impacted by the failure at time T. Time T is defined from the start of the SLA period, say, for example, the beginning of each calendar year, for the connections. For each connection f in set F, step 32 considers various alternate routes unaffected by the failure to recover that connection. At time T these alternate routes typically carry other network connections and might have available or free bandwidth, i.e., unused capacity, to carry more connections. To recover or protect the network connection f, by using an alternate route, the available bandwidth may be used, or other connections may be preempted or "bumped off the alternate routes. [22] In accordance with one embodiment of the present invention, step 34 calculates the priorities of the network connections i at time T. Priority can be defined in many different ways and the following is a simple example:

Priority(i, T) = 1 if ActualDown(i, T) > SLA(i); or Priority(i, T) = 0 otherwise;

where SLA(i) is the maximum allowed outage, or downtime, according to the SLA, and ActualDown(i,T) is the actual amount of downtime that the connection i has experienced from the beginning of the SLA period. The connections i in the set I include not only the connections fin the failed set F, but also the connections which already occupy the alternate routes. Such prioritization preempts users who have not yet exhausted their allowable outage time, SLA(i), without regard to how close/far they are from violating their SLA. This priority determination is very simple, but it is still much better than the current situation of fixed priorities. [23] Of course, other variable prioritizations are possible. For example, a function Slack(i, T) can be defined which measures the amount of tolerance a connection i has for an outage at time T under the network connection's SLA. Slack is used to determine priority responsive to the allowed downtime, the actual downtime and even the expected downtime of the connection. For example:

Slack(i,T) = SLA(i) - ActualDown(i,T) - (l-T>ExpectedDown(i) where ExpectedDown(i) is the expected downtime for the entire SLA period for the connection I and the likely downtime through the remainder of the year is (l-T)»ExpectedDown(i,T). In this example, priority is also responsive to the expected downtime, in addition to the allowed downtime and the actual downtime. Priority might then be set as:

Priority(i, T) = MaxPriority - Slack(i,T) where MaxPriority is the highest priority which a connection can receive. [24] Returning to the Fig. 2 flow chart, step 34 thus determines the priorities of all the connections F. The preempting network connections and the set of victims V, the set of preempted network connections, are selected by step 38. First, priority is used for the network connections f to first consume the available free bandwidth and then to preempt the previously occupied alternate routes. The preempted network connections fall into set V. The process flow then awaits the next link failure. Even if the network chooses to preempt a connection and the downtime is extensive enough to risk a violation of the connection's SLA, the preempted connection may be reconsidered at the next failure event based on its new higher priority and a different connection may be preempted. [25] Ring Network Operation

[26] The present invention is adaptable to ring networks. In a ring network the operation according to the present invention is simpler since alternate routing is trivial compared to a mesh network. Nonetheless, determination of an optimal V, the set of connections which are to be preempted, is still difficult. The exemplary flow chart of Fig. 3 shows the operations according to one embodiment of the present invention.

[27] Step 40 detects the failure of a link in the ring network during the normal operation of the network as indicated by the dotted arrow from the top of the drawing. As stated previously, failure detection is dependent upon the particular communications network protocol and is well established and known to network designers. Step 41 calculates the slack(i,T) defined above for all connections, including the network connections which failed and the alternate route connections, typically the network connections in the opposite direction around the ring network from the failed connections. In step 42 a failed network connection f with lowest slack(f,T) is selected, and by step 44 the connection f is alternately routed around the other side of the ring network. Any network connection c with the highest slack(c,T) is selected for preemption. [28] Decision step 46 tests whether if slack(f,T) > slack(c,T). If the statement is true, the process is stopped. If the statement is false, then the alternate routing of the failed connection f is completed in step 48 by preempting all, or parts of, the network connection c. If parts of connection c, i.e., links for connection c, are not used by connection f, they are marked as free bandwidth (to be used by subsequent connections).

[29] Step 49 tests whether the alternate routing of the network connection f reaches its intended destination, i.e., whether the alternate routing for connection f is completed. If the connection is completed, the process returns to step 42 wherein another failed connection f with the lowest slack(f,T) is selected. If the connection is not completed, the process moves to step 44 to continue the alternate routing of connection f with the preemption of additional network connections c.

[30] Even though the flow chart above describes a single failure event on a ring network, the present invention returns the process flow for future failure events as in the case for mesh networks above.

[31] Network Operations in General

[32] The descriptions above with respect to the Figs. 2 and 3 networks refer to the operations of the entire network. The priority determining operations might be specifically performed by

the nodes of the network, a network management station, or a network billing system. If the priority determinations are performed by the network nodes, the node which detects the link failure in a mesh network performs the priority determinations for its network connections or priority may also be determined by the node originating the failed network connection in the failed link. In the latter case, the node detecting the failure must, of course, relay this information to all nodes affected by the failure.

[33] Rather than the network nodes, a network management station may determine the network connection priorities. Any tracking information of downtime per network connection through the SLA period, which allows an SP to prove an SLA has been met, is most likely to reside in the network management station, rather than at the network node level where the connection preemptions are performed. Therefore it may be easier to have the network management station determine the network connection priorities.

[34] Fig. 4 illustrates an exemplary mesh network with nodes 50 and a network management station 51 which supervises the operation of the network by communicating directly to each of the nodes 50, as shown by the dotted lines. For nodes which are not connected directly to the station 51 , network management is performed through the network links by an optical service channel. The network management station 51 tracks the network failures and measures the actual downtime for each connection. In operation, the network management station provisions the network nodes with the priorities of each of the network connections and then periodically updates the network nodes 50 with new priority values in response to the failures which have occurred in the intervening period. As stated previously, the network management station 51 determines the priority of each network connection responsive to the connection's actual downtime and allowed downtime from the connection's SLA and actual downtime. Even the connection's expected downtime might also be used. The new priority values prepare the nodes for the next failure. Alternatively, the network management station 51 can re -provision the nodes after each failure.

[35] A network billing system 52 might also be used to determine the priorities of the network connections. Since a service provider charges the users of the network, the network operates with a network billing system installed within, or as shown in Fig. 3 connected to, the network management station 51. The network billing system 52 keeps track of the each connection, including its allowed downtime in accordance with its SLA and its actual downtime, to be able to

justify the bills to the users. Information for determining connection priority is thus readily available to make priority determinations and communication to the nodes 50 is through the network management station 51.

[36] The network nodes, network management station or network billing system are adapted to determine the connection priorities. One or more integrated circuits, such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), can be used in the nodes, station or system to carry out these determination operations. This implementation in hardware allows for high speed operations.

[37] On the other hand, the present invention might be implemented in more conventional arrangements of integrated circuits. Firmware, such as the ROM (Read-Only Memory) of a microcontroller, or software can be used with these arrangements and offer certain advantages. For instance, a processor unit instructed by the software might also perform operations. Fig. 5 shows a block diagram of a representative computer system 60 that may be used to execute the software of an embodiment of the invention. The computer system 60 includes memory 62 which can store and retrieve software programs incorporating computer code that implements aspects of the invention, data for use with the invention , and the like. Exemplary computer readable storage media include CD-ROM, floppy disk, tape, flash memory, semiconductor system memory, and hard drive. The computer system 60 further includes subsystems such as a central processor 61, fixed storage 64 (e.g., hard drive), removable storage 66 (e.g., CD-ROM drive), and one or more network interfaces 67, all connected by a system bus 68. Additional or fewer subsystems may be used. For example, computer system 60 may include more than one processor 61 (i.e., a multi-processor system) or a cache memory. [38] Further Embodiments of the Present Invention

[39] There are many ways of determining priority in accordance with the present invention. If the network connections are not operating under SLAs, only the actual downtime of a network connection is used to determine the priority of the connection at a link failure. This allows the downtime caused by the failure to be shared among the network connections in prescribed and orderly manner.

[40] As a further example of priority determination according to the present invention, the definition above for Slack assumes that there is some data on the reliability of an end-to-end path, i.e., the expected downtime of a connection for the year. In case such information is not

known, one way of accounting for the lack of information to simply assume that the expected downtime for the year is the allowed downtime for the year. This represents a more conservative approach that ensures a connection is on track to meet its SLA irrespective of how much its path is likely to fail.

[41] Another embodiment of the present invention determines the priority of a network connection based upon its permitted downtime from its SLA and the connection's link failures themselves. Instead of tracking the history of outages for a connection over an extended period of time (e.g., a year) and determining the connection's priority based upon the tracked downtime, the priority of a connection simply decays exponentially in time if the connection does not experience failures and rises upon failure. This permits a variable priority without the requirement of extensive memory to track the downtimes experienced by the connections. [42] In the descriptions above it was assumed that the SLA is defined in terms of downtime per year. A single significant outage might allow the priorities of the connections to be set and still meet the terms of the SLAs. In contrast, the expectations of the customers might not be met due to the long interruption of services. Accordingly, the Slack of each connection might be recalculated during the outage period so that if a connection that was not recovered based on its initial priority is now close to violating its SLA, its priority will be reconsidered. Alternatively, an SLA can be defined with two time periods: (1) the total permitted outage during a year, and (2) the maximum contiguous permitted outage. In such a case, the Slack function accounts for both time frames in the determination of a connection's priority. This guarantees that the downtime of a failure is limited and the outage period for a connection is not excessive. [43] Finally, since at least one embodiment of the present invention operates to preempt connections with maximum Slack, it may be desirable to reroute connections based on knowledge of Slack in the network. This requires changes to the topology dissemination protocols, such as OSPF (Open Shortest Path First), of the network. There are two possible approaches without incorporating per connection data in the disseminated information. For networks which operate with pre-defined backup paths, such as MPLS (Multiprotocol Label Switching) FRR (Fast Rerouting) or SONET mesh protection, there is no need for real-time routing decisions, so there is no need to change OSPF. The same is true for ring networks. For other networks, it is sufficient to add failure statistics per link in the network, instead of adding

per connection data. Since most failures are due to fiber cuts, this minimal information suffices to estimate the Slack for a particular route.

[44] Therefore, while the description above provides a full and complete disclosure of the preferred embodiments of the present invention, various modifications, alternate constructions, and equivalents will be obvious to those with skill in the art. Thus, the scope of the present invention is limited solely by the metes and bounds of the appended claims. [45] This description of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the teaching above. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications. This description will enable others skilled in the art to best utilize and practice the invention in various embodiments and with various modifications as are suited to a particular use. The scope of the invention is defined by the following claims.