Structured id-based and topology adaptive control plane for 5G

TECHNICAL FIELD

The present invention relates to the control of 5G networks, and more particularly to a structured id- based control plane.

BACKGROUND

According to the industry consensus, the 5th generation (5G) mobile technology will be standardized and deployed by 2020. Compared to the 4th generation (4G) mobile technology, the devices and applications of the next generation network is expected to support many new types of connections between various devices such as cars, wearables, sensors and actuators, from both private and industrial environments. The new types of connections usually imply very distinct service requests about, for example, latency and data rate, which naturally asks for different treatments and thereby poses challenges to the control of the 5G networks.

In particular, supporting various new types of services means a deep impact on the core network architecture. In today's mobile networks, services mainly refer to the access of portable devices in human hands for data and voice services only. Core networks (CN) can basically apply the same treatment to portable devices following the best-effort principle. Hence, those services are predictable and the way of responding to them can be well pre-planned. However, the various new service requests cause their properties to be heterogeneous. For example, in the future 5G networks, at a certain point, the sensor nodes may request to join the network in order to transmit sensing results, while there could be remote high speed vehicles communicating with each other at the same time. Effortlessly, distinct traffic patterns will be spontaneously generated to the network. Here, the traffic patterns can be affected by the service duration, the node mobility patterns, and the requirements of quality-of-service (QoS) of the connection. Consequently, the service requests to the 5G network are hard to be predicted and a proper way to meet the requirements is to respond with agility to the incoming services in a dynamic way. However, efficiently responding to distinct and unpredictable service requests is not an easy task. Obviously, administratively pre-planning approaches with static network deployment do not work. Recently, network functions virtualization (NFV) and software-defined networking (SDN), as disclosed in: Open Networking Foundation, "Openflow Switch Specification", Version 1.5.1, 2015, have been mentioned as the central two enablers to realizing a flexible and programmable network infrastructure. The idea behind the SDN is to abstract everything as a flow and to move the complexity of the flow treatment towards a single logical element called SDN controller (SDNC). Hence, the SDN view reduces all the network elements to dump the flow treatment devices, which are only responsible for the flow processing. With these concepts, the SDN fuses all the management and control plane intelligence at the SDNC. In return, the common abstraction and the locally available data simplify the development of the network control and management applications. The NFV further turns existing network functions (NF), which were mainly implemented by dedicated and specialized hardware formerly, into virtualized function modules that can be easily deployed or removed, as disclosed in: H. Hawilo, A. Shami, M. Mirahmadi and R. Asal, "NFV: state of the art, challenges, and implementation in next generation mobile networks (vEPC)", Network, IEEE, vol. 28, pp. 18-26, 2014. With these two enabling technologies, responding to new service requests becomes easier than formerly given that any network function (NF) modules can be deployed or allocated on demand (features of NFV) and their forwarding behaviors can be remotely programmed (features of SDN), just like installing and running a computer program.

With the new enabling technologies, changes are also expected in the network infrastructure per se. In their struggle to reduce operational and capital expenses, mobile network operators are seeking for possibilities to align the available overall infrastructure capacities and the actual usage. More agility in both infrastructure and architecture together with operational decisions is preferable. For example, green networking advocates switching off underused resources or dynamically reroutes traffic to cheap energy nodes and links. Beyond that, new approaches to allow both sharing of excess resources and leasing extra resources lead to generalized usage of virtualization of both links and nodes in the infrastructures. Green networking and virtual networking call for support for network structural dynamics since links, nodes and their services cannot be presumed permanent, which are brought by SDN and NFV as well.

Given the diversification of new services and the transition happening on the network infrastructure, resource control becomes the central task for future 5G networks. In particular, NFV orchestrator and SDNC have to rely on a robust control plane connecting every network resource to transmit and realize their strategies operating the network behaviors. While the control plane deployment could be achieved manually or through some management solutions like, for example, the configuration management, the command line interface (CLI) management or the NETCONF configuration protocol as found in: M. Wasserman and T. Goddard, "Using the NETCONF configuration protocol over secure shell (SSH)", 2006, the result would be a fixed (i.e., not programmable, not adaptive, not dynamic) control plane supposed to handle a data plane exhibiting high dynamics due to the continuous and frequent changes by both NFV's virtual infrastructure management (VIM) and the SDNC. However, this would result in the following unsolved problems.

First of all, this view would not be holistic. As a result, not only it does require out-of-band mechanisms in the network resource elements, uncovered by the actual NFV and SDN specifications, but it also results in the well-known chicken-egg problem during the initial deployment (namely, does SDN/NFV require a network or does the network require SDN/NFV?), complicates the maintenance of the quality of the control service and raises new challenges with regard to the resilience of the latter.

Secondly, both OpenFlow-enabled SDN and European Telecommunication Standards Institute (ETSI) NFV currently neglect their own needs for control, presuming, on the one hand, distinct resources available for SDNC or VIM execution and, on the other hand, end-to-end connectivity from every resource element to the location of the controller or VIM, respectively. The current specifications to SDN and NFV actually do not provide any mechanism for the control that they themselves require. For instance, in OpenFlow specifications, the control channel establishment is not covered by the protocol itself, but requires additional, pre-provisioned and separate (physical or logical) network and compute resources (for the controller element execution and the controller-switch

communications). Similarly, in NFV, the VIM is supposedly well-connected to the worker resources (compute and storage nodes), but the provision of this connectivity and the deployment of the VIM itself are nevertheless not covered. Failing to provide initial self-organization means requires more complex initial configurations, which result in unrelia ble manual interventions. In a nutshell, while NFV and SDN in principle promise ease of operations, without the auto-provision of control plane (CP), they would rather offload their complexity to our customers, the operators.

Thirdly, different control applications will ask for different QoS on the path from switches to the controller. As an example, a monitoring application (e.g., an application of the sample theorem) might be interested in specific guarantees on the latency of the control channel, while another application (e.g., the NetFlow standard for network traffic monitoring) might be interested in high data rates. Beyond the path, different requirements will emerge in terms of necessary computational capacity and memory in the resource and in its controller. For example, in the current OpenFlow- enabled SDN, each switch connects by means of a TCP connection to the SDNC. Over this same connection, several different packet types such as PACKETJN for notification and FLOWMOD for flow rule modification are then sent. Today, such packets are usually grouped together in a TCP segment, even though, semantically speaking, they might be destined to different control applications. It is difficult to provide QoS guarantees in this setup. As a result, the control plane needs be engineered in order to fulfill QoS and resilience requirements in a pragmatic manner.

Last but not least, beyond the initial connectivity deployment and the QoS, neither SDN nor NFV today protect the required control means and control channels from user traffic, general failures, and so on. An aggressive protocol could easily block control plane communications without further provisions. The problem is that such provisions cannot be easily pre-configured because the control needs and the traffic map of a SDN switch could change with the deployment of new control applications. More specifically, neither of current technologies considers failures stemming from the control decisions. Indeed, beyond the classical error sources, software networks are characterized by intrinsic dynamics due to the continuous reconfigurations. As an example, in OpenFlow SDN, it is possible to upload flow rules into a switch, which will cut down OpenFlow flows to this same switch and/or the downlink switches, thereby breaking control and making corrections impossible. All these errors are called self-inflicted. If such errors are not prevented, the actual development of the control applications (cApps) will become a much more tedious task consisting of increased OPEX (operational expenses) or CAPEX (capital expenditures) for operators according to whom is developing the cApps.

A solution to all the aforementioned problems would be to design a self-organizing control plane that can bootstrap autonomously without pre-configuration, provide multiple paths for failed control channel switching and be topological^ adaptive to different service QoS requirements. Then, those key features should be supported by the resources, independently of the SDNC and the VIM, and they will therefore require a control specific intelligence in all controllable resources. In other words, the control plane should be spawned and constantly maintained by the resources without the need for the SDNC and VIM intervention. However, the problem is challenging in the software networks because both the control realm (infrastructure, resources) and the control needs (network functions, operator needs, end-user services) are supposed to evolve in time due to usage dynamics, but also due to reconfigurations and the transient nature of resources. Such a problem can be designated as a unified resilient control problem or an in-resource control problem, thereby meaning that the control plane and the controller functionalities are provided from the same execution, storage and communication resource pool as the actual services and the data plane, for the control of which the controller and the control plane are used.

To solve such a problem, an IETF working group called autonomic networking integrated model and approach (ANIMA) suggests that future networks shall be capable to manage themselves by some autonomous functions deployed in the infrastructure. Moreover, those autonomous functions shall be automatically connected and communicate with each other over an autonomous control plane (ACP) as found in: M. H. Behringer, T. Eckert and S. Bjarnason, "An Autonomic Control Plane", Internet Engineering Task Force, 2016. Although the documents US-9,043,884-B2, US-9,130,837-B2 and US-9,391,959-B2 are specifically related to the control plane solution, the existing routing protocol RFC6550 for low-power and lossy networks is suggested as the protocol to maintain an ACP, as found in T. Winter, P. Thuber, B. Brandt and others, "RFC 6550: IPv6 Routing Protocol for Low- Power and Lossy Networks," Internet Engineering Task Force (IETF) Request For Comments, 2008.

In fact, there are much more alternatives to the protocol in RFC6550 since any existing routing protocol can in principle be a candidate. A large class of autonomous routing protocols was devised from the world of infrastructured networks. For example, for switches in local area network (LAN) and wide area network (WAN), RSTP as found in: W. Wojdak, "Rapid Spanning Tree Protocol: A new solution from an old technology", Reprinted from CompactPCI Systems, 2003), OSPF as found in: J. Moy, "Open Shortest Path First Routing Protocol (Version 2)", 1994, RIP as found in: C. L. Hedrick, "Routing information protocol", 1988 and IS-IS as found in: D. Oran, "OSI IS-IS intra-domain Routing Protocol", 1990, can be the choice. Another class of autonomous routing protocols is from the world of wireless ad-hoc networks. For example, AODV as found in: C. Perkins, E. Belding-Royer and S. Das, "Ad hoc on-demand distance vector (AODV) routing", 2003, DSDV as found in: C. E. Perkins and P. Bhagwat, "Highly dynamic destination-sequenced distance-vector routing (DSDV) for mobile computers", ACM SIGCOMM computer communication review, 1994, OLSR as found in: T. Clausen and P. Jacquet, "Optimized link state routing protocol (OLS )", 2003 and DSR as found in: D. B. Johnson and D. A. Maltz, "Dynamic source routing in ad hoc wireless networks", Mobile computing, Springer, 1996, pp. 153-181, can be the choice. The idea of directly relying on existing routing protocols to maintain the control plane connectivity can be found in existing works such as: S.

Sharma, D. Staessens, D. Colle, M. Pickavet and P. Demeester, "Automatic bootstrapping of OpenFlow networks", Local & Metropolitan Area Networks (LANMAN), 2013, such as: S. Sharma, D. Staessens, D. Colle, M. Pickavet and P. Demeester, "Enabling fast failure recovery in OpenFlow networks", Design of Reliable Communication Networks (DRCN), 2011, such as: S. Sharma, D.

Staessens, D. Colle, M. Pickavet and P. Demeester, "Fast failure recovery for in-band OpenFlow networks", Design of Reliable Communication Networks (DRCN), 2013, and such as: C.-C. Tu, P.-W. Wang and T.-C. Chiueh, "In-band control for an ethernet-based software-defined network", Proceedings of International Conference on Systems and Storage, 2014. However, although the existing protocols are natural candidates, they typically fall short in fulfilling certain requirements, specifically imposed by the need to bootstrap a control plane.

First of all, most of the existing protocols (e.g., the link-state protocol or the distance-vector protocol), are flooding-based. Every node floods either their local topology information or their routing table information to its neighboring nodes across the whole network. Also, the flooding- based mechanism slows down the convergence of the network in order to build any-to-any connectivity if the topology dynamic happens.

More importantly, most of the existing protocols are topology dependent. For example, the switches in LAN/WAN use RSTP or OSPF for interconnection. They both generate a spanning tree as an overlay. For datacenter networks, a fat-tree topology is used to connect computing resources. For small mobile ad hoc networks (MANETs), ad hoc on-demand distance vector (AODV), destination- sequenced distance-vector (DSDV) and optimized link state routing (OLSR) also generate a hierarchical tree overlay, respectively. However, the structure has a major impact on the node state and the possible communications in the control plane as well as the resilience of the whole. The ACP needs be engineered in order to fulfill the QoS and resilience requirements in a pragmatic manner. However, existing routing protocols are not capable of adapting their own overlay topology to strike a desired balance between the QoS and the resilience. With the apparent lack of autonomy, another possibility is to proactively protect the control domain controllability. This can be made by pre-analyzing all the possible threats to the control plane connectivity. For example, one can examine all the flow rule instructions that will be installed into the network elements in order to prevent any of them to break the pre-deployed control channel, as found in: A. R. Curtis, J. C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma and S. Banerjee,

"DevoFlow: scaling flow management for high-performance networks", ACM SIGCOMM Computer Communication Review, 2011, and in: R. Sherwood, G. Gibb, K.-K. Yap, G. Appenzeller, M. Casado, N. McKeown and G. Parulkar, "Flowvisor: A network virtualization layer", OpenFlow Switch Consortium, Tech. Rep, pp. 1-13, 2009. However, this is related to a serious scalability problem when the size of the network increases. Moreover, one may also consider to increase the dedicated resource for controlling, which is equivalent to a kind of pre-allocation wherein the control point is put closer to unstable areas so that the network elements are not easily disconnected due to network failures, as found in: A. Dixit, F. Hao, S. Mukherjee, T. V. Lakshman and R. Kompella, "Towards an elastic distributed SDN controller", ACM SIGCOMM Computer Communication Review, 2013. Nevertheless, this cannot fundamentally resolve the controllability issue because the unstable areas themselves can be totally dynamic. Consequently, where to dedicate more resources is unpredictable.

Thus, few prior art solutions focus on the autonomous control plane problem and, when they do so, those solutions remain very basic. In addition, for the control plane requirements, important functionalities are still missing and do not meet the needs of the future 5G networks.

SUMMARY

It is therefore an object of the present invention to provide a self-organizing and adaptive control plane, by means of which an id-based logical structure can be autonomously constructed and the topology of the id-based logical structure can be rendered adaptive by doing rewiring.

The object is achieved by the features of the independent claims. Further embodiments of the invention are apparent from the dependent claims, the description and the figures.

According to a first aspect, the invention relates to a system comprising multiple resource elements within a communication network, wherein each resource element is adapted to support a resource- to-resource (R2R) protocol. The R2R protocol is adapted to assign a respective identification (id) value (rii) to each resource element amongst the multiple resource elements, each identification (id) value (n,) being a unique value, adapted to establish, for each resource element amongst the multiple resource elements, a connection to each of its physical or virtual neighbors by transmitting an id- based message (Hello message) from each resource element towards each of its physical or virtual neighbors, the physical or virtual neighbors of a resource element being the resource elements located in an underlay and with a network connection to it which is available for a connection establishment, and adapted to establish, by means of each resource element amongst the multiple resource elements and for each pair of logical neighbors, a logical connection between the respective two logical neighbors in order to form an id-based logical structure as an overlay and based on the order of the identification (id) values (n,) of each resource element amongst the multiple resource elements, the two logical neighbors corresponding to two resource elements with adjacent identification values (n,) on the id-based logical structure in such a manner that one of the two logical neighbors is either a successor of the other one or a predecessor of the other one on the id-based logical structure.

Thereby, all the controllable resource elements within the network can autonomously establish and maintain an id-ordered logical structure. Each resource element can be defined as a node or a network node like, for example, a routing or forwarding element such as a switch and a probe. Each resource element can also be identified by a distinct unsigned integer identification value, namely an integer id regardless of length (in terms of bits). The integer id can, for example, be a public key of any binary length. For simplicity purpose, the id value (n,) can be randomly drawn from a virtual id space. The resource-to-resource ( 2 ) protocol can be defined as an inter-resource protocol.

It should be noted that the network connection between the physical or virtual neighbors of a resource element and the resource element itself can also be designated as a network path in a generic manner. On the other hand, for each resource element, the connection to each of its physical or virtual neighbors by transmitting an id-based message, such as a Hello message, can be specifically designated as a neighboring link, so that the network connection to the resource element which is available for a connection establishment can correspond to the network path to the resource element which is available for a neighboring link. According to a first implementation of the system according to the first aspect, each resource element amongst the multiple resource elements supports the 2 protocol through a respective resource control agent (RCA).

Thereby, each resource element can be a controllable resource element.

Furthermore, each resource control agent (RCA) can be locally implemented inside each respective resource element amongst the multiple resource elements as a local daemon. Thereby, each resource element can intrinsically use some of its own components as an RCA. Then, those components will be locally modified by extending their functionalities in order to support the R2R protocol.

According to a second implementation of the system according to the first implementation of the first aspect, each resource element amongst the multiple resource elements performs a local pairing operation in a distributed manner by transmitting a notification message (NotifyNb message) towards each of the two logical neighbors of each pair in order to establish the logical connection between the respective two logical neighbors, the logical connection between the respective two logical neighbors being established once the transmitted notification message (NotifyNb message) has been received by each of them.

Thereby, a logical path between each consecutive pair can be established by each resource element doing the pairing operation.

According to a third implementation of the system according to the second implementation of the first aspect, the R2R protocol is adapted to solve a conflict of resource elements pairing the same pair of logical neighbors by allowing each pair of logical neighbors to be paired by means of a single resource element.

Thereby, a pairing-based conflict resolution can be achieved and the formation of the id-based logical structure through the pairing operation can be improved. According to a fourth implementation of the system according to the third implementation of the first aspect, the conflict is solved by allowing each of the two logical neighbors constituting the pair to make a same and independent choice to select the single resource element and to reject any other resource element by transmitting each a respective rejection message ( ejectNb message) towards each rejected resource element.

Thereby, the formation of the id-based logical structure can be improved using transmission of messages from the R2R protocol.

According to a fifth implementation of the system according to the fourth implementation of the first aspect, the choice is based on calculating a respective logical distance from each of the pairing resource elements towards each of the two logical neighbors, the selected single resource element corresponding to the shortest logical distance between itself and one of the two logical neighbors of the pair.

Thereby and with respect to other conflict resolution methods, the present method based on the calculation of a logical distance offers the advantage to require no overhead and to accelerate the formation of the id-based logical structure by accepting the closer resource element doing the pairing operation.

According to a sixth implementation of the system according to any one of the second to fifth implementations of the first aspect, each resource element amongst the multiple resource elements has a routing table comprising all the pairs of logical neighbors paired by itself, all the logical paths going through itself and all the logical paths between its logical neighbors, and consisting of a set of data routing rules to be applied to the incoming data packets and a set of control routing rules to be applied to the incoming control packets, the set of control routing rules being maintained by the resource control agent (RCA).

Thereby, control channels can be established amongst all the resource elements, the routing table comprising control flow rules establishing the id-based structured overlay. According to a seventh implementation of the system according to the first aspect or any one of the implementations of the first aspect, the 2 protocol is adapted to establish, for at least one resource element amongst the multiple resource elements, logical connections to at least one finger resource element, the finger resource element corresponding to a resource element other than one of the two logical neighbors.

Thereby, the id-based logical structure can be grown with finger links or finger paths or finger connections.

According to an eighth implementation of the system according to the seventh implementation of the first aspect, the logical connections to at least one finger resource element are selectively established using an algorithm of selection of finger resource elements.

Thereby, a resource element can select specific finger resource elements dynamically and keep connections to them so as to form a certain topology of the id-based logical structure, in particular for the control plane.

According to a ninth implementation of the system according to the eighth implementation of the first aspect, a topology of the id-based logical structure is adaptively changed according to the network conditions.

The change in the topology can be carried out in several ways.

Thus, the topology of the id-based logical structure can be changed by adding or removing at least one logical connection to at least one finger resource element. Thereby, a simple rewiring method can be obtained. The change in the topology can also be triggered either manually or autonomously and achieved using the algorithm of selection of finger resource elements. Thereby, the finger links can be added or removed adaptively.

The change in the topology can also be manually triggered by a controller or an administrator of the communication network. Thereby, the finger links can be added or removed adaptively according to the own needs of the controller or the administrator.

The change in the topology can also be autonomously triggered by each resource element amongst the multiple resource elements based on a performance indicator (KPI) of its connections to the logical neighbors and the finger resource elements. Thereby, the finger links can be added or removed adaptively according to a quality or performance indicator.

According to a tenth implementation of the system according to the first aspect or any one of the implementations of the first aspect, the id-based logical structure is an id-ordered ring-based logical structure.

Thereby, a simple id-based logical structure can be obtained owing to a closed-loop logical structure.

Furthermore, each resource element amongst the multiple resource elements can be a network- capable node. Thereby, the resource element can be any network-capable element. Thus, the resource element can be any node like, for example, a network element, a compute node, a server, a storage server, a file server, a compute cluster or a domain controller amongst others.

The above object is also solved in accordance with a second aspect.

According to the second aspect, the invention relates to a control plane comprising the system as specified in the first aspect or any one of the implementations of the first aspect. Thereby, a simple and self-organizing control plane can be obtained based on a full infrastructure controllability. Indeed, it is supported by the in-resource intelligence from all the controllable resources, with a distributed 2 protocol. Any network resource element locally hosts a resource control agent (RCA) talking to each other through the proposed R2R protocol. Thus, the self- organizing control plane is adapted to bootstrap autonomously without pre-configuration, provide multiple paths for failed control channel switching and be topological^ adaptive to different service QoS requirements. A control-based intelligence is available in all the controllable resource elements such as the nodes, network-capable nodes and so on, which allows the control plane to be spawned and constantly maintained by the resources independently of the SDNC and VIM.

The above object is also solved in accordance with a third aspect.

According to the third aspect, the invention relates to a resource element of a system comprising multiple resource elements within a communication network according to the first aspect or any one of the implementations of the first aspect.

The above object is also solved in accordance with a fourth aspect.

According to the fourth aspect, the invention relates to a method for controlling multiple resource elements within a communication network. The method comprises the steps of assigning a respective identification (id) value (n,) to each resource element amongst the multiple resource elements, each identification (id) value (n,) being a unique value, establishing, for each resource element amongst the multiple resource elements, a connection to each of its physical or virtual neighbors by transmitting an id-based message (Hello message) from each resource element towards each of its physical or virtual neighbors, the physical or virtual neighbors of a resource element being the resource elements located in an underlay and with a network connection to it which is available for a connection establishment, and establishing, by means of each resource element amongst the multiple resource elements and for each pair of logical neighbors, a logical connection between the respective two logical neighbors in order to form an id-based logical structure as an overlay and based on the order of the identification (id) values (n,) of each resource element amongst the multiple resource elements, the two logical neighbors corresponding to two resource elements with adjacent identification values (n,) on the id-based logical structure in such a manner that one of the two logical neighbors is either a successor of the other one or a predecessor of the other one on the id-based logical structure.

According to a first implementation of the method according to the fourth aspect, the method comprises the step of esta blishing, for at least one resource element amongst the multiple resource elements, logical connections to at least one finger resource element, the finger resource element corresponding to a resource element other than one of the two logical neighbors.

The above object is also solved in accordance with a fifth aspect.

According to the fifth aspect, the invention relates to a computer program comprising a program code for performing the method according to the fourth aspect or the first implementation of the fourth aspect when executed on a computer.

Thereby, the method can be performed in an automatic and repeatable manner.

The computer program can be performed by the above apparatus.

More specifically, it should be noted that the above apparatus may be implemented based on a discrete hardware circuitry with discrete hardware components, integrated chips or arrangements of chip modules, or based on a signal processing device or chip controlled by a software routine or program stored in a memory, written on a computer-readable medium or downloaded from a network such as the Internet. Moreover it can be a fully virtualized appliance, that is, for example, a virtual machine executed by any other machine including another virtual machine. The resource element can also be any computer program executed in some appropriate way on some hardware, which might in some situations be seen as not belonging to the resource element. That means that the resource element can, if necessary and appropriate for the specific control purpose, be operationally delimited both in size (e.g., physical, geographical or logical protocol reach) and in depth (spanning all hardware and software or being rather limited to a specific layer in the protocol stack), while the definition of the resource element per se does not limit its applicability and generality.

It shall further be understood that a preferred embodiment of the invention can also be any combination of the dependent claims or above embodiments with the respective independent claim.

These and other aspects of the invention will be apparent and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following detailed portion of the present disclosure, the invention will be explained in more detail with reference to the exemplary embodiments shown in the drawings, in which:

Fig. 1 shows a structured id-based control plane 100 consisting of an id-ordered ring-based logical structure (overlay) and a physical network structure (underlay) according to an embodiment of the present invention;

Fig. 2 shows an architecture 200 of the extended OVS node within an SDN scenario according to an embodiment of the present invention;

Fig. 3 shows a procedure of establishment of a physical path connecting two OVS nodes (S,, S, ) according to an embodiment of the present invention;

Fig. 4 shows a pairing procedure of establishment of a logical path connecting two logical neighbors (a,, b,) according to an embodiment of the present invention; and

Fig. 5 shows an exemplary procedure of conflict resolution related to a pairing operation according to an embodiment of the present invention; Fig. 6 shows a procedure of handling an OVS node joining (n _x ) according to an em bodiment of the present invention;

Fig. 7 shows an exemplary procedure of handling an OVS node leaving (n _x ) according to an em bodiment of the present invention;

Fig. 8 shows an exemplary procedure of topological adaptation with finger selection according to an embodiment of the present invention; and

Fig. 9 shows a system 300 of multiple resource elements with a distributed resource-to- resource ( 2 ) protocol according to an embodiment of the present invention.

Identical reference signs are used for identical or at least functionally equivalent features.

DETAI LED DESCRI PTION OF EM BODI M ENTS OF THE I NVENTION

Fig. 1 shows a structured id-based control plane 100 consisting of an id-ordered ring-based logical structure (overlay) and a physical network structure (underlay) according to an em bodiment of the present invention.

The present invention rests on the design of the structured id-based control plane 100, in which each resource element is a node such as a network-capa ble node (e.g., a routing element, a forwarding element, a network element, a compute node, a server, a storage server, a file server, a compute cluster or a domain controller amongst others) and is identified with a respective and unique identification (id) value (n,). The resource elements located in the physical network (denoted hereafter by u nderlay) are arranged in an embedded logical structure (denoted hereafter by overlay) through an id-based ma pping. The core structure of the overlay is an id-ordered ring-based topology grown with finger links connecting, for example, ¾ to n ₅ to n ₇ a nd n ₇ to n¾ as depicted in the exemplary extended ring of Fig. 1. In another em bodiment, the ring-based topology can be replaced with any other topology exhibiting a circular form or with any other topology exhibiting a form other than circular. Generally speaking, it should be noted that the terms "underlay" and "overlay" are not limited to a respective physical and logical network, but are also applicable to any layers. For example, it is possible to have a physical layer as an underlay and to construct the overlay on top of it using a 2 protocol, so that the resource elements are physical elements. In another example, it is possible to have the resource elements as virtual machines running on any support, and to have them connected by virtual links, possibly using TCP/IP networking or VPN between each other. The corresponding virtual network or structure can then be used as an underlay, and the R2R protocol will span a new overlay on top for the control of the resource elements in this scenario. In addition, it should be noted that the resource elements are considered unstable and their connectivity is considered transient. The feature "unstable" means that the resource elements can come and go (this process is commonly called churn, and sometimes attrition) more or less at any time. The resource elements can also move, and the links or connections can also change, be dropped or new links or connections can be created by something other than the R2R protocol. As will be described in the following, the proposed R2R protocol is adapted to handle all these issues: new nodes, nodes leaving, links or connections dropped, links or connections failing, links or connections coming, and so on. However, it should be further noted that this instability would be rather atypical if the resources were classical forwarding elements only. Indeed, a physical switch as a forwarding element cannot disappear from the rack, and cannot be easily duplicated. Thus, it is the essence of the proposed R2R protocol to mainly target non-physical networks and to handle dynamics created by programmability.

The assignment of each identification (id) value (n,) is achieved by the proposed distributed R2R protocol, which is supported by the in-resource intelligence from each resource element through a respective resource control agent (RCA).

In the following, the present invention will be described in the illustrative case, where the resource element is an Open vSwitch (OVS) node. As a forwarding element, the OVS node represents a well- known switch, which can be instantiated and run on a standard computing platform (e.g., in a virtual machine (VM)). The OVS node has also the advantage to be already a commercial-grade product widely deployed in the real networks of many IT companies such as Google.

Firstly, the OVS node will be extended by integrating the R2R protocol therein, as it is depicted in Fig. 2 showing the architecture 200 of the extended OVS node within a SDN scenario. A virtual network, using, for example, the prototyping environmental tool Mininet as described in: B. Lantz, B. Heller and N. McKeown, "A network in a laptop: rapid prototyping for software-defined networks", Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks, 2010, and consisting of a number of extended OVS nodes connected according to a predefined network topology, will be then created.

As shown in Fig. 2, the architecture 200 of the extended OVS node consists of two main parts: a kernel part and a userspace part, which are graphically separated by the dotted line. The kernel part contains a kernel datapath module 210, which implements the forwarding engine responsible for per-packet lookup, modification and forwarding. The userspace part contains an ovsdb-server 220, which is responsible for storing the information about the configuration of the switch, and an ovs- vswitchd module 230 as a local daemon, which can modify the kernel part by modifying, for example, the flow rule entries in the flow table. It should be noted that, in a generalization context with respect to the specific SDN scenario, the flow table could then be a normal routing table consisting of a set of data routing rules to be applied to the incoming data packets and a set of control routing rules to be applied to the incoming control packets. In addition, the local daemon also exposes the external interface in order to allow the extended OVS node to communicate with a remote SDN controller through the depicted control port.

As shown in Fig. 2, the architecture 200 of the extended OVS node further comprises a RCA module (231), which is locally implemented inside the ovs-vswitchd module 230 in order to extend the functionalities of the local daemon (i.e., the ovs-vswitchd module 230) and allow the extended OVS node to support the R2R protocol of the present invention. The local daemon has been chosen because it has already been used successfully as a local agent responsible for many basic control tasks such as implementing the forwarding logic including media access control (MAC) learning, load balancing over bonded interfaces and communicating with the external SDN controller using the OpenFlow protocol. Thus, with minimum effort, the extended local daemon can support the functionality to construct the structured id-based control plane of the present invention.

In order to complete its task, the proposed R2R protocol utilizes four kinds of messages, which are designated as a Hello message, a NotifyNb message, a RejectNb message and a RemovePath message. The Hello message represents an id-based message that is periodically broadcast by an OVS node towards its physical or virtual neighbors in order to inform them about its own id value. The NotifyN b message represents a notification message that is broadcast by an OVS node towards its logical neighbors in order to inform them a bout other OVS nodes. The RejectNb message represents a rejection message that is broadcast by an OVS node having received a NotifyN b message towards the logical neighbor that has sent it the NotifyN b message in order to reject the received NotifyNb message. The RemovePath message represents a message that is broadcast by an OVS node towards all the peers whose connections take a leaving node as the next hop in order to remove a physical path towards the leaving node.

In the following, the sym bol n, will refer to the id value of an OVS node. Furthermore, since the proposed R2R protocol is fully distributed, it will be described from the point of view of the OVS node and all the other OVS nodes will behave identically to that OVS node.

The R2R protocol starts with a step of bootstrapping, wherein each OVS node first generates its id value (n,) and afterwards, locates and esta blishes connections to its logical neigh bors and finger OVS nodes in order to form the structured overlay.

The id generation ca n be carried out in different methods. For example, the id value (n,) can be preconfigured in the OVS node or the id value (n,) can consider the location of the OVS node in the network such that the physically closer OVS nodes get closer id values (n,). In the following and for illustrative purpose only, another example of the id value (n,) being randomly d rawn from a virtual id space will be chosen due to its simplicity. However, the id value (n,) should be generated with the aim of being unique thanks to a method allowing to minimize as much as possible the collision of generating the same id values (n,).

Once the id generation has finished, each OVS node first identifies its physical or virtual neighbors that have availa ble network connections to itself and broadcasts a respective Hello message over all its port interfaces towards each of them, as illustrated in Fig. 3 showing the procedure of esta blishment of a physical path or connection between the two OVS nodes: S, and Sv with a respective port h and port i. The Hello message contains the id value (n,) of the sending OVS node (i.e., id value = n,). Once the Hello message from the port h is received by the receiving OVS node at the port i, the receiving OVS node learns that on the incoming port (i.e., the port h) it can reach the sending OVS node S, with the id value: n,. The method to form the structured overlay is inspired by the linearization algorithm as found in: M. Onus, A. W. icha and C. Scheideler, "Linearization: Locally Self-Stabilizing Sorting in Graphs", ALENEX, 2007. The algorithm turns any connected graph into a linear graph. The idea of the algorithm is that, based on its local view, each OVS node links its two peer OVS nodes if their id values (n,) are adjacent on the logical ring. Derived from that linearization algorithm, the pairing operation can be illustrated in Fig. 4, which shows a pairing procedure of establishment of a logical path connecting two logical neighbors (a,, b,) in the case of one iteration. Let us assume that, at a time slot t, an OVS node S, has a peer set N, (in Fig. 4, N, = {..., a, < b,, ...}), to which the OVS node S, has already respective connections to those peers. In a first step (1), the OVS node S, sorts the peer set N, according to the id values (n,) of the peers of the set N,. In a second step (2) and for each consecutive pair, denoted by (a,, b,) in Fig. 4 and called a logical pair, the OVS node S, sends a respective NotifyNb message towards a, and b, over the respective paths or connections between itself (S,) and a, and between itself (S,) and b,. Thus, the OVS node S, notifies a, about b, and b, about a,, respectively. The OVS node performing the pairing operation (e.g., the OVS node S, in Fig. 4) will be denoted in the following as a notifying node. When both endpoints in the logical pair (a,, b,) receive their respective NotifyNb message, they add each other as logical neighbors if there is no better candidate so far.

It should be noted that, in one iteration, the logical pair (a,, b,) might not be globally correct in the occurrence of another OVS node with an id value (n, = x,) closer than b, to a,, but unknown yet.

Nevertheless, the theoretical results show that, if each OVS node achieves the local pairing operation in a distributed manner, the suggested linearization-based algorithm guarantees the convergence, and the graph will be globally linearized. Thus, expressed differently, the suggested linearization- based algorithm can be considered self-stabilizing, and each OVS node will be able to find its global logical neighbor(s) eventually. The logic behind this is that the linearization-based pairing operation grows the peer set of other logical neighbors by notifying about a possible logical neighbor information. Similarly, the notifying node (e.g., the OVS node S, in Fig. 4) may receive NotifyNb messages from other ones, which in turn grows the logical neighbor set itself. With a newer logical neighbor set, the pairing and notification procedures will be repeatedly executed until there is no further change to the view of the logical neighbor set.

In addition to the connections forming the structured overlay, virtual paths to finger OVS nodes can be established in a similar way. Still considering the suggested linearization-based algorithm, the linearization of a globally incorrect logical pair (a,, b,) by the OVS node S, makes both endpoints have a finger path or connection, though later the OVS node a, and the OVS node b, will find out that the OVS node x, is their better logical neighbor. However, an information a bout the finger OVS node wil not be particularly notified.

In the present em bodiment, it will be assumed the provision of a centralized scheduler adapted to schedule the pairing operation such that, at each time, there cannot be two OVS nodes pairing the same pair of OVS nodes. For example, if an OVS node S, tries to pair (a,, b,) with a logical connection, then another OVS node S will be una ble to pair (a,, b,). However, in a distributed system, it is classically difficult to synchronize such a pairing operation. In other words, when any OVS node asynchronously pairs the same possible logical neighbors, there can be linearization conflicts happening. To that extent, the suggested 2 protocol is further adapted to solve such a conflict of OVS nodes pairing the same pair of logical neighbors by allowing each pair of logical neighbors to be paired by means of a single OVS node.

Fig. 5 shows an exemplary proced ure of conflict resolution related to a pairing operation according to an em bodiment of the present invention. This em bodiment provides a conflict resolution algorithm in the case that, when two notifying OVS nodes pair the same logical pair, two endpoints of that logical pair can make the same decision and independently agree to accept a pairing operation from the sa me notifying node.

in a first step (1), as depicted in Fig. 5, each OVS node a, and b, receives a NotifyNb message from both the notifying OVS node x and the notifying OVS node y. Then, in a second step (2), the OVS nodes a, and b, use the minimum logical distances from each of the two notifying OVS nodes x and y to their respective endpoints to make, in a third step (3), their decision. More specifically, in the second step (2), the OVS node a, calculates the logical distance from x to a,, the OVS node b, calculates the logical distance from x to b,, and the minimu m logical distance is chosen as δ _χ . The same calculation applies to the OVS node y in order to calculate the logical distance from y to a, and bi, respectively, and obtaining the minimum logical distance δ _γ . Between x a nd y, if δ _χ < δ _γ , then both a, and b, choose in the third step (3) to linearize the OVS node x correspond ing to the minimum distance δ _χ by sending each a respective RejectNb message towards the notifying OVS node y, otherwise (i.e., if δ _γ < δ _χ ) they would have chosen the OVS node y. In a fou rth step (4), the pairing procedure by the accepted notifying OVS node x can be u ndertaken in order to pair (a,, b,) with a logical connection. It should be noted that the a bove conflict resolution can also be based on alternative criteria. For example, each endpoint can use the path length given by the two notifying OVS nodes in order to make the decision. However, the criteria of determining the minimum logical distance between δ _χ and 8y, as used in the em bodiment of Fig. 5, presents the advantage of requiring no overhead and also the advantage of accelerating the formation of the structured overlay, in particular the id- ordered ring-based overlay, due to the proximity with the notifying OVS node that is accepted.

When the topology of the network changes, the present invention needs no special treatment to maintain the structured overlay. Indeed, any network dynamic results in logical neighbor set changes of related OVS nodes, which further trigger new linearization-based pairing operations and more particularly new notifications. In the following, the link change will be considered through the case of a node joining and another case of a node leaving.

A node joining can be defined as a new node associating with some existing nodes in the network by creating new links. As depicted in Fig. 6 showing a procedure of hand ling a node joining according to an em bodiment of the present invention, the new OVS node will be assumed to have its own id value n _x , which is different from all the other id values of the existing OVS nodes in the network. In a first step (1), the OVS node n _x starts to esta blish its physical paths to the OVS nodes that are directly associated with it by sending and receiving Hello messages according to the procedure of Fig. 3. This makes the peer sets of those associated OVS nodes changed, as illustrated in the second step (2) where the OVS node S, sorts its peer set and finds the new logical pair (a,, n _x ). As a result, the adjacent logical neighbors execute the linearization-based pairing process and continuously notify the OVS node n _x with the information a bout its possible logical neighbor(s). Similarly, since the logical neighbor set of the OVS node n _x grows iteratively, the OVS node n _x also executes its own

linearization-based pairing process. Eventually, the OVS node n _x will be merged into the globally linearized logical structure by linking to its global logical neighbors.

A node leaving can be defined as an existing node leaving the network while its adjacent links are removed. Such an event triggers the peer set changes of the nodes originally associated with the leaving node. Those nodes that directly observe the node leaving not only have to remove the physical paths to the leaving node, but also those virtual paths passing through the leaving node. As depicted in Fig. 7 showing a procedure of handling a node leaving according to an embodiment of the present invention, the OVS node S, observes the OVS node leaving n _x . Thus, the invalid physical path from Si to n _x will be removed from the flow table of S,, as illustrated in the first step (1). For the virtual paths, any endpoint of a path that takes the leaving node as the next hop will be notified with a RemovePath message to remove the respective invalid virtual path, as illustrated in the second step (2). In addition to the path removing, all the rest is the same as the case of the node joining where the linearization-based pairing and notification processes triggered by the logical neighbor set changes will continue and eventually correct the globally linearized structure by removing the leaving node.

By default, as introduced above, the finger paths are established, when a node is kept being notified by possible logical neighbors from other nodes. Every node maintains paths or connections to all the notified logical neighbors. The topological adaptation requires a rewiring method. The present invention introduces a finger selection algorithm. More specifically, after the bootstrapping step of the suggested R2R protocol, a node will select particular finger nodes dynamically and keep connections to them so as to form a certain overlay topology for the control plane. Such a finger selection procedure takes a certain topological mode as its argument, which can generally switch between the balanced, scale-free and centralized modes.

Fig. 8 shows a procedure of topological adaptation with finger selection according to an embodiment of the present invention.

As can be gathered therefrom, the finger selection algorithm starts with a first step (1) during which, when the OVS node S, with an id value establishes its finger paths, it starts by performing a lookup to know which are the OVS nodes n, with an id value equal to or greater than id + 2 ^1"1 , Vi 6 [l, k] . Then, instead of selecting this OVS node n, as the finger node directly, the finger node is picked up from an interval, and either the most or the least connected OVS node within the interval is chosen, as inspired by: S. Ktari, A. Hecker and H. Labiod, "A construction scheme for scale free dht-based networks", Global Telecommunications Conference, 2009. This leads to either an amplification or a mitigation of the OVS node degree skewing, which in turn yields either scale-free power graphs or uniform degree distributions, respectively. The size of the considered interval determines the number of super OVS nodes according to the following algorithm: - If the interval equals the ID space and encompasses all the OVS nodes with a high probability, then by preferring the most connected OVS node, the system will become a fully centralized system with essentially one central OVS node, to which each other OVS node will be connected;

- By preferring each time the least connected OVS node, the system will become a fully distributed system, where all the OVS nodes have an equal degree. This also works for smaller intervals, so that the global ID space as interval is not a necessary requirement for that;

- If the interval is of zero length, then a standard distributed hash table (DHT) is obtained regardless of the choice criterion. If the node IDs are irregular (e.g., if they are randomly chosen from a typical random distribution, e.g., uniform or normal), then the nodes are not placed within the space in an equidistant manner, which leads to an amplification of the topological importance of nodes responsible for bigger parts of the ID space. It is the reason why typical DHT nodes using random ID assignment are known to have small world structures.

If we choose the interval to go from one calculated OVS node to the next (e.g., in the Chord system, which is an efficient distributed lookup service based on the Chord protocol, for an OVS node with an id value, the interval would be: [id + 2 ^{1" 1} , id + 2' [), then decentralized systems can be created by increasing or decreasing the system centralization or distribution, respectively (i.e., the average OVS node's betweenness centrality). Different graph-theoretical properties (such as random graph, scale free graph, ring, tree and bus) can be achieved in this way at the system level by only using relatively simple and local algorithms within the respective resource nodes.

It should be noted that this finger selection procedure of Fig. 8 does not require the OVS nodes to keep any additional information. What an OVS node only needs to know is to maintain the connections to its logical neighbors. Moreover, in a third step (3), the selection procedure can be performed at the remote node n, that is found in the lookup process (step 1) in a transparent way without adding any extra overhead. In particular, when the OVS node S, issues, in a second step (2), a query to set up a finger path and this query reaches the destination node (i.e., n,), instead of returning its own id, it can, in a fourth step (4), merely return the selected id and build the connection for the OVS node S,.

In the present invention, changing to a certain overlay topology can be triggered either manually or autonomously. Manually changing the overlay topology for the control plane requires to propagate the decision from the controller or administrator of the network. Since the connectivity of the control plane will be always maintained by the control plane itself, the topology argument can be distributed across the network. After receiving the topology-changing command, each node runs the finger selection algorithm taking the topology argument.

Autonomously changing the overlay topology for the control plane is actually a promising mechanism, because each node can have a different perception of its local network conditions (e.g., it could experience heterogeneous QoS connections in the control plane). Some parts of the network can be more dynamic, for example, at the edge of a mobile network, thereby rendering the connections not very relia ble. In contrast, some parts of the network can be more static, for example, at the core of the mobile network. With heterogeneous network conditions, each node

autonomously senses the conditions of its connections to its logical neighbors and finger nodes. Once the KPI of the connections drops until reaching a threshold, each concerned node starts to run the finger selection algorithm switching to a balanced mode for resilience. Conversely, it switches to a centralized mode for efficiency if the situation is stable.

It should be noted that the 2 protocol is designed to establish a connectivity in any possible situation. Thus, even if different nodes were to be optimized for different structures (e.g., during the propagation of the administrative command or because of the different perceptions of the conditions), this would be not a problem as the graph stays connected at all times.

Fig. 9 shows a system 300 of multiple resource elements with a distributed resource-to-resource (R2R) protocol according to an embodiment of the present invention.

As depicted therein, the resource control agent (RCA) on each resource element keeps running the distributed R2R protocol to build and maintain the flow table consisting of a set of control flow rules, such that control channels are established amongst all the resource elements.

Finally, the suggested structured overlay forms an in-resource control plane and, thus, resolves the aforementioned chicken-egg problem for the control need of both SDN and NFV, since it is established by a self-organizing and distributed R2R protocol, independent of SDNC and VIM. Thus, the operators do not have any longer to dedicate external resources, in particular, for establishing the control channel. Once the network is deployed, its control plane is built autonomously. This applies to physical networks, logical networks and virtual networks equally.

Moreover, the forwarding policy becomes simple and straightforward. Indeed, given a packet, whose destination is dst.id, the node greedily chooses the path to the node, whose id is the closest in the id- based logical structure to dst.id. Because of the logical order among all the nodes, the packet is forwarded by getting closer and closer towards the destination, until the logical distance becomes zero. Such a routing mechanism can be called an id-based routing. It is, thus, clear that each node only needs a partial information of the network topology, but it can still achieve any-to-any routing. In addition, since the control messages are routed with an id-value addressing, each node is naturally independent to any data plane address strategies.

Furthermore, the suggested structured overlay is immune to any topology change (except for full disconnect of network areas), and, for example, the node joining and node leaving will not cause fluctuations to the connectivity of the control plane. This is due to the fact that, once there is a node joining in the network, only its logical neighbors on the id-based logical structure will be involved to merge the new node, while all the other nodes do not have to update their routing information.

Also, just because of the local impact to the whole established routing structure, the suggested structured overlay is quite scalable to the network size, which is a critical feature being required for the future control plane in 5G networks due to the expected large number of network nodes (e.g., small cells, smaller components in both software and data plane, dynamically composed to different slices, amongst others) and their high dynamics.

In addition, the suggested overlay provides a built-in multiple-path option. Specifically, when network dynamics (e.g., link failures resulting in broken paths) occurs, the affected node usually can route around the failed paths without requiring them to be repaired, as there are usually many routes between each pair of nodes. Since the R2R protocol keeps maintaining the structured overlay, the affected path between two logical neighbors will be reconnected eventually, if the disconnected logical neighbor is still in the network. Another advantage of the suggested structured overlay enables a topological adaptation, unlike the well-known routing protocols. The structured overlay provides enough flexibility for topology adaptation of the control plane. Specifically, the network nodes can add or remove finger connections in a distributed way but globally change the topology of the structured overlay. For example, if each node connects to all the other nodes, the control plane becomes a full mesh network. If each node randomly chooses its finger nodes, the control plane becomes a balanced network. If a few nodes are selected by other nodes as finger nodes based on some criteria, the structured overlay will become a scale-free network. Such a topology adaptation enables the control plane to be adaptive to the QoS requirements of different control applications.

In summary, the present invention relates to a self-organizing and adaptive control plane (CP) solution, which allows network nodes, namely all the controllable resource elements within the communication network, to autonomously establish and maintain an id-ordered ring-based logical structure without any pre-configuration. Each network node is identified by a distinct unsigned integer identification value, namely an integer id, and the entirety of the network nodes bootstrap and maintain the id-ordered ring-based logical structure for the control plane (CP), in which each node has logical connections to its predecessor and its successor as well as some finger peers. The assignment of each identification value is achieved by a distributed resource-to-resource ( 2 ) protocol, which is supported by the in-resource intelligence from each network node through a respective resource control agent (RCA). Following a manual or autonomous trigger, the network nodes can adaptively add or remove logical connections to some finger peers in order to change the topology of the id-ordered ring-based logical structure and thereby to balance between resilience and efficiency.

While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments and the OVS node, and is also applicable to any resource element or any node like, for example, a routing element, a forwarding element, a network-capa ble element, a network element, a compute node, a server, a storage server, a file server, a compute cluster or a domain controller amongst others. From reading the present disclosure, other modifications will be apparent to a person skilled in the art. Such modifications may involve other features, which are already known in the art and may be used instead of or in addition to features already described herein.

The invention has been described in conjunction with various embodiments herein. However, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

Although the present invention has been described with reference to specific features and embodiments thereof, it is evident that various modifications and combinations can be made thereto without departing from the spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded simply as an illustration of the invention as defined by the appended claims, and are contemplated to cover any and all modifications, variations, combinations or equivalents that fall within the scope of the present invention.