Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
FIRST NETWORK NODE, AND METHOD PERFORMED THEREBY, FOR HANDLING A PERFORMANCE OF A COMMUNICATIONS NETWORK
Document Type and Number:
WIPO Patent Application WO/2020/218956
Kind Code:
A1
Abstract:
A method, performed by a first network node (111), for handling a performance of a communications network (10). The first network node (111) determines (201) a predictive model of a metric of a target performance of an application supported by the communications network (10). The determining (201) uses a machine-implemented learning procedure, based on: i) crowdsourced data obtained from one or more entities (150) in the communications network (10), and ii) data on an infrastructure of the network. The first network node (111) also identifies(202), based on a second set of the crowdsourced data, one or more operations underperforming in the network. The first network node (111) then performs (203), based on the predictive model, a simulated optimization for each of the identified one or more operations, by modifying one or more features comprised in the predictive model, the optimization being based on one or more performance thresholds.

Inventors:
FARAJ ALAN NADHMI (ID)
MONDAL SURAJIT (IN)
Application Number:
PCT/SE2019/050372
Publication Date:
October 29, 2020
Filing Date:
April 24, 2019
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
ERICSSON TELEFON AB L M (SE)
International Classes:
H04W24/02; G06N20/00
Foreign References:
US20170244777A12017-08-24
US20180205631A12018-07-19
US20160217022A12016-07-28
US9439081B12016-09-06
US20160242053A12016-08-18
Other References:
AMARAN SATYAJITH ET AL.: "Simulation optimization: a review of algorithms and applications", ANNALS OF OPERATIONS RESEARCH, 23 September 2015 (2015-09-23), XP035899202, DOI: 10.1007/s10479-015-2019-x
PAPADOPOULOU NIKELA ET AL.: "Predictive communication modeling for HPC applications", CLUSTER COMPUTING, 24 March 2017 (2017-03-24), XP036337571, DOI: 10.1007/s10586-017-0821-8
Attorney, Agent or Firm:
HEDLUND, Claes (SE)
Download PDF:
Claims:
CLAIMS:

1. A method, performed by a first network node (1 11), for handling a

performance of a communications network (10), the first network node (1 11) operating in the communications network (10), the method comprising:

- determining (201) a predictive model of a metric of a target performance of an application supported by the communications network (10), the determining (201) using a machine-implemented learning procedure, based on:

a. a first set of crowdsourced data obtained from one or more entities (150) comprised in the communications network (10), and used to support the application, and b. data on an infrastructure of the communications network

(10),

- identifying (202), based on a second set of the crowdsourced data obtained from the one or more entities (150), one or more operations underperforming in the communications network (10) with regard to, respectively, one or more performance thresholds, and

- performing (203), based on the predictive model, a simulated

optimization for each of the identified one or more operations, by modifying one or more features comprised in the determined predictive model, the optimization being based on the one or more performance thresholds.

2. The method according to claim 1 , further comprising:

- initiating (204) providing, to a second network node (1 12) operating in the communications network (10), one or more indications of at least one of:

a. the determined predictive model;

b. the identified one or more operations;

c. one or more first modifications of the one or more features to achieve a respective target performance for the metric; and d. an estimated improvement in the metric, based on one or more second modifications of the respective one or more features, to achieve a higher performance than a respective current performance.

3. The method according to claim 2, wherein the one or more first modifications comprise at least one of:

a. a physical modification of the infrastructure of the communications network (10),

b. a change in layer strategy, the layer being one of one or more layers used in the communications network (10) to support the application, c. a change in configuration in a network node comprised in the

communications network (10),

d. a change in a configuration related to a user profile,

e. a change in a configuration in an application server, and

f. a change in a configuration in a server of a content provider.

4. The method according to any of claims 1-3, wherein the determining (201) further comprises engineering at least a first feature comprised in the predictive model to improve a predictive power of the predictive model, and wherein the determined predictive model comprises the engineered first feature.

5. A computer program (809), comprising instructions which, when executed on at least one processor (805), cause the at least one processor (805) to carry out the method according to any one of claims 1 to 4.

6. A computer-readable storage medium (810), having stored thereon a

computer program (809), comprising instructions which, when executed on at least one processor (805), cause the at least one processor (805) to carry out the method according to any one of claims 1 to 4.

7. A first network node (111) configured to handle a performance of a

communications network (10), the first network node (111) being further configured to operate in the communications network (10), the first network node (111) being further configured to: - determine a predictive model of a metric of a target performance of an application supported by the communications network (10), the determining being configured to use a machine-implemented learning procedure, based on:

a. a first set of crowdsourced data obtained from one or more entities (150) comprised in the communications network (10), and used to support the application, and b. data on an infrastructure of the communications network

(10),

- identify, based on a second set of the crowdsourced data obtained from the one or more entities (150), one or more operations underperforming in the communications network (10) with regard to, respectively, one or more performance thresholds, and

- perform, based on the predictive model, a simulated optimization for each of the identified one or more operations, by modifying one or more features comprised in the determined predictive model, the optimization being based on the one or more performance thresholds.

8. The first network node (111) according to claim 7, being further configured to:

- initiate providing, to a second network node (112) configured to operate in the communications network (10), one or more indications of at least one of:

a. the determined predictive model;

b. the identified one or more operations;

c. one or more first modifications of the one or more features to achieve a respective target performance for the metric; and d. an estimated improvement in the metric, based on one or more second modifications of the respective one or more features, to achieve a higher performance than a respective current performance.

9. The first network node (111) according to claim 8, wherein the one or more first modifications comprise at least one of:

a. a physical modification of the infrastructure of the communications network (10), b. a change in layer strategy, the layer being one of one or more layers used in the communications network (10) to support the application, c. a change in configuration in a network node comprised in the

communications network (10),

d. a change in a configuration related to a user profile,

e. a change in a configuration in an application server, and

f. a change in a configuration in a server of a content provider.

10. The first network node (11 1) according to any of claims 7-9, wherein to

determine further comprises to engineer at least a first feature comprised in the predictive model to improve a predictive power of the predictive model, and wherein the determined predictive model comprises the engineered first feature.

Description:
FIRST NETWORK NODE, AND METHOD PERFORMED THEREBY, FOR HANDLING A PERFORMANCE OF A COMMUNICATIONS NETWORK

TECHNICAL FIELD

The present disclosure relates generally to a first network node and methods performed thereby for handling a performance of a communications network. The present disclosure further relates generally to a computer program product, comprising instructions to carry out the actions described herein, as performed by the node. The computer program product may be stored on a computer-readable storage medium.

BACKGROUND

Computer systems in a communications network may comprise one or more network nodes, which may also be referred to simply as nodes. A network node may comprise one or more processors which, together with computer program code may perform different functions and actions, a memory, a receiving and a sending port. Network nodes may be comprised in a telecommunications network.

The telecommunications network may cover a geographical area which may be divided into cell areas, each cell area being served by another type of node, a network node or Transmission Point (TP), for example, an access node such as a Base Station (BS), e.g. a Radio Base Station (RBS), which sometimes may be referred to as e.g., gNB, evolved Node B (“eNB”),“eNodeB”,“NodeB”,“B node”, or BTS (Base Transceiver Station), depending on the technology and terminology used. The base stations may be of different classes such as e.g. Wide Area Base Stations, Medium Range Base Stations, Local Area Base Stations and Home Base Stations, based on transmission power and thereby also cell size. A cell is the geographical area where radio coverage is provided by the base station at a base station site.

One base station, situated on the base station site, may serve one or several cells. Further, each base station may support one or several communication technologies. The telecommunications network may also be a non-cellular system, comprising network nodes which may serve receiving nodes, such as wireless devices, with serving beams.

The telecommunications network may also comprise wireless devices, e.g., stations (STAs), User Equipments (UEs), mobile terminals, wireless terminals, terminals, and/or Mobile Stations (MS). Wireless devices are enabled to

communicate wirelessly in a cellular communications network or wireless communication network, sometimes also referred to as a cellular radio system, cellular system, or cellular network. The communication may be performed e.g. between two wireless devices, between a wireless device and a regular telephone, and/or between a wireless device and a server via a Radio Access Network (RAN), and possibly one or more core networks, comprised within the telecommunications network. Wreless devices may further be referred to as mobile telephones, cellular telephones, laptops, or tablets with wireless capability, just to mention some further examples. The wireless devices in the present context may be, for example, portable, pocket-storable, hand-held, computer-comprised, or vehicle-mounted mobile devices, enabled to communicate voice and/or data, via the RAN, with another entity, such as another terminal or a server.

With the advancement of technology, cellular networks, now with an increased capacity, may be able to deliver higher data throughput, that is, data volumes. The rapid increase of data volumes, the high expectation from end-users on increased bitrates, connectivity and reliability of a connection, including the use of new services and smartphones, require mobile operators to optimize their assets further and be more efficient when managing network coverage, capacity, and quality.

A problem experienced by an end-user in a cellular network, for example, between an end-user and a content server may manifest itself and impact an end- user in many ways. In the context of network design and optimization in cellular networks, the measurement approach used, to identify and therefore also address a problem, are either classified as a network-centric or an end-user-experience centric type of view.

Network centric view may be understood as referring to a system, e.g., by means of communications within or amongst network elements, that focuses on a collective set of metrics, that if improved, may impact the experience of end-users using that system. In the case of an end-user experience centric view, the quality of experience metrics that may be used, may give a true quality of experience representation of an end-user or group of end-users. A network-centric view may be referred to as a bottom up approach, that is, that usually focuses on representing the communication on a segment by segment basis, while an end-user-experience centric view may be understood to follow a top down approach, that is, that it usually focuses on representing the communication between two endpoints; between end- users, or an end-user and an application content server.

Therefore, from an analytical standpoint, it may be considered advantageous to shift from a“network” centric, to an“end-user-experience” centric point of view when working with performance optimization. Performance within a cellular network may be understood to reflect the capacity, functionality, and strategy taken.

Different applications may require and pose application specific demands on network resources. Based on various use cases, this demand is in a continuously increasing trend. Achieving optimal network resource utilization through a well- balanced load on resources may be understood as the first step to improve application-based services.

Some initial studies have been conducted with this aim. A first existing approach deals with the challenges experienced in Multi-access edge computing or Mobile Edge Computing (MEC) due to the mobility of the users within a given network. An aspect of the study was to identify the similarity of the edge servers, base stations in our terminology, based on the server density or inter-server distance, for predicting the QoS. This deals with the limitation in MEC due to user mobility and lack of historical information stored in the edge servers. So, one of the problems which is addressed in this work is the application of MEC in a real-world mobile network expecting user mobility. By service recommendation, which may be associated to a SON feature deployed in base stations, it is intended to trigger or suggest one of the following options: i) handover to a better server in real time, predicting the new one may be able to provide better QoS, or ii) prevent the handover to a new server in real time, predicting the current one may be able to provide better QoS.

A second existing approach presents a lightweight, no-reference, loadable kernel module that derives the Quality of Experience (QoE) of a video stream in transit, in real time. The method uses a specific set of parameters and human survey with a different set of video clips. Subsequently, a model is developed to predict the QoE. The problem which is addressed in this work is the realization of a lightweight module, deployable within the network core, which is capable of deriving Video Quality with substantial accuracy, considering multiple parameters in k- dimensional space. In the section‘Improving QoE’, the authors propose to adapt the bitrate, in real time, to improve the QoE. These studies are respectively focused on addressing the challenges in MEC due to the mobility of users within the network , and a lightweight module deployable within network core to derive the QoE of a video stream in real time.

To provide best in class end-user experience, with optimal usage of network resources, many operators are focusing on application performance and its impact on end-user experience. The existing approaches, however, may still result in poor performance of a network, and in turn, result in poor QoE, which may be more or less pronounced based on the requirements different applications may have.

SUMMARY

It is an object of embodiments herein to improve the performance of a communications network. It is a particular object of embodiments herein to improve the target performance of an application supported by the communications network.

According to a first aspect of embodiments herein, the object is achieved by a method, performed by a first network node. The method is for handling a

performance of a communications network. The first network node operates in the communications network. The first network node determines a predictive model of a metric of a target performance of an application supported by the communications network. The determining uses a machine-implemented learning procedure, based on: i) a first set of crowdsourced data obtained from one or more entities comprised in the communications network, and used to support the application, and ii) data on an infrastructure of the communications network. The first network node also identifies, based on a second set of the crowdsourced data obtained from the one or more entities, one or more operations underperforming in the communications network with regard to, respectively, one or more performance thresholds. The first network node then performs, based on the predictive model, a simulated

optimization for each of the identified one or more operations, by modifying one or more features comprised in the determined predictive model. The optimization is based on the one or more performance thresholds.

According to a second aspect of embodiments herein, the object is achieved by the first network node. The first network node is configured to handle the performance of the communications network. The first network node is further configured to operate in the communications network. The first network node is also configured to determine the predictive model of the metric of the target performance of the application supported by the communications network. The determining is configured to use the machine-implemented learning procedure, based on: i) the first set of crowdsourced data obtained from the one or more entities comprised in the communications network, and used to support the application, and ii) the data on the infrastructure of the communications network. The first network node is also configured to identify, based on the second set of the crowdsourced data obtained from the one or more entities, the one or more operations underperforming in the communications network with regard to, respectively, the one or more performance thresholds. The first network node is further configured to perform, based on the predictive model, the simulated optimization for each of the identified one or more operations, by modifying the one or more features comprised in the determined predictive model. The optimization is based on the one or more performance thresholds.

According to a third aspect of embodiments herein, the object is achieved by a computer program, comprising instructions which, when executed on at least one processor, cause the at least one processor to carry out the method performed by the first network node.

According to a fourth aspect of embodiments herein, the object is achieved by a computer-readable storage medium, having stored thereon the computer program, comprising instructions which, when executed on at least one processor, cause the at least one processor to carry out the method performed by the first network node.

By the first network node determining the predictive model, identifying the one or more operations underperforming in the communications network, and then performing the simulated optimization for the identified one or more operations, the first network node is thereby enabled to quantify the required amount of network optimization to achieve a desired level of application-specific performance, e.g., quality of experience. The first network node is also enabled to recommend actions in terms of required network optimization that may be able to solve existing network problems or, for example, improve the current QoE for a specific service or application, to acquire the best possible application performance with optimal usage of network resources. Therefore, the first network node makes it possible to significantly enhance end-user experience and optimize, or balance, the use of systems and resources in the communications network. BRIEF DESCRIPTION OF THE DRAWINGS

Examples of embodiments herein are described in more detail with reference to the accompanying drawings, according to the following description.

Figure 1 is a schematic diagram illustrating a non-limiting example of a

communications network, according to embodiments herein.

Figure 2 is a flowchart depicting a method in a first network node, according to

embodiments herein.

Figure 3 is a schematic diagram depicting a non-limiting particular example of the method performed by the first network node, according to embodiments herein.

Figure 4 is a schematic diagram depicting aspects of problem identification,

according to embodiments herein.

Figure 5 is a schematic diagram depicting aspects of problem identification, such as overshooting and low dominance according to embodiments herein.

Figure 6 is a schematic diagram depicting a non-limiting particular example of

aspects of the method performed by the first network node, according to embodiments herein.

Figure 7 is a schematic diagram depicting a non-limiting particular example of

aspects of the method performed by the first network node, according to embodiments herein.

Figure 8 is a schematic block diagram illustrating embodiments of a first network node, according to embodiments herein.

DETAILED DESCRIPTION

As part of the development of embodiments herein, one or more problems with the existing technology will first be identified and discussed.

As stated earlier, the primary objective of embodiments herein may be understood as to find and unlock the underlying factors driving application performance. Using an exploratory level analysis of the data sources, improvement may be made, with an emphasis on end-user perceived quality. By balancing the conditions amongst coverage, capacity, and quality within the network, optimum performance may be achieved. The three International Telecommunication Union- Telecommunications (ITU-T) areas accessibility, retainability, and integrity complemented with mobility, utilization, and availability are the main six areas for performance monitoring on a macro level. Existing crowdsourcing data from crowdsourcing providers, may provide insights into the application performance and associated end-user experience. However, the available crowdsourcing data and existing solutions do not translate this information into actionable work items, that is, required network optimization activities, in order to achieve their performance target.

Vendors or owners of crowdsourcing data, at present, provide insights on the user experience only. For example, for a given network, they address following questions, not limited to: a) what is the current end-user experience in terms of various QoE?, b) where, geographically, is end-user experience bad?, c) how are the competitors performing?.

However, none of the existing solutions provide insights in terms of required network optimization that may be able to solve existing network problems or improve the current QoE for a specific service or application to acquire best possible application performance with optimal usage of network resources by providing performance gap insights.

Embodiments herein provide for a method that serves this purpose, With the use of the new approach disclosed herein, it may be possible to recommend actions aimed at significantly enhancing end-user experience and optimizing, or balancing, the use of systems and network resources.

As summarized overview, embodiments herein may be understood to relate to a method that uses machine learning to model the crowdsourcing data and quantify the required amount of network optimization to achieve the desired level of application-specific quality of experience, delivering the best possible application performance and end-user satisfaction to meet the performance goals of the operators.

The actions in an optimization service may need to be prioritized to deliver information that also enables improvements in quality of service aspects.

As a summarized overview, embodiments herein may be understood to relate to predictive modelling of crowdsourcing data and quantification of required network optimization for application QoE improvement.

Several embodiments and examples are comprised herein. It should be noted that the embodiments and/or examples herein are not mutually exclusive.

Components from one embodiment or example may be tacitly assumed to be present in another embodiment or example and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments and/or examples.

Figure 1 depicts two non-limiting examples, in panels“a” and“b”, respectively, of a communications network 10, in which embodiments herein may be implemented. In some example implementations, such as that depicted in the non limiting example of Figure 1a), the communications network 10 may be a computer network. In other example implementations, such as that depicted in the non-limiting example of Figure 1b), the communications network 10 may be implemented in a telecommunications network 100, sometimes also referred to as a cellular radio system, cellular network or wireless communications system.

In some examples, the telecommunications network 100 may for example be an application or content provider-oriented network, a network such as 5G system, or Next Gen network or an Internet service provider (ISP)-oriented network, or application or content provider-oriented network. The telecommunications network 100 may also support other technologies, such as a Long-Term Evolution (LTE) network, e.g. LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), LTE Half-Duplex Frequency Division Duplex (HD-FDD), LTE operating in an unlicensed band, Wideband Code Division Multiple Access (WCDMA), Universal Terrestrial Radio Access (UTRA) TDD, GSM/Enhanced Data Rate for GSM

Evolution (EDGE) Radio Access Network (GERAN) network, Ultra-Mobile

Broadband (UMB), EDGE network, network comprising of any combination of Radio Access Technologies (RATs) such as e.g. Multi-Standard Radio (MSR) base stations, multi-RAT base stations etc., any 3rd Generation Partnership Project (3GPP) cellular network, Wireless Local Area Network/s (WLAN) or WFi network/s, Worldwide Interoperability for Microwave Access (WiMax), IEEE 802.15.4-based low-power short-range networks such as IPv6 over Low-Power Wreless Personal Area Networks (6LowPAN), Zigbee, Z-Wave, Bluetooth Low Energy (BLE), or any cellular network or system. In some examples, the telecommunications network 100 may comprise network nodes which may serve receiving nodes, such as wireless devices, with serving beams.

The communications network 10 comprises a plurality of network nodes, whereof a first network node 111 and a second network node 112 are depicted in the non-limiting examples of Figure 1. Each of the first network node 111 and the second network node 112 may be understood as a first computer system and a second computer system. Each of the first network node 111 and the second network node 112 may be implemented as a standalone server in e.g., a host computer in the cloud 110, as depicted in the non limiting example of Figure 1 b). In other examples, any of the first network node 111 and the second network node 112 may be a distributed node or distributed server, such as a virtual node in the cloud 110, and may perform some of its respective functions being locally, e.g., by a client manager, and some of its functions in the cloud 110, by e.g., a server manager. In other examples, any of the first network node 111 and the second network node 112 may perform its functions entirely on the cloud 110, or partially, in collaboration or collocated with a radio network node. Yet in other examples, any of the first network node 111 and the second network node 112 may also be implemented as processing resource in a server farm. Any of the first network node 111 and the second network node 112 may be under the ownership or control of a service provider, or may be operated by the service provider or on behalf of the service provider.

In the particular examples depicted in Figure 1 , any of the first network node 111 and the second network node 112 may be a core network node. In 5G, for example, any of the first network node 111 and the second network node 112 may be located in the OSS (Operations Support Systems).

The first network node 111 may be understood to have the capability to perform machine-implemented learning procedures, which may be also referred to as“machine learning”. The model used for prediction may be understood as a predictive model, e.g., a predictive regression model such as Random Forest. In some embodiments, the system that may be used for training the model and the one used for prediction may be different. The system used for training the model may require more computational resources than the one to use the built/trained model to make predictions. Therefore, the first network node 111 may, for example, support running python/Java.

The second network node 112 may be another core network node, or, in some examples not depicted in Figure 1 , a radio network node, such as the radio network node described below.

In some examples of the communications network 10, which are not depicted in Figure 1 , the first network node 111 and the second network node 112 may be co located, or be a same node. The communications network 10 has users, e.g., subscribers. Any of the users may access the communications network 10, respectively, via a communication device 140, as depicted in the non-limiting example scenario of Figure 1. The communication device 140 may be a UE or a Customer Premises Equipment (CPE) which may be understood to be enabled to communicate data, with another entity, such as a server, a laptop, a Machine-to-Machine (M2M) device, device equipped with a wireless interface, or any other radio network unit capable of communicating over a wired or radio link in a communications system such as the communications network 10. The communication device 140 may be also e.g., a mobile terminal, wireless terminal and/or mobile station, mobile telephone, cellular telephone, or laptop, just to mention some further examples. The communication device 140 may be, for example, portable, pocket-storable, hand-held, computer-comprised, a sensor, camera, or a vehicle-mounted mobile device, enabled to communicate voice and/or data, via a RAN, with another entity, such as a server, a laptop, a Personal Digital Assistant (PDA), or a tablet computer, sometimes referred to as a tablet with wireless capability, or simply tablet, a Machine-to-Machine (M2M) device, a device equipped with a wireless interface, such as a printer or a file storage device, modem, Laptop Embedded Equipped (LEE), Laptop Mounted Equipment (LME), USB dongles or any other radio network unit capable of communicating over a wired or radio link in the communications network 10. The communication device 140 may be enabled to communicate wirelessly in the communications network 10. The communication may be performed e.g., via a RAN and possibly one or more core networks, comprised within the communications network 10.

The communications network 10 comprises one or more entities 150 that are used to support an application supported by the communications network 10. In the particular non-limiting example of Figure 1 , the one or more entities 150 comprises a radio network node. The communications device 140 may access the network 10 via a radio network node, e.g., an access node, or radio network node, such as, for example, the radio network node, depicted in Figure 1 b). The telecommunications network 100 may cover a geographical area, which in some embodiments may be divided into cell areas, wherein each cell area may be served by a radio network node, although, one radio network node may serve one or several cells. The radio network node may be e.g., a gNodeB. That is, a transmission point such as a radio base station, for example an eNodeB, or a Home Node B, a Home eNode B or any other network node capable to serve a wireless device, such as the communications device 140 in the communications network 10. The radio network node may be of different classes, such as, e.g., macro eNodeB, home eNodeB or pico base station, based on transmission power and thereby also cell size. In some examples, the radio network node may serve receiving nodes with serving beams. The radio network node may support one or several communication technologies, and its name may depend on the technology and terminology used. The radio network node may be directly connected to one or more core networks in the telecommunications network 100. In other examples not depicted in Figure 1 , However, in other examples, the one or more entities 150 may comprise of any network node such as the radio network node, or a user device such as the communication device 140, or an application server or content provider server that may comprise network or service or application specific configuration information which may have influence on signalling and/or user-plane data pertaining to a user session.

The first network node 111 is configured to communicate with the second network node 112 over a first link 161 , e.g., a radio link, an infrared link, a wired link. The first link 161 may be comprised of a plurality of individual links. The first network node 111 is configured to communicate within the communications network 10 with radio network node over a second link 162, e.g., a radio link, an infrared link, or a wired link. The second link 162 may be comprised of a plurality of individual links. Each communication device 140 may be understood to

communicate with the communications network 10 via the radio network node over a respective link, which is not depicted in Figure 1 to simplify the Figure.

Any of the first link 161 , the second link 162 and any of the respective links may be a direct link or it may go via one or more computer systems or one or more core networks in the communications network 10, which are not depicted in Figure 1 , or it may go via an optional intermediate network. The intermediate network may be one of, or a combination of more than one of, a public, private or hosted network; the intermediate network, if any, may be a backbone network or the Internet; in particular, the intermediate network may comprise two or more sub-networks, which is not shown in Figure 1.

In general, the usage of“first”,“second”, etc. herein may be understood to be an arbitrary way to denote different elements or entities, and may be understood to not confer a cumulative or chronological character to the nouns they modify.

Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

Embodiments of a method, performed by the first network node 111 , will now be described with reference to the flowchart depicted in Figure 2. The method is for handling a performance of the communications network 10. The first network node 111 operates in the communications network 10.

Several embodiments are comprised herein. In some embodiments all the actions may be performed. In some embodiments, one or more actions may be optional. In Figure 2, optional actions are indicated with dashed lines. It should be noted that the examples herein are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. Some actions may be performed in a different order than that shown in Figure 2.

Action 201

In this Action 201 , the first network node 111 determines a predictive model of a metric of a target performance of an application supported by the communications network 10. The determining in this Action 201 uses a machine-implemented learning procedure, based on: i) a first set of crowdsourced data obtained from the one or more entities 150 comprised in the communications network 10, and used to support the application, and ii) data on an infrastructure of the communications network 10.

Determining in this Action 201 may be understood as e.g., calculating or deriving. In some examples, determining in this Action 201 may be understood as retrieving, e.g., from a memory source.

A predictive model may be understood as a mathematical model or function that aims to best fit a set of data, such that inputting observed data, and output predicted or estimated data with a certain level of accuracy

The application may be understood as a software program located in the communication device 140 and that may be used by an end-user to access the services provided by the telecommunication network 100 or ISP service providers or application vendors or content providers.

A performance of an application may be understood herein as quality and integrity of initiation, quality and integrity of channel/communication and ability to maintain the channel/communication and the level of overall end-user experience. A target performance may be understood to refer to a satisfactory level of end-user experience associated with an application or service.

The metric may be understood to be a variable that indicates a measure of performance of the communications network 10. An example of the metric may be throughput. In some embodiments, the metric may be related to Quality of

Experience (QoE). The QoE may be defined as a translation of a measure of end- user perception and experience, while accessing or using a service or application, into a quantitative scale. For example, these measures may be VMOS, Video Quality, MOS, Webpage Download Time, Voice Drop Call Rate, etc.

Data sources for the machine-implemented learning procedure

Embodiments herein may use two primary data sources: data on the infrastructure of the communications network 10, also referred to herein as network node metadata, and crowdsourced data, also referred to herein as crowdsourcing data. Network node metadata comprises information related to infrastructure of the communications network 10, e.g., site locations, height, azimuth and tilt of serving cells, configured transmit power, etc. The infrastructure of the communications network 10 may be understood as any hardware component of the communications network 10, as described for example on Figure 1 , either explicitly or implicitly for supporting the performance of the operations described herein.

Crowdsourcing in this context may be defined as a collection of metrics either actively or passively obtained from any end-user related triggered services or interactions, e.g., web, social media, streaming, voice, in the communications network 10. In general, crowdsourcing data may originate from the endpoints, e.g., the c device 140, content delivery, server, or any intermediate communication point(s) within the end-to-end chain. The information collected through

crowdsourcing data that comprises, but is not limited to, e.g., end-user state, device state/information, radio access performance, application/connectivity/service performance, location information, e.g., cell served, geographical location, server information, procedure timestamp information, which is correlated with the application or service performance, may be considered of particular interest. In some context, an end-user may not even, at least not directly, be involved but instead, it may be the communication, e.g., business interactions, social media, mobile services, from a group of entities, that is, a crowd, that may be captured and analyzed. This information, combined with the data on the infrastructure of the communications network 10, may in return be used to understand the pain points and identify gaps within the communication channel.

The machine-implemented learning procedure: Algorithms

The machine implemented procedure may be any algorithm or set of algorithms that may perform a specific task in the absence of explicit instructions, relying on an analysis of patterns and inference. It is seen as a subset of artificial intelligence. The machine implemented procedure may be understood to build a mathematical model of sample data, such as the first set of crowdsourced data and the data on an infrastructure of the communications network 10, in order to make predictions or decisions without being explicitly programmed to perform the task.

The determining in this Action 201 with the machine implemented procedure may be understood to comprise one or more iterations. A first group of iterations may be understood as a training phase of the model. Subsequently, the trained model may be used again with subsequent data to further improve the accuracy of the predictive model. The training phase of the predictive model may be understood to result in a first predictive model. Subsequent iterations of the predictive model with subsequent data may be understood to result in a second predictive model, third predictive model, etc. Accordingly, any reference herein to“the predictive model” herein may be understood to refer to one or more iterations of the model.

In some examples, the Random Forest regressor may be used in the determining of this Action 201 to model the crowdsourcing data to predict the application QoE with acceptable accuracy, and it may be later also used in Action 203 to evaluate or quantify the possible gain through suggested network

optimization. The decision for selecting a specific algorithm may be based on a trial and error approach. K-fold cross validation may be, for example, applied to three regression models: Support Vector, Decision Tree and Random Forest, and each time the mean and standard deviation of the accuracy factor may be measured.

This may then be used to decide on a suitable model. The features used in the predictive model may be available in almost all the crowdsourcing data, e.g., connectivity metric, transport metric, service metric, measurement location, air interface measurements like signal strength, quality, serving cell, technology, band, etc. These set of information may be combined with sites metadata to derive the actual feature set. The actual feature set may be understood as the set of transformed features used for modelling. Some of the information available in crowdsourcing data may not have relevance with application or service performance unless combined with sites meta data and transformed into meaningful features. To decide on the list of features used in the model, the following approach may be adopted. Firstly, a data source used may comprise some information that may apparently not influence the dependent variable, e.g., application QoE, if considered in isolation and without any transformation. For example, measurement location. Unless it is converted to DISTANCE FROM SERVING CELL and RELATIVE ORIENTATION W.R.T SERVING CELL, the model may not be able to relate this feature with the dependent variable. Therefore, in some embodiments, feature engineering may be applied. The site metadata may be merged with the crowdsourced data so that the raw information may be converted into meaningful features based on a particular domain knowledge. In accordance with the foregoing, in some embodiments, the determining in this Action 201 may further comprise engineering at least a first feature comprised in the predictive model to improve a predictive power of the predictive model. In such embodiments, the determined predictive model may comprise the engineered first feature.

Secondly, at the initial attempt, all the features may be used, which may influence the dependent variable in the model. Initially, it may be advantageous to not filter out any feature knowing the fact that some of the features may have a high correlation.

With this iteration, it may be possible to determine the feature importance, and to start eliminating the features having the lowest significance. However, there may be situations where the two features have very similar significance. In those situations, different combinations may be tested to maximize the accuracy and decision taken based on domain knowledge and understanding of the influence of the feature on the dependent variable.

At the same time, the accuracy of the predictive model achieved with each set of features used may be measured in each iteration. Finally, once an acceptable accuracy and accurate prediction in the lower QoE ranges is achieved, the determining in this Action 201 may be concluded, until that is, additional source data may be obtained to potentially increase the accuracy of the predictive model in further iterations.

Table 1 depicts the list of features that may be used in a particular non-limiting example of the predictive model, according to embodiments herein, as well as their description.

Table 2 describes the direct or indirect influence of the features listed in Table 1 on service or application QoE, in a particular non-limiting example. While applying feature engineering, the following strategy may be adopted: a) different features may be derived that may have a significant impact on end-user experience, and it may be based on domain expertise; Domain expertise may be understood to refer to the knowledge and information of the telecommunication product functionality, associated protocols and procedures as described by the standard bodies; b) also, the problem and solution associated with each of the features may be considered.

Table 2

By performing this Action 201 , the first network node 111 may be enabled to, based on a considerable amount of crowdsourcing data, prepare a machine learning model, that is, the predictive model, and predict the metric of the target performance of the application supported by the communications network 10, e.g., the user experience or application level Quality of Experience (QoE). This predictive model, that is, this initial version of the predictive model, may then be used to evaluate and compare the results achieved through virtual optimization in Action 203 in order to improve the performance of the communications network 10.

Action 202

In this Action 202, the first network node 111 identifies, based on a second set of the crowdsourced data obtained from the one or more entities 150, one or more operations underperforming in the communications network 10 with regard to, respectively, one or more performance thresholds. That is, the one or more operations may be identified as underperforming if their performance is below a respective performance threshold of the one or more thresholds.

The one or more thresholds may be understood to be configurable based on a desired or target performance.

An operation may be understood as a measure of the performance of network nodes that may have direct or indirect influence on the target application or service performance. Examples of operations may be: service provided by a cell, coverage provided by cells, radio quality, mobility, etc...

The problem identification module in this Action 202 may be understood as a group of functions that may be used to identify problems and root-cause of poor performance observed in the second set of the crowdsourced data. The second set of the crowdsourced data, may be entirely different from the first set of the crowdsourced data. For example, the second set of the crowdsourced data may comprise additional data, such as more recent data, if Action 201 was performed considerably earlier in time. The predictive model may be built and trained with data that may have been collected in the recent past for a considerable period of time.

For example, it may be collected for a duration of a month. This data may be used only to build and train the model in Action 201. The second set of the crowdsourced data may have been collected more recently, e.g., for the current week or previous week, and used in Action 202 and Action 203, then and output of Action 203 then fed to Action 204, as will be described later.

The problem identification module may comprise a set of functions

responsible for identification of different issues in the communications network 10. Each of these functions may be designed to identify each type of problem-based on the available set of information in crowdsourcing data. In some examples, the identifying in this Action 202, may be further based on a second the data on the infrastructure of the communications network 10.

For example, overshooting cells may be identified based on a function that identifies all the serving cells at a given geographical location and based on the distance from the serving cell and distance of first tier neighbors of each serving cell, it may detect the overshooting cell.

Similarly, another set of functions may identify operational issues, low dominance, high interference, and mobility strategy related issues. Based on information available with the second set of crowdsourced data, additional functions may be added to identify other types of network issues.

In some examples, this Action 202 may also comprise classifying the identified underperforming one or more operations, according to type of operation to be optimized. For example, overshooting and interference issues may be classified as part of a physical operations, that is operations subject to a physical issues, whereas tuning mobility thresholds or parameters may be classified as layer strategy operations, that is operations relating to layer strategy.

The classification may be performed with classification algorithms.

Classification algorithms may be also tested and selected, such as the Decision Tree and Random Forest. The latter may generate an acceptable result. However, based on the problem that is solved, the limitation may be understood to be may be the continuous nature of the dependent variable. When the dependent variable is digitized to apply the classification algorithm, it may impact the subsequent method of quantification of achievable gain, as it may be understood to be sensitive to the ranges of predicted result.

By identifying the one or more operations underperforming in the

communications network 10 in this Action 202, the first network node 111 may then be enabled to resolve the identified problems with the one or more operations with a virtual optimization, as described in the next Action 203.

Action 203

In this Action 203, the first network node 111 performs, based on the predictive model determined in Action 201 , a simulated optimization for each of the identified one or more operations in Action 202, by modifying one or more features comprised in the determined predictive model. The optimization is based on the one or more performance thresholds.

A simulated optimization may be understood as a virtual optimization. In other words, the identified problems in Action 202 may be resolved in this Action 203 using a virtual optimization for each type of problem in a sequential manner, and may yield an optimized set of crowdsourced data, which may be referred to as a test crowdsourcing data. That the simulated optimization is performed based on the predictive model may be understood to mean that this optimized test crowdsourcing data may then be fed to the determined predictive model in Action 201 , as represented by the arrow from Action 203 to Action 201 in Figure 2, and the resulting application QoE may be predicted for comparison with the baseline values and quantification of the achieved improvement. The baseline values may be understood to refer to the initial values of the metric that may have been observed in the communications network 10 based on the second set of crowdsourced data before virtual optimization is performed.

To quantify the possible improvement in application QoE or user experience, a virtual optimization of the communications network 10 may be performed based on the second set of crowdsourced data. This virtual optimization may be defined as a set of logical modifications, see Figure 6 described later, in the baseline

crowdsourcing data for each type of problem identified in Action 202, that may be expected when optimization of the communications network 10 may be performed in the real world. That is, changes in the value of the various measures or fields captured in crowdsourcing data pertaining to the nodes, e.g., a radio node or cell, for which virtual optimization may have been performed.

As mentioned earlier, the problems identified in Action 202 may be grouped in terms of actionable work items. For example, overshooting and interference issues may be grouped as part of a physical optimization, as most of these problems may be resolved through physical optimization, whereas tuning mobility thresholds or parameters may resolve layer strategy related issues. So, this type of issues may be added as a separate optimization activity, that is, layer strategy. The above two used cases have been implemented based on the available set of information, that is, the second set of crowdsourced data.

As a non-limiting example, with an example test dataset, the following virtual optimization results depicted in Table 3 may be achieved. In this example, application throughput has been considered as target performance metric which may be understood as one of the QoE indicators. The test data, collected for a very recent time period, shows present, considered as baseline, mean application throughput is 6.26 Mbps. Using Action 202, multiple network performance related issues have been identified, namely, the one or more operations underperforming in the communications network 10 in this particular example. To resolve these issues, virtual optimization workflow is applied on the test data set. This helps to quantify that: 1) with physical optimization of 44 cells, a 1.8% improvement may be achieved in application throughput, 2) with a change in layer strategy, here, extending the capacity layer for 32 cells, a 2.5% improvement may be achieved in application throughput.

Table 3 Once an acceptable accuracy and accurate prediction in the lower QoE ranges is achieved, the iterations with the optimized data through the determining in Action 201 may be concluded.

By performing the simulated optimization in this Action 203 for each of the identified one or more operations in Action 202, the first network node 111 may be enabled to predict a gain in the performance of the metric, in comparison with the original or baseline, that is, non-optimized, values, and quantify the improvement that may be achieved, if such optimization were to be realized in the real world. In particular, the simulated optimization in this Action 203 may enable the first network node 111 to identify or determine one or more first modifications of the one or more features to achieve a respective target performance for the metric. In some examples, the simulated optimization in this Action 203 may enable the first network node 111 to identify or determine an estimated improvement in the metric, based on one or more second modifications of the respective one or more features, to achieve a higher performance than a respective current performance. In other words, while there may be an initial target performance which may be achieved with the one or more first modifications, embodiments herein may enable to identify an even further gain, that is, an even higher or better performance, that may be achieved, if the two or more second modifications were to be implemented.

In some embodiments, the one or more first modifications may comprise at least one of: a) a physical modification of the infrastructure of the communications network 10, b) a change in layer strategy, the layer being one of one or more layers used in the communications network 10 to support the application, c) a change in configuration in a network node comprised in the communications network 10, d) a change in a configuration related to a user profile, e) a change in a configuration in an application server, and f) a change in a configuration in a server of a content provider.

The one or more second modifications may be understood to correspond to similar type of modifications as the one or more first modifications, however, with at least partially different values. The one or more second modifications may therefore comprise at least one of: a) a physical modification of the infrastructure of the communications network 10, b) a change in layer strategy, the layer being one of one or more layers used in the communications network 10 to support the application, c) a change in configuration in a network node comprised in the communications network 10, d) a change in a configuration related to a user profile, e) a change in a configuration in an application server, and f) a change in a configuration in a server of a content provider.

Action 204

In this Action 204, the first network node 11 1 may initiate providing, to the second network node 1 12 operating in the communications network 10, one or more indications of at least one of: i) the determined predictive model in Action 201 , ii) the identified one or more operations in Action 202, iii) the one or more first

modifications of the one or more features to achieve a respective target

performance for the metric; and iv) the estimated improvement in the metric, based on the one or more second modifications of the respective one or more features, to achieve the higher performance than the respective current performance.

Initiating providing may be understood as e.g., triggering, enabling, or starting the providing by the first network node 11 1 itself, or by another network node.

Providing may be understood as e.g., sending, for example, via the first link 161.

For example, the first network node 11 1 may, according to this Action 204, itself provide, or enable another network node in the communications network 10 to provide, an instruction to a radio network node in the telecommunications network 100, to change a number of antennas used, a power of transmission, etc., in order to correct overshoot. The first network node 1 11 may for example enable a core network node in the communications network 10 to provide such an instruction to the radio network node.

Figure 3 is a schematic workflow overview of a non-limiting example of a method performed by the first network node 11 1 , according to embodiments herein. As schematically represented in Figure 3, the method may be understood to comprise three modules: the problem identification module of Action 202, the virtual optimization module of Action 203 and the ML model module of Action 201. Based on a considerable amount of crowdsourcing training data 301 , that is, the first set of crowdsourced data, an ML model, that is, the predictive model, may be prepared in Action 201 to predict the user experience or application level QoE. This predictive model may later be used to evaluate and compare the results achieved through virtual optimization in Action 203. Based on the second set of the crowdsourced data 302 obtained from the one or more entities 150, here from application vendors, and based on the data 303 on the infrastructure of the communications network 10, the one or more operations underperforming in the communications network 10 are identified in Action 202, with regard to, respectively, the one or more performance thresholds. In this case, the identification is based on whether the data indicates at 304 that the target QoE is below a threshold, with respect to a desired application QoE 305. At 203, the first network node 111 performs, based on the predictive model, a simulated optimization for each of the identified one or more operations, by modifying one or more features comprised in the determined predictive model. This optimized test crowdsourcing data may then be fed to the determined predictive model, and the result application QoE may be predicted at 306 for comparison with the baseline values and quantification of the achieved improvement 307. For example, as stated earlier, overshooting and interference issues may be grouped as part of a physical optimization 308, as most of these problems may be resolved through physical optimization, whereas tuning mobility thresholds or parameters may resolve layer strategy related issues 309. Therefore, this type of issues may be added as a separate optimization activity, layer strategy. The problems identified in Action 202 may be grouped in terms of actionable work items, and optimized individually, in a stepwise fashion, iterating the predictive model. Once all work items have been implemented at 310, the method may conclude.

Figure 4 is a schematic illustration of the identifying of Action 202 performed according to embodiments herein. As depicted in Figure 4, the second set of crowdsourced data 302 from application vendors and the data on the infrastructure of the communications network 10, represented as network infrastructure data, may be used to identify problems in the communications network 10, that is, the one or more operations underperforming in the communications network 10, as well as for classifying the problematic features causing the underperformance. The graph on the right side of Figure 4 shows a result of the Root Cause Analysis (RCA) that may be performed in the identifying the one or more operations underperforming in the communications network 10 of Action 202. The graph depicts the count for different root causes identified and classified, such as closest cell not serving, overshooting, no dominance due to high overlap, interference and mobility strategy.

Figure 5 is a schematic illustration of some aspects of the identifying the one or more operations underperforming in the communications network 10 of Action 202, performed according to embodiments herein. In particular, panel a) depicts how an underperforming operation such as dominance may be detected due to high overlap between a first cell and a neighboring cell with a dominance of +3dB. In panel a), the circle on the left depicts the coverage area of Cell-1 , and the circle on the right depicts the Coverage area of Cell-2. The striped intersection area depicts the coverage overlap area of Cell-1 and Cell-2, where abs(S ceii -i - S ceii-2 ) <= 3 dB, S ceii -i and S ceii denote the signal strength of Cell-1 and Cell-2, respectively. Therefore, this indicates none of the serving cells are dominant. The reported signal strengths of the respective cells are very close to each other. Performance of an optimization to make either of the serving cells as a dominant serving cell appears to be necessary. Panel b) depicts how an overshooting cell may be detected. In panel b), the small depicted rectangle at the top represents a geographical area where a target performance metric is below threshold. Cell-1 , depicted by a circle at the top, and Cell-5, depicted by a circle at the top, are serving at the location, as indicated by the arrows. As Cell-5 is serving far from the location, and there are three cells, that is, Cell-2, Cell-3, and Cell-4 in between, as depicted by the three circles enjoined in the larger rectangle, which are much closer to the location, Cell-5 may be identified as an overshooting cell.

Figure 6 is a schematic illustration of a non-limiting example of a virtual optimization workflow performed by the first network node 11 1 , according to embodiments herein. As schematically represented in Figure 6, based on the second set of the crowdsourced data 302 obtained from the one or more entities 150, here from application vendors, and based on the data 303 on the infrastructure of the communications network 10, the one or more operations underperforming in the communications network 10 are identified in Action 202, with regard to, respectively, the one or more performance thresholds. At 203, the first network node 1 11 performs, based on the predictive model, a simulated optimization for each of the identified one or more operations, by modifying one or more features comprised in the determined predictive model, e.g., such as the features depicted in Table 1. This optimized test crowdsourcing data may then be fed to the determined predictive model, and the result application QoE or target Key Performance Indicator (KPI) may be predicted at 306 for comparison at 601 with the baseline values, and generation of statistics and reports at 602, such as quantification of the achieved improvement 307. For example, as stated earlier, overshooting and interference issues may be grouped as part of a physical optimization 308, as most of these problems may be resolved through physical optimization, whereas tuning mobility thresholds or parameters may resolve layer strategy related issues 309. Since the optimization is performed underperforming operation by underperforming operation, at 602, it is checked if all use cases have been executed. If not, the method iterates again from Action 203. Once all work items have been implemented at 310, the method may conclude.

Figure 7 is a schematic illustration of the determining of Action 201 performed according to embodiments herein. As depicted in the Figure, the predictive model used in this example method has been prepared and trained using the first set of crowdsourced data 301 , here from application vendors, and the data 303 on the infrastructure of the communications network 10, represented as network infrastructure data, to predict the application or service QoE. The graph on the right side of Figure 7 shows the test results, depicted as“y_test”, versus the prediction results, depicted as“y_pred”, for predicting goodput by test samples. To build and train the predictive model, the first set of crowdsourced data is used. To test or validate the predictive model, that is, to check whether it is capable of predicting the dependent variable with acceptable accuracy, the whole first set of crowdsourced data may be divided into two parts, e.g., 80% and 20%. The first part, the training set, may be used to train the model and the second part, the test set, may be used to test the capability of the predictive model in terms of prediction accuracy. In the example depicted in Figure 7, the test result may be understood to refer to the comparison of the values y_test and y_pred, where y_test is the actual (known) values of the dependent variable in the test set and y_pred is the predicted values of the dependent variable by the predictive model. The test sample may be understood to refer to the set of independent variables associated with the dependent variable. Here, the test samples are fed to the predictive model to predict the dependent variable. Each instance of such set may be called a test sample. The instances of independent variables in test set, e.g., 20% of the first set of crowdsourced data, may be termed as“Test Samples”. Application bitrate may be understood to refer to a target performance metric. From Figure 7, it may be concluded that the predictive model is capable of predicting the dependent variable with an acceptable accuracy. Ninety-seven percent (R 2 =0.97) of the variance in the dependent variable is predictable from the independent variable using the predictive model. As a summary of the description provided herein, embodiments herein may be understood to enable modelling the crowdsourcing data to predict the target performance metric with an acceptable accuracy factor. Furthermore, the predictive model may be used to quantify the achievable improvement in the target performance metric through virtual optimization based on the identified network performance issues grouped by type of action items in a sequential and logical manner.

The described method may also be adaptable to various crowdsourcing data sources that provide different types of performance metrics. Embodiments herein may be understood to comprise the following distinct attributes. First, to model the crowdsourcing data which already comprises the measured application and network level performance indicators and application-specific QoE. Second, to identify, from the perspective of end-user or impacting end-user experience, the poor performing geographical areas and network nodes with specific problems that may be addressed through network optimization. Third, to quantify the required network optimization to achieve a desired level of QoE, translating to actionable work items on network nodes, e.g., physical optimization, change in layer strategy etc. All the decisions may be understood to not be performed in real time, and therefore to not require changes in real time.

Certain embodiments may provide one or more of the following technical advantage(s). Embodiments herein enable to improve the performance of the communications network 10, as they are focused towards yielding the best possible application performance by providing performance gap insights and necessary network-wide optimization. A further advantage of embodiments herein is that they enable integration and adaptability. Embodiments herein may be understood to use one or more ML algorithms combined with domain knowledge, which may be understood to be practically adaptable to various crowdsourcing data. Different blocks and functionality, described and used in embodiments herein, may be based on Python and may be easily integrated and deployed with existing solutions or data sources with minimal adaptation for any network, such as, a combination of technology, available capacity, and infrastructure. Yet another advantage of embodiments herein is that they enable improved efficiency. With the automation embodiments herein may also result into cost and time savings. Figure 8 depicts two different examples in panels a) and b), respectively, of the arrangement that the first network node 111 may comprise to perform the method actions described above in relation to Figure 2. In some embodiments, the first network node 111 may comprise the following arrangement depicted in Figure 8a. The first network node 111 is configured to handle the performance of the communications network 10. The first network node 111 is further configured to operate in the communications network 10.

Several embodiments are comprised herein. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the first network node 111 , and will thus not be repeated here. For example, the metric of the target performance of the application supported by the communications network 10, may be e.g., the user experience or application level Quality of Experience (QoE).-ln Figure 8, optional modules are indicated with dashed boxes.

The first network node 111 is configured to, e.g. by means of a determining unit 801 within the first network node 111 configured to, determine the predictive model of the metric of the target performance of the application supported by the communications network 10. The determining is configured to use the machine- implemented learning procedure, based on: i) the first set of crowdsourced data obtained from the one or more entities 150 comprised in the communications network 10, and used to support the application, and ii) the data on the infrastructure of the communications network 10.

In some embodiments, to determine may further comprise to engineer at least the first feature comprised in the predictive model to improve the predictive power of the predictive model. In such embodiments, the determined predictive model may comprise the engineered first feature.

The first network node 111 is configured to, e.g. by means of an identifying unit 802 within the first network node 111 configured to, identify, based on the second set of the crowdsourced data obtained from the one or more entities 150, the one or more operations underperforming in the communications network 10 with regard to, respectively, the one or more performance thresholds. The first network node 11 1 may be further configured to, e.g. by means of a performing unit 803 within the first network node 1 11 configured to, perform, based on the predictive model, the simulated optimization for each of the identified one or more operations, by modifying one or more features comprised in the determined predictive model. The optimization is based on the one or more performance thresholds.

In some embodiments, the first network node 11 1 may be further configured to, e.g. by means of an initiating unit 804 within the first network node 11 1 configured to, initiate providing, to the second network node 1 12 configured to operate in the communications network 10, the one or more indications of at least one of: i) the determined predictive model; ii) the identified one or more operations; iii) the one or more first modifications of the one or more features to achieve a respective target performance for the metric; and iv) the estimated improvement in the metric, based on the one or more second modifications of the respective one or more features, to achieve the higher performance than the respective current performance.

In some embodiments, the one or more first modifications may comprise at least one of: a) the physical modification of the infrastructure of the communications network 10, b) the change in layer strategy, the layer being the one of one or more layers used in the communications network 10 to support the application, c) the change in configuration in the network node comprised in the communications network 10, d) the change in the configuration related to a user profile, e) the change in the configuration in the application server, and f) the change in the configuration in the server of the content provider.

Other modules may be comprised in the first network node 11 1.

The embodiments herein in the first network node 11 1 may be implemented through one or more processors, such as a processor 805 in the first network node 11 1 depicted in Figure 8a, together with computer program code for performing the functions and actions of the embodiments herein. A processor, as used herein, may be understood to be a hardware component. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the first network node 1 11. One such carrier may be in the form of a CD ROM disc. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on a server and downloaded to the first network node 11 1. The first network node 111 may further comprise a memory 806 comprising one or more memory units. The memory 806 is arranged to be used to store obtained information, store data, configurations, schedulings, and applications etc. to perform the methods herein when being executed in the first network node 111.

In some embodiments, the first network node 111 may receive information from, e.g., the second network node 112, the one or more entities 150, or another network node in the communications network 10, through a receiving port 807. In some embodiments, the receiving port 807 may be, for example, connected to one or more antennas in first network node 111. In other embodiments, the first network node 111 may receive information from another structure in the communications network 10 through the receiving port 807. Since the receiving port 807 may be in communication with the processor 805, the receiving port 807 may then send the received information to the processor 805. The receiving port 807 may also be configured to receive other information.

The processor 805 in the first network node 111 may be further configured to transmit or send information to e.g., the second network node 112, the one or more entities 150, or another network node or another structure in the communications network 10, through a sending port 808, which may be in communication with the processor 805, and the memory 806.

Those skilled in the art will also appreciate that the determining unit 801 , the identifying unit 802, the performing unit 803, the initiating unit 804 and any of the other units described above may refer to a combination of analog and digital modules, and/or one or more processors configured with software and/or firmware, e.g., stored in memory, that, when executed by the one or more processors such as the processor 805, perform as described above. One or more of these processors, as well as the other digital hardware, may be included in a single Application- Specific Integrated Circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a System-on-a-Chip (SoC).

Also, any of the units 801-804 described above may be respectively implemented as the processor 805 of the first network node 111 , or an application running on such processor.

Thus, the methods according to the embodiments described herein for the first network node 111 may be respectively implemented by means of a computer program 809 product, comprising instructions, i.e. , software code portions, which, when executed on at least one processor 805, cause the at least one processor 805 to carry out the actions described herein, as performed by the first network node 11 1. The computer program 809 product may be stored on a computer-readable storage medium 810. The computer-readable storage medium 810, having stored thereon the computer program 809, may comprise instructions which, when executed on at least one processor 805, cause the at least one processor 805 to carry out the actions described herein, as performed by the first network node 11 1.

In some embodiments, the computer-readable storage medium 810 may be a non- transitory computer-readable storage medium, such as a CD ROM disc, or a memory stick. In other embodiments, the computer program 809 product may be stored on a carrier containing the computer program 809 just described, wherein the carrier is one of an electronic signal, optical signal, radio signal, or the computer- readable storage medium 810, as described above.

The first network node 11 1 may comprise an interface unit to facilitate communications between the first network node 11 1 and other nodes or devices, e.g., the first network node 11 1 , or any of the other nodes. In some particular examples, the interface may, for example, include a transceiver configured to transmit and receive radio signals over an air interface in accordance with a suitable standard.

In other embodiments, the first network node 11 1 may comprise the following arrangement depicted in Figure 8b. The first network node 11 1 may comprise a processing circuitry 805, e.g., one or more processors such as the processor 805, in the first network node 11 1 and the memory 806. The first network node 1 11 may also comprise a radio circuitry 811 , which may comprise e.g., the receiving port 807 and the sending port 808. The processing circuitry 805 may be configured to, or operable to, perform the method actions according to Figure 2, and/or any of Figures 4, 5, and/or 7, in a similar manner as that described in relation to Figure 8a. The radio circuitry 811 may be configured to set up and maintain at least a wireless connection any of the other nodes in the communications network 10. Circuitry may be understood herein as a hardware component.

Hence, embodiments herein also relate to the first network node 11 1 operative to handle the performance of the communications network 10. The first network node 11 1 may be further operative to operate in the communications network 10.

The first network node 11 1 may comprise the processing circuitry 805 and the memory 806, said memory 806 containing instructions executable by said processing circuitry 805, whereby the first network node 11 1 is further operative to perform the actions described herein in relation to the first network node 11 1 , e.g., in Figure 2, and/or Figures 4, 5, and/or 7.

Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and/or is implied from the context in which it is used. All references to a/an/the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and/or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following description.

As used herein, the expression“at least one of:” followed by a list of alternatives separated by commas, and wherein the last alternative is preceded by the“and” term, may be understood to mean that only one of the list of alternatives may apply, more than one of the list of alternatives may apply or all of the list of alternatives may apply. This expression may be understood to be equivalent to the expression “at least one of:” followed by a list of alternatives separated by commas, and wherein the last alternative is preceded by the“or” term.