TECHNICAL FIELD

The present disclosure relates to a new generation mobile network, e.g., to a 5 ^th generation (5G) mobile network, and to the generation of analytics information in the mobile network. The disclosure is concerned with tracing an analytics identifier (ID) or one or more analytics outputs for the analytics ID, and with performing a rollback action in case on an unstable analytics ID or at least one unstable analytics output for the analytics ID. The disclosure is especially concerned with a continuity of the tracing and the rollback action, if the analytics ID is transferred from one network analytics entity to the other and/or in case of mobility of a user equipment (UE) that is associated with the analytics ID.

To this end, this disclosure presents a first network analytics entity, a second network analytics entity, a network data analytics consumer entity, a network repository entity, and corresponding methods.

BACKGROUND

A network analytics entity like the Network Data Analytics Function (NWDAF) provides various analytics functions, which can be used by several Network Functions (NF) in the mobile network to make or improve decisions. For example, analytics information may be provided by the NWDAF to support the NFs to assist on Radio Access Technology (RAT) selection, Universal Software Radio Peripheral (USRP) update, determining a back-off timer provided to UEs etc.). Each NWDAF has its own identifier (ID), which any NF function can use to indicate the analytics output that it requests by the NWDAF. The NWDAF mainly consists of two functionalities that can be part of the same NWDAF instance or can be placed at different NWDAF instances: (a) a network analytics inference entity, which is configured to provide analytics and/predictions, for instance, using trained machine learning models; (b) a network analytics training entity, which is configured to generate the machine learning models using collected data and/or a training data set.

The quality of the NWDAF analytics output is affected by various factors, e.g., the quality and amount of the collected data to train the model, the configuration of the machine learning model, etc. Consequently, the quality of a NWDAF machine learning model can affect the network performance or status, according to the usage of the NWDAF analytics output. A successful (or an efficient, or reliable) analytics ID can lead or maintain a stable network status, while a less successful (or an inefficient, or unreliable, or unstable) analytics ID can lead to or create an unstable network status. For a stable network status, the system key performance indicators (KPIs) and/or metrics are kept within the expected pattern of usage (or improve). For an unstable network status, the KPIs/metrics of the system load remain at expected patterns, but KPIs indicating specific situations are unstable (or decrease from expected pattern).

It is important to detect and address when an analytics ID leads to an unstable network status. For that reason, the monitoring of the quality of an analytics ID and/or the quality of the usage of the analytics ID and/or the analytics output(s) for the analytics ID is necessary. The monitoring results can, for example, be provided from an analytics monitoring entity to other entities (e.g., an analytics tracing entity), where the indication of the quality of usage of the analytics can be evaluated (e.g., via Analytics ID Grade Information (AidGI), or indications about Unstable Analytics ID Information (UAiDI), etc.). Then, based on the monitoring information, the analytics tracing entity can determine if there is a need for the update of the analytics ID (e.g., via a retraining), or also for an analytics rollback action according to the obtained analytics ID tracing activation (e.g., a low performance analytics ID, an unstable analytics ID etc.). An analytics rollback action may allow a network data analytics consumer entity to receive an adequate quality of the respective NWDAF analytics output during, e.g., the retraining of the model of the analytics ID. The analytics rollback action may be executed by the network analytics inference entity and/or network analytics training entity of the NWDAF.

However, a problem is that the tracing and/or rollback of an analytics ID may be interrupted, when the analytics ID is transferred from one NWDAF to a different NWDAF. Moreover, triggering multiple tracing processes for the exact same analytics ID may occur, when an analytics service is transferred from one NWDAF to a different NWDAF, and this may waste signaling. Notably, such an analytics ID transfer for one or more analytics subscriptions from one NWDAF to another NWDAF (or NWDAF instance) can happen due to internal triggers (e.g., NWDAF load balancing, graceful shutdown) or due to external triggers (e.g., UE mobility of a UE associated with the analytics ID, wherein the mobility causes the handover from one NWDAF to another NWDAF). For instance, the subscriptions to analytics IDs with “target of Analytics Information” per UE basis (i.e., for one or more specific UEs) can be transferred from a source NWDAF to a target NWDAF according to the definition in TS 23.288 V17.1.0 Clause 6. IB.2. NFs subscribe to the NWDAF requesting analytics with “Target of Analytics Information” set to one or more specific UE (identified by their Subscription Permanent Identifier (SUPI)). Examples of NFs (analytics consumers) that use analytics IDs per UE basis are: Policy Control Function (PCF) - as in Interim Draft TS 23.503 VI 7.1.0 Clause 6.1.1.3) - consuming Service Experience Abnormal Behavior; Access and Mobility Management Function (AMF) - Interim Draft TS 23.501 V17.1.0 Clause 5.3.5, 5.4.1.3, 5.4.3.1 - consuming Abnormal Behavior, UE mobility and/or UE communication; Session Management Function (SMF) - Interim Draft TS 23.502 V17.1.0 Clause 5.6.5, 5.19.7, 5.33.2.2 - consuming UE mobility, Session Management Congestion Control Experience, Redundant Transmission Experience.

When an analytics ID for one or more specific UEs is relocated to a new NWDAF (from source NWDAF to target NWDAF) all services related to this analytics ID should be preserved in the target NWDAF, including the mentioned tracing and rollback actions. And also for a new data analytics consumer entity (from source to target analytics consumer entity) all services related to such analytics ID should be preserved in the target consumer entity. Due to the UEs mobility, the changes of analytics consumer entity and/or allocated NWADF happens many times.

Conventional solutions to the above problem so far focus on: (a) detecting the reliability of labels in the training samples, but this is a specific issue only to the training phase, and only this, is neither able to detect the effect of a trained model (i.e., analytic ID) in the system KPIs nor able to trace the wrong labels to changes in the network status; (b) collecting batches of data from multiple sources, in order to allow rollback to a previously collected batch of data. However, such a solution does not consider mechanisms to specifically detect unreliable/unstable analytics IDs or control configurations of an analytics ID; (c) executing rollback to previous known good state upon receiving a request for resources in a computer system (where resources are defined as databases, load balancers, scaling group machines). Specific solutions to rollback configurations of analytics IDs are not considered at all.

Regarding the 3GPP implementations, when an analytics ID (e.g., for one or more specific UE) is relocated to a new NWDAF (from source NWDAF to target NWDAF), the services related to this analytics ID should be preserved in the target NWDAF. For that reason, the Analytics Subscription Transfer and the Analytics Context Information Transfer has been introduced. However, there is no specific solution for the above-mentioned tracing and rollback capability continuity, during a relocation or handover procedure from a source NWDAF to a target NWDAF.

SUMMARY

In view of the above, this disclosure aims to provide a solution for tracing and rollback continuity in case of a transfer of an analytics ID and/or in case of UE mobility associated with the analytics ID. An objective is to extend an existing analytics transfer mechanism from one network analytics entity to another network analytics entity (e.g., from a provider or source NWDAF to a consumer or target NWDAF) to support tracing and rollback. Another objective is to introduce parameters for indicating analytics monitoring structures and analytics rollback structures, in order to enable the continuous usage of the tracing and rollback mechanisms for the analytics ID, which is transferred.

These and other objectives are achieved by the solution of this disclosure as described in the independent claims. Advantageous implementations are further defined in the dependent claims.

A first aspect of this disclosure provides a first network analytics entity, configured to: provide analytics tracing information to a second network analytics entity; wherein the first network analytics entity is configured to, based on the analytics tracing information, at least one of the following: trace one or more analytics outputs for an analytics identifier, ID, and/or trace the analytics ID; determine at least one unstable analytics output for the analytics ID and/or determine that the analytics ID is unstable; and determine and/or execute an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the unstable analytics ID.

The analytics tracing information can be sent directly to the second network analytics entity or via another network entity (e.g., an analytics model repository entity or an analytics data repository function). The analytics tracing information enables the second network analytics entity to continue the tracing and/or rollback performed by the first network analytics entity. Thus, continuity in case of a transfer of the analytics ID and/or in case of UE mobility associated with the analytics ID can be provided. The solution is compatible with existing analytics transfer mechanism from one network analytics entity to another network analytics entity.

In an implementation form of the first aspect, the analytics tracing information is associated with the one or more analytics outputs for the analytics ID and/or is associated with the analytics ID.

Thus, the second network analytics entity may ensure continuity of tracing and/or rollback with respect to this analytics ID.

In an implementation form of the first aspect, the analytics tracing information enables the second network analytics entity to at least one of the following: trace the one or more analytics outputs for the analytics ID and/or the analytics ID; determine at least one unstable analytics output for the analytics ID and/or the analytics ID; and determine and/or execute an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the analytics ID.

Accordingly, the continuity of tracing and performing rollback actions at the second network analytics entity is ensured, in case the analytics ID is transferred from the first to the second network analytics entity.

In an implementation form of the first aspect, the first network analytics entity is configured to provide to the second network entity the analytics tracing information included in an analytics subscription transfer message or in an analytics context information transfer message.

In this way, conventional transfer mechanisms may support the solution of this disclosure.

In an implementation form of the first aspect, the first network analytics entity is configured to: receive an analytics tracing information request or an analytics context transfer request from the second network analytics entity, and provide the analytics tracing information to the second network analytics entity in response to the analytics tracing information request or the analytics context transfer request; and/or provide an analytics subscription transfer with the analytics tracing information to the second network analytics entity and receive a response of the analytics subscription transfer. In an implementation form of the first aspect, the analytics ID is associated with one or more user equipments, UEs; and the first network analytics entity is configured to provide the analytics tracing information to the second network analytics entity, when at least one of the one or more UEs is allocated from the first network analytics entity to the second network analytics entity.

Thus, the continuity of the tracing and/or rollback actions is ensured in case of UE mobility.

In an implementation form of the first aspect, the analytics ID output is associated with one or more network functions, NF; and the first network analytics entity is configured to provide the analytics tracing information to the second network analytics entity, when at least one of the one or more NFs is allocated from the first network analytics entity to the second network analytics entity.

Thus, the continuity of the tracing and/or rollback actions is ensured in case of NF mobility, which may imply that the analytics ID is transferred.

In an implementation form of the first aspect, the first network analytics entity and the second network analytics entity are respectively associated with a first NWDAF and a second NWDAF.

In an implementation form of the first aspect, the inference rollback action is an action for changing an inference configuration about the at least one unstable analytics output and/or about the analytics ID at a network analytics inference entity; and the training rollback action is an action for changing a training configuration about the at least one unstable analytics output and/or about the analytics ID at a network analytics training entity.

The rollback action may allow reaching a stable analytics ID or analytics output for the analytics ID.

In an implementation form of the first aspect, the first network analytics entity is further configured to provide a rollback notification related to the analytics ID based on the analytics tracing information, wherein the rollback notification comprises one or more of the at least one unstable analytics output for the analytics ID and/or the analytics ID, wherein the rollback notification is provided to a network analytics consumer entity, a network analytics inference entity, or a network analytics training entity; the inference rollback action for the at least one unstable analytics output for the analytics ID and/or for the analytics ID, wherein the rollback notification is provided to the network analytics inference entity; the training rollback action for the at least one unstable analytics output for the analytics ID and/or for the analytics ID, wherein the rollback notification is provided to the network analytics training entity.

In an implementation form of the first aspect, the first network analytics entity is further configured to receive a rollback status notification comprising at least one of a status of the inference rollback action executed by the network analytics inference entity and a status of the training rollback action executed by the network analytics training entity.

The first network analytics entity may have one or both roles of the network analytics inference entity and the network analytics training entity.

In an implementation form of the first aspect, the analytics tracing information further comprises at least one of a tracing capability of the first network analytics entity related to the analytics ID; a rollback capability of the first network analytics entity related to the analytics ID; an analytics monitoring capability of the first network analytics entity related to the analytics ID.

The second network analytics entity may check if/whether it can support any of provided capabilities and provide a response whether the indicated capabilities can be supported or not.

In an implementation form of the first aspect, the analytics tracing information comprises at least one of the analytics ID; an association of the analytics ID to the one or more analytics outputs for the analytics ID; one or more quality indications about at least one of the one or more analytics outputs for the analytics ID and the analytics ID; an inference configuration about the analytics ID and/or about the one or more analytics outputs for the analytics ID; a training configuration about the analytics ID and/or about the one or more analytics outputs for the analytics ID. The analytics tracing information may further comprise: a performance of the one of the one or more analytics outputs for the analytics ID and the analytics ID; and/or an accuracy of the one of the one or more analytics outputs for the analytics ID and the analytics ID.

In an implementation form of the first aspect, the analytics tracing information further comprises at least one of an Analytics ID tracing data structure; analytics ID inference configuration information; analytics ID training configuration information; an inference rollback notification; a training rollback notification; an unstable notification.

In an implementation form of the first aspect, the first network analytics entity is further configured to provide to the second network analytics entity analytics monitoring information comprising at least one of analytics ID performance information; analytics ID grade information; unstable analytics ID information.

In an implementation form of the first aspect, the analytics tracing information further comprises location information and an association of the location information to at least one of: the training configuration about at least one of the one or more analytics outputs for the analytics ID and/or the analytics ID; the inference configuration about at least one of the one or more analytics outputs for the analytics ID and/or the analytics ID; the one or more quality indications about at least one of the one or more analytics outputs for the analytics ID and/or the analytics ID.

In an implementation form of the first aspect, the location information comprises at least one of: a layered description of an environment, where a specific training and/or inference configuration has been applied and respective quality indications of analytics performance; geographical map; information about one or more parameters that describe the network configuration and UE configuration where a specific training and/or inference configuration has been applied.

The layered description of the environment may include static, semi-static and/or dynamic information (e.g., of different types) of the physical and communication related environment. For instance, it may comprise a geographical map, e.g., with a physical location or terrain, road, sensor information, the time of the day, UE capabilities, network entities capabilities, etc. In an implementation form of the first aspect, the analytics tracing information further comprises at least one of a status of the inference rollback action; a status of the training rollback action; statistics of a rollback action, or an inference configuration, or a training configuration; one or more inference rollback actions, and/or one or more training rollback actions associated with the analytics ID and/or associated with one or more analytics outputs for the analytics ID; one or more inputs for the analytics ID and/or corresponding analytics outputs, for at least one of a rollback action, an inference configuration, and a training configuration.

In an implementation form of the first aspect, the analytics tracing information further comprises an association of the analytics ID and/or of the one or more analytics outputs for the analytics ID to at least one of a timestamp; an identification of a network analytics inference entity; an identification of a network analytics training entity; an identification of a network analytics consumer entity; an identification of the one or more analytics outputs for the analytics ID; an identification of a reason that the analytics ID and/or of the one or more analytics outputs for the analytics ID are used by a network analytics consumer entity.

In an implementation form of the first aspect, the first network analytics entity is further configured to discover the second network analytics entity based on at least one of the following: a tracing capability of the second network analytics entity related to the analytics ID; a rollback capability of the second network analytics entity related to the analytics ID; an analytics monitoring capability of the second network analytics entity related to the analytics ID.

In an implementation form of the first aspect, the first network analytics is configured to discover the second network analytics entity by sending a discovery request, indicating the required tracing and/or rollback and/or analytics monitoring capability related to the analytics ID, to a network repository entity.

In an implementation form of the first aspect, the first network analytics entity is configured to send a registration request, indicating the tracing and/or rollback and/or analytics monitoring capability of the first network analytics entity related to one or more analytics IDs, to a network repository entity. A second aspect of this disclosure provides a second network analytics entity, configured to receive analytics tracing information from a first network analytics entity; wherein the second network analytics entity is configured to, based on the analytics tracing information, at least one of the following: trace one or more analytics outputs for an analytics identifier, ID, and/or trace the analytics ID; determine at least one unstable analytics output for the analytics ID and/or determine that the analytics ID is unstable; and determine and/or execute an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the unstable analytics ID.

The analytics tracing information enables the second network analytics entity to continue the tracing and/or rollback performed by the first network analytics entity. Thus, continuity in case of a transfer of the analytics ID and/or in case of UE mobility associated with the analytics ID can be provided. The solution is compatible with existing analytics transfer mechanism from one network analytics entity to another network analytics entity.

In an implementation form of the second aspect, the analytics tracing information is associated with the one or more analytics outputs for the analytics ID and/or is associated with the analytics ID.

In an implementation form of the second aspect, the second network analytics entity is configured to receive from the first network entity the analytics tracing information included in an analytics subscription transfer message or in an analytics context information transfer message.

In an implementation form of the second aspect, the second network analytics entity is configured to: provide an analytics tracing information request or an analytics context transfer request to the first network analytics entity, and receive the analytics tracing information from the first network analytics entity in response to the analytics tracing information request or the analytics context transfer request; and/or receive an analytics subscription transfer with the analytics tracing information from the first network analytics entity and receive a response of the analytics subscription transfer.

In an implementation form of the second aspect, the analytics ID is associated with one or more user equipments, UEs; and the second network analytics entity is configured to receive the analytics tracing information from the first network analytics entity, when at least one of the one or more UEs is allocated from the first network analytics entity to the second network analytics entity.

In an implementation form of the second aspect, the analytics ID output is associated with one or more network functions, NF; and the second network analytics entity is configured to receive the analytics tracing information from the first network analytics entity, when at least one of the one or more NFs is allocated from the first network analytics entity to the second network analytics entity.

In an implementation form of the second aspect, the first network analytics entity and the second network analytics entity are respectively associated with a first NWDAF and a second NWDAF.

In an implementation form of the second aspect, the inference rollback action is an action for changing an inference configuration about the at least one unstable analytics output and/or about the analytics ID at a network analytics inference entity; and the training rollback action is an action for changing a training configuration about the at least one unstable analytics output and/or about the analytics ID at a network analytics training entity.

In an implementation form of the second aspect, the analytics tracing information further comprises at least one of: a tracing capability of the first network analytics entity related to the analytics ID; a rollback capability of the first network analytics entity related to the analytics ID; an analytics monitoring capability of the first network analytics entity related to the analytics ID.

In an implementation form of the second aspect, the analytics tracing information comprises at least one of the analytics ID; an association of the analytics ID to the one or more analytics outputs for the analytics ID; one or more quality indications about at least one of the one or more analytics outputs for the analytics ID and the analytics ID; an inference configuration about the analytics ID and/or about the one or more analytics outputs for the analytics ID; a training configuration about the analytics ID and/or about the one or more analytics outputs for the analytics ID. In an implementation form of the second aspect, the analytics tracing information further comprises at least one of an Analytics ID tracing data structure; analytics ID inference configuration information; analytics ID training configuration information; an inference rollback notification; a training rollback notification; an unstable notification.

In an implementation form of the second aspect, the second network analytics entity is further configured to receive from the first network analytics entity analytics monitoring information comprising at least one of analytics ID performance information; analytics ID grade information; unstable analytics ID information.

In an implementation form of the second aspect, the analytics tracing information further comprises location information and an association of the location information to at least one of: the training configuration about at least one of the one or more analytics outputs for the analytics ID and/or the analytics ID; the inference configuration about at least one of the one or more analytics outputs for the analytics ID and/or the analytics ID; the one or more quality indications about at least one of the one or more analytics outputs for the analytics ID and/or the analytics ID.

In an implementation form of the second aspect, the location information comprises at least one of a layered description of an environment, where a specific training and/or inference configuration has been applied and respective quality indications of analytics performance; geographical map; information about one or more parameters that describe the network configuration and UE configuration where a specific training and/or inference configuration has been applied.

In an implementation form of the second aspect, the analytics tracing information further comprises at least one of a status of the inference rollback action; a status of the training rollback action; statistics of a rollback action, or an inference configuration, or a training configuration; one or more inference rollback actions, and/or one or more training rollback actions associated with the analytics ID and/or associated with one or more analytics outputs for the analytics ID; one or more inputs for the analytics ID and/or corresponding analytics outputs, for at least one of a rollback action, an inference configuration, and a training configuration. In an implementation form of the second aspect, the analytics tracing information further comprises an association of the analytics ID and/or of the one or more analytics outputs for the analytics ID to at least one of a timestamp; an identification of a network analytics inference entity; an identification of a network analytics training entity; an identification of a network analytics consumer entity; an identification of the one or more analytics outputs for the analytics ID; an identification of a reason that the analytics ID and/or of the one or more analytics outputs for the analytics ID are used by a network analytics consumer entity.

The implementation forms of the second network analytics entity provide the same advantages as described above for the implementation forms of the first network analytics entity.

A third aspect of this disclosure provides a network data analytics consumer entity, configured to: provide, to a first network analytics entity, an indication with information to activate an analytics tracing and/or a rollback of one or more analytics outputs for an analytics ID and/or of the analytics ID; and receive, from a second network analytics entity, an indication that the analytics tracing and/or the rollback are supported.

Thus, the network analytics consumer entity can receive a continuous service of tracing and/or rollback, even if the analytics ID is transferred from the first network analytics entity to the second network analytics entity, or in case of mobility of the network analytics consumer entity (e.g. UE).

A fourth aspect of this disclosure provides a network repository entity, configured to store a profile of each of one or more network analytics entities, wherein the profile of each network analytics entity comprises at least one of the following: tracing capability of network analytics entity related to one or more analytics IDs; rollback capability of the network analytics entity related to the one or more analytics ID; an analytics monitoring capability of the network analytics entity related to the one or more analytics ID.

The network repository entity allows the first network analytics entity to select the second network analytics entity, from multiple network analytics entities, in view of a desired continuity of the tracing and rollback support for an analytics ID. In this way, the network repository entity supports the continuity. In an implementation form of the fourth aspect, the network repository entity is further configured to: receive a discovery request, indicating a required tracing and/or rollback and/or analytics monitoring capability related to an analytics ID, from a first network analytics entity or a network data analytics consumer entity; and provide a profile of at least one of one or more network analytics entities to the first network analytics entity or the network data analytics consumer entity in response.

In an implementation form of the fourth aspect, the network repository entity is further configured to: receive a registration request, indicating a tracing and/or rollback and/or analytics monitoring capability of a first network analytics entity related to one or more analytics IDs, from the first network analytics entity; and store a profile for the first network analytics entity based on the capabilities in the registration request.

A fifth aspect of this disclosure provides a method for a first network analytics entity, the method comprising providing analytics tracing information to a second network analytics entity; wherein the method further comprises, based on the analytics tracing information, at least one of the following: tracing one or more analytics outputs for an analytics ID and/or tracing the analytics ID; determining at least one unstable analytics output for the analytics ID and/or determining that the analytics ID is unstable; and determining and/or executing an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the unstable analytics ID.

The method of the fifth aspect may have implementation forms that correspond to the implementation forms of the first network analytics entity of the first aspect. The method of the fifth aspect provides the same advantages as described for the first network analytics entity of the first aspect.

A sixth aspect of this disclosure provides a method for a second network analytics entity, the method comprising: receiving analytics tracing information from a first network analytics entity; wherein the method further comprises, based on the analytics tracing information, at least one of the following: tracing one or more analytics outputs for an analytics ID, and/or tracing the analytics ID; determining at least one unstable analytics output for the analytics ID and/or determining that the analytics ID is unstable; and determining and/or execute an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the unstable analytics ID.

The method of the sixth aspect may have implementation forms that correspond to the implementation forms of the second network analytics entity of the second aspect. The method of the sixth aspect provides the same advantages as described for the second network analytics entity of the second aspect.

A seventh aspect of this disclosure provides a method for a network data analytics consumer entity, the method comprising: providing, to a first network analytics entity, an indication with information to activate an analytics tracing and/or a rollback of one or more analytics outputs for an analytics ID and/or of the analytics ID; and receiving, from a second network analytics entity, an indication that the analytics tracing and/or the rollback are supported.

The method of the seventh aspect may have implementation forms that correspond to the implementation forms of the network data analytics consumer entity of the third aspect. The method of the seventh aspect provides the same advantages as described for the network data analytics consumer entity of the third aspect.

An eighth aspect of this disclosure provides a method for a network repository entity, the method comprising storing a profile of each of one or more network analytics entities, wherein the profile of each network analytics entity comprises at least one of the following: tracing capability of network analytics entity related to one or more analytics IDs; rollback capability of the network analytics entity related to the one or more analytics ID; an analytics monitoring capability of the network analytics entity related to the one or more analytics ID.

The method of the eighth aspect may have implementation forms that correspond to the implementation forms of the network repository entity of the fourth aspect. The method of the eighth aspect provides the same advantages as described for the network repository entity of the fourth aspect.

A ninth aspect of this disclosure provides a computer program comprising instructions which, when the program is executed by a computer, cause the computer to perform the method according to one of the fifth, sixth, seventh, or eighth aspect, or any implementation form thereof.

A ninth aspect of this disclosure provides a storage medium storing executable program code which, when executed by a processor, causes the method according to one of the fifth, sixth, seventh, or eighth aspect, or any implementation form thereof to be performed.

As an example of the above aspects and implementation forms, this disclosure proposes a first network data analytics entity (e.g., source NWDAF) transfers developed and/or stored tracing and rollback information of one or more analytics ID to a second network data analytics entity (e.g., target NWDAF) during, for instance, a UEs mobility with NWDAF reallocation. This may ensure consistent and continuous tracing of one or more analytics IDs and also the performance (or operation) one or more analytics ID with a rollback action applied, thus helping to maintain a stable network status.

An Analytics Subscription Transfer extension may indicate the tracing and/or rollback and/or analytics capability for the source NWDAF to the target NWDAF. An Analytics Context Information Transfer extension with new parameters may indicate the analytics monitoring structures and analytics rollback structures provided from the source NWDAF to the Target NWDAF. An Analytics Context Information may be extended to support tracing and rollback triggering and execution in the target NWDAF.

It has to be noted that all devices, elements, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof. BRIEF DESCRIPTION OF DRAWINGS

The above described aspects and implementation forms will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which

FIG. 1 shows a first network analytics entity, a second network analytics entity, and a network data analytics consumer entity, according to this disclosure.

FIG. 2 shows a network repository entity according to this disclosure.

FIG. 3 shows an example extension on the Analytics Subscription Transfer initiated by a first analytics network entity according to this disclosure, with rollback and tracing capability enabled.

FIG. 4 shows an example of a flow analytics context information exchange with tracing, monitoring and rollback information.

FIG. 5 shows an exemplary method at a second network analytics entity of this disclosure, to check an analytics rollback action indication received by the first network analytics entity.

FIG. 6 shows an example of a visualization of analytics context with analytics tracing and rollback information.

FIG. 7 shows an example for the retrieval of the description of an active rollback action and/or configuration by another entity (not the first network analytics entity).

FIG. 8 shows an example for the retrieval of the description of an active rollback action and/or configuration by another entity.

FIG. 9 shows a method for a first network analytics entity according to this disclosure.

FIG. 10 shows a method for a second network analytics entity according to this disclosure.

FIG. 11 shows a method for a network data analytics consumer entity according to this disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

FIG. 1 shows a first network analytics entity 100 according to an embodiment of this disclosure, a second network analytics entity 110 according to an embodiment of this disclosure, and a network data analytics consumer entity 120 according to an embodiment of this disclosure. The network analytics entities 100, 110 may be NWDAFs, e.g., respectively a source NWDAF 100 and a target NWDAF 110. The network data analytics consumer entity 120 may be a UE or NF. The first network analytics entity 100 is configured to provide analytics tracing information 101 to the second network analytics entity 110, which is accordingly configured to receive the analytics tracing information 101 from the first network analytics entity 100.

Both the first network analytics entity 100 and the second network analytics entity 110 are configured to, based on the analytics tracing information 101, perform at least one of the following steps: (i) tracing one or more analytics outputs for an analytics ID and/or tracing the analytics ID; (ii) determining at least one unstable analytics output for the analytics ID and/or determining that the analytics ID is unstable; (iii) determining and/or executing an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the unstable analytics ID.

That is, each network analytics entity 100, 110 is configured to perform the tracing step, the determining step, and/or the executing step by using the analytics tracing information 101. In this way, continuity of performing these steps is ensured at the second network analytics entity 110, even in case of an analytics ID transfer or UE or NF mobility.

The analytics ID may be associated with one or more UEs or one or more NFs, for instance, with the network data analytics consumer entity 120 shown in FIG. 1. The first network analytics entity 100 may be configured to provide the analytics tracing information 101 to the second network analytics entity 110, particularly, when at least one UE or NF of the one or more UEs or NFs, for instance, the network data analytics consumer entity 120, is allocated from the first network analytics entity 100 to the second network analytics entity 110. In this case, the analytics ID associated with the network data analytics consumer entity 120 may be transferred from the first network analytics entity to the second network analytics entity.

As also shown in FIG. 1, the network data analytics consumer entity 120 is configured to provide, to the first network analytics entity 100, an indication 121 with information to activate an analytics tracing and/or a rollback of one or more analytics outputs for an analytics ID and/or of the analytics ID. Further, the network data analytics consumer entity 120 is configured to receive, from the second network analytics entity 110, an indication 122 that the analytics tracing and/or the rollback are supported. The second network analytics entity 110 may be able to provide the indication 122 based on the analytics tracing information 101, which it received from the first network analytics entity 100. For instance, the first network analytics entity 100 may be configured to select the second network analytics entity 110 as a recipient of the analytics tracing information 101 based on a tracing capability of the second network analytics entity 110 related to the analytics ID, and/or based on a rollback capability of the second network analytics entity 110 related to the analytics ID, and/or based on an analytics monitoring capability of the second network analytics entity 110 related to the analytics ID. In this way, the tracing and rollback continuity in case of a transfer of the analytics ID from the first network analytics entity 100 to the second network analytics entity 110 and/or in case of mobility of the network data analytics consumer entity 120 associated with the analytics ID.

FIG. 2 shows a network repository entity 200 according to an embodiment of this disclosure. The network repository entity 200 may be a network repository function (NRF). The network repository entity 200 is configured to store a profile 201, 202 of each of one or more network analytics entities 100, 110, for example, a first profile 201 of the first network analytics entity 100 and a second profile 202 of the second network analytics entity 110. The profile 201, 202 of each respective network analytics entity 100, 110 comprises at least one of the following: the tracing capability of the respective network analytics entity 100, 110 related to one or more analytics IDs; the rollback capability of the respective network analytics entity 100, 110 related to the one or more analytics ID; and the analytics monitoring capability of the respective network analytics entity 100, 110 related to the one or more analytics ID. The profiles 201, 202 allow, for example, the first network analytics entity 100 to select the second network analytics entity 110 as the recipient of the analytics tracing information 101 based on at least one of the tracing capability, the rollback capability, and the analytics monitoring capability of the second network analytics entity 110 related to the analytics ID. This may ensure the tracing and rollback continuity.

In an example, as shown also in FIG. 2, the network repository entity 200 may receive a discovery request 203 from the first network analytics entity 100 (or from a network data analytics consumer entity 120), wherein the discovery request 203 indicates a required tracing and/or rollback and/or analytics monitoring capability related to the analytics ID. The network repository entity 200 may then provide the profile 202 of the second network analytics entity 110 to the first network analytics entity 100 in response, in particular, if the second network analytics entity 110 meets the required tracing and/or rollback and/or analytics monitoring capability related to the analytics ID. Each of the entities 100, 110, 120, 200 shown in FIG. 1 and FIG. 2 may comprise a processor or processing circuitry (not shown) configured to perform, conduct or initiate the various operations of the entity 100, 110, 120, 200 described herein. The processing circuitry may comprise hardware and/or the processing circuitry may be controlled by software. The hardware may comprise analog circuitry or digital circuitry, or both analog and digital circuitry. The digital circuitry may comprise components such as application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), digital signal processors (DSPs), or multipurpose processors. Each entity 100, 110, 120, 200 may further comprise memory circuitry, which stores one or more instruction(s) that can be executed by the processor or by the processing circuitry, in particular under control of the software. For instance, the memory circuitry may comprise a non-transitory storage medium storing executable software code which, when executed by the processor or the processing circuitry, causes the various operations of the entity 100, 110, 120, 200 to be performed. In one embodiment, the processing circuitry comprises one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code which, when executed by the one or more processors, causes the entity 100, 110, 120, 200 to perform, conduct or initiate the operations or methods described herein.

FIG. 3 shows an example for an extension of the Analytics Subscription Transfer, which may be initiated by a first network analytics entity 100 (here a provider NWDAF, or, as in the following, source NWDAF) to a second network analytics entity 110 (here a consumer NWDAF, or, as in the following, a target NWDAF)) according to this disclosure, with rollback and tracing capability enabled (“Analytics Tracing and Rollback Support”). In particular, the first network analytics entity 100 is configured to provide, to the second network analytics entity 110, the analytics tracing information 101 included in an analytics subscription transfer message 301.

The target NWDAF 110 may use the transferred analytics tracing information 101 to continue the tracing over time of the analytics ID grade (extending Analytics ID Tracing Data Structure of the previous NWDAF(s)) and/or to apply to the inference (or a ML model) an active rollback configuration, which may be decided at the source NWDAF 100. The target NWDAF 110 may also become aware of a last known stable network state, in case this is needed to be applied. The source NWDAF 100, with the Analytics Subscription Transfer to the target NWDAF 110, may indicate whether the tracing and rollback is needed by the target NWDAF 110, for example, by providing:

• Tracing and rollback capability of the source NWDAF 100 per one or more analytics ID.

• Rollback capability or rollback capability of the source NWDAF 100 per one or more analytics ID.

• Analytics monitoring capability or analytics monitoring capability of the source NWDAF 100 per one or more analytics ID.

The source NWDAF(s) 100 and/or a network data analytics consumer entity 120 (here an NF consumer) can also trigger the target NWDAF 110 to check, if the tracing and rollback capability for a specific analytics ID is supported.

An example of the proposed extensions on the Analytics Subscription Transfer initiated by the source NWDAF (TS 23.288, 6. IB.2.2,) is provided in FIG. 3. The source NWDAF 100, after a decision to request an analytics subscription transfer and after selecting the target NWDAF 110, may firstly provide an indication about its analytics tracing and/or rollback, i.e., the analytics tracing information 101, via the NWDAF AnalyticsSubscription Transfer message 301 (message number 4 in FIG. 3) to the target NWDAF 110. The target NWDAF 110 may take over the Analytics Subscription and may check and enable analytics tracing and rollback Support, based on the analytics tracing information 101 provided by the source NWDAF 100, and based, e.g., on its own capabilities (i.e., target NWDAF capabilities). The target NWDAF 110 can also (optionally) notify the network analytics consumer entity 120 (e.g., NF Consumer) whether or not the analytics tracing and rollback will be continued to be provided via the NWDAF_AnalyticsSubscription_Notify message 302 (message number 6 in FIG. 3), which may include the indication 122 shown in FIG. 1, that the analytics tracing and/or the rollback are supported.

More information for the analytics monitoring, tracing and/or analytics rollback configurations or rollback actions may be provided from the source NWDAF 100 to the target NWDAF 110 in the context of procedure number 7 of FIG. 3 (more details about the content is provided below). Finally, the target NWDAF 110 can inform the source NWDAF 100 whether analytics tracing and rollback are supported or enabled, via the message NWDAF AnalyticsSubscription Transfer response 303 (message number 9 in FIG. 3). The target NWDAF 110 may also provide a NWDAF_Analytics Sub scri ption JSiotify message 304 to the NF Consumer 120 including information about the analytics subscription, e.g., an analytics output of a analytics ID.

FIG. 4 shows an example of a flow analytics context information exchange with tracing, monitoring and rollback information.

The target NWDAF 110 may request analytics context information, which may include a list of analytics context identified s), a set of SUPI and associated analytics ID for UE related analytics or an analytics ID for NF related analytics. Thereby, the target NWDAF 110 can indicate if tracing, monitoring and rollback information is needed by the source NWDAF 100. For example, as shown in FIG. 4, the source NWDAF 100 may receive an analytics context transfer request 401 from the target NWDAF 110 with this indication.

The source NWDAF 100 may respond with analytics context information to the consumer target NWDAF 1100, for example, by adding to its response message 402 the analytics tracing information 101. That is, the source NWDAF 100 may provide the analytics tracing information 101 to the target NWDAF 110 in response to the analytics context transfer request 401. For instance, the source NWDAF 100 may add one or more of the following contextual information (CI) for the analytics monitoring and analytics rollback:

• CI 1 : Analytics monitoring information

■ Analytics ID Performance Information (API)

■ Analytics ID Grade Information (AidGI)

■ Unstable Analytics ID Information (UAidl)

The Analytics Monitoring Information may be used by the target NWDAF 110 to continue consistent monitoring the effect of an analytics ID consumption in the changes of network status (stable, unstable analytics ID).

• CI2: Analytics tracing information

■ Analytics ID Tracing Data Structure (ATDS)

■ Analytics ID Inference Configuration Information (AICI)

■ Analytics ID Training Configuration Information (ATCI) ■ Inference Rollback Notification (IRN)

■ Training Rollback Notification (TRN)

■ Unstable Notification (UN)

The analytics tracing information may be used by the target NWDAF 110 to continue consistent tracing analytics ID grade taking into the inference engine and/or ML training model configurations. And to inform the target NWDAF 110 about last known stable network state for an analytics ID output and its associated configurations (inference and/or training configurations).

CI3: Indication about the active rollback configuration, in case a rollback action has been applied by the source NWDAF 100.

This may be used by the target NWDAF 110 to continue using an active rollback configuration, decided at the source NWDAF 100.

CI4: Layered Description of the environment, where a specific configuration has been applied and the respective network quality indications has been received:

■ LI : geographical map: one-dimensional or multi-dimensional (e.g., to represent a physical location or terrain, road)

■ L2: map can include or be described by additional dimensions (not only geographical dimensions) to provide information about other parameters that may describe the situation where a configuration has been used (e.g., sensor information, the time of the day, UE capabilities etc.).

This may be used by the target NWDAF 110 to assess if the active (or another provided) rollback configuration matches with the current (network) environment conditions, like those triggered the rollback at the source NWDAF 100.

CI5: Statistics of a Rollback Action/Configuration (active or not).

More information about this type of information is provided below.

CI6: NFs that consume an analytics ID and reason the analytics used (e.g., URSP update). This may be used by the target NWDAF 110 to assess if rollback should be enabled for a specific NF ID and/or reason (i.e. action supported by analytics ID) or applied to all cases. This may help to saves signaling and resources. • CI7: (optional) Inputs for Analytics ID and corresponding analytics outputs, for each identified rollback configuration (state)

This may be useful to allow the target NWDAF 110 to make verification of the source NWDAF 100 reliability about associated rollbacks and network quality indicators. This type of information is useful especially in case of multi-vendor environments.

An example method at the target NWDAF 110 with reception of analytics context with analytics tracing and rollback support is presented in FIG. 5. In particular, FIG. 5 shows how the above described contextual information CI1-CI8 may be used at the target NWDAF 110.

The target NWDAF 110 may check, if a rollback configuration is enabled using CI3. If “No”, it may use a current ML configuration. If “Yes”, the target NWDAF 110 may check if the same ML model type is used. If “No”, it may use the current ML configuration or may deactivate. If “Yes”, the target NWDAF 110 may verify whether to trust a source ML using CI7. If “Yes”, it may further check a similarity of (network) environment descriptors and impact of rollback to one or more analytics using CI4, CI5, and CI6. In case of an adequate similarity, it may enable an active rollback configuration (inference and/or training) using CI3, CI1, and CI2. In case of a low similarity, it may use another rollback configuration provided by the source NWDAF 100 or may keep a default configuration of the ML model or deactivate by using CI1 and CI2.

An example structure of analytics context with analytics tracing and rollback Support, is presented in FIG. 6.

FIG. 7 shows an example for the retrieval of the description of an active rollback action/configuration by another entity (not the first network analytics entity).

The indication about the currently active rollback configuration (CI3), in case rollback has been activated by the source NWDAF 100, and is provided from the source NWDAF 100 to the target NWDAF 110 to which the UE will be transferred.

Two options have been identified to provide the description of the active rollback from the source NWDAF 100 to the target NWDAF 110. • Option 1 : Send from the source NWDAF 100 to the target NWDAF 110 an “Explicit Description of the Rollback Configuration”.

The configuration can include: parameters related to the ML mode (e.g., type of algorithm, gradients, weights, architecture information about the mode, ...), parameters related to the requested analytics ID to be trained or modeled, other parameters e.g., interval of data collection for generating the analytics, whether pre-processing of collected data for analytics generation has been applied, type of collected data for generating the analytics etc. The above information can be part of the NWDAF AnalyticsInfo ContextTransfer response message.

• Option 2: The source NWDAF 100 provides to the target NWDAF 110 an indication (e.g., via a flag parameter) that a rollback configuration has been activated at the source NWDAF 100 as well as an identifier of the Activated Rollback Configuration (e.g., ID number). The above information can be part of the NWDAF AnalyticsInfo ContextTransfer response message.

The target NWDAF 110 can request and receive the detailed or explicit description of the rollback configuration, using the provided identifier of the Activated Rollback Configuration (e.g., Rollback Configuration lD), by contacting an entity that associates the identifier of the Activated Rollback Configuration with the explicit description of the rollback configuration and/or the ML model. The network repository entity 200 (called here Model Configuration Repository) that keeps this information (i.e., the association) can be another NF e.g., Training NF, Training Model Training logical function (MTLF), NWDAF, Analytics Data Repository Function (ADRF), Models repository etc.). The source NWDAF 100 or any other NWDAF can provide the detailed or explicit description of a rollback configuration to this NF. An example flow chart is provide in FIG. 7.

Furthermore, each rollback configuration can be associated with statistics that provide an indication or a value of the success rate of a specific rollback configuration (or rollback action) on the performance of an analytics ID. The success rate can be understood in terms of the impact of an analytics ID on the decision that is made by an analytics consumer (e.g., NF decision, AF decision etc.), receiving the respective analytics ID outputs. For example, whether the analytics ID output(s) leads to successful or accurate decision by the analytics consumer, or even whether the analytics ID outputs are accurate enough comparing e.g., with the ground truth (e.g., analytics ID prediction accuracy). For instance, the SMF may use Session Management Congestion Control Experience analytics provided by the NWDAF, to determine back-off timer provided to UEs, to support DNN based congestion control (according to 23.501 TS). In this case the “success rate of a specific analytics ID configuration” could be identified by the impact of the SMF decision, based on received analytics to the NAS level congestion criterion. In that case the network data analytics consumer entity 120 can report to the NWDAF 100, 110 information about the impact (or performance) of the usage of an analytics ID, according to the defined criteria per analytics consumer or analytics ID. Based on this information the NWDAF 100, 110 that is aware of the used rollback configuration or rollback action can calculate the statistics e.g., success rate of a specific “Rollback Configuration of the per analytics ID.

The Nnwdaf AnalyticsSubscription Transfer message (as shown in FIG. 3) or the Nnwdaf AnalyticsSubscription Transfer Request message may be used to request the target NWDAF 110 to transfer analytics subscription(s) from the source NWDAF 100. In the case that a source NWDAF 100 has enabled/configured the tracing and/or rollback capability, this may be indicated via the Nnwdaf AnalyticsSubscription Transfer (or Nnwdaf AnalyticsSubscription Transfer Request) message with information about tracing and rollback capability, including one or more of the following information:

• Tracing capability or tracing capability per one or more analytics ID;

• Rollback capability or Rollback capability per one or more analytics ID;

• Analytics monitoring capability or Analytics Monitoring capability per one or more analytics ID.

Based on this information the target NWDAF 110 can check to takes over Analytics Subscription and whether and how to enable Analytics Tracing and Rollback Support.

In the response from the target NWDAF 110 to the source NWDAF 100 (e.g., Nnwdaf AnalyticsSubscription Transfer Response message) it may be indicated whether the indicated capabilities can be supported or not.

FIG. 8 shows an example for the retrieval of the description of an active rollback action/configuration by another entity. In particular, as shown in FIG. 8, when one or more analytic IDs are transferred to a different NWDAF, the source (provider) NWDAF 100 may have to discover and select the target NWDAF 110. This means that the source NWDAF 100 may have to discover NWDAFs with tracing and rollback capabilities in case the latter has been enabled. For that reason, it is beneficial to make enhancements of the Network Repository Function (NRF) - i.e., network repository entity 200 - and the NWDAF discovery and selection to enable the identification of NWDAFs with tracing and rollback capabilities. Specifically the NRF extensions can be involved:

• To update the NWDAF profile 201, 292 with information to indicate tracing and/or rollback and/or analytics monitoring capability.

• The NRF discovery service extended with new parameters to discover the tracing, rollback and analytics monitoring capabilities.

• The NRF notification service of changes in NWDAF capabilities to support notification on tracing and/or rollback and/or analytics monitoring capabilities of NWDAF.

The NWDAF NF profile 201, 202 should also include one or more of the following information:

• Tracing capability for all analytics IDs or per analytics IDs that can be supported by an NWDAF AnLF (i.e., exposing the AnaltyicsSubscription/ Analyticsinfo service)

• Rollback capability for all analytics IDs or per analytics IDs that can be supported by an NWDAF AnLF (i.e., exposing the AnaltyicsSubscription/ Analyticsinfo service).

• Analytics monitoring capability for all analytics IDs or per analytics IDs that can be supported by an NWDAF AnLF (i.e., exposing the AnaltyicsSubscription/ Analyticsinfo service).

• Rollback capability for ML models, such ML models being associated with all analytics IDs or per analytics IDs supported by an NWDAF MTLF (i.e., exposing the MLModel Provisioning service).

The network repository entity 200 (NRF) may receiver a registration of NWDAF profile enhanced with any of the following information:

• Tracing capability for all analytics IDs or per analytics IDs that can be supported by an NWDAF AnLF (i.e., exposing the AnaltyicsSubscription/ Analyticsinfo service).

• Rollback capability for all analytics IDs or per analytics IDs that can be supported by an NWDAF AnLF (i.e., exposing the AnaltyicsSubscription/ Analyticsinfo service). • Analytics monitoring capability for all analytics IDs or per analytics IDs that can be supported by an NWDAF AnLF (i.e., exposing the Analtyics Sub scription/ Analyticsinfo service).

• Rollback capability for ML models, such ML models being associated with all analytics IDs or per analytics IDs supported by an NWDAF MTLF (i.e., exposing the MLModel Provisioning service).

The discovery of an NWDAF with rollback and/or tracing and/or analytics monitoring support as well as the selection of an NWDAF based on tracing and/or rollback and/or analytics monitoring support could be initiated by a NF consumer (e.g., any NF, AF etc.) and or another NWDAF.

The NF consumers and/or another NWDAF search (/discovers) NRF including in the search (discover) request the following enhanced parameters:

• Tracing capability or Tracing capability per one or more analytics ID.

• Rollback capability or Rollback capability per one or more analytics ID.

• Analytics monitoring capability or Analytics Monitoring capability per one or more analytics ID.

NF consumers 120 or another NWDAF may select the NWDAFs based on the discovery NWDAF profiles 201, 202 with the enhanced required capabilities (i.e., tracing and/or rollback and/or analytics monitoring).

In addition, the NF consumers subscribe to NRF to be notified on changes of the tracing and/or rollback and/or monitoring capabilities of associated to an NWDAF, or associated to the analytics IDs supported by an NWDAF, or associated with specific analytics IDs supported by an NWDAF. The NF consumers use the notifications from NRF about the changes in the tracing and/or rollback and/or monitoring capabilities to re-select an NWDAF.

Finally, extension of the NRF services (TS 23.502) such as Nnrf_NFManagement_NFRegister, Nnrf_NFDiscovery_Request, Nnrf_NFManagement_NFStatusSubscribe can be may by including one more of the following parameters: Tracing capability for all analytics IDs or per analytics IDs, Rollback capability for all analytics IDs or per analytics IDs, Analytics Monitoring capability for all analytics IDs or per analytics IDs.

FIG. 9 shows a method 900 for a first network analytics entity 100 according to this disclosure. The method 900 comprises a step 901 of providing analytics tracing information 101 to a second network analytics entity 110. The method 900 further comprises, based on the analytics tracing information 101, at least one of the following steps 902, 903 and 904. A step 902 of tracing one or more analytics outputs for an analytics ID and/or tracing the analytics ID. A step 903 of determining at least one unstable analytics output for the analytics ID and/or determining that the analytics ID is unstable. A step 904 of determining and/or executing an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the unstable analytics ID.

FIG. 10 shows a method 1000 for a second network analytics entity 110 according to this disclosure. The method 1000 comprises a step 1001 of receiving analytics tracing information 101 from a first network analytics entity 100. The method 1000 further comprises, based on the analytics tracing information 101, at least one of the following steps 1002, 1003 and 1004. A step 1002 of tracing one or more analytics outputs for an analytics ID, and/or tracing the analytics ID. A step 1003 of determining at least one unstable analytics output for the analytics ID and/or determining that the analytics ID is unstable. A step 1004 of determining and/or execute an inference rollback action and/or a training rollback action for the at least one unstable analytics output for the analytics ID and/or for the unstable analytics ID.

FIG. 11 shows a method 1100 for a network data analytics consumer entity 120 according to this disclosure. The method 1100 comprises a step 1101 of providing, to a first network analytics entity, an indication 121 with information to activate an analytics tracing and/or a rollback of one or more analytics outputs for an analytics ID and/or of the analytics ID. Further, the method 1100 comprises a step 1102 of receiving, from a second network analytics entity 110, an indication 122 that the analytics tracing and/or the rollback are supported.

Advantages of the entities 100, 110, 120, 200, and of the methods 900, 1000, 1100 of this disclosure are as follows: • Avoid interruption on the tracing of an analytics status if the target NWDAF 110 is not aware about the need for supporting tracing and rollback for the transferred analytics ID especially.

■ Change of NWDAF can happen due to UEs mobility that could also include change of the NF consumer (e.g., AMF).

• Reduce signaling of triggering multiple times the tracing and rollback mechanisms for the exact the same analytics ID in multiple NWDAFs due to move from source NWDAF 100 to target NWDAF 110.

• Reduce risk of inconsistent tracing of analytics in the system.

■ The source NWDAF 100 might use a complete different set of configurations and ML model.

■ If the target NWDAF 110 triggers in a unilateral manner its own tracing support without any knowledge of the previous traces for the same analytics subscription, there might exist inconsistences on the stable state of an analytics.

• NF Consumers 120 of analytics IDs have assurances to keep consuming the analytics output that is leading to a stable network status (even if the NF sent a feedback indicating a problem for a previous analytics output).

• No interruption or break on the criteria used by NFs for the decision-making (e.g., by removing the analytics output as a criteria when NF sent feedback of problem to NWDAF), especially during NWDAF relocation.

• Assurances that analytics ID usage deteriorating system KPIs (i.e., leading to unstable network status) become visible and traceable to NWDAFs with inference and training capability without delays due to data collection, even during NWDAF relocation.

• Allow the chains/trees of NWDAFs with training capability sharing the ML model associated with unstable analytics ID to become automatically aware of such relationship.

DEFINITIONS

In the following, some explanations of terms (e.g., entities) are provided, and what these may be configured to, which may be useful for this disclosure.

Inference NF:

• (Mandatory) Obtain the analytics rollback actions, examples of how it obtain are: ■ Receive a message from the analytics tracing entity with the analytics rollback actions. Where the messages can be: Unstable Analytics Notification, Inference Rollback Notification

■ Be configured with analytics rollback actions

■ Identification of the analytics rollback actions.

(Mandatory) Execute actions based on the information from the obtained analytics rollback actions, which can comprise any of the following:

■ Exchange old with new obtained AICI and/or ATCI and use the new AICI and/or ATCI for the generation of the analytics (ID and/or output(s));

■ Determine the new AICI and use this information for the generation of the analytics (ID and/or output(s));

■ Determine (e.g., reselect) a new ML model and/or model to be used for the generation of the analytics (ID and/or output(s)), where the determination relates to the interaction between Inference NF and Training NF in order to obtain a new ML model and/or model for the analytics (ID and/or output(s));

(Mandatory) Based on the obtained analytics rollback actions and/or execute actions, provide and/or forward to further entities (e.g., Analytics Consumer, Training NF, Inference NFs) any of the following:

■ Unstable Analytics Notification,

■ Confirmation Unstable Analytics Notification

■ Analytics Status Notification

(Optional) Provide and/or forward the indication of analytics ID tracing activation for other entities (e.g., Analytics Tracing Entity)

(Optional) In order to determine the new ML model and/or model to be used for the generation of analytics (ID and/or output(s)), provide to a Training NF an indication of the reason for re-selection related to an unstable analytics ID, where the indication can be of any of the following types:

■ An Unstable Analytics Notification

■ A request for ML model and/or model selection (and/or reselection) with information about the reason for the reselection (e.g., unstable analytics ID)

(Optional) In order to determine the new ML model and/or model to be used for the generation of analytics (ID and/or output(s)), Provide to a Training NF an indication of the reason for re-training related to an unstable analytics ID, where the indication can be a request for ML model and/or model training (and/or re-training) with information about the reason for the retraining (e.g., unstable analytics ID)

• (Optional) Obtain from a further entity (e.g., Analytics Consumer) an indication for analytics subscription reactivation and use this information for activating an analytics subscription that has been previously suspended.

• (Optional) Provide from other entities (e.g., Analytics Tracing Entity) the Rollback Status Notification.

• (Optional) Obtain from a further entity (e.g., Analytics Tracing Entity) an Indication of Inference Tracing Activation and based on it generate the Analytics Inference Configuration Information (AICI)

• (Optional) Provide to the Analytics Tracing Entity any of the following information:

■ the Analytics Inference Configuration Information (AICI) for analytics output(s) and/or analytics IDs

■ analytics output identification or the analytics output itself and analytics identification;

Training NF:

• (Mandatory) Obtain from another entity (e.g., Analytics Tracing Entity) the analytics rollback actions, examples of how analytics rollback actions are obtained:

■ Receive a message from other entity (e.g., Analytics Tracing Entity, Inference NF, training NF) with the analytics rollback actions. Where the messages can be: Unstable Analytics Notification, Training Rollback Notification

■ Be configured with analytics rollback actions

• (Mandatory) Execute actions based on the information from the obtained analytics rollback actions, which can comprise any of the following:

■ Exchange old with new obtained ATCI and use the new ATCI for the training or model tuning of ML model and/or model associated with the analytics (ID and/or output(s));

■ Identify the ML model and/or model associated with the analytics (ID and/or output(s)) of the new obtained ATCI, select this new ML model and/or model and its associated ATCI to be used by consumers of ML model and/or model for the analytics (ID and/or output(s)); ■ Determine the new ATCI to be used for the ML model and/or model associated with the analytics (ID and/or output(s)) via training or model tuning;

■ Determine based on shared training with other Training NFs the new ATCI to be used for the ML model and/or model associated with the analytics (ID and/or output(s)), where shared training involves the exchange of an indication of the reason for re-selection related to an unstable analytics ID and/or an indication of the reason for re-training related to an unstable analytics ID among the Training NFs performing shared training.

■ Mark the ML model and/or model associated with the analytics (ID and/or output(s)) of the old obtained ATCI as deactivated;

(Mandatory) Based on obtained analytics rollback actions and/or executed actions, provide and/or forward to further entities (e.g., other Inference NF, Training NFs) any of the following:

■ Unstable Analytics Notification

■ Confirmation Unstable Analytics Notification

■ Training Rollback Information

■ Analytics Status Notification

■ An indication of the reason for re-selection related to an unstable analytics ID

■ An indication of the reason for re-training

(Optional) Obtain from another entity (e.g., Inference NF, other Training NFs) an indication of the reason for re-selection related to an unstable analytics ID, where the indication can be of any of the following types:

■ An Unstable Analytics Notification

■ A request for ML model and/or model selection (and/or reselection) with information about unstable analytics (ID and/or output(s))

(Optional) Obtain from another entity (e.g., Inference NF, Training NF) an indication of the reason for re-training related to an unstable analytics ID, where the indication can be a request for ML model and/or model training (and/or re-training) with information about the reason for the retraining (e.g., unstable analytics ID)

(Optional) Based on the obtained indication of the reason for re-selection related to an unstable analytics ID and/or an indication of the reason for re-training related to an unstable analytics ID determine the local ATCI (i.e., the ATCI that is calculated determined by the one Training NF that obtained the indications) of the ML model and/or model for the analytics (ID and/or output(s)).

• (Optional) Provide to other entity (e.g., Analytics Tracing Entity, Inference NF) the indication of analytics ID tracing activation

• (Optional) Provide from other entities (e.g., Analytics Tracing Entity) the Rollback Status Notification

• (Optional) Obtain from a further entity (e.g., Analytics Tracing Entity) an Indication of Training Tracing Activation and based on it generate the Analytics Training Configuration Information (ATCI)

• (Optional) Provide to the Analytics Tracing Entity any of the following information:

■ the Analytics Training Configuration Information (ATCI) for analytics output(s) and/or analytics IDs

Analytics Tracing Entity:

• (Mandatory) Obtain an indication for analytics ID tracing activation

• (Mandatory) Determine the analytics rollback actions based on the obtained analytics ID tracing activation

• (Mandatory) Provide the analytics rollback actions to further entities (e.g., Analytics Consumer, Inference NF, Training NF), where the analytics rollback actions can be any of the following:

■ Unstable Analytics Notification

■ Inference Rollback Notification

■ Training Rollback Notification

• (Optional) Obtain from other entities (e.g., Inference NF, Training NF) the Rollback Status Notification

• (Optional) Provide to further entities (e.g., Analytics Consumer) any of the following:

■ Analytics Status Notification

■ Confirmation of Unstable Analytics Notification

■ Indication of Inference Tracing Activation

■ Indication of Training Tracing Activation • (Optional) Obtain the Analytics Tracing Data Structure (ATDS) records that are associated to an ATDS and further use this information for the determination of the analytics rollback actions.

■ Examples of how ATDS record is obtained are:

■ be configured with Analytics Tracing Data Structure, by a Network Management Entity

■ compose or assemble the ATDS record by combining any of the following information:

• timestamp;

• analytics identification from Inference NF;

• analytics output identification or the analytics output itself from Inference NF;

• an indication of the quality of usage of the analytics (ID and/or output(s)) in the mobile network, from a Analytics Consumer and/or from a Analytics Monitoring Entity

• Analytics Inference Configuration Information (AICI) for the analytics output(s) and/or analytics IDs from Inference NF

• Analytics Training Configuration Information (ATCI) for the analytics output(s) and/or analytics IDs from Training NF

• (Optional) Provide to Analytics Monitoring Entity an indication for collecting data related to Analytics Performance Information.

Analytics Monitoring Entity:

• (Mandatory) Provide the indication of the quality of usage of the analytics (ID and/or output(s)) for other entities (e.g., Analytics Tracing Entity), where the indication of the quality of usage of the analytics can be any of the following:

■ Analytics ID Grade Information (AidGI)

■ Unstable Analytics ID Information (UAiDI)

• (Optional) Obtain from other entity (e.g., Analytics Tracing Entity, Inference NF, Training NF) the Analytics Performance Information (API), where any of the following can be examples of how the API is obtained:

■ a configuration from a Management Entity; ■ combination of configuration from a Management Entity and receiving a message from another entity comprising the specific analytics (ID and/or output(s)) and/or analytics consumer that should be associated with the configured API

■ a message received from other entity (e.g., Analytics Tracing Entity, , Inference NF, Training NF) comprising the API, the specific analytics (ID and/or output(s)) and/or analytics consumer

• (Optional) Based on the obtained indication for collecting data related to Analytics Performance Information, determine or calculate the Analytics ID Grade Information (AidGI) and/or Unstable Analytics ID Information (UAiDI)

Unstable Analytics ID: One or more outputs of an Analytics ID (e.g., an analytics type) and/or an Analytics ID that are associated with unstable network status. Synonyms of the term unstable analytics ID are: unstable analytics output, inefficient analytics output (or ID, or type), unreliable analytics output (or ID, or type), low performance analytics outputs (or ID, or type). The one or more outputs of an analytics ID and/or analytics ID that is considered unstable analytics ID are associated with indications of low network performance. Examples of these indications are: feedback from an NF consumer (NF Feedback), Analytics ID grades, notification about analytics ID grade that deviates from defined/configured/expected threshold values.

Stable Analytics ID: One or more output of an Analytics ID (e.g., an analytics type) and/or analytics ID that leads to (e.g., that is related to or associated with) stable network status. Synonyms of the term stable analytics ID are: stable analytics output, efficient analytics output (or ID, or type), reliable analytics output (or ID, or type), high performance analytics outputs (or ID, or type).

Network Status: Is the set of information that quantify the network performance. Examples of information that can be used for representing the network status are: KPIs and/or metrics associated with a network.

Stable Network Status: Defines a network status where the KPIs and/or metrics related to the network are kept within the expected pattern of usage (or improve). Unstable Network Status: Defines the network status where KPIs and/or metrics related to the overall performance of the network remain at expected patterns but specific KPIs and/or metrics indicating specific situations are oscillating out of expected pattern or decrease from expected pattern. Examples of overall performance metrics are: total number of accepted PDU sessions, average link throughput, total number of UE registrations per slice. Examples of specific KPIs or metrics that are specific are: number of rejected sessions per type of application; link usage per group of UEs. KPIs or metrics related to overall performance of the network can be generally related to performance information that can be obtained from OAM, Management Entities. While the specific KPIs and/or metrics can be generally related to performance information that can be obtained from Control Plane Entities (e.g., SMF, PCF, AMF).

Information about unstable analytics ID: Defines the set of data and/or parameters and/or properties, and/or configurations of an analytics ID (e.g., analytics type, analytics output(s)) at inference and/or at training where this set is associated with an unstable analytics ID.

Information on how to repair an unstable analytics ID: Defines the set of data and/or parameters and/or properties, and/or configurations of an analytics ID (e.g., analytics type, analytics output(s)) at inference and/or training associated with a last known stable network state in the mobile system.

Tracing of an analytics ID: Denotes the process executed for associating an analytics ID to analytics output(s), an indication of the quality of usage of the analytics (ID and/or output(s)) in the mobile network, and the configuration at inference and/or training for such analytics ID. In other words, is the process for creating an ATDS for a given analytics ID and creating over time the ATDS records for such analytics ID.

Indication for analytics ID tracing activation: A message that contains the analytics ID tracing activation.

Analytics ID tracing activation: One or more parameters or information that denotes the tracing of an analytics ID needs to be started. Examples of these parameters are: a flag (e.g., tracing flag or rollback flag), a tuple with analytics ID and flag. Indication of inference tracing activation: One or more parameters or information that denotes the tracing of inference configurations for an analytics (ID and/or output(s) needs to be started. For instance, this one or more parameter indicates to the Inference NF that it needs to create AICI information for the indicated analytics and provide such information to the Analytics Tracing Entity.

Indication of training tracing activation: One or more parameters or information that denotes the tracing of training configurations for an analytics (ID and/or output(s) needs to be started. For instance, this one or more parameter indicates to the training NF that it needs to create ATCI information for the indicated analytics and provide such information to the Analytics Tracing Entity.

Configuration of analytics ID tracing activation: Is the set of information provided by a Management Entity (e.g., OAM) that defines the tracing of an analytics ID needs to be started.

Analytics output: Is the set of information that denotes the result of a generated, calculated, predicted instance of an analytics ID or type. Different analytics IDs have a different set of information denoting the analytics output. For instance, and analytics ID such as “Observed Service experience information” as defined in 3GPP TS 23.288 Release 17 defines that the output for such analytics ID is the set of the following information: S-NSSAI, Slice instance service experiences (O...max), and for each of the NSI IDs related to an S-NSSAI there are other information such as: NSI ID, Slice instance service experiences (O...max)„ SUPI List, (O..SUPImax), Estimated percentage of UEs with similar service experience, etc.

Analytics ID or Analytics Type: Is a type of an analytics that can be generated by an Inference NF (e.g., NWDAF). For instance, 3GPP TS 23.288 Release 17 defines the Observed Service experience information, Slice Load level information, Network Performance information, etc.

ATDS Record: ATDS record defines a network state at a given point in time when using a given analytics (e.g., analytics ID or analytics type or analytics output(s)). ATDS record is a tuple that has the mapping in a given point in time of an analytics ID to: one or more analytics output identifications and/or one or more analytics output, an indication of the quality of usage of the analytics (ID and/or output(s)) in the mobile network (e.g., Analytics ID Grade Information (AidGI) or unstable analytics ID information (UAiDI) for analytics output instance(s) or a Network Function (NF) feedback), and inference and/or training configuration information (e.g., AICI and/or ATCI) for the analytics output(s) or analytics IDs. Optionally the ATDS record can also include the rollback status and the analytics rollback actions.

Last known stable network state: Is a network state (e.g., ATDS record) at a given point in time when using a given analytics (e.g., analytics ID or analytics type or analytics output(s)) with the most appropriated Analytics Inference Configuration Information (AICI) and/or Analytics Training Configuration Information (ATCI) for analytics (ID and/or output(s)).

Analytics Rollback Actions: Is the set of one or more information that define the possible actions that can be taken in order to change, revert, and/or reconfigure the configuration associated with an analytics (e.g., analytics output(s) or analytics ID, or analytics type). This action or actions can help to improve the performance of analytics ID. Examples of this information and how they denote an action are described as follows:

• One set of information is related to the old/current configuration, parameters, properties associated with an analytics (ID and/or output(s)) and a second set of information is related to the new possible configuration, parameters, properties associated with an analytics (ID and/or output(s)). This two sets of information defines that an action of replacing the old information with the new information should be executed.

• Only one set of information related to the new configuration, parameters, properties associated with an analytics (ID and/or output(s)). This denotes that the new information should be used to replace the existing one related to the analytics (ID and/or output(s)).

• Only one set of information related to the old/current configuration. This denotes that a specific action for rollback or reconfiguration or repairing the analytics was not identified.

Rollback Status Notification: Is the set of information describing the results of performing Analytics Rollback Actions. Examples of the set of information describing the results of the actions are the following:

• Successful rollback, optionally indicating whether the actions and/or configurations included in the Inference Rollback Notification or Training Rollback Notification have been implemented (e.g. used to change the configuration) or if a different actions and/or configurations from the included in Inference Rollback Notification or Training Rollback Notification have been used, optionally indicating the different actions and/or configurations used.

• Unsuccessful rollback, optionally indicating whether the indications of actions and/or configurations included in the Inference Rollback Notification or Training Rollback Notification have not been implemented (e.g. suggested configuration was not able to be used).

• No rollback, which indicates that the one or more suggested Analytics Rollback Actions by the Analytics Tracing Entity were not used.

Analytics Status Notification: is the information defining an analytics (ID and/or analytics output(s)) is considered as Stable Analytics ID, which means that there is no problem with the analytics, or that the analytics is not anymore considered Unstable Analytics ID, which means there is problem with the analytics.

Indication of the quality of usage of the analytics: Is the information that defines the quality of an analytics ID (or analytics type) and/or analytics output(s) in relationship to the network of such analytics (analytics ID or analytics type, and/or analytics output(s)).

Analytics ID Performance Information (API): Is a data structure that comprises the KPIs, metrics to be monitored and associated with a consumed analytics (ID and/or analytics output(s)), optionally for a specific consumer of such analytics (e.g., Analytics Consumer). The API serves as input for the AidGI calculation and support the identification of the effect of an analytics ID consumption in the changes of network status.

Analytics ID Grade Information (AidGI): Set of information that quantifies, e.g., in the format of a grade the effects of a consumed analytics (ID and/or output(s) on the changes in network status after the consumption of analytics ID. Examples of the set of information comprising the AidGI are any of the following:

• Identification of AidGI

• Identification of Analytics ID type: the type analytics ID for which a grade has to be calculated • (List of) Identification of analytics ID output: identification of the one or more analytics output information for the analytics ID that were consumed in the interval of time for the grade calculation

• Grade Value: a single value that represents how the APIs monitored for the consumed analytics ID diverge from the expected pattern of network status. For instance:

■ Grade value can be a real number between 1 and -1, where

■ 0 indicates that the consumption of the analytics ID had no significant effect in the expected network status pattern

■ 1 indicates that the consumption of the analytics ID had significant positive effect in the expected network status pattern (e.g., improved the pattern of the KPIs)

■ -1 indicates that the consumption of the analytics ID had significant negative effect in the expected network status pattern (e.g., degraded the pattern of the KPIs)

Unstable Analytics ID Information (UAidl): Is the set of information that identifies the grade associated with a consumed analytics (ID and/or outputs) crossing, deviating thresholds that denote an analytics has been identified as leading to unstable network status. Examples of information comprising the UAidl are any of the following:

• Identification of UAidl

• Identification of Analytics ID type: the type analytics ID for which a grade has to be calculated

• (List of) Identification of analytics ID output: identification of the one or more analytics output information for the analytics ID that were consumed in the interval of time for the grade calculation

• Calculated grade

• Grade value deviation from thresholds

NF Feedback: Is the information that qualifies the consumption of an analytics (ID and/or output(s)) in the perspective of the consumer of such analytics. Examples of the NF feedback information can be any of the following: • NF Feedback can be a set of information equivalent to the Analytics ID Grade Information (AidGI), where the NF (e.g., Analytics Consumer) based on its internal information defines a grade for the consumed analytics (ID and/or output(s))

• NF Feedback can be a set of information equivalent to the Unstable Analytics ID Information (UAidl), where the NF (e.g., Analytics Consumer) based on its internal information identifies grades for the consumed analytics (ID and/or output(s)) and thresholds being crossed indicating problems with the analytics.

• NF Feedback can be a tuple with the information about the analytics (ID and/or output(s)) and a flag indicating problems with the analytics.

Analytics Tracing Data Structure (ATDS): Is a set of network states for an analytics (e.g., analytics ID, analytics type, analytics output(s)). ATDS is a historical set of ADTS records that have the association of an analytics ID to analytics output(s), an indication of the quality of usage of the analytics in the mobile network, and the configuration at inference and/or training for the analytics (which can mean analytics output(s) or analytics IDs or analytics types), and optionally the analytics rollback actions and/or rollback status. With this data structure it is possible to traces over time for an analytics ID, the different association of analytics output(s) of this analytics ID, analytics ID grade values (AidGI) and/or UAidl alerts and/or NF feedbacks, the information of the inference and/or training configurations (e.g., AICI and/or ATCI) for the analytics output(s) or analytics IDs, and optionally the rollback status if an analytics rollback has been performed and associated with a ATDS record. . The analytics ID grade values (AidGI) and/or UAidl alerts are examples of indication of the quality of usage the analytics ID in the mobile network.

Analytics ID Inference Configuration Information (AICI): Set of information related to configuration and/or parametrization of used by an Inference NF for generating an analytics (ID and/or output). An AICI comprises any of the examples of information listed below:

• Configurations are: interval of data collection for generating the analytics; whether preprocessing of collected data for analytics generation has been applied; type of collected data for generating the analytics; mappings of Access Network properties (such as TAs, Cells, Radio Type, Radio Frequency) to Core Network entities (e.g., associated TAs to AMFs ) or properties (e.g., restricted or unrestricted S-NSSAIs per TAs); geographical aggregation level (e.g., per UEs, per Aol), temporal aggregation (i.e., per minute, per hour).

• Parametrization are related to the Analytics Filter Information and Analytics Reporting Information (as defined in 3 GPP TS 23.288 Release 17) for the generation of an analytics, or related to the Data Specification or Format and Processing (as defined in 3GPP TS 23.288 Release 17) definitions when collecting the data for the generation of an analytics

Analytics ID Training Configuration Information (ATCI): Set of Information related to configuration and/or parametrization for a Machine Learning (ML) Model or simply Model (e.g., optimization model) used by for an analytics (ID and/or output(s)) in a given point in time. An ATCI comprises any of the examples of information listed below:

• Configurations are: interval of data collection for training the ML model or model for the analytics; whether pre-processing of collected data for analytics training has been applied; type of collected data for training the analytics; mappings of Access Network properties (such as TAs, Cells, Radio Type, Radio Frequency) to Core Network entities (e.g., associated TAs to AMFs ) or properties (e.g., restricted or unrestricted S-NSSAIs per TAs), geographical aggregation level (e.g., per UEs, per Aol), temporal aggregation (i.e., per minute, per hour).

• Parametrization related to the requested analytics ID to be trained or modeled: Analytics Filter Information and Analytics Reporting Information (as defined in 3GPP TS 23.288 Release 17) for the analytics, or related to the Data Specification or Format and Processing (as defined in 3GPP TS 23.288 Release 17) definitions when collecting the data for the analytics training or statistics calculation

• Parametrization related to the ML model or model: type of algorithm; gradients; weights; architecture information about the model (e.g., if neural networks the number of layers, the amount of neurons per layer), type of activation functions, type of loss functions, number of epochs used for training, configuration of the model validation (e.g., 5- crossfold validation), distribution of the amount of data for the different phases (training, validation testing), score for all phases training, validation, and test; type of used metrics (e.g., median absolute error, mean square error, coefficient of determination R ^A 2, Fl- Score, Accuracy, Recall, Precision, etc.) Inference Rollback Notification (IRN): Is the set of information, possibly enclosed in a message, defining the rollback actions related to an existing analytics ID that can be enforced by an Inference NF. The Inference Rollback Notification, can be for instance:

• a message containing the one ATDS record with the unstable analytics ID information and another ATDS record with the ATDS record associated with the last known stable network state for the analytics ID.

• Another possible embodiment for an Inference Rollback Notification is to have parameter such as tuple with the identification of analytics ID and/or analytics output and a flag, where this flag denotes that there is a problem (or an error) with the analytics (ID and/or output(s)). Examples of flags that can be used are: unstable analytics, halt analytics usage, invalid analytics, temporary invalid analytics, trigger inference rollback.

Training Rollback Notification (TRN): Is the set of information, possibly enclosed in a message, related the rollback actions related to an existing analytics (ID and/or output(s)) that can be enforced by a Training NF. The Training Rollback Notification can be for instance:

• A message containing the training information (e.g., ATCI) of an ATDS record with the ML model configuration or parametrization for the unstable analytics ID and another training information (ATCI) of the ATDS record with ATDS record associated with the last known stable network state for a ML model for the analytics ID.

• Another possible embodiment for a Training Rollback Notification is to have parameter such as tuple with the identification of analytics ID and/or analytics output and a flag, where this flag denotes that there is a problem (or an error) with the analytics (ID and/or output(s)). Examples of flags that can be used are: unstable analytics, halt analytics usage, invalid analytics, temporary invalid analytics, trigger training rollback.

Unstable Analytics Notification (UN): Is the set of information that identifies an analytics ID and/or analytics output(s) as Unstable Analytics ID, i.e., an analytics (ID and/or output(s)) that are related to problems in the network status. The Unstable Analytics Notification comprises any of the information listed as examples below:

• The ATDS record with the Analytics Inference Configuration Information (AICI) and/or Analytics Training Configuration Information (ATCI) for the analytics ID of the ATDS record that has been considered as Unstable Analytics ID. This information will indicate that there is a problem (and/or an error) with the analytics ID and/or analytics outputs. • A parameter such as tuple with the identification of analytics ID and/or analytics output and a flag, where this flag denotes that there is a problem (or an error) with the analytics (ID and/or output(s)). Examples of flags that can be used are: unstable analytics, halt analytics usage, invalid analytics, temporary invalid analytics.

Indication for re-selection: Is the set of information that relates an ML model and/or model to an analytics ID and/or analytics output(s) that is considered Unstable Analytics ID, i.e., an analytics (ID and/or output(s)) that are related to problems in the network status.

Indication for re-training: Is the set of information that relates the need (e.g., request) for ML model and/or model training (and/or retraining) to an analytics ID and/or analytics output(s) that is considered Unstable Analytics ID, i.e., an analytics (ID and/or output(s)) that are related to problems in the network status.

Training Rollback Information: Is the set of information that indicates a change in the ML model and/or model related to an analytics ID and/or analytics output(s) due to the ML model and/or model has been identified as associated with an Unstable Analytics ID, i.e., an analytics (ID and/or output(s)) that are related to problems in the network status.

The present disclosure has been described in conjunction with various embodiments as examples as well as implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed matter, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.