NODES, AND METHODS FOR HANDLING A PERFORMANCE EVALUATION OF AN AE-ENCODER

BACKGROUND

In a typical wireless communication network, wireless devices, also known as wireless communication devices, mobile stations, stations (STA) and/or User Equipments (UE), communicate via a Local Area Network such as a Wi-Fi network or a Radio Access Network (RAN) to one or more core networks (CN). The RAN covers a geographical area which is divided into service areas or cell areas. Each service area or cell area may provide radio coverage via a beam or a beam group. Each service area or cell area is typically served by a radio access node such as a radio access node e.g., a Wi-Fi access point or a radio base station (RBS), which in some networks may also be denoted, for example, a NodeB, eNodeB (eNB), or gNB as denoted in 5G. A service area or cell area is a geographical area where radio coverage is provided by the radio access node. The radio access node communicates over an air interface operating on radio frequencies with the wireless device within range of the radio access node.

Specifications for the Evolved Packet System (EPS), also called a Fourth Generation (4G) network, have been completed within the 3rd Generation Partnership Project (3GPP). A Fifth Generation (5G) network also referred to as 5G New Radio (NR) has also been specified and work is now directed to further specifications of the 5G network. This work will continue in the coming 3GPP releases.

For reference, the EPS comprises the Evolved Universal Terrestrial Radio Access Network (E-UTRAN), also known as the Long Term Evolution (LTE) radio access network, and the Evolved Packet Core (EPC), also known as System Architecture Evolution (SAE) core network. E-UTRAN/LTE is a variant of a 3GPP radio access network wherein the radio access nodes are directly connected to the EPC core network rather than to RNCs used in 3G networks. In general, in E-UTRAN/LTE the functions of a 3G RNC are distributed between the radio access nodes, e.g. eNodeBs in LTE, and the core network. As such, the RAN of an EPS has an essentially “flat” architecture comprising radio access nodes connected directly to one or more core networks, i.e. they are not connected to RNCs. To compensate for that, the E-UTRAN specification defines a direct interface between the radio access nodes, this interface being denoted the X2 interface. Wireless communication systems in 3GPP

Figure 1 illustrates a simplified wireless communication system. Consider the simplified wireless communication system in Figure 1, with a UE 12, which communicates with one or multiple access nodes 103-104, which in turn is connected to a network node 106. The access nodes 103-104 are part of the radio access network 10.

For wireless communication systems pursuant to 3GPP Evolved Packet System, (EPS), also referred to as Long Term Evolution, LTE, or 4G, standard specifications, such as specified in 3GPP TS 36.300 and related specifications, the access nodes 103-104 corresponds typically to Evolved NodeBs (eNBs) and the network node 106 corresponds typically to either a Mobility Management Entity (MME) and/or a Serving Gateway (SGW). The eNB is part of the radio access network 10, which in this case is the E-UTRAN (Evolved Universal Terrestrial Radio Access Network), while the MME and SGW are both part of the EPC (Evolved Packet Core network). The eNBs are inter-connected via the X2 interface, and connected to EPC via the S1 interface, more specifically via S1-C to the MME and S1-U to the SGW.

For wireless communication systems pursuant to 3GPP 5G System, 5GS (also referred to as New Radio, NR, or 5G) standard specifications, such as specified in 3GPP TS 38.300 and related specifications, on the other hand, the access nodes 103-104 corresponds typically to an 5G NodeB (gNB) and the network node 106 corresponds typically to either an Access and Mobility Management Function (AMF) and/or a User Plane Function (UPF). The gNB is part of the radio access network 10, which in this case is the NG-RAN (Next Generation Radio Access Network), while the AMF and UPF are both part of the 5G Core Network (5GC). The gNBs are inter-connected via the Xn interface, and connected to 5GC via the NG interface, more specifically via NG-C to the AMF and NG-U to the UPF.

To support fast mobility between NR and LTE and avoid change of core network, LTE eNBs may also be connected to the 5G-CN via NG-U/NG-C and support the Xn interface. An eNB connected to 5GC is called a next generation eNB (ng-eNB) and is considered part of the NG-RAN. LTE connected to 5GC will not be discussed further in this document; however, it should be noted that most of the solutions/features described for LTE and NR in this document also apply to LTE connected to 5GC. In this document, when the term LTE is used without further specification it refers to LTE-EPC. NR uses Orthogonal Frequency Division Multiplexing (OFDM) with configurable bandwidths and subcarrier spacing to efficiently support a diverse set of use-cases and deployment scenarios. With respect to LTE, NR improves deployment flexibility, user throughputs, latency, and reliability. The throughput performance gains are enabled, in part, by enhanced support for Multi-User Multiple-Input Multiple-Output (MU-MIMO) transmission strategies, where two or more UEs receives data on the same time frequency resources, i.e. , by spatially separated transmissions.

A MU-MIMO transmission strategy will now be illustrated based on Figure 2. Figure 2 illustrates an example transmission and reception chain for MU-MIMO operations. Note that the order of modulation and precoding, or demodulation and combining respectively, may differ depending on the implementation of MU-MIMO transmission.

A multi-antenna base station with N x antenna ports is simultaneously, e.g., on the same OFDM time-frequency resources, transmitting information to several UEs: a sequence S ⁽¹⁾ is transmitted to is transmitted to UE(2), and so on. An antenna port may be a logical unit which may comprise one or more antenna elements. Before modulation and transmission, precoding is applied to each sequence to mitigate multiplexing interference - the transmissions are spatially separated.

Each UE demodulates its received signal and combines receiver antenna signals to obtain an estimate of the transmitted sequence. This estimate for UE / may be expressed as (neglecting other interference and noise sources except the MU-MIMO interference)

The second term represents the spatial multiplexing interference, due to MU-MIMO transmission, seen by UE(j). A goal for a wireless communication network may be to construct a set of precoders to meet a given target. One such target may be to make the norm large (this norm represents the desired channel gain towards user i); and the norm || H ^<J) Wy ’ || ,j = i small (this norm represents the interference of user i’s transmission received by user j).

In other words, the precoder Wy ¹ - ¹ shall correlate well with the channel observed by UE(j) whereas it shall correlate poorly with the channels observed by other UEs.

To construct precoders Wy ^L> , i = 1, ... that enable efficient MU -Ml MO transmissions, the wireless communication network may need to obtain detailed information about the users’ downlink (DL) channels H(j), i = 1, . . ,J. The wireless communication network may for example need to obtain detailed information about all the users downlink channels H(j), i = 1, . . ,J.

In deployments where full channel reciprocity holds, detailed channel information may be obtained from uplink (UL) Sounding Reference Signals (SRS) that are transmitted periodically, or on demand, by active UEs. The wireless communication network may directly estimate the uplink channel from SRS and, therefore (by reciprocity), the downlink channel Full channel reciprocity may be obtained in time division duplex (TDD) deployments for UEs with same number of transmitters (TX chains) as receive branches (RX chains). However, a typical scenario is that UEs have fewer TX chains than RX chains, so the radio access network may only be able to estimate part of the uplink channel using SRS (in which case only certain columns of a precoding matrix may be estimated using SRS). This situation is known as partial channel knowledge.

Thus, the wireless communication network cannot always accurately estimate the downlink channel from uplink reference signals. Consider the following examples:

In frequency division duplex (FDD) deployments, the uplink and downlink channels use different carriers and, therefore, the uplink channel may not provide enough information about the downlink channel to enable MU-MIMO precoding.

In TDD deployments, the wireless communication network may only be able to estimate part of the uplink channel using SRS because UEs typically have fewer TX branches than RX branches (in which case only certain columns of the precoding matrix may be estimated using SRS). This situation is known as partial channel knowledge. If the wireless communication network cannot accurately estimate the full downlink channel from uplink transmissions, then active UEs may report channel information to the wireless communication network over the uplink control or data channels. In LTE and NR, this feedback is achieved by the following signalling protocol:

The radio access network configures a UE to report CSI in a certain way. The wireless communication network transmits Channel State Information reference signals (CSI-RS) over the downlink, e.g., using N ports.

The UE estimates the downlink channel (or important features thereof, such as eigenvectors of the channel or the Gram matrix of the channel, one or more eigenvectors that correspond to the largest eigenvalues of an estimated channel covariance matrix, one or more Discrete Fourier Transform (DFT) base vectors (described on the next page), or orthogonal vectors from any other suitable and defined vector space, that best correlates with an estimated channel matrix, or an estimated channel covariance matrix, the channel delay profile), e.g., for each of the N antenna ports from the transmitted CSI-RS.

The UE reports CSI (e.g., channel quality index (CQI), precoding matrix indicator (PMI), rank indicator (Rl)) to the wireless communication network over an uplink control channel and/or over a data channel.

The wireless communication network uses the UE’s feedback, e.g., the CSI reported from the UE, for downlink user scheduling and precoding, such as MIMO precoding.

In NR, both Type I and Type II reporting is configurable, where the CSI Type II reporting protocol has been specifically designed to enable MU -Ml MO operations from uplink UE reports, such as the CSI reports. The CSI Type II may be configured in a normal reporting mode or in a port selection reporting mode.

The CSI Type II normal reporting mode is based on the specification of sets of Discrete Fourier Transform (DFT) basis functions in a precoder codebook. The UE selects and reports L DFT vectors from the codebook that best match its channel conditions (like the classical codebook precoding matrix indicator (PMI) from earlier 3GPP releases). The number of DFT vectors L is typically 2 or 4 and it is configurable by the wireless communication network. In addition, the UE reports how the L DFT vectors should be combined in terms of relative amplitude scaling and co-phasing.

Algorithms to select L, the L DFT vectors, and co-phasing coefficients are outside the specification scope -- left to UE and network implementation. Or, put another way, the 3gpp Rel. 16 specification only defines signaling protocols to enable the above message exchanges.

In the following, “DFT beams” will be used interchangeably with DFT vectors. This slight shift of terminology is for example appropriate whenever the base station has a uniform planar array with antenna elements separated by half of the carrier wavelength.

The CSI type II normal reporting mode is illustrated in Figure 3 and described in 3gpp TS 38.214 “Physical layer procedures for data” (Release 16). The selection and reporting of the L DFT vectors b _n and their relative amplitudes a _n is done in a wideband manner; that is, the same beams are used for both polarizations over the entire transmission frequency band. The selection and reporting of the DFT vector co-phasing coefficients are done in a subband manner; that is, DFT vector co-phasing parameters are determined for each of multiple subsets of contiguous subcarriers. The co-phasing parameters are quantized such that e ^jdn is taken from either a Quadrature phase-shift keying (QPSK) or 8-Phase Shift Keying (8PSK) signal constellation.

With k denoting a sub-band index, the precoder W _v [k] reported by the UE to the network can be expressed as follows:

The Type II CSI report may be used by the network to co-schedule multiple UEs on the same OFDM time-frequency resources. For example, the network can select UEs that have reported different sets of DFT vectors with weak correlations. The CSI Type II report enables the UE to report a precoder hypothesis that trades CSI resolution against uplink transmission overhead.

As mentioned above, NR 3GPP Release 15 supports Type II CSI feedback using port selection mode, in addition to the above normal reporting mode. In this case,

The base station transmits a CSI-RS port in each one of the beam directions.

The UE does not use a codebook to select a DFT vector (a beam), instead the UE selects one or multiple antenna ports from the CSI-RS resource of multiple ports.

Type II CSI feedback using port selection gives the base station some flexibility to use non-standardized precoders that are transparent to the UE. For the port-selection codebook, the precoder reported by the UE can be described as follows Here, the vector e is a unit vector with only one non-zero element, which can be viewed as a selection vector that selects a port from the set of ports in the measured CSI- RS resource. The UE thus feeds back which ports it has selected, the amplitude factors and the co-phasing factors.

Autoencoders for Al-enhanced CSI reporting

Recently neural network based autoencoders (AEs) have gained large interests in the wireless communications research community for its capability of compressing and decompressing (reconstructing) Ml MO radio channels accurately even at high compression ratios. For example, neural network based AEs have shown promising results for compressing downlink MIMO channel estimates for uplink feedback. That is, the AEs may be used to compress downlink MIMO channel estimates. The compressed output of the AE may then be used as uplink feedback. In other words, the use of the AE is here in the context of CSI compressing where a UE provides CSI feedback to a radio access network node by sending a CSI report that include a compressed and encoded version of the estimated downlink channel, or of important features thereof. For example, prior art document Zhilin Lu, Xudong Zhang, Hongyi He, Jintao Wang, and Jian Song, “Binarized Aggregated Network with Quantization: Flexible Deep Learning Deployment for CSI Feedback in Massive MIMO System”, arXiv, 2105.00354 v1 , May, 2021 provides a recent summary of academic work.

An AE is a type of Neural Network (NN), e.g., a type of machine learning algorithm, that may be used to compress and decompress data in an unsupervised manner.

Unsupervised learning is a type of machine learning in which the algorithm is not provided with any pre-assigned labels or scores for the training data. As a result, unsupervised learning algorithms may first self-discover any naturally occurring patterns in that training data set. Common examples include clustering, where the algorithm automatically groups its training examples into categories with similar features, and principal component analysis, where the algorithm finds ways to compress the training data set by identifying which features are most useful for discriminating between different training examples and discarding the rest. This contrasts with supervised learning in which the training data include pre-assigned category labels, often by a human, or from the output of non-learning classification algorithm. Figure 4a illustrates a fully connected (dense) AE (fully connected layers). The AE may be divided into two parts: an encoder (used to compress the input data ), and a decoder (used to recover important features of the input data).

The encoder and decoder are separated by a bottleneck layer that holds a compressed representation, Y in Figure 4a, of the input data X. The variable Y is sometimes called the latent representation of the input X. More specifically,

The size of the bottleneck (latent representation) Y is smaller than the size of the input data X. The AE encoder thus compresses the input features X to Y. The decoder part of the AE tries to invert the encoder’s compression and reconstruct X with minimal error, according to some predefined loss function.

AEs may have different architectures. For example, AEs may be based on dense NNs (like Figure 4a), multi-dimensional convolution NNs, recurrent NNs, transformer NNs, or any combination thereof. However, all AEs architectures possess an encoder- bottleneck-decoder structure, like the one presented in Figure 4a.

Figure 4b illustrates how an AE may be used for Al-enhanced CSI reporting in NR during an inference phase (that is, during live network operation).

The UE estimates the downlink channel (or important features thereof) using configured downlink reference signal(s), e.g., CSI-RS. As mentioned above, important features of the channel may be eigenvectors of the channel or the Gram matrix of the channel, one or more eigenvectors that correspond to the largest eigenvalues of an estimated channel covariance matrix, one or more Discrete Fourier Transform (DFT) base vectors (described on the next page), or orthogonal vectors from any other suitable and defined vector space, that best correlates with an estimated channel matrix, or an estimated channel covariance matrix, the channel delay profile). For example, the UE estimates the downlink channel as a 3D complex-valued tensor, with dimensions defined by the gNB’s Tx-antenna ports, the UE’s Rx antenna ports, and frequency units (the granularity of which is configurable, e.g., subcarrier or subband).

The UE uses a trained AE encoder to compress the estimated channel or important features thereof down to a binary codeword. The binary codework is reported to the network over an uplink control channel and/or data channel. In practice, this codeword will likely form one part of a channel state information (CSI) report that might also include rank, channel quality, and interference information.

The network uses a trained AE decoder to reconstruct the estimated channel or the important features thereof. The decompressed output of the AE decoder is used by the network in, for example, MIMO precoding, scheduling, and link adaption.

The architecture of an AE (e.g., structure, number of layers, nodes per layer, activation function etc) may need to be tailored for each particular use case. For example, properties of the data (e.g., CSI-RS channel estimates), the channel size, uplink feedback rate, and hardware limitations of the encoder and decoder may all need to be considered when designing the AE’s architecture.

After the AE’s architecture is fixed, it needs to be trained on one or more datasets. To achieve good performance during live operation in a network (the so-called inference phase), the training datasets need to be representative of the actual data the AE will encounter during live operation in the network.

The training process involves numerically tuning the AE’s trainable parameters (e.g., the weights and biases of the underlying NN) to minimize a loss function on the training datasets. The loss function may be, for example, the Mean Squared Error (MSE) loss calculated as the average of the squared error between the UE’s downlink channel estimate H and the network’s reconstruction H, i.e. , ,,\\H - H\\ ² . The purpose of the loss function is to meaningfully quantify the reconstruction error for the particular use case at hand.

The training process is typically based on some variant of the gradient descent algorithm, which, at its core, comprises three components: a feedforward step, a back propagation step, and a parameter optimization step. We now review these steps using a dense AE (i.e., a dense NN with a bottleneck layer, see Figure 4a) as an example.

Feedforward: A batch of training data, such as a mini-batch, (e.g., several downlink-channel estimates) is pushed through the AE, from the input to the output. The loss function is used to compute the reconstruction loss for all training samples in the batch. The reconstruction loss may be an average reconstruction loss for all training samples in the batch. The feedforward calculations of a dense AE with N layers (n = 1,2, .... N) may be written as follows: The output vector of layer n is computed from the output of the previous layer using the equations

In the above equation, W and b^ are the trainable weights and biases of layer n, respectively, and g is an activation function applied elementwise (for example, a rectified linear unit).

Back propagation (BP): The gradients (partial derivatives of the loss function, L, with respect to each trainable parameter in the AE) are computed. The back propagation algorithm sequentially works backwards from the AE output, layer-by-layer, back through the AE to the input. The back propagation algorithm is built around the chain rule for differentiation: When computing the gradients for layer n in the AE, it uses the gradients for layer n + 1.

For a dense AE with N layers the back propagation calculations for layer n may be expressed with the following well-known equations where * here denotes the Hadamard multiplication of two vectors.

Parameter optimization: The gradients computed in the back propagation step are used to update the AE’s trainable parameters. An approach is to use the gradient descent method with a learning rate hyperparameter (a) that scales the gradients of the weights and biases, as illustrated by the following update equations

A core idea here is to make small adjustments to each parameter with the aim of reducing the average loss over the (mini) batch. It is common to use special optimizers to update the AE’s trainable parameters using gradient information. The following optimizers are widely used to reduce training time and improving overall performance: adaptive subgradient methods (AdaGrad), RMSProp, and adaptive moment estimation (ADAM).

The above process (feedforward, back propagation, parameter optimization) is repeated many times until an acceptable level of performance is achieved on the training dataset. An acceptable level of performance may refer to the AE achieving a pre-defined average reconstruction error over the training dataset (e.g., normalized MSE of the reconstruction error over the training dataset is less than, say, 0.1). Alternatively, it may refer to the AE achieving a pre-defined user data throughput gain with respect to a baseline CSI reporting method (e.g., a MIMO precoding method is selected, and user throughputs are separately estimated for the baseline and the AE CSI reporting methods).

The above actions use numerical methods (e.g., gradient descent) to optimize the AE’s trainable parameters (e.g., weights and biases). The training process, however, typically involves optimizing many other parameters (e.g., higher-level hyperparameters that define the model or the training process). Some example hyperparameters are as follows:

• The architecture of the AE (e.g., dense, convolutional, transformer).

• Architecture-specific parameters (e.g., the number of nodes per layer in a dense network, or the kernel sizes of a convolutional network).

• The depth or size of the AE (e.g., number of layers).

• The activation functions used at each node within the AE.

• The mini-batch size (e.g., the number of channel samples fed into each iteration of the above training steps).

• The learning rate for gradient descent and/or the optimizer.

• The regularization method (e.g., weight regularization or dropout) Additional validation datasets may be used to tune the AE’s architecture and other hyperparameters.

SUMMARY

An object of embodiments herein may be to obviate some of the problems related to evaluation, such as testing and/or validation, of AEs in wireless communication networks. The evaluation may evaluate the performance of the AE. The evaluation may also be referred to as a model evaluation. According to a first aspect, the object is achieved by a method, performed by a wireless communications device operating in a wireless communication network, for handling a performance evaluation of a trained AE-encoder of the wireless communication device. The AE-encoder may be trained to provide encoded data to a compatible trained AE-decoder of a network node of the wireless communications network. The method may comprise receiving a trigger message for the performance evaluation of the AE-encoder. The method may further comprise providing, to a node, a report comprising an indication of a status of the performance evaluation.

According to a second aspect, the object is achieved by a method, performed by a node for handling a performance evaluation of a trained Auto Encoder, AE, -encoder of a wireless communications device. The AE-encoder may be trained to provide encoded data to a compatible trained AE-decoder of a network node of the wireless communications network. The method may comprise receiving, from the wireless communications device, a report comprising an indication of a status of the performance evaluation of the AE-encoder in the wireless communications network.

According to a third aspect, the object is achieved by a wireless communications device for operating in a wireless communications network, for handling a performance evaluation of a trained Auto Encoder, AE, -encoder of the wireless communications device. The AE-encoder may be trained to provide encoded data to a compatible trained AE-decoder of a network node of the wireless communications network. The wireless communications device may comprise processing circuitry. The wireless communications device and/or processing circuitry may be configured to receive a trigger message for the performance evaluation of the AE-encoder. The wireless communications device and/or processing circuitry may further be configured to provide, to a node, a report comprising an indication of a status of the performance evaluation.

According to a fourth aspect, the object is achieved by a node for handling a performance evaluation of a trained Auto Encoder, AE, -encoder of a wireless communications device. The AE-encoder may be trained to provide encoded data to a compatible trained AE-decoder of a network node of the wireless communications network. The node may comprise processing circuitry. The node and/or processing circuitry may be configured to receive, from the wireless communications device, a report comprising an indication of a status of the performance evaluation of the AE-encoder in the wireless communications network.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 illustrates a simplified wireless communication system

Figure 2 illustrates an example transmission and reception chain for MU -Ml MO operations

Figure 3 illustrates the CSI type II normal reporting mode

Figure 4a illustrates a fully connected AE

Figure 4b illustrates how an AE may be used for Al-enhanced CSI reporting in NR during an inference phase

Figure 4c illustrates training and inference pipelines

Figure 5 is a schematic overview depicting a wireless communications network

Figure 6aa illustrates a wireless communications device, a network node, and further nodes in accordance with some embodiments

Figure 6ab illustrates a wireless communications device, a network node, and further nodes in accordance with some embodiments

Figure 6b illustrates a wireless communications device, a network node, and further nodes in accordance with some embodiments

Figure 7a illustrates a signaling diagram in accordance with some embodiments

Figure 7b illustrates a flowchart in accordance with some embodiments

Figure 7c illustrates a flowchart in accordance with some embodiments

Figure 7d illustrates a flowchart in accordance with some embodiments

Figure 7e illustrates a flowchart in accordance with some embodiments

Figure 8 illustrates a flowchart in accordance with some embodiments

Figure 9a illustrates a flowchart in accordance with some embodiments

Figure 9b illustrates band combinations linked to feature set combinations

Figure 10 illustrates an example of a wireless communications device in accordance with some embodiments

Figure 11 illustrates an example of a second node in accordance with some embodiments

Figure 12 illustrates a communication system including a telecommunication network

Figure 13 illustrates example implementations of a UE, base station and host computer in accordance with some embodiments Figure 14 illustrates a flowchart of a method implemented in a communication system in accordance with some embodiments

Figure 15 illustrates a flowchart of a method implemented in a communication system in accordance with some embodiments

Figure 16 illustrates a flowchart of a method implemented in a communication system in accordance with some embodiments

Figure 17 illustrates a flowchart of a method implemented in a communication system in accordance with some embodiments

DETAILED DESCRIPTION

Typically, the AE training process is a highly iterative process that may be expensive - consuming significant time, compute, memory, and power resources. Therefore, it may be expected that AE architecture design and training will largely be performed offline, e.g., in a development environment, using appropriate compute infrastructure, training data, validation data, and test data.

A development-domain may refer to a software/simulation-based environment used by a vendor to develop algorithms and functionality to be implemented in a product.

Data for training, validation, and testing may be collected from one or more of the following examples: real measurements recorded in live networks, synthetic radio channel data from, e.g., 3GPP channel models or ray tracing models and/or digital twins, and mobile drive tests.

Validation data may be part of the development and tuning of the NN, whereas the test data may be applied to the final NN. For example, a “validation dataset” may be used to optimize AE hyperparameters (like its architecture). For example, two different AE architectures may be trained on the same training dataset. Then the performance of the two trained AE architectures may be validated on the validation dataset. The architecture with the best performance on the validation dataset may be kept for the inference phase. In other words, validation may be performed on the same data set as the training, but on “unseen” data samples (e.g. taken from the same source). Test may be performed on a new data set, usually from another source and it tests the NN ability to generalize. The neural network-based AE algorithm, the backpropagation gradient descent algorithm, the training data, and the validation data, as described above, result in a machine learning (ML)-based AE model with trainable weights and biases taking specific values. This AE model represents the outcome of an iterative training process, or training pipeline, from which the AE model is ready to be deployed and used to perform inference.

Building the AE model, or any machine learning model, includes several development steps where the actual training of the AE model is just one step in a training pipeline. An important part in AE developing is the ML model lifecycle management. Figure 4c illustrates training and inference pipelines, and their interactions within a model lifecycle management procedure. The AE model lifecycle management typically consists of

• A training pipeline, which may also be a re-training pipeline o With data ingestion referring to gathering raw (training) input data, like wireless channel data, from a data storage. After data ingestion, there may also be an action that controls the validity of the gathered data. o With data pre-processing referring to some feature engineering applied to the gathered data. For example, it may include data normalization and possibly a data transformation required for the input data to the AE. o With the actual model training actions as previously outlined. o With model evaluation referring to benchmarking the performance to some AE baseline. The iterative steps of model training and model evaluation may continue until the acceptable level of performance (as previously exemplified) is achieved. o With model registration referring to register the AE model, including any corresponding AE-meta data that provides information on how the AE model was developed, and possibly performance outcomes of the AE model evaluations.

• A deployment stage to make the trained (or re-trained) AE model part of the inference pipeline.

An inference pipeline, o With data ingestion referring to gathering raw (inference) input data from a data storage, which for the AE means gathering raw channel data from a data buffer storage of a UE. o With data pre-processing stage that is typically identical to corresponding processing that occurs in the training pipeline. o With model operational referring to using the trained and deployed model in an operational mode, which for the AE may mean that the UE feedforward the input data through its AE-encoder, sends the output data of the AE-encoder to the radio access network, which use the received output data as input to its AE-decoder. o With data & model monitoring referring to validating that the inference data are from a distribution that aligns well with the training data, as well as monitoring model outputs for detecting any performance, or operational, drifts. For the AE model, this may include both the AE-encoder operation in the UE and the AE- decoder operation in the radio access network.

• A drift detection stage that informs about any drifts in the model operations.

One problem with existing AE solutions is that they require a very tight cooperation between the UE (AE-encoder) and the base station (AE-decoder) to conduct performance validation in a test environment setup. Firstly, this may not always be possible since not all UE manufacturers may have direct contacts with the different base station manufacturers in question. Secondly, the AE-encoder may have been tested in a lab setting with limited sets of test data, or perhaps more importantly the lab test may have occurred before new relevant deployment scenarios were considered e.g., in terms of new radio propagations and antenna configurations, and thereby not captured in the lab test. The performance may then need to be verified or validated for UEs in the field, e.g., operating in a wireless communications system, directly rather than in the lab environment.

Another problem is that the AE-encoder, or AE-decoder, design may have a relatively short lifecycle, which will be more likely with machine learning driven designs. In other words, the ML research area is advancing so rapidly that there is a large risk that the design available in the field, e.g. in nodes operating in the wireless communication network, will be outdated quickly in terms of performance. Therefore, new designs need to be considered. In some scenarios it may not be possible to test a new design towards the other side (UE or network) in a lab environment when units, such as UEs and base stations, are already deployed, and hence the new design may need to be tested on already deployed units (UEs and base stations). This is particularly true for UEs if the base station design is new as there may be older models of the UEs in the field that are not available in the lab environment.

As mentioned above, there are challenges and issues with developing AE models for encoded reporting.

An object of embodiments herein is therefore to improve encoded reporting in communications networks. In particular, embodiments herein disclose improved handling of performance evaluation of AE-encoders and/or AE-decoders in a wireless communication network. The evaluation may be a model evaluation. Evaluating the performance may also be referred to as evaluating the AE-model.

Embodiments herein are mainly described from an CSI reporting use case when using AE-encoder. However, embodiments are not limited to that use case given that the AE-encoder may be used in other use cases as well, such as for positioning, data transfer and so on. For CSI reporting AEs may for example be used to compress a channel that is decompressed at the network side. A base station may be able to derive PM I from the decompressed channel. It is believed that this is a more efficient manner of representing the channel for PM I reporting than the hand-designed/theoretical approach that is currently used within LTE and NR. For positioning AE may be used to derive a more efficient representation of the RSTD measurement, i.e., the channel impulse response (channel delay profile). For data transfer AE may be used for transferring the data in an efficient manner instead as current standards wherein the components are designed theoretically.

Embodiments herein relate to communication networks in general, and specifically to wireless communication networks. Figure 5 is a schematic overview depicting a wireless communications network 100 wherein embodiments herein may be implemented. The wireless communications network 100 comprises one or more RANs and one or more CNs. The wireless communications network 100 may use a number of different technologies, such as Wi-Fi, Long Term Evolution (LTE), LTE-Advanced, 5G, New Radio (NR), Wideband Code Division Multiple Access (WCDMA), Global System for Mobile communications/enhanced Data rate for GSM Evolution (GSM/EDGE), Worldwide Interoperability for Microwave Access (WiMax), or Ultra Mobile Broadband (UMB), just to mention a few possible implementations. Embodiments herein relate to recent technology trends that are of particular interest in a 5G context, however, embodiments are also applicable in further development of the existing wireless communication systems such as e.g. WCDMA and LTE.

Access nodes operate in the wireless communications network 100 such as a radio access node 111. The radio access node 111 provides radio coverage over a geographical area, a service area referred to as a cell 115, which may also be referred to as a beam or a beam group of a first radio access technology (RAT), such as 5G, LTE, Wi-Fi or similar. The radio access node 111 may be a NR-RAN node, transmission and reception point e.g. a base station, a radio access node such as a Wireless Local Area Network (WLAN) access point or an Access Point Station (AP STA), an access controller, a base station, e.g. a radio base station such as a NodeB, an evolved Node B (eNB, eNode B), a gNB, a base transceiver station, a radio remote unit, an Access Point Base Station, a base station router, a transmission arrangement of a radio base station, a stand-alone access point or any other network unit capable of communicating with a wireless device within the service area depending e.g. on the radio access technology and terminology used. The respective radio access node 111 may be referred to as a serving radio access node and communicates with a UE with Downlink (DL) transmissions on a DL channel 123-DL to the UE and Uplink (UL) transmissions on an UL channel 123-UL from the UE.

A number of wireless communications devices operate in the wireless communication network 100, such as a UE 121.

The UE 121 may be a mobile station, a non-access point (non-AP) STA, a STA, a user equipment and/or a wireless terminal, that communicate via one or more Access Networks (AN), e.g. RAN, e.g. via the radio access node 111 to one or more core networks (CN) e.g. comprising a CN node 130, for example comprising an Access Management Function (AMF). It should be understood by the skilled in the art that “UE” is a non-limiting term which means any terminal, wireless communication terminal, user equipment, Machine Type Communication (MTC) device, Device to Device (D2D) terminal, or node e.g. smart phone, laptop, mobile phone, sensor, relay, mobile tablets or even a small base station communicating within a cell. Embodiments herein will now be described in relation to Figure 6aa, Figure 6ab and Figure 6b. Figures 6aa, 6ab and 6b all illustrate a wireless communications device 601 comprising hardware and/or software implementing a trained Neural Network, NN,- based Auto Encoder, AE, -encoder 601-1. When the AE-encoder have been trained the trainable parameters are fixed for a specific version of the AE-encoder.

As mentioned above, the trained AE-encoder 601-1 have been trained to provide compressed encoded output data, such as coded CSI, based on input data, such as channel measurements. The compressed coded output data is sent from the wireless communications device 601 to one or more network nodes, such as the radio access node 111, over a communications channel, such as the UL channel 123-LIL, in a communications network, such as the wireless communications network 100. The compressed coded output data is then processed by an AE-decoder of the one or more network nodes to produce an uncompressed output.

The wireless communications device 601 may further implement a reference AE- encoder 601-2 illustrated in Figure 6aa, which may be used to produce reference reports to the network.

Figures 6aa, 6ab and 6b further illustrates a network node 602 comprising hardware and/or software implementing a trained NN-based AE-decoder 602-1. The AE-decoder 602-1 is compatible with the AE-encoder 601-1 of the wireless communications device 601. For example, the NN-based AE-decoder 602-1 may comprise a same number of input nodes as a number of output nodes of the AE-encoder 601-1. The network node 602 may correspond to a radio access network node, such as the radio access node 111 in Figure 5. However, the network node 602 may also correspond to a relay node, a donor base station, or a gNB distributed unit or central unit, or similar entities. The network node 602 may also refer to multiple RAN nodes and units, including those previously mentioned single nodes and units, as well as including core network nodes and cloud servers, and other network architecture entities.

Further, the network node 602 and/or the wireless communications device 601 may have access to test data, such as channel data. Figures 6aa and 6ab illustrate a scenario in which a test data source 602-2 is located in the network node 602. In this scenario the network node 602 may provide the test data to the wireless communications device 601 or the network node 602 may provide the wireless communications device 601 with a network address of the test data source 602-2 or some other indication of how the wireless communications device 601 may retrieve the test data itself.

Figure 6b illustrates a scenario in which a second test data source 601-3 is located in the wireless communications device 601. In this scenario the wireless communications device 601 may provide the test data to the network node 602 or the wireless communications device 601 may provide the network node 602 with a network address of the second test data source 601-3 or some other indication of how the network node 602 may retrieve the test data itself.

The wireless communications device 601 may have access to one or more trained NN-based AE-encoder models, e.g., for encoding CSI. The network node 602 may have access to one or more trained NN-based AE-decoder models for decoding the encoded CSI provided by the wireless communications device 601.

In Figure 6aa the network node 602 may further comprise a reference AE encoder

602-3 that it may use as a benchmark to evaluate the performance of the AE encoder 601-1 of the wireless communications device 601. The reference AE encoder 602-2 may be used together with a reference AE decoder 602-4 also comprised in the network node 602.

If the AE encoder 601-1 beats the performance of the reference AE encoder 602-2 of the network node 602, then the network node 602 may indicate to the wireless communications device 601 that the performance is good enough.

Figures 6ab and 6b further illustrate a further node 603 implementing a digital twin

603-1. In the context of embodiments herein the digital twin 603-1 may be a virtual representation of the physical wireless communications device 601 , or of some objects of the physical wireless communications device 601, or of some functionality that may be processed by the physical wireless communications device 601, within the digital domain. In this case the object may be the AE-encoder 601-1 and the functionality may be the functionality of the AE-encoder 601-1. The digital twin 603-1 may yield the same response (output) as the corresponding function/object of the physical wireless communications device 601 given the same input. With that said, the digital twin 603-1 representation may thus not be a full virtual representation of the complete physical wireless communications device 601 in the sense that it may have support for only a subset of the physical functionality of the wireless communications device 601. Alternatively, or additionally, the digital twin 603-1 may have some additional functionality implemented that is not supported by the physical wireless communications device 601. The reason for this is that the digital twin 603-1 may in part be another software implementation and hence may have a slightly different set of supported features. For example, the digital twin may support functionality that is not yet release into the wireless communication device. In this way the new functionality may be tested before releasing it in the wireless communication device. If the functionality passes one or several tests the functionality may then be made available within the wireless device.

Further the digital twin 603-1 of the function and/or object may not be constrained by the processing and/or memory constrains of the physical wireless communications device 601 since it may be hosted on a computer, cloud server or similar entity. The digital twin 603-1 may therefore be able to process more data or process the data quicker or both in comparison to the physical wireless communications device 601. As an example, the digital twin 603-1 may have the possibility to run multiple (including two or more) functions at the same time while the wireless communications device 601 will not. For example, creating a reference report and the AE-encoder report with the same input data while the wireless communications device 601 may not have that possibility.

Figures 6aa, 6ab and 6b further illustrate a second node 604 with which the wireless communications device 601 may communicate. The second node 604 may be a server. The second node 604 may be accessible by the UE or chipset developer. The information that is provided to the server from the wireless communications device 601 may be used to determine whether or not the AE-encoder that is designed is functional in the field. If the network is testing the AE-encoder over multiple UEs the second node 604 may have aggregated information from multiple UEs and it may from that aggregated information be able to determine whether or not an AE-encoder has passed or failed the test.

In some embodiments the second node 604 is the same node as the further node 603 implementing the digital twin 603-1.

In Figures 6aa, 6ab and 6b the wireless communications device 601, the network node 602 and the nodes 603, 604 have been illustrated as single units. However, as an alternative, each node 601, 602, 603, 604 may be implemented as a Distributed Node (DN) and functionality, e.g. comprised in a cloud 140 as shown in Figure 6, may be used for performing or partly performing the methods. There may be a respective cloud for each node.

Details of the network node 602 and the NN-based AE-decoder 602-1 , such as a reconstructed channel H, a loss function, and a method to compute gradients may be transparent to the wireless communications device 601 .

Exemplifying methods for handling a performance evaluation of the trained AE- encoder 601-1 in the wireless communications network 100 will now be described.

In embodiments herein the wireless communications device 601 may evaluate the performance of the AE-encoder 601-1 and determine a pass or fail indication related to the performance of the AE-encoder 601-1 . Following that the wireless communications device 601 determines the pass or fail indication, the wireless communications device 601 indicates the status of the AE-encoder to the second node 604. With the pass/fail indication a set of parameters associated with the evaluation may also be indicated to the second node 604.

Embodiments herein enable Al-based AE-encoders to be verified in the field. Further advantages are

• Ability to report back the test outcome of an AE-encoder to the developer of a UE or chipset of the UE,

• This enables the developer to verify that the design is functioning in the field or not functioning in the field and then redesign it.

• the data that is reported back is possible to use to redesign the AE-encoder to either a functioning design if it failed or to further enhance its performance if it succeeded.

There may be multiple reasons to initiate a performance/validation test of an AE version, with the AE encoder being implemented in a wireless communications device 601 and the AE decoder being implemented at the network side, e.g., in a base station. As mentioned above, a test may be initiated whenever there has been an update of the AE encoder or/and the AE decoder (e.g., after a re-training of the AE encoder or/and the AE decoder), or that the characteristics of the inference data (i.e. , the input data to a deployed AE encoder) may have changed with respect to the data used in a predeployment training. One reason for initiating a performance validation may be that the neural network AE encoder of the UE model has been re-trained, fully, or partly, meaning that some or all AE encoder trainable parameters have been changed after that the UE model was released at the first time. For example, the UE’s AE-encoder model may first be tested in a lab environment together with a network vendor’s AE-decoder. Later the UE’s AE- encoder has been re-trained. In such scenario, the radio access capability signalling may include one or more parameters indicating a re-training status of the UE model, or more specifically of the AE encoder version. For example, this may be signalled by a version number of the AE-encoder. This may, alternatively, be done outside the radio access network. For example, the radio access network may access the retraining status of the AE encoder from a repository when a UE is connecting to the network. This type of information may be used by the network to conclude whether to initiate a test.

Another reason for initiating the performance validation may be that the lab test, or offline testing, was limited to testing the AE encoder as part of a software module rather than then testing the UE with the software being implemented in a final product. In this case, the UE and/or chipset vendor may have tested its AE encoder software with a network vendor’s AE decoder software in e.g., a lab or in a development domain, while the final performance test of the UE, with the AE encoder being implemented, may be done in a live network. However, some UE testing in between may be needed to ensure a minimum AE encoder performance before a certain UE model is released.

Yet another reason for initiating the performance validation is that the AE-decoder in the network may have been retrained without considering all the AE-encoders on the market and hence needs to be tested with the AE-encoders in UEs on the market that it was not able to be tested on within a lab-environment.

Embodiments herein will now be described with reference to a first signaling diagram in Figure 7a and flowcharts in Figures 7b, 7c, 7d and 7e and with continued reference to Figures 5 and 6aa, 6ab and 6b. The signaling diagram illustrates a method for supporting evaluation of performance of the trained AE-encoder 601-1 in the wireless communications network 100.

In a scenario, the wireless communications device 121 operates in the wireless communication network 100. As mentioned above, the AE-encoder 601-1 is trained to provide encoded data to the compatible trained NN-based AE-decoder 602-1 comprised in one or more network nodes, such as the network node 602 in Figures 6a and 6b, of the wireless communication network 100. In the actions below, the wireless communications device 601 communicates with the wireless communications network 100. For example, the wireless communications device 601 may communicate with any network node of the wireless communications network 100, such as the network node 602. The wireless communications device 601 may further communicate with the node 603 in which the digital twin 603-1 is implemented.

In action 701 of Figure 7a, the wireless communications device 601 sends an indication that it supports an AE-encoder framework and/or version of an AE-encoder and/or a version of the UE model that indicates the AE-encoder framework and/or version of the AE-encoder to the network.

A UE connecting to a wireless network may report its UE radio access capability to the network. Therefore, the indication of the AE-encoder framework and/or version of the AE-encoder and/or the version of the UE model may be sent as capability information, e.g., as UE capability information. Thus, the wireless communications device 601 may indicate its capabilities with respect to a specific version of a trained AE-encoder, e.g., in a UE capability report.

The UE may further report some specific details around its UE model to the network, such as the number of receive antennas, supported number of downlink transmission layers, supported number of CSI-RS antenna ports, chipset model or other unique parameters that could be used to identify a specific UE model. The unique information may also be secondary type information that may be used by the network to identify a specific UE, for example, a full list of UE radio access capabilities.

An example framework for signalling UE capabilities will be given below in the section “Further detailed example embodiments” with NR as the example radio access technology.

Among the reported UE radio access capability parameters is one or more parameters indicating support of an AE-based framework potentially for a given purpose, including the support of one or more AE-encoder versions.

The purpose may for example be CSI reporting, positioning, data transfer, beam management.

The framework may consist of a type of formatting that is defined as the output of the AE-encoder, e.g. number of bits, bit representation (e.g. several bits together represent a number how this is done), use case for the AE-encoder (as there may be separate capabilities for separate use case (e.g. CSI reporting, positioning, data transfer and soon, in addition there may be multiple frameworks within the same use case). The one or more AE encoder versions may further be identified either via specification or via a pre-registered (unique) identity that have been stored in some repository which may either be part of the network or accessible by the network.

In the following, a specific AE-encoder version is considered with the understanding that the UE may support more than one AE-encoder version. For example, the indicated support may be split up into two parts, wherein one capability signalling is for indicating support for a specific type of AE-encoder framework and the second part is a version number for that AE-encoder. The UE may or may not support multiple versions. If the UE supports multiple versions the UE may then need to indicate which version numbers the UE supports. The version number may be used to indicate if the UE has modified its implementation of the AE-encoder in some form that may warrant to indicate it as a new version. In practise the version number may be implemented as a counting value that is increased, but other options are also possible. The specific AE-encoder design used in the UE may not be known to the network. Instead, the specific AE-encoder design may be identified by a version. Alternatively, parts of the AE-encoder design may be known by the network for example to aid the design of an AE-decoder. The specific design details may be given by either the AE-encoder framework capability or the AE-encoder version. Alternatively, both together.

The UE may further report a parameter that indicates its capability for conducting a certain performance test. There may be multiple capability parameters associated with this. One being that the UE may support performance test based on a given AE-encoder framework and AE-encoder version. Alternatively, it may be based on only the AE- encoder framework. In addition, for the test purpose there may be additional parameters that may be provided among the UE capability parameters. These may be a parameter related to test buffer capacity of the UE, whether or not the UE may sample test data by itself for the test, receive test data, etc.

The test buffer capacity may be two different parameters or a single parameter and covers the aspects of how much test data samples the UE may measure of the channel and/or how much output from the AE-encoder the UE may store. This UE capability may be needed to be indicated by the UE independently if the UE may measure its own test data or not. Since if the UE cannot measure its own test data the UE will be provided with test data and the capability may then indicate the amount of test data it may be provided with. On the other hand, if the UE measures its own test data this parameter may set a limit on how many measurements the UE is able to make. The parameter of whether or not the UE may sample test data by itself, is whether the UE can measure and/or generate data that results in input data to the AE-encoder.

The parameter of receiving test data is whether the UE is able to receive test data from the network.

The UE may in addition provide a capability signalling indicating a passed test status of the UE. In more detail it may mean that the UE’s AE-encoder has been tested by one or multiple network’(s) AE-decoder(s) and not by only a development test towards a reference AE-encoder. The network may use this info to select which AE-encoders to test. The passed test status may be sent before the other capabilities.

An additional capability may be whether the UE is able to feedback additional information with the output of the AE-encoder results. Such additional information may be used as a reference of the performance of the AE-encoder. Within the use case of CSI reporting this may be for example another CSI report that is for example based on CSI report type II and uses the similar or same input data as the AE-encoder. It should here be understood that the input data may vary slightly depending on how the algorithms are designed. It may also vary depending on if some form of filtering such as averaging is done on the input data for one of the reports but not the other one. Taking the use case of positioning as another example, the measurements described above may be a representation of the impulse response. Then the reference report that is indicated to be supported may be an RSTD measurement. The receiving network may compare the performance between the two reporting modes and the network based on that may conclude whether the AE-encoder is functioning sufficiently good. The other reporting mode may also be another AE-encoder that the UE supports.

In yet in another example one additional capability is the number of AE-encoders the UE may test at once since multiple AE-encoders may be tested at the same time within the UE. The capability may indicate how many AE-encoders the UE may test simultaneously, and it may further identify these AE-encoders that may be tested simultaneously. This may be advantageous since the AE-encoders may then be tested with the same input data which may provide a better comparison. Further, using the same input data may reduce memory requirements. One or more of the AE-encoders running in parallel may be used as a reference. The test may be initiated by the network in that the UE first reports the abovedescribed UE capabilities, after which the UE receives a request from the network to perform a test of one or multiple AE-encoder(s) in accordance with the sent UE capabilities (i.e. , the UE has indicated support for the AE-encoders it is being tested for). The request will be described further below in action 703.

In action 702 the network may determine, based on the indication of the AE- encoder framework and/or version of the AE-encoder and/or the version of the UE mode whether or not it is required to evaluate the performance of the AE-encoder 601-1.

The network may conclude from the UE capability report that the UE is of a specific UE model, or has a particular AE-encoder, that needs to undergo a performance evaluation, such as a validation or a test, before it may be used to provide AE-based encoded data in an operational mode. The network may retrieve information associated with received AE-encoder framework and/or version of the AE-encoder and/or version of the UE mode from a memory in the network or from a repository in a cloud.

The operational mode may for example be implemented for CSI reporting and the UE may in such case construct a CSI report or parts of a CSI report based on an AE- encoder. As mentioned above, other operational modes of the AE-encoder are also possible such as using it for data transfer, positioning and so on. One possibility for the network for validating the AE-encoder is to configure the UE to report test output from the encoder but not use the output from the AE-encoder for other purposes than testing the AE-encoder. That is, the network doesn’t need to take the test information into account when scheduling the UE. For scheduling the UE the network may use other methods such as CSI type II reporting. This is here exemplified for CSI feedback. The UE receives a configuration from the network to provide AE-based CSI feedback with the AE-encoder, but the network may not use that CSI feedback other than for the purpose of testing the AE-encoder. Whether or not the CSI report based on the AE-encoder is used for something else than testing may be unknown to the UE. To conduct the testing the UE may need to be configured with some intermediate, or additional, CSI reporting at least until the UE has passed the performance test of the specific AE encoder version to be tested. The network may then compare the difference between the additional CSI report and the tested AE-encoder CSI report.

One reason for initiating a performance test may be that the neural network AE encoder of the UE model has been re-trained, fully, or partly, meaning that some or all AE encoder trainable parameters have been changed after that the UE model was released at the first time. For example, the UE’s AE-encoder model may first be tested in a lab environment together with a network vendor’s AE-decoder. Later the UE’s AE-encoder has been re-trained. In such scenario, the radio access capability signalling may include one or more parameters indicating a re-training status of the UE model, or more specifically of the AE encoder version. For example, this may be signalled by a version number of the AE-encoder. This may, alternatively, be done outside the radio access network. For example, the radio access network may access the retraining status of the AE encoder from a repository when a UE is connecting to the network. This type of information may be used by the network to conclude whether to initiate a test.

Another reason for initiating the performance test may be that the lab test, or offline testing, was limited to testing the AE encoder as part of a software module rather than then testing the UE with the software being implemented in a final product. In this case, the UE and/or chipset vendor may have tested its AE encoder software with a network vendor’s AE decoder software in e.g., a lab or in a development domain, while the final performance test of the UE, with the AE encoder being implemented, may be done in a live network. However, some UE testing in between may be needed to ensure a minimum AE encoder performance before a certain UE model is released.

Yet another reason for initiating the performance test is that the AE-decoder in the network may have been retrained without considering all the AE-encoders on the market and hence needs to be tested with the AE-encoders in UEs on the market that it was not able to be tested on within a lab-environment.

As mentioned above, in action 703 the wireless communications device 601 receives a request from the network to perform a test of one or multiple AE-encoder(s) in accordance with the sent UE capabilities. The request may also be to let the digital twin 603-1 perform the test.

Further, the request to perform the test may trigger a test report to the second node 604. The test report may comprise: an indication of pass or fail of the test, or an indication that the test has started or has been performed. The test may be performed in the wireless communications device 601 or in the digital twin as described herein.

If the test is not completed within a certain time period then the UE may trigger a negative test report. Therefore, in some embodiments the UE may receive a request from the second node 604 to start a timer and follow the procedure described herein to trigger either a positive or a negative test report. Thus, the timer may for example be started when the UE receives the request from the second node 604 to start the timer, or when the UE starts the performance evaluation, or when the UE transmits the output data to the network node 602. The timer may expire after the time period if the UE did not obtain or determine the pass status of the performance evaluation. The timer may be understood as a time period that should have passed between two different events. This time period may be defined as between a trigger received at the start of the performance evaluation until a report containing a negative indication of the performance of the AE-encoder should be sent assuming that the UE did not obtain or determine a pass status of the performance evaluation. If however a positive indication of the performance of the AE- encoder is detected before the negative report is supposed to be sent a report indicating a positive status of the AE-encoder is instead sent.

The request may further include more details related to the UE capabilities, e.g., if the UE should measure its own test data or if the test data is provided by the network. The amount of test data the UE should collect if the UE is requested to collect test data and so on. The request may further include information of any reference format (e.g., reference AE-encoder(s) or CSI reporting modes) that may have its results jointly reported with the tested AE-encoder(s).

The UE may perform the test and report the result of the test, i.e. the output, to the network.

In response to determining that it is required to validate the performance of the AE- encoder 601-1 , the network node 602 may transmits a test data set, or a test data segment, (e.g., including multiple H’s) to one or more wireless communication devices, such as the wireless communication device 601. The test data set may comprise input data for the test, such as channel data. Alternatively, the network node 602 transmits a network address from where the test data may be downloaded by one or more UEs.

Thus, in action 703 the wireless communications device 601 may also receive or retrieve the test data H and processes each H through its AE-encoder which results in a set of encoded and compressed output data Y that is derived based on the input test data set H. Each H being representative of each individual input data vector H. Each such input produces one output vector Y. If one takes the example of CSI estimation each H would be the one single channel measurement represented in some format.

The test input data H may not necessarily be a representation of channels observed by the UE when estimating channels from CSI-RS. It may for example be a representation of uplink channels estimated by base stations from uplink transmissions such as from SRS transmissions. It may also be a representation of channels in some other manners, such as being synthetically generated channels with or without noise, or impairments, included. The test input data H may also be a representation that corresponds to some stage of a CSI report, such as a NR CSI type II report.

In action 704 the wireless communications device 601 may process the test input data set H to produce the output data Y.

The processing of the test data set, or a test data segment, may be done within the UE as a background process over a certain time as it is not too time critical.

The output Y of the AE-encoder, based on the test data set, may be sent to the network node 602 in action 705. If there are several outputs, e.g., several Ys based on several Hs, then the outputs may be sent in a big batch rather than being sent individually.

In action 706 the wireless communications device 601 may send a message, e.g., an AE-encoder test report, to the second node 604 indicating that the evaluation of the AE-encoder has been triggered.

The AE-encoder test report may include one or more of the following information:

• Parts or all of information related to the test, such as configuration information for the test

• Parts or all the information related to the test

• AE-encoder version

• AE-encoder type

• AE-decoder type

• Size of test data set

• Type of test, e.g. based on digital twin or UE based test

• Test data set, with the additional indication that the test data was provided by the network or measured by the UE.

• A reference report type, such as for example CSI report(s) based on the test data set, a RSTD measurement, or more generic a reference AE-encoder output based on the test data set

• Position of the UE

• Network information such as PLMN-id, cell-id, attached frequencies during the collection of test data • Observed cells by the UE during the collection of test data (for positioning)

• Observed wifi nodes by the UE (for positioning)

• Time of test

• Information received from the network about AE-decoder the network used

• Network equipment information, if available. For example, UE context with the network. The manufacturers of the network equipment may support different versions of the AE.

• Specific configurations of the UE such as number of CSI-RS antenna ports

• Frequency band information

• Phone version and specific details around it, e.g. IMEI (International Mobile Equipment Identity)

The AE-encoder type and/or the AE-decoder type may include a definition of the interface between the AE-encoder and the AE-decoder, e.g. the structure of the interface as defined for example by the number of nodes, the bit width of each node and so forth.

In some embodiments the network tests the AE-encoder not in a single UE but distributes the test over many UEs. However, this may create a specific problem for an individual UE to determine whether or not the specific UE’s AE-encoder passed or failed the test. The second node 604 may when receiving the test report trigger a message to UEs operating within the same operator and supporting the same AE-encoder type to start their respective timer. This may be done to collect information about whether or not the AE-encoder is functioning. The message may not be to all the UEs within an operator but instead to a subset of them. The subset may be based on a user preference when configuring the UE (user or operator approval to be part of testing), it may also be based on geographical area the UE is in and other factors. The UEs themselves may also ping the second node 604 from time to time to check whether a test is being performed and start their timer based on that.

In response to receiving the AE-encoder test report, indicating that the evaluation of the AE-encoder has been triggered, the second node 604 may, in action 707, send messages to other wireless communications devices, such as a second wireless communications device 612 and a third wireless communications device 613, which triggers the other wireless communications devices with the same AE-encoder to start their timers associated with evaluation of the AE-encoder. There may be a requirement to test AE-encoders of a certain number of wireless communications devices.

In action 708 the wireless communications device 601 may start a timer associated with the AE-encoder 601-1 of the wireless communications device 601. The second wireless communications device 612 and the third wireless communications device 613 may also start their timers.

For example, when a new AE-encoder is made available in a UE the UE may have a timer indication internally associated with the AE-encoder to initiate a negative report, such as an evaluation report, indicating failure of the test of the AE-encoder to the second node 604, such as a server, after the timer expires. A lapse of the timer may be an indication of that the AE-encoder is not functioning properly.

The timer may be associated with operation in certain networks, geographical areas, spectrum ranges or technologies. Thus, the timer may be started for certain networks, geographical areas, spectrum ranges or technologies, but not for other. The networks that are selected may be networks that support the AE-encoder based operation according to the new AE-encoder in the UE. This information may be known by the UE in some manner and hence be used to determine whether or not to start the timer. Similar reasoning is possible to make around the geographical areas and spectrum ranges.

Further, if the UE is only connected to a radio access technology (RAT) that does not support AE-encoder based operation, the timer will not trigger, or count down, as there is no testing possibility. Note also that the timer may either count upwards or downwards. Embodiments herein will be described as if the timer counts downwards and when it expires, and action is then triggered. It may however be the opposite way around that the timer is counting until a threshold is reached and then triggers an event. Another option is that there is no timer, rather the UE will follow the actions associated with the timer described here without there being an explicit timer.

In a yet another alternative, the timer is triggered to start after the UE has been asked to perform an evaluation of its AE-encoder by the network. If the timer expires before it has been determined that the AE-encoder has passed the evaluation then the expiration of the timer may trigger a negative report to the server.

In some embodiments the network node 602 indicates a successful evaluation of the AE-encoder 601-1 to the wireless communications device 601 by configuring the AE- encoder 601-1. Thus, if the AE-encoder is configured it may trigger a positive report, such as a positive evaluation report, to the second node 604.

In action 709 the network node 602 may then process the test output data from the AE encoder test to validate the performance of the AE-encoder. The network node 602 may for example

• compare the received output data, such as the compressed channel data Y, from the UE with a sequence Y ₂ , wherein Y ₂ is obtained by the network node 602 by processing the test input data set H with an AE-encoder reference model,

• alternatively, the network may input the compressed channel data Y into an AE- decoder and compare the output of the AE-decoder H, to some form reference for the output Href or the test input data set H. The reference may either be a fixed value or based on a reference AE-encoder together with the AE-decoder.

• Compute a loss function L, or use the specific loss function that was used to train the AE-decoder.

If the performance of the AE-encoder in the UE is deemed sufficient the network node 602 may record this for future use.

For example, the network node 602 may record that the specific AE-encoder functions sufficiently well and may be used within the deployments that are deemed representative of the test data. This information may in the future be shared among network nodes within the whole operator’s deployment or between a limited set of base stations within an operator’s deployment. The information may also be shared across different operators.

The network node 602 may in the future use this information to check that a specific AE-encoder works with its AE-decoder implementation. Hence if a UE reports that it supports a specific AE-encoder or AE-encoder version in its UE capability and the network is able to identify that it has previously tested that AE-encoder via the test data described earlier, then the network may configure the UE to use that AE-encoder.

In action 710 the network node 602 may indicate to the wireless communications devices that the AE-encoder’s performance is deemed sufficient. This indication may be in the form of a pass/fail indication (e.g. in terms of loss), a relative value of the loss (compared to a reference model AE-encoder), an absolute loss value, a single bit indication, etc. The network node 602 may also indicate the performance by configuring the UE to use the AE-encoder 601-1 that has been tested according to the above description. For example, the network node 602 may configure CSI reporting with the tested AE-encoder or indicate a test failure. In Figure 7a the network node 602 configures CSI reporting with the tested AE-encoder for the second wireless communications device 612.

The wireless communications devices may determine and set a test status indication, e.g. register AE test status with tested AE-encoder. The test status may be any one or more of: test started, test passed and test failed. The wireless communications device 601 may determine the test status based on the indication received from the network node 602, e.g., based on the configuration or lack of configuration (e.g., within a certain time).

The wireless communications devices may transmit the test status indication to the network node 602.

In action 711 the wireless communications device 601 transmits the report, such as the evaluation report or a pass/fail report, to the second node 604. In Figure 7a the second wireless communications device 612 transmits the report including the test status indication to the second node 604.

A pass or negative test report when triggered by the UE to the server may include one or more of the following information:

• Successful/failed indication o Type of indication, e.g. explicit indication by network or implicit by configuration of the AE-encoder

• AE-encoder version

• AE-encoder type

• AE-decoder type

• Size of test data set

• Type of test, e.g. digital twin or measured based test

• Test data set, with the additional indication the test data was provided by the network or measured by the UE.

• A reference report type, such as for example CS/ report(s) based on the test data set, a RSTD measurement, or more generic a reference AE-encoder output based on the test data set

• Position • Network information such as PLMN-id, cell-id, attached frequencies

• Observed cells by the UE

• Observed wifi nodes by the UE

• RSRP/RSRQ/SINR levels for the connected cell(s) and neighbour cell(s)

• Time of test

• Information received from the network about AE-decoder the network used

• Network equipment information, if available

• Specific configurations of the UE such as number of CSI-RS antenna ports

• Frequency band information

• Phone version and specific details around it, e.g. IMEI (International Mobile Equipment Identity).

The UE may further relay information it has received from the network. The information may be any of the following pass/fail indication (in terms of loss), a relative value of the loss (compared to a reference model AE-encoder or another type of reference report for example CSI report type II), an absolute loss value, a single bit indication, etc.

In action 712 the second node 604 may store the status of the test.

If a negative report is received by the second node 604 or if sufficiently many negative report(s) are received the second node 604 may indicate to all UEs of the type being tested that they should not indicate support any longer for the specific AE-encoder. Based on receiving such a message these UEs will not report support for that AE-encoder any longer to the network within its UE capability signalling. Thus, the UE may remove the UE capabilities relating to AE-encoders from the signalled UE capabilities. That information may however be specific to certain properties such as operator specific (e.g., PLMN-ID specific), geographical area, frequency band specific, etc. A negative report may also be the absence of a report to the server as this in itself would indicate that no AE-encoder has been configured.

If the second node 604 receives a positive report or sufficiently many positive reports the second node 604 may trigger a message to all the UEs that it has requested to start the new AE-encoder or has the new AE-encoder configured to halt their timer. Alternatively or in addition the second node 604 may not ask for any reports from the UE. Thus, in action 713 second node 604 may indicate to the wireless communications device 601 that it should either stop its timer or not indicate support any longer for the specific AE-encoder 601-1.

The flowcharts of Figures 7b, 7c, 7d and 7e illustrate some specific embodiments of methods, performed by the UE, for handling performance evaluation of the AE-encoder.

As mentioned above, the methods described herein for handling a performance evaluation are also valid for the case where the evaluation is based on the digital twin 603-1. Thus, in some embodiments herein the UE capability includes information about a digital twin representation. The digital twin may be used for testing purposes. The testing may then be conducted in the digital twin rather than in the UE. The digital twin may for example be implemented within a cloud server. The network may or may not be able to communicate directly with the digital twin. If the network is not able to communicate directly with the digital twin then the network may communicate with the digital twin via the UE.

The test of the AE-encoder may be initiated by the network communicating with the UE. The UE may forward the information to its digital twin. The information that the UE forwards may be information on how to setup a test. The actual transfer of the test data set and the results of the test may be sent directly between the network and the digital twin. It may be so that the digital twin retrieves the test data set and uploads the results.

An alternative is that the UE indicates that it has a digital twin and further provides the accessible network address of the digital twin to the network. The network may then communicate input and output test data directly with the digital twin. For example, the network may initiate the test in the digital twin and the digital twin may provide the result of the test to the network. The test initiated to be conducted by the digital twin is handled from the network side via communication with the UE. For example, other communication than communication of test data may be provided via the UE.

The wireless communications device 601 may send an indication to the network that it supports evaluation or testing of at least one AE-encoder framework and/or version of an AE-encoder by the digital twin 603-1. The wireless communications device 601 may further report some specific details around the model of the wireless communications device 601 to the network as mentioned above in action 701.

For the above-described UE capabilities, it may be so that the digital twin has a separate set of UE capabilities then the UE has. For example, not all the features may be supported by a UE, or more features may be represented in the digital twin than may be configured for live operation in the network by the UE. All features that are associated with the digital twin may be indicated separately, such as for example the test buffer capacity, whether or not the UE may sample/collect test data by itself for the test, receive test data, reference of the performance of the AE-encoder or reference CSI report, number of AE- encoders possible to test at once, etc.

In response to determining that it is required to evaluate the performance of the AE- encoder 601-1 , the network node 602 transmits a request to the wireless communication device 601 to initiate a test of one or multiple AE-encoder(s) within the digital twin 603-1.

The request may or may not include a network address together with access information where test data may be retrieved by the further node 603, such as a cloud server, wherein the digital twin 603-1 is operated. The access information may include information needed to setup a secure connection.

More information may also be indicated with the request as described further below.

In response to receiving the request the UE sends a request to the further node 603, such as a cloud server, and/or the digital twin 603-1 to test one or multiple AE- encoder(s). With the request, the UE may forward the information on where the test data may be retrieved, if it has received such information from the network together with the access information.

Based on this information the further node 603, such as the cloud server, retrieves the test data.

The digital twin 603-1 may process the test input data set H to produce the output data Y. In other words the cloud server may process the test data through the AE- encoder(s). For example, the digital twin 603-1 may processes each H through its AE- encoder which results in a set of encoded and compressed output data Y that is derived based on the input test data set H. The UE may receive a complete message, indicating that the AE-encoder of the digital twin 603-1 has finished processing the data and produced an output, from the cloud server. The complete message may potentially comprise a network address at which the processed test data may be located with the associated access information.

The UE may then send a message to the network node 602 indicating that the AE- encoder(s) test is completed with the network address where the network may retrieve the output data of the test. The UE either knows the network address before or has received the network address from the cloud server with associated access information.

The output Y of the AE-encoder within the digital twin 603-1 , based on the test data set, is then retrieved or received by the network node 602. If there are several outputs, e.g., several Ys based on several Hs, then the outputs may be sent in a big batch rather than being sent individually.

The network node 602 may then process the test output data from the AE encoder test to validate the performance of the AE-encoder in a similar manner as described above.

The network node 602 may indicate to the UE that the AE-encoder’s performance is deemed sufficient. For example, the network node 602 may configure CSI reporting with the tested AE-encoder or indicate a test failure.

Within the testing procedure the UE may determine a test indication status as described above. The UE may determine this status either by receiving signalling indicating the test status from the network or detecting an absence of receiving such signalling from the network.

Thus, in some embodiments according to the above the UE receives signalling indicating the test status from the network. This may in its simplest form be an indication of pass, alternatively either pass or fail. In addition, the signalling from the network may include more information such as how good the AE-encoder performance was, what type of AE-decoder the AE-encoder was used together with, the version of the AE-decoder.

The performance of the AE-encoder may for example be provided in relation to a reference AE-encoder that is specified, in relation to a type of CSI report (e.g. CSI report type II), or another AE-encoder the UE supports and was simultaneously tested with. The specific results may be indicated as a relative value of the loss (compared to a reference model AE-encoder), an absolute loss value, a single bit indication, etc. In case there is no signalling directly indicating a pass or fail indication, the UE may for such a case detect that the AE-encoder test was successful if the UE receives a configuration message configuring the AE-encoder. If the UE is not configured with the AE-encoder the UE may detect that the test of the AE-encoder has failed.

An alternative to the above is that the network does not provide test data. Instead, the network includes a message to the UE that may trigger the digital twin to initiate a procedure to collect test data from one or multiple UE(s). The network message may indicate that test data should be collected, and/or it may include details on how the collection of test data should be collected.

With the request above the UE may indicate to the digital twin that test data should be collected. If the network has indicated details on how the test data should be collected the UE may include such information. The digital twin may initiate a test data collection procedure asking either one or multiple UEs to collect test data and provide it to the digital twin. The request to collect test data may include one or more of the information provided as examples for the request received by the UE above from the network to collect test data.

One or multiple UEs may then start to collect test data according to the instructions and provide it to the digital twin. The digital twin processes the test data in some form. After enough test data is collected processing of the test data is complete. The digital twin either indicates to a UE that the test is complete or connects to the network directly and indicates that the test is complete.

If the digital twin indicates to the UE or another UE in the same network that the test is complete the UE that receives this indication may indicate that to the network and the UE and the network may follow a similar procedure as when the test data was not collected by the UE(s). The collected test data set may further be provided to the network in a similar manner as the output of the AE-encoder(s).

Specific for this embodiment is that the network may further save the test data set H collect by the one or multiple UEs for future use together with the additional information collected with it. The test data set H and its additional information may however be anonymized, so it is not possible to derive a specific user from it, to retain privacy. For example, the collected test data set H by a set of UEs may later be used to evaluate yet another UE model’s AE-encoder without requiring that UE model to measure the test data itself. As indicated above, the request from the network to the UE may also include information that the test data (or reformatted version of the test data) should be processed by another AE-encoder, such as reference AE-encoder, or a non-AE-encoder based CSI report. This independent of whether or not the test data is provided by the network or collected by UEs to the digital twin. The UE may indicate this information to the digital twin. The digital twin may derive the additional AE-encoder and/or a non-AE-encoder based CSI report, e.g. CSI report II on the same test data set as the AE-encoder(s) are tested. The output from these reports may be provided in a similar manner as the output of the AE-encoder(s) described above. The network may for example use the CSI report(s) from the UE to calculate a relative difference between the CSI report(s) and the AE-decoder output the network calculates based on a specific AE-encoder. For example, the AE-decoder together with the AE-encoder may be aiming to represent the phase and amplitude of each CSI-RS. The network may use this to derive the corresponding PMI to use and compare this to the UE reported PMI value from the CSI report of type II. A similar example for positioning is that the AE-encoder together with AE-decoder aim to represent taps (or some taps) of the channel (measured a set of reference signals from a cell) with their strength and difference to each other in connection to a reference time that may be retrieved from a cell. The network may then compare this to a RSTD measurement report wherein the UE reports the time difference of arrival between two cells. Such a report may include the first occasion of the cell of interest to measure on or multiple occasion. The occasion represents here taps in the channel.

The network may further only deem the performance of the AE-encoder to be sufficient if that relative difference is sufficiently large or positive in favour of the AE- encoder.

If the network node identifies, i.e., determines, that the AE-encoder has failed or has not previously passed the test, the network node may configure the UE with an alternative reporting mechanism for the data that is encoded by the AE-encoder. For CSI reporting that alternative reporting mechanism may use another supported AE-encoder that has a pass indication. The alternative reporting mechanism may also be based on a CSI report not based on an AE-encoder, such as a CSI type II report.

Exemplifying methods according to embodiments herein will now be described with reference to a flow chart in Figure 8 and with continued reference to Figures 5 and 6aa, 6ab and 6b. The flow chart illustrates a method, performed by the wireless communications device 121, 601, for handling a performance evaluation of the trained NN-based AE-encoder 601-1. The trained NN-based AE-encoder may be implemented in the wireless communications device 121, 601. Evaluation may for example mean validation or testing.

As mentioned above, the evaluation of the performance may be based on running the AE-encoder in the digital twin 603-1.

In action 801 the wireless communications device 121, 601 triggers the performance evaluation of the AE-encoder 601-1 by receiving a trigger message for performance evaluation of the AE-encoder 601-1.

The wireless communications device 121 , 601 may trigger the performance evaluation in response to a request from the network node 602, such as the request described above in action 703. The performance may be evaluated based on the AE- encoder of the wireless communications device 121, 601 or based on the corresponding AE-encoder of the digital twin 603-1.

In action 802 the wireless communications device 121, 601 obtains or determines the status of the performance evaluation.

In some embodiments the wireless communications device 121 , 601 determines the status of the performance evaluation by determining that the performance evaluation has been triggered based on receiving the trigger message. Then the indication of the status of the performance evaluation may indicate that the performance evaluation has been triggered. In other words, when the UE has previously triggered a test report to the second node 604, the report may be triggered based on that the UE is requested to perform a test of its AE-encoder, i.e. , the test report may be provided in response to triggering the test of the AE-encoder.

In some embodiments the wireless communications device 121 , 601 determines the status of the performance evaluation by determining whether or not the AE-encoder 601-1 passed or failed the performance evaluation. Then the indication of the status of the performance evaluation may indicate a pass status or a fail status of the performance evaluation.

In some embodiments the wireless communications device 121 , 601 determines that the AE-encoder 601-1 passed the performance evaluation based on receiving a configuration of the AE-encoder from the network node 111. Then determining the AE- encoder 601-1 failed the performance evaluation may be based on absence of the configuration of the AE-encoder.

In some embodiments the provided indication of the status of the performance evaluation indicates the pass status if the wireless communications device 121 obtains or determines the pass status of the evaluation within a time period. Then the provided indication of the status of the performance evaluation indicates the fail status if the wireless communications device 121 does not obtain or determine the pass status of the evaluation within the time period.

The time period may be associated with a timer of the wireless communications device 121 , 601. For example, in some embodiments the wireless communications device 121 , 601 determines the status of the performance evaluation based on a timer associated with the status of the performance evaluation. Then determining the status of the performance evaluation may comprise determining the fail status if the timer has expired and determining the pass status if the timer has not expired.

For example, the UE may detect that the AE-encoder test was successful if the UE receives a configuration message configuring the AE-encoder for operation. Taking the use case of CSI or positioning as an example the UE may be configured to report CSI based AE-encoder or a representation of the radio channel for positioning based on an AE-encoder. The opposite of this may then be a detection of that the test was failed, i.e. the UE is not configured with the AE-encoder. The later may then be within a time frame so that the UE can determine that the test has actually failed, i.e. the timer has expired.

In action 803 the wireless communications device 121, 601 provides, to the second node 604, a report, such as an evaluation report, comprising an indication of the status of the performance evaluation.

The provided indication of pass or fail status of the performance evaluation may be associated with any of: a network identity of the wireless communications network 100, such as a PLMN-ID, a frequency band or range for wireless transmissions in the wireless communications network 100, or a geographic area in which the wireless communications device 121 is located.

The wireless communications device 121 , 601 may report the PLMN-ID of the network it is connected to while performing the AE-encoder test. This to be able to later identify which network the wireless communications device 121, 601 was operating with and further which infrastructure the wireless communications device 121, 601 was operating with. The infrastructure provided in an operator network may be deployed within a unified geographical area and/or frequency range.

The wireless communications device 121 , 601 may further report PLMN-ID from other networks which it detects in connection with performing the AE-encoder test. This may be used by the second node 604 to locate the position of wireless communications device 121, 601 while performing the AE-encoder test.

The provided report may be provided in response to obtaining or determining the status of the performance evaluation. For example, the provided report may be provided in response to a triggered test of the AE-encoder 601-1 , or e.g. in response to receiving a pass/fail indication from the network node 602.

In some embodiments the provided report includes further information about the performance evaluation of the AE-encoder.

Exemplifying methods according to embodiments herein will now be described with reference to a flow chart in Figure 9a and with continued reference to Figures 5 and 6aa, 6ab and 6b. The flow chart illustrates a method, performed by the second node 604, for handling the performance evaluation of the trained NN-based AE-encoder 601-1 of the wireless communications device 121.

In action 901 the second node 604 receives, from the wireless communications device 121, the report comprising the indication of the status of the performance evaluation of the AE-encoder 601-1 in the wireless communications network 100.

In action 902 the second node 604 may determine, based on the received indication of the status of the performance evaluation, whether or not the AE-encoder 601-1 is functional in the wireless communications network 100.

In some embodiments the indication of the status of the performance evaluation indicates a pass status of the performance evaluation. Then in action 903 the second node 604 may transmit a request to stop a timer associated with the status of the performance evaluation to a second wireless communications device 612 in the wireless communications network 100. For example, if the second node 604 receives a positive report or sufficiently many positive reports the second node 604 may trigger a message to all the UEs that it has requested to start the new AE-encoder or has the new AE-encoder configured to halt their timer. Alternatively or in addition the second node 604 may not ask for any reports from the UE.

If instead the indication of the status of the performance evaluation indicates a fail status of the performance evaluation, the second node 604 may transmit a request to stop indicating support for the AE-encoder 601-1 to the second wireless communications device 612 in the wireless communications network 100.

For example, if a negative report is received by the second node 604 or sufficiently many negative reports are received the second node 604 may indicate to all UEs of the type being tested that they should not indicate support any longer for the specific AE- encoder.

The proposed solution enables AE-encoders to be verified in the field. For example, it is possible to verify the AE-encoder 601-1 implemented in the wireless communications device 121 operating in the wireless communications network 100. This enables the AE- encoders to be

• tested towards test data that is representative of channel propagation conditions that were not considered within the test and training data when the AE encoder was trained,

• enables re-designs of the AE-decoder and testing of it with AE-encoders deployed in the field

• enable identification of the best AE-decoder for the associated AE-encoder within UEs in the field.

Since the UE provides the UE capabilities for the testing the network will know what is supported by the UE.

Details of the example embodiments

A UE that needs to evaluate the performance of a supported AE-encoder may first be added to a list of UEs that share the same AE-encoder version before the actual test is initiated. The network may then decide whether or not to initiate a validation of the performance of the AE-encoder with the UE based on the number of active UEs that are of the same UE model or have the same AE-encoder version. For example, a testing may be initiated first after that the number of such active UEs are above a threshold value that has been set by the network to control validation conditions when performing the performance test. With such an approach, it will be possible to push a test to many UEs that are in the same test status with respect to a specific UE model or AE-encoder version.

One advantage of testing many UEs with same AE-encoder version, possibly at the same time, is that one UE does not need to handle a large test data set to perform a test with high confidence. Instead, a set, or sets, of test data may be partitioned into segments in which one or a few segments are handled by a single UE. This will lower the requirements on the UE’s capability to buffer validation data (both input and output), and thus lower the memory requirements for the UE. This also leads to a lower energy consumption by the UE as less amount of test data needs to be processed. Hence, by distributing the validation data segments over several UEs, each UE may contribute to an aggregated validation result from which the network may determine the AE performance. One may group UEs to be tested into validation groups such that one validation group of UEs process the same validation data segment(s).

The network may have multiple AE-decoder implemented and may perform the network related functions described above once per AE-decoder. The network may afterwards rank the AE-decoders after the performance they have with a specific AE- encoder. The network may for example identify that a specific AE-decoder should be used with the specific AE-encoder for a specific UE. Alternatively, the network may rank an AE- decoder in some other manner further based on additional parameters. For example, if not only pure performance is considered but also other aspects such as required processing power in the network node is considered then the network may rank the AE- decoder also based on required processing power in the network node.

To further expand the above, the UE may indicate support for multiple AE-encoders in its UE capabilities to the network. The network may request the UE to test a specific or multiple, or all AE-encoders with the same test data set. The UE then applies the abovedescribed features on all the applicable AE-encoders. The UE may further send one response message per AE-encoder with the output of the AE-encoder or a single message with all the outputs of all request AE-encoders. The network will then apply the above steps for all the AE-encoders.

A complete and concise embodiment will now be described. In some embodiments as disclosed herein the UE sends UE capability information to the network indicating that UE supports an AE-encoder framework and/or AE-encoder version. The UE capability information may comprise radio access capability information. Further the UE capabilities for the AE-encoder may indicate that the UE supports for the AE-encoder to be tested or retrained. The testing or retraining may either occur within the UE or within a digital twin, e.g., as indicated by a UE capability support. Based on the sent UE capabilities the UE receives a request from the network to conduct a test of an AE-encoder or to retrain an AE-encoder. The UE may in response perform the test or retrain the AE-encoder. If the UE receives a request to perform an AE-encoder test or retrain the AE-encoder within a digital twin the UE initiates the test or retraining process within the digital twin. The UE capabilities of the UE may be separately indicated for the UE and for the digital twin. For example, it is not necessary that the same set of UE features are supported by the UE and by the digital twin. The proposed solution comprises a signalling framework that enables AE-encoders of different UE models to be tested and validated with the network AE-decoder within the field. The signalling involves indicating to the UE from the network that a test of the AE-encoder will occur.

The operational mode may for example be implemented for CSI reporting and the UE may in such case construct a CSI report or parts of a CSI report based on an AE- encoder. As mentioned above, other operational modes of the AE-encoder are also possible such as using it for data transfer, positioning and so on. One possibility for the network for evaluating the AE-encoder is to configure the UE to report test output from the encoder but not use the output from the AE-encoder for other purposes than evaluating the AE-encoder. That is, the network doesn’t need to take the evaluation information into account when scheduling the UE. For scheduling the UE the network may use other methods such as CSI type II reporting. This is here exemplified for CSI feedback. The UE receives a configuration from the network to provide AE-based CSI feedback with the AE- encoder, but the network may not use that CSI feedback other than for the purpose of evaluating the AE-encoder. Whether or not the CSI report based on the AE-encoder is used for something else than evaluation may be unknown to the UE. To conduct the evaluation the UE may need to be configured with some intermediate, or additional, CSI reporting at least until the UE has passed the performance evaluation of the specific AE encoder version to be tested, e.g. based on the digital twin. The network may then compare the difference between the additional CSI report and the tested AE-encoder CSI report.

UE capability handling

The UE capability framework specified in NR is given on a high-level in chapter 14 of TS 38.300 16.7.0. For simple reference below is an extract of that part: 14 UE Capabilities

The UE capabilities in NR rely on a hierarchical structure where each capability parameter is defined per UE, per duplex mode (FDD/TDD), per frequency range (FR1/FR2), per band, per band combinations, ... as the UE may support different functionalities depending on those (see TS 38.306 [11]).

NOTE 1 : Some capability parameters are always defined per UE (e.g.

SDAP, PDCP and RLC parameters) while some other not always (e.g. MAC and Physical Layer Parameters).

The UE capabilities in NR do not rely on UE categories: UE categories associated to fixed peak data rates are only defined for marketing purposes and not signalled to the network. Instead, the peak data rate for a given set of aggregated carriers in a band or band combination is the sum of the peak data rates of each individual carrier in that band or band combination, where the peak data rate of each individual carrier is computed according to the capabilities supported for that carrier in the corresponding band or band combination.

For each block of contiguous serving cells in a band, the set of features supported thereon is defined in a Feature Set (FS). The UE may indicate several Feature Sets for a band (also known as feature sets per band) to advertise different alternative features for the associated block of contiguous serving cells in that band. The two-dimensional matrix of feature sets for all the bands of a band combination (i.e. all the feature sets per band) is referred to as a feature set combination. In a feature set combination, the number of feature sets per band is equal to the number of band entries in the corresponding band combination, and all feature sets per band have the same number of feature sets. Each band combination is linked to one feature set combination. This is depicted in Figure 9b.

In addition, for some features in intra-band contiguous CA, the UE reports its capabilities individually per carrier. Those capability parameters are sent in feature set per component carrier and they are signalled in the corresponding FSs (per Band) i.e. for the corresponding block of contiguous serving cells in a band. The capability applied to each individual carrier in a block is agnostic to the order in which they are signalled in the corresponding FS.

• NOTE 2: For intra-band non-contiguous CA, there are as many feature sets per band signalled as the number of (groups of contiguous) carriers that the UE is able to aggregate non-contiguously in the corresponding band. To limit signalling overhead, the gNB can request the UE to provide NR capabilities for a restricted set of bands. When responding, the UE can skip a subset of the requested band combinations when the corresponding UE capabilities are the same.

If supported by the UE and the network, the UE may provide an ID in NAS signalling that represents its radio capabilities for one or more RATs in order to reduce signalling overhead. The ID may be assigned either by the manufacturer or by the serving PLMN. The manufacturer-assigned ID corresponds to a pre-provisioned set of capabilities. In the case of the PLMN-assigned ID, assignment takes place in NAS signalling.

The AMF stores the UE Radio Capability uploaded by the gNB as specified in TS 23.501 [3],

The gNB can request the UE capabilities for RAT-Types NR, EUTRA, UTRA-FDD. The UTRAN capabilities, i.e. the INTER RAT HANDOVER INFO, include START-CS, START-PS and "predefined configurations", which are "dynamic" lEs. In order to avoid the START values desynchronisation and the key replaying issue, the gNB always requests the UE UTRA-FDD capabilities before handover to UTRA-FDD. The gNB does not upload the UE UTRA-FDD capabilities to the AMF.

Figure 10 shows an example of the wireless communications device 601 and Figure 11 shows an example of the second node 604. The wireless communications device 601 may be configured to perform the method actions of Figure 8 above. The second node 604 may be configured to perform the method actions of Figure 9a above.

The wireless communications device 601 and the second node 604 may comprise a respective input and output interface, IF, 1006, 1106 configured to communicate with each other, see Figures 10-11. The input and output interface may comprise a wireless receiver (not shown) and a wireless transmitter (not shown).

The wireless communications device 601 and the second node 604 may comprise a respective processing unit 1001, 1101 for performing the above method actions. The respective processing unit 1001 , 1101 may comprise further sub-units which will be described below.

The second node 604 may further comprise a receiving unit 1110, and transmitting unit 1120, see Figure 11, which may receive and transmit messages and/or signals. The second node 604 may further comprise a determining unit 1130 which for example may determine based on the received indication of the status of the performance evaluation, whether or not the AE-encoder 601-1 is functional in the wireless communications network 100.

The wireless communications device 601 may further comprise a triggering unit 1010 which for example may triggering the performance evaluation of the AE-encoder 601-1.

The wireless communications device 601 may further comprise a providing unit 1020 which for example may provide the report comprising the indication of the status of the performance evaluation to the second node 604.

The wireless communications device 601 may further comprise a status obtaining unit 1030 and/or a status determining unit 1040 which for example may obtain or determine the status of the evaluation of the performance of the AE-encoder 601-1.

The embodiments herein may be implemented through a respective processor or one or more processors, such as the respective processor 1004, and 1104, of a processing circuitry in the wireless communications device 601 and the second node 604, and depicted in Figures 10-11 together with computer program code for performing the functions and actions of the embodiments herein. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the respective wireless communications device 601 and second node 604. One such carrier may be in the form of a CD ROM disc. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on a server and downloaded to the respective wireless communications device 601 and second node 604.

The wireless communications device 601 and the second node 604 may further comprise a respective memory 1002, and 1102 comprising one or more memory units. The memory comprises instructions executable by the processor in the wireless communications device 601 and second node 604. Each respective memory 1002 and 1102 is arranged to be used to store e.g. information, data, configurations, and applications to perform the methods herein when being executed in the respective wireless communications device 601 and second node 604.

In some embodiments, a respective computer program 1003 and 1103 comprises instructions, which when executed by the at least one processor, cause the at least one processor of the respective wireless communications device 601 and second node 604 to perform the actions above.

In some embodiments, a respective carrier 1005 and 1105 comprises the respective computer program, wherein the carrier is one of an electronic signal, an optical signal, an electromagnetic signal, a magnetic signal, an electric signal, a radio signal, a microwave signal, or a computer-readable storage medium.

Those skilled in the art will also appreciate that the units described above may refer to a combination of analog and digital circuits, and/or one or more processors configured with software and/or firmware, e.g. stored in the respective wireless communications device 601 and second node 604, that when executed by the respective one or more processors such as the processors described above. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuitry (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip (SoC).

NUMBERED EMBODIMENTS

1. A method, performed by a wireless communications device (121) operating in in a wireless communications network (100), for handling a performance evaluation of a trained Neural Network, NN, -based Auto Encoder, AE, -encoder (601-1) of the wireless communications device (121), wherein the AE-encoder (601-1) is trained to provide encoded data to a compatible trained NN-based AE-decoder (602-1) of a network node (111 , 130, 602) of the wireless communications network (100), the method comprises: receiving a trigger message for the performance evaluation of the AE-encoder (601-1); and providing, to a node (604), a report comprising an indication of a status of the performance evaluation. The method according to embodiment 1, further comprising: obtaining or determining the status of the performance evaluation. The method according to embodiment 2, wherein determining the status of the performance evaluation comprises determining that the performance evaluation has been triggered based on receiving the trigger message and wherein the indication of the status of the performance evaluation indicates that the performance evaluation has been triggered. The method according to any of the embodiments 2-3, wherein determining the status of the performance evaluation comprises determining whether or not the AE-encoder (601-1) passed or failed the performance evaluation and wherein the indication of the status of the performance evaluation indicates a pass status or a fail status of the performance evaluation. The method according to embodiment 4, wherein determining the AE-encoder (601-1) passed the performance evaluation is based on receiving a configuration of the AE- encoder from the network node (111), or wherein determining the AE-encoder (601-1) failed the performance evaluation is based on absence of the configuration of the AE- encoder. The method according to any of the embodiments 1-5, wherein the provided indication of the status of the performance evaluation indicates the pass status if the wireless communications device 121 obtains or determines the pass status of the performance evaluation within a time period, or wherein the provided indication of the status of the performance evaluation indicates the fail status if the wireless communications device 121 does not obtain or determine the pass status of the performance evaluation within the time period. The method according to any of the embodiments 1-6, wherein the provided indication of pass or fail status of the performance evaluation is associated with any of: a network identity of the wireless communications network (100), a frequency band or range for wireless transmissions in the wireless communications network (100), or a geographic area in which the wireless communications device (121) is located.

8. The method according to any of the embodiments 1-7, wherein the provided report is provided in response to obtaining or determining the status of the performance evaluation.

9. The method according to any of the embodiments 1-8, wherein the provided report includes further information about the performance evaluation of the AE-encoder.

10. A method, performed by a node (604) for handling a performance evaluation of a trained Neural Network, NN, -based Auto Encoder, AE, -encoder (601-1) of a wireless communications device (121), wherein the AE-encoder (601-1) is trained to provide encoded data to a compatible trained NN-based AE-decoder (602-1) of a network node (111) of the wireless communications network (100), the method comprising: receiving, from the wireless communications device (121), a report comprising an indication of a status of the performance evaluation of the AE-encoder (601-1) in the wireless communications network (100).

11. The method according to embodiment 10, wherein the indication of the status of the performance evaluation indicates a pass status of the performance evaluation, the method further comprising transmitting a request to stop a timer associated with the status of the performance evaluation to a second wireless communications device (612) in the wireless communications network (100).

12. The method according to embodiment 10, wherein the indication of the status of the performance evaluation indicates a fail status of the performance evaluation, the method further comprising transmitting a request to stop indicating support for the AE- encoder (601-1) to a second wireless communications device (612) in the wireless communications network (100).

13. The method according to any of the embodiments 10-12, further comprising: determining, based on the received indication of the status of the performance evaluation, whether or not the AE-encoder (601-1) is functional in the wireless communications network (100). With reference to Figure 12, in accordance with an embodiment, a communication system includes a telecommunication network 3210, such as a 3GPP-type cellular network, which comprises an access network 3211 , such as a radio access network, and a core network 3214. The access network 3211 comprises a plurality of base stations 3212a, 3212b, 3212c, such as the source and target access node 111, 112, AP STAs NBs, eNBs, gNBs or other types of wireless access points, each defining a corresponding coverage area 3213a, 3213b, 3213c. Each base station 3212a, 3212b, 3212c is connectable to the core network 3214 over a wired or wireless connection 3215. A first user equipment (UE) such as a Non-AP STA 3291 located in coverage area 3213c is configured to wirelessly connect to, or be paged by, the corresponding base station 3212c. A second UE 3292 such as a Non-AP STA in coverage area 3213a is wirelessly connectable to the corresponding base station 3212a. While a plurality of UEs 3291, 3292 are illustrated in this example, the disclosed embodiments are equally applicable to a situation where a sole UE is in the coverage area or where a sole UE is connecting to the corresponding base station 3212.

The telecommunication network 3210 is itself connected to a host computer 3230, which may be embodied in the hardware and/or software of a standalone server, a cloud-implemented server, a distributed server or as processing resources in a server farm. The host computer 3230 may be under the ownership or control of a service provider, or may be operated by the service provider or on behalf of the service provider. The connections 3221, 3222 between the telecommunication network 3210 and the host computer 3230 may extend directly from the core network 3214 to the host computer 3230 or may go via an optional intermediate network 3220. The intermediate network 3220 may be one of, or a combination of more than one of, a public, private or hosted network; the intermediate network 3220, if any, may be a backbone network or the Internet; in particular, the intermediate network 3220 may comprise two or more subnetworks (not shown).

The communication system of Figure 12 as a whole enables connectivity between one of the connected UEs 3291, 3292 such as e.g. the UE 121 , and the host computer 3230. The connectivity may be described as an over-the-top (OTT) connection 3250. The host computer 3230 and the connected UEs 3291 , 3292 are configured to communicate data and/or signaling via the OTT connection 3250, using the access network 3211, the core network 3214, any intermediate network 3220 and possible further infrastructure (not shown) as intermediaries. The OTT connection 3250 may be transparent in the sense that the participating communication devices through which the OTT connection 3250 passes are unaware of routing of uplink and downlink communications. For example, a base station 3212 may not or need not be informed about the past routing of an incoming downlink communication with data originating from a host computer 3230 to be forwarded (e.g., handed over) to a connected UE 3291. Similarly, the base station 3212 need not be aware of the future routing of an outgoing uplink communication originating from the UE 3291 towards the host computer 3230. Example implementations, in accordance with an embodiment, of the UE, base station and host computer discussed in the preceding paragraphs will now be described with reference to Figure 13. In a communication system 3300, a host computer 3310 comprises hardware 3315 including a communication interface 3316 configured to set up and maintain a wired or wireless connection with an interface of a different communication device of the communication system 3300. The host computer 3310 further comprises processing circuitry 3318, which may have storage and/or processing capabilities. In particular, the processing circuitry 3318 may comprise one or more programmable processors, application-specific integrated circuits, field programmable gate arrays or combinations of these (not shown) adapted to execute instructions. The host computer 3310 further comprises software 3311, which is stored in or accessible by the host computer 3310 and executable by the processing circuitry 3318. The software 3311 includes a host application 3312. The host application 3312 may be operable to provide a service to a remote user, such as a UE 3330 connecting via an OTT connection 3350 terminating at the UE 3330 and the host computer 3310. In providing the service to the remote user, the host application 3312 may provide user data which is transmitted using the OTT connection 3350.

The communication system 3300 further includes a base station 3320 provided in a telecommunication system and comprising hardware 3325 enabling it to communicate with the host computer 3310 and with the UE 3330. The hardware 3325 may include a communication interface 3326 for setting up and maintaining a wired or wireless connection with an interface of a different communication device of the communication system 3300, as well as a radio interface 3327 for setting up and maintaining at least a wireless connection 3370 with a UE 3330 located in a coverage area (not shown in Figure 13) served by the base station 3320. The communication interface 3326 may be configured to facilitate a connection 3360 to the host computer 3310. The connection 3360 may be direct or it may pass through a core network (not shown in Figure 13) of the telecommunication system and/or through one or more intermediate networks outside the telecommunication system. In the embodiment shown, the hardware 3325 of the base station 3320 further includes processing circuitry 3328, which may comprise one or more programmable processors, application-specific integrated circuits, field programmable gate arrays or combinations of these (not shown) adapted to execute instructions. The base station 3320 further has software 3321 stored internally or accessible via an external connection.

The communication system 3300 further includes the UE 3330 already referred to. Its hardware 3335 may include a radio interface 3337 configured to set up and maintain a wireless connection 3370 with a base station serving a coverage area in which the UE 3330 is currently located. The hardware 3335 of the UE 3330 further includes processing circuitry 3338, which may comprise one or more programmable processors, applicationspecific integrated circuits, field programmable gate arrays or combinations of these (not shown) adapted to execute instructions. The UE 3330 further comprises software 3331 , which is stored in or accessible by the UE 3330 and executable by the processing circuitry 3338. The software 3331 includes a client application 3332. The client application 3332 may be operable to provide a service to a human or non-human user via the UE 3330, with the support of the host computer 3310. In the host computer 3310, an executing host application 3312 may communicate with the executing client application 3332 via the OTT connection 3350 terminating at the UE 3330 and the host computer 3310. In providing the service to the user, the client application 3332 may receive request data from the host application 3312 and provide user data in response to the request data. The OTT connection 3350 may transfer both the request data and the user data. The client application 3332 may interact with the user to generate the user data that it provides. It is noted that the host computer 3310, base station 3320 and UE 3330 illustrated in Figure 13 may be identical to the host computer 3230, one of the base stations 3212a, 3212b, 3212c and one of the UEs 3291, 3292 of Figure 12, respectively. This is to say, the inner workings of these entities may be as shown in Figure 13 and independently, the surrounding network topology may be that of Figure 12.

In Figure 13, the OTT connection 3350 has been drawn abstractly to illustrate the communication between the host computer 3310 and the use equipment 3330 via the base station 3320, without explicit reference to any intermediary devices and the precise routing of messages via these devices. Network infrastructure may determine the routing, which it may be configured to hide from the UE 3330 or from the service provider operating the host computer 3310, or both. While the OTT connection 3350 is active, the network infrastructure may further take decisions by which it dynamically changes the routing (e.g., on the basis of load balancing consideration or reconfiguration of the network).

The wireless connection 3370 between the UE 3330 and the base station 3320 is in accordance with the teachings of the embodiments described throughout this disclosure. One or more of the various embodiments improve the performance of OTT services provided to the UE 3330 using the OTT connection 3350, in which the wireless connection 3370 forms the last segment. More precisely, the teachings of these embodiments may improve the data rate, latency, power consumption and thereby provide benefits such as reduced user waiting time, relaxed restriction on file size, better responsiveness, extended battery lifetime.

A measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 3350 between the host computer 3310 and UE 3330, in response to variations in the measurement results. The measurement procedure and/or the network functionality for reconfiguring the OTT connection 3350 may be implemented in the software 3311 of the host computer 3310 or in the software 3331 of the UE 3330, or both. In embodiments, sensors (not shown) may be deployed in or in association with communication devices through which the OTT connection 3350 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software 3311 , 3331 may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 3350 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not affect the base station 3320, and it may be unknown or imperceptible to the base station 3320. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling facilitating the host computer’s 3310 measurements of throughput, propagation times, latency and the like. The measurements may be implemented in that the software 3311, 3331 causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 3350 while it monitors propagation times, errors etc.

FIGURE 14 is a flowchart illustrating a method implemented in a communication system, in accordance with one embodiment. The communication system includes a host computer, a base station such as a AP STA, and a UE such as a Non-AP STA which may be those described with reference to Figure 12 and Figure 13. For simplicity of the present disclosure, only drawing references to Figure 14 will be included in this section. In a first action 3410 of the method, the host computer provides user data. In an optional subaction 3411 of the first action 3410, the host computer provides the user data by executing a host application. In a second action 3420, the host computer initiates a transmission carrying the user data to the UE. In an optional third action 3430, the base station transmits to the UE the user data which was carried in the transmission that the host computer initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In an optional fourth action 3440, the UE executes a client application associated with the host application executed by the host computer.

FIGURE 15 is a flowchart illustrating a method implemented in a communication system, in accordance with one embodiment. The communication system includes a host computer, a base station such as a AP STA, and a UE such as a Non-AP STA which may be those described with reference to Figure 12 and Figure 13. For simplicity of the present disclosure, only drawing references to Figure 15 will be included in this section. In a first action 3510 of the method, the host computer provides user data. In an optional subaction (not shown) the host computer provides the user data by executing a host application. In a second action 3520, the host computer initiates a transmission carrying the user data to the UE. The transmission may pass via the base station, in accordance with the teachings of the embodiments described throughout this disclosure. In an optional third action 3530, the UE receives the user data carried in the transmission.

FIGURE 16 is a flowchart illustrating a method implemented in a communication system, in accordance with one embodiment. The communication system includes a host computer, a base station such as a AP STA, and a UE such as a Non-AP STA which may be those described with reference to Figure 12 and Figure 13. For simplicity of the present disclosure, only drawing references to Figure 16 will be included in this section. In an optional first action 3610 of the method, the UE receives input data provided by the host computer. Additionally or alternatively, in an optional second action 3620, the UE provides user data. In an optional subaction 3621 of the second action 3620, the UE provides the user data by executing a client application. In a further optional subaction 3611 of the first action 3610, the UE executes a client application which provides the user data in reaction to the received input data provided by the host computer. In providing the user data, the executed client application may further consider user input received from the user. Regardless of the specific manner in which the user data was provided, the UE initiates, in an optional third subaction 3630, transmission of the user data to the host computer. In a fourth action 3640 of the method, the host computer receives the user data transmitted from the UE, in accordance with the teachings of the embodiments described throughout this disclosure.

FIGURE 17 is a flowchart illustrating a method implemented in a communication system, in accordance with one embodiment. The communication system includes a host computer, a base station such as a AP STA, and a UE such as a Non-AP STA which may be those described with reference to Figures 32 and 33. For simplicity of the present disclosure, only drawing references to Figure 17 will be included in this section. In an optional first action 3710 of the method, in accordance with the teachings of the embodiments described throughout this disclosure, the base station receives user data from the UE. In an optional second action 3720, the base station initiates transmission of the received user data to the host computer. In a third action 3730, the host computer receives the user data carried in the transmission initiated by the base station.

When using the word "comprise" or “comprising” it shall be interpreted as nonlimiting, i.e. meaning "consist at least of".

The embodiments herein are not limited to the above described preferred embodiments. Various alternatives, modifications and equivalents may be used.