Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
VISION ASSISTED USER CLUSTERING IN MMWAVE/THZ-NOMA SYSTEMS
Document Type and Number:
WIPO Patent Application WO/2023/017301
Kind Code:
A1
Abstract:
In one aspect of the teachings herein, a method for use in a base station having at least one associated imaging device includes receiving (602) image data of a wireless communication environment from the at least one associated imaging device, and detecting each of two or more users within the image data, each of the users being associated with a wireless communication device. The method further includes predicting a beam direction for each of the two or more users based upon the image data, and clustering the two or more users into one or more groups based upon the predicted beam direction associated with each user. The method further includes assigning a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group.

Inventors:
YANIKOMEROGLU HALIM (CA)
MARAQA OMAR (SA)
AL-AHMADI SAAD (SA)
RAJASEKARAN ADITYA SRIRAM (CA)
SOKUN HAMZA (CA)
Application Number:
PCT/IB2021/057455
Publication Date:
February 16, 2023
Filing Date:
August 12, 2021
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
ERICSSON TELEFON AB L M (SE)
YANIKOMEROGLU HALIM (CA)
MARAQA OMAR (SA)
AL AHMADI SAAD (SA)
International Classes:
H04B7/06; G06N3/02; H04W16/28; H04W64/00; H04W72/04
Domestic Patent References:
WO2020065384A12020-04-02
Foreign References:
US20180284217A12018-10-04
EP3110031A12016-12-28
Other References:
MUHAMMAD ALRABEIAH ET AL: "Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 14 November 2019 (2019-11-14), XP081532625
Attorney, Agent or Firm:
HOMILLER, Daniel P. (US)
Download PDF:
Claims:
CLAIMS

What is claimed is:

1. A method for use in a base station (12) having at least one associated imaging device (16), the method comprising: receiving (602) image data of a wireless communication environment from the at least one associated imaging device; detecting (604) each of two or more users within the image data, each of the users being associated with a wireless communication device; predicting (606) a beam direction for each of the two or more users based upon the image data; clustering (608) the two or more users into one or more groups based upon the predicted beam direction associated with each user; and assigning (610) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group.

2. The method of claim 1 or 2, wherein the clustering is performed, for at least one of the users, without regard to channel information for that user.

3. The method of claims 1 or 2, further comprising: receiving channel information associated with at least one of the users, wherein the predicting of the beam direction for each user is further based upon the channel information associated with the at least one user.

4. The method of claims 1-3, wherein the predicted beam direction for each user is further based on a weighted combination of the image data and the channel information.

5. The method of claim 4, wherein the channel information includes location information associated with each user.

6. The method of any of claims 1-5, wherein clustering the one or more users into one or more groups is based upon a clustering metric. 7. The method of claim 6, wherein the clustering metric is calculated based on a score assigned to each user, where the score is based upon which of at least two categories applies to a beam serving a cluster to which the user is assigned.

8. The method of claim 7, wherein the at least two categories include (a) the best beam for the user, given the predicted beam direction, and (b) a beam spatially adjacent to the best beam for the user, given the predicted beam direction.

9. The method of claim 7, wherein the at least two categories further include (c) any beam other than the beast beam for the user and a beam spatially adjacent to the best beam for the user, given the predicted beam direction.

10. The method of claim 7, wherein the method comprises determining the score assigned to each user by: assigning a first score to a particular user based upon a determination that the assigned beam for the particular user is a best beam for the particular user from the set of candidate beams; assigning a second score to a particular user based upon a determination that the predicted beam for the particular user is adjacent to the best beam for the particular user; and assigning a third score to a particular user based upon a determination that the predicted beam for the particular user is any other beam that is not the best beam or adjacent to the best beam for the particular user.

11. The method of claim 10, wherein the first score is greater than the second score, and the second score is greater than the third score.

12. The method of any of claims 1-11, wherein the image data includes at least one first image having a plurality of users depicted therein.

13. The method of any of claim 12, wherein detecting each of two or more users within the image data further comprises: detecting two or more of the plurality of users within the at least one first image uses an object detection procedure; and generating a plurality of second images, each of the second images including one of the detected users, the predicting of the beam direction for each of the two or more users being based upon the plurality of second images.

14. The method of any of claims 1-13, wherein the image data includes one or more RGB images.

15. The method of any of claims 1-14, wherein predicting the beam direction for each of the two or more users utilizes a neural network.

16. The method of claim 15, wherein the neural network is a convolutional neural network.

17. The method of an of claims 1 -16, wherein the base station is associated with at least one of a non-orthogonal multiple access (NOMA) communication network or a millimeter wave (mmWave) communication network.

18. A method for use in a base station (12) having at least one associated imaging device (16), the method comprising: receiving (702) image data of a wireless communication environment from the at least one associated imaging device; detecting (704) each of two or more users within the image data, each of the users being associated with a wireless communication device; and for each particular user of the two or more users: determining (708), for the particular user, whether the particular user is detected within the image data; determining (710), for the particular user, whether channel information associated with the particular user is available; and predicting (712) a beam direction for the particular user based upon at least one of the image data and the channel information; clustering (714) the two or more users into one or more groups based upon the predicted beam direction associated with each user; and assigning (716) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group. 19. The method of claim 18, further comprising: responsive to determining that the particular user is detected within the image data and channel information associated with the particular user is not available, predicting the beam direction for the particular user based upon the image data.

20. The method of claim 18, further comprising: responsive to determining that the particular user is not detected within the image data and channel information associated with the particular user is available, predicting the beam direction for the particular user based upon the channel information.

21. The method of claim 18, further comprising: responsive to determining that the particular user is detected within the image data and channel information associated with the particular user is available, predicting the beam direction for the particular user based upon the image data and the channel information.

22. The method of claim 21, wherein the predicated beam direction for the particular user is further based on a weighted combination of the image data and the channel information.

23. The method of any of claims 18-22, wherein the channel information includes location information associated with each user.

24. The method of any of claims 18-23, wherein clustering the one or more users into one or more groups is based upon a clustering metric.

25. The method of claim 24, wherein the clustering metric is calculated based on a score assigned to each user, where the score is based upon which of at least two categories applies to a beam serving a cluster to which the user is assigned.

26. The method of claim 25, wherein the at least two categories include (a) the best beam for the user, given the predicted beam direction, and (b) a beam spatially adjacent to the best beam for the user, given the predicted beam direction. 27. The method of claim 25, wherein the at least two categories further include (c) any beam other than the beast beam for the user and a beam spatially adjacent to the best beam for the user, given the predicted beam direction.

28. The method of claim 25, wherein the method comprises determining the score assigned to each user by: assigning a first score to a particular user based upon a determination that the assigned beam for the particular user is a best beam for the particular user from the set of candidate beams; assigning a second score to a particular user based upon a determination that the predicted beam for the particular user is adjacent to the best beam for the particular user; and assigning a third score to a particular user based upon a determination that the predicted beam for the particular user is any other beam that is not the best beam or adjacent to the best beam for the particular user.

29. The method of claim 28, wherein the first score is greater than the second score, and the second score is greater than the third score.

30. A base station (12) configured for operation in a wireless communication network, the base station comprising: processing circuitry (902); and memory (904) storing instructions, which when executed by the processing circuitry, cause the processing circuitry to: receive (602) image data of a wireless communication environment from the at least one imaging device associated with the base station; detect (604) each of two or more users within the image data, each of the users being associated with a wireless communication device; predict (606) a beam direction for each of the two or more users based upon the image data; cluster (608) the two or more users into one or more groups based upon the predicted beam direction associated with each user; and assign (610) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group. 31. The base station (12) of claim 30, wherein the processing circuitry is further caused to: receive channel information associated with at least one of the users, wherein the predicting of the beam direction for each user is further based upon the channel information associated with the at least one user.

32. The base station (12) of claims 30-31, wherein the predicted beam direction for each user is further based on a weighted combination of the image data and the channel information.

33. The base station (12) of any of claims 30-32, wherein clustering the one or more users into one or more groups is based upon a clustering metric.

34. The base station (12) of claim 33, wherein the clustering metric is calculated based on a score assigned to each user, where the score is based upon which of at least two categories applies to a beam serving a cluster to which the user is assigned.

35. The base station (12) of claim 34, wherein the at least two categories include (a) the best beam for the user, given the predicted beam direction, and (b) a beam spatially adjacent to the best beam for the user, given the predicted beam direction.

36. The base station (12) of claim 34, wherein the at least two categories further include (c) any beam other than the beast beam for the user and a beam spatially adjacent to the best beam for the user, given the predicted beam direction.

37. The base station (12) of claim 34, wherein the processing circuitry is further caused to determine the score assigned to each user by: assigning a first score to a particular user based upon a determination that the assigned beam for the particular user is a best beam for the particular user from the set of candidate beams; assigning a second score to a particular user based upon a determination that the predicted beam for the particular user is adjacent to the best beam for the particular user; and assigning a third score to a particular user based upon a determination that the predicted beam for the particular user is any other beam that is not the best beam or adjacent to the best beam for the particular user.

38. A base station (12) configured for operation in a wireless communication network, the base station comprising: processing circuitry (902); and memory (904) storing instructions, which when executed by the processing circuitry, cause the processing circuitry to: receive (702) image data of a wireless communication environment from the at least one associated imaging device; detect (704) each of two or more users within the image data, each of the users being associated with a wireless communication device; and for each particular user of the two or more users: determine (708), for the particular user, whether the particular user is detected within the image data; determine (710), for the particular user, whether channel information associated with the particular user is available; and predict (712) a beam direction for the particular user based upon at least one of the image data and the channel information; cluster (714) the two or more users into one or more groups based upon the predicted beam direction associated with each user; and assign (716) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group.

39. The base station (12) of claim 38, wherein the processing circuitry is further caused to: responsive to determining that the particular user is detected within the image data and channel information associated with the particular user is not available, predict the beam direction for the particular user based upon the image data.

40. The base station (12) of claim 38, wherein the processing circuitry is further caused to: responsive to determining that the particular user is not detected within the image data and channel information associated with the particular user is available, predict the beam direction for the particular user based upon the channel information.

41. The base station (12) of claim 38, wherein the processing circuitry is further caused to: responsive to determining that the particular user is detected within the image data and channel information associated with the particular user is available, predict the beam direction for the particular user based upon the image data and the channel information.

42. A computer-readable medium storing instructions that, when executed by processing circuitry (902) in a base station (12), configures the base station to: receive (602) image data of a wireless communication environment from the at least one imaging device associated with the base station; detect (604) each of two or more users within the image data, each of the users being associated with a wireless communication device; predict (606) a beam direction for each of the two or more users based upon the image data; cluster (608) the two or more users into one or more groups based upon the predicted beam direction associated with each user; and assign (610) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group.

43. A computer-readable medium storing instructions that, when executed by processing circuitry (902) in a base station (12), configures the base station to: receive (702) image data of a wireless communication environment from the at least one associated imaging device; detect (704) each of two or more users within the image data, each of the users being associated with a wireless communication device; and for each particular user of the two or more users: determine (708), for the particular user, whether the particular user is detected within the image data; determine (710), for the particular user, whether channel information associated with the particular user is available; and predict (712) a beam direction for the particular user based upon at least one of the image data and the channel information; cluster (714) the two or more users into one or more groups based upon the predicted beam direction associated with each user; and assign (716) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group.

Description:
VISION ASSISTED USER CLUSTERING IN MMWAVE/THZ-NOMA SYSTEMS

TECHNICAL HELD

The present invention relates to wireless communication networks, and particularly relates to the clustering of users for use in connecting to such networks.

BACKGROUND

With the increasing demand for bandwidth in wireless networks, millimeter-wave (mmWave) communications is an attractive approach for fifth-generation (5G) wireless communications. mmWave communications typically utilize frequencies between 30 GHz and 300 GHz which results in highly directional transmissions between base stations and wireless communication devices within the wireless network environment. As a result, the mmWave spectrum provides for wireless transmissions with reduced interference and higher data rates. Due to the rapid growth of wireless devices, it is desirable to apply new multiple access techniques to mmWave communications to enhance the availability of the mmWave spectrum. One promising multiple access technique is that of non-orthogonal multiple access (NOMA). A key idea of NOMA is to serve multiple users at the same resource (e.g., frequency and time) and exploiting user difference in the power domain. By utilizing successive interference cancellation (SIC) techniques within user equipment receivers, channel differences between users can be exploited to service multiple users.

Note that the 3GPP documentation refers to wireless communication devices as items of

“user equipment,” where “UE” denotes a single wireless device and “UEs” denotes plural wireless devices. The term “wireless communication device” as used herein encompasses the term “UE” and more. Indeed, unless otherwise noted, the term encompasses essentially any type of wireless communication apparatus that is configured to communicate within a wireless communication network. Without limitation, then, the term “wireless communication device” encompasses smart phones, feature phones, cellular network modems and dongles, Machine Type Communication (MTC) or Machine-to- Machine (M2M) devices, along with wireless- enabled computers, laptops, tablets, and the like.

An effective approach for enhancing the performance of NOMA networks is the use of user clustering. User clustering refers to the selection of users to service in a cluster (e.g., a NOMA cluster), typically in a beam via beamforming techniques. User ordering refers to the order in which SIC is applied at the users in the downlink. Power allocation techniques are used to allocate a certain amount of power to each user in the cluster, so that SIC decoding can used successfully and the target rates of each user are met. As users are grouped in clusters, the weakest user only has to decode its own signal, while the strongest user has to decode the signals of all other users in the SIC procedure. In mmWave systems, the user channels are highly correlated due to the directional nature of mmWave transmissions. User clustering schemes in mmWave-NOMA systems typically exploit tiie high correlation amongst user channels to cluster correlated users together. The high level of correlation between user channels in mmWave makes them very suitable for the formation of user clusters to be served by a single beam and separated in the power domain through NOMA.

Such a system presents the problem of grouping the users optimally, such that the users in one cluster have highly correlated channels and less correlation with users in other clusters. Recent work in mmWave-NOMA system have used machine learning (ML) clustering techniques to identify correlated users and group the users in NOMA clusters. Existing approaches, including those utilizing ML-based approaches, make clustering decisions based on the channel information of all of the users available in the system. A problem with these existing approaches is that they fail to perform in cases in which access to channel information for all of the users is unavailable or inadequate. In addition, obtaining the channel information of all users in order to perform the user clustering requires a high overhead.

SUMMARY

In various embodiments of the techniques, apparatuses, and systems described herein, a radio network node utilizes image data captured by an imaging device to solve the user clustering problem in mmWave-NOMA system or other wireless communication systems. In some wireless communication systems, for example, camera equipped base stations (BSs) are used to capture red-green-blue (RGB) images, and deep learning algorithms are utilized on these RGB images for performing wireless communication tasks. The radio network node may use visual information of users in the wireless network environment to make decisions regarding how to form NOMA clusters of the users (i.e., groups of the users) and assign a suitable beam to each cluster to be used for communication with the radio network node. In an example case in which no access to visual information is available, the radio network node may rely on channel information, if available, to make clustering decisions. In another example case in which access to channel information is not available, tire radio network node may rely on visual information to make clustering decisions. In another example case in which access to both channel information and visual information is available, the radio network node may fuse the channel information and visual information to make better clustering decisions. In an example method, as might be carried out in a base station having at least one associated imaging device, image data of a wireless communication environment is received from the at least one associated imaging device, and each of two or more users are detected within the image data, each of the users being associated with a wireless communication device. This method further comprises predicting a beam direction for each of the two or more users based upon the image data, clustering the two or more users into one or more groups based upon the predicted beam direction associated with each user, and assigning A method for use in a base station (12) having at least one associated imaging device (16), the method comprising:

In another example method, again as might be carried out in a base station having at least one associated imaging device, image data of a wireless communication environment is received from the at least one associated imaging device, and each of two or more users are detected within the image data, each of the users being associated with a wireless communication device. In this example method, for each particular user of the two or more users, the base station determines, for the particular user, whether the particular user is detected within the image data and determines, for the particular user, whether channel information associated with the particular user is available. The base station then predicts a beam direction for the particular user based upon at least one of the image data and the channel information, clusters the two or more users into one or more groups based upon the predicted beam direction associated with each user, assigns a beam from a set of candidate beams to each of the one or more groups, and uses each assigned beam for communication with users in the respective group.

Corresponding base station apparatuses, computer program products, and computer- readable mediums are also described below.

Of course, the present invention is not limited to the above features and advantages. Indeed, those skilled in the art will recognize additional features and advantages upon reading the following detailed description, and upon viewing the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a diagram of one embodiment of a wireless communication network for vision assisted user clustering.

Fig. 2 is a block diagram of an embodiment of a user clustering procedure.

Fig. 3 is a diagram of a wireless communication environment illustrating vision assisted user clustering.

Fig. 4 is a diagram of an example beam assignment for clustered users in a wireless communication network. Fig. 5 is a diagram of another wireless communication environment for vision assisted user clustering.

Fig. 6 is a logic flow diagram of one embodiment of a method for vision assisted user clustering in a wireless communication environment.

Fig. 7 is a flow diagram of another embodiment of a method for vision assisted user clustering in a wireless communication environment.

Fig. 8 is a diagram illustrating simulation results of an example vision assisted user clustering procedure.

Fig. 9 is a diagram of one embodiment of a radio network node configured for vision assisted user clustering.

Fig. 10 is a block diagram of one embodiment of a radio network node comprising one or more operational or functional modules.

DETAILED DESCRIPTION

Fig. 1 illustrates one embodiment of a wireless communication network 10 that is configured to use vision assisted user clustering. While not so limited, the network 10 may be a mmWave-NOMA cellular radio network based on the Long Term Evolution (LTE) standard, or based on another Third Generation Partnership Project (3GPP) standard. The network 10 includes one or more radio network nodes 12, which comprise base stations, access points, or the like. The radio network node 12 provides radio service in a corresponding coverage area 14, e.g., a “cell.” While any given one or more of the radio network nodes 12 may provide more than one cell, e.g., using different radio resources for each one, the diagram depicts each radio network node 12 as corresponding to a single overall coverage area 14. The wireless communication network 10 further includes one or more images device 16, such as a camera, associated with the radio network node 12 configured to capture image data corresponding to one or more images representative of the coverage area 14. The radio network node 12 is in communication with a plurality of wireless communication devices 20-1 through 20-5 within the coverage area 14. While Fig. 1 illustrates wireless communication devices 20-1 through 20-5, it shall be appreciated that a lesser or greater number of wireless communication devices 20 may be operating within the network 10 at any given time. It shall also be understood that the wireless communication devices 20 are not necessarily all of the same type or function. Example wireless communication devices 20 include any one or more of user equipments (UEs), smartphones, feature phones, wireless computers, communication network adaptors, dongles, Machine-Type Communication (MTC) devices, which are also referred to as M2M devices, etc. In one or more embodiments, the radio network node 12 utilizes image data captured by imaging device 16 to solve the user clustering problem in mmWave-NOMA system or other wireless communication systems. In some wireless communication systems, camera equipped base stations (BSs) are used to capture red-green-blue (RGB) images, and deep learning algorithms are utilized on these RGB images for performing wireless communication tasks. A synthetic data generation framework for RGB images and associated user channels for mobile users has been developed to facilitate these tasks. In one example, convolutional neural networks (CNNs) based on residual networks (ResNets) are applied to solve a beam prediction problem. In a particular example, an 18-layer ResNet is adopted and customized to fit the beam prediction problem.

Learning beam prediction from images is essentially an image classification task, where an algorithm relies on knowing a user’s location at a scene within a wireless network environment. A set of candidate beam vectors divide the spatial dimensions of the scene into multiple sectors. Depending on the known user location in the scene, an algorithm attempts to identify to which of the sectors a user belongs. In other words, a goal of tire algorithm is to find a user’s “best beam direction.” In mathematical terms, the goal of the algorithm is to select the best beam ƒ* from a set of candidate beams F. Let h represent the channel information for the user. Then,

One approach to leam the beam prediction function is based on deep convolutional neural networks and transfer learning. A pre-trained 18-layer ResNet (Resnet-18) trained on an image database is customized for beam prediction. The final fully connected layer in the ResNet- 18 is removed and replaced with another fully connected layer of 64 neurons to represent 64 beams. Using training data of images of users explicitly marked against their best beam in a supervised learning setting, the model learns a new classification function, that maps an image to a beam index. Thus, the beam prediction function parameterized by a set of parameters outputs a probability distribution P = {p 0 , p 1, ... , p 63 }. The best beam corresponds to the index of the element with maximum probability, where

The parameters of the neural network are tuned to maximize the probability that i.e., chosen to maximize When generating an image-beam dataset using image data from a framework, every image in the dataset is paired with a beam from a candidate beam list using channel information to generate a dataset that contains “image-beam pairs” which are used to train a neural network.

As discussed above, in mmWave systems the users’ channels are highly correlated due to tiie highly directional nature of mmWave transmission. The user clustering schemes in mmWave-NOMA systems typically exploit the high correlation amongst user channels to cluster correlated users together. Recent work in mmWave-NOMA systems have also used machine learning clustering techniques to identify correlated users and group them in NOMA clusters. These approaches, including the ML based approaches, all make clustering decisions based on channel information of all the users available in the system.

Existing solutions for NOMA clustering use only channel information with no utilization of visual information. Hence, these existing solutions fail to perform in cases access to the channel information is problematic. In addition, existing solutions requires the high overhead of obtaining the channel information of all users in order to perform user clustering.

One or more embodiments described herein provide for NOMA clustering for mmWave systems and benefit not only from channel information, if available, but also from visual information obtained by one or more imaging devices associated with a radio network node such as a BS. In an embodiment, the radio network node utilizes the visual information of users in the wireless network environment to make decisions regarding how to form NOMA clusters of the users (i.e., groups of the users) and assign a suitable beam to each cluster to be used for communication with the radio network node. In an example case in which no access to visual information is available, the radio network node may rely on channel information, if available, to make clustering decisions. In another example case in which access to channel information is not available, the radio network node may rely on visual information to make clustering decisions. In another example case in which access to both channel information and visual information is available, the radio network node may fuse the channel information and visual information to make better clustering decisions.

An advantage that may be provided by one or more embodiments is to address overhead problems caused by requesting and receiving channel information for each user, by allowing the determination of user clusters based on images captured at the radio network node rather than requiring channel information. Accordingly, NOMA clustering may be performed without the overhead of requesting each user’s channel. Another advantage that may be provided by various embodiments is that the allows the exploitation of advances in deep learning techniques for image processing, such as object recognition and detection, to improve the performance of user clustering techniques by utilizing the visual information. Still another advantage that may be provided in one or more embodiments is increased cell throughput by using visual information for user clustering when access to channel information is problematic.

Fig. 2 is a block diagram of an embodiment of a user clustering procedure 200 implemented by a radio network node 12. The procedure 200 includes receiving (Block 210), from the at least one imaging device 16, image data of an image of a wireless communication environment having visual representations of multiple wireless communication device users depicted within the image. In a particular embodiment, the image data includes RGB image data. The procedure 200 further includes generating (Block 220) multiple images from the received image in which each generated image depicts a single user. In one or more embodiments, an object detection procedure is used to identify the individual users in the received image and generate the multiple images, each including a single user. In a particular embodiment, the radio network node 12 uses a deep neural network (DNN) to perform object detection on the received image.

The procedure 200 further includes performing (Block 230) beam prediction for each user based upon the multiple images. In an embodiment, a set of possible beam vectors, called a beam-forming codebook, divides the spatial dimension of the wireless network environment into multiple sectors. With this pre-defined beam-forming codebook, learning beam prediction from the RGB images degenerates to an image classification task in which the goal is to identify to which sector a user belongs. For an example problem in which there are N users (e.g., 100 users), a goal is to identify which users can be served by the same best beam and cluster those users into a group. In an embodiment, once multiple images with single users are obtained, a CNN, such as a ResNet-18, is used to perform beam prediction for each user in order identify a predicted best beam for each of the users. Once the network is sufficiently trained, i.e., when the accuracy on a validation set is above a predetermined threshold, the beam prediction results may be used to cluster users that all identify the same best beam together.

Fig. 3 is a diagram of a wireless communication environment 30 illustrating vision- assisted user clustering. The wireless communication environment 30 includes a radio network node 12 and associated imaging device 16. The radio network node 12 includes a plurality of beams 40-1 through 40-4 for serving a plurality of wireless communication devices associated each of users 50-1 through 50-4. Fig. 3 illustrates that it is desired to cluster users that are all in the same beam direction into the same group, with an optimal strategy of clustering users who all have the same “best beam” into the same beam direction, using RGB images captured by imaging device 16 and without the overhead of collecting channel information of each user 50-1 through 50-4. Referring again to Fig. 2, the radio network node 12 clusters (Block 240) the users into two or more groups based upon the predicted beam direction for each user. In an embodiment, clustering the users into one or more groups is based upon evaluating a clustering metric. In a particular embodiment, the clustering metric rewards a given clustering the highest amount if it places a user in the user’s best beam, rewards the clustering a lesser amount if it places a user in any adjacent beam to its best beam, and gives no score or a negative score if a user is placed in a cluster served by any other beam than the best beam or adjacent beam.

Hence ƒ u , represents a score that is assigned per-user as follows

Upon clustering two or more users into one or more groups based upon the predicted beam direction associated with each user, the radio network node may assign a beam from the set of beam candidates to each of the one or more groups, and each user in the respective group may use the assigned beam for communication.

Fig. 4 is a diagram of an example beam assignment for clustered users in a wireless communication network. In a particular embodiment, the wireless communication network is a NOMA/mmWave network. A radio network node 12 is configured to transmit a plurality of directional beams 40, where directional beams 40-1 through 40-12 are shown by way of example for illustration. Fewer beams or more beams may be configured and not all radio network nodes 12 will necessarily operate with the same number, shape, or configuration of beams. Further, the number of beams used by a given radio network node 12, or the beam parameters associated therewith, may be adapted from time to time. Here, it shall be understood that the phrase “beam direction” encompasses azimuthal directions (horizontal angles), or elevational directions (vertical angles), or both. Thus, a given beam direction may be defined by horizontal and/or vertical angles or angular ranges. Each directional beam 40 provides coverage for a portion of the overall coverage area of the radio network node 12.

The radio network node 12 serves a number of wireless communication devices 20-1 through 20-9 are clustered in a number of groups to be served by a particular beams 40-1 through 40-12. In the example of Fig. 4, wireless communication device 20-1 is clustered in a first group and communicates using directional beam 40-1, and wireless communication device 20-2 is clustered in a second group and communicates using directional beam 40-3. Wireless communication devices 20-3 and 20-4 are clustered in a third group and communicate using directional beam 40-6. Wireless communication devices 20-5 and 20-6 are clustered in a fourth group and communicate using directional beam 40-9. Wireless communication devices 20-7, 20-8, and 20-9 are clustered in a fifth group and communicate using directional beam 40-10. Fig. 5 is a diagram of another wireless communication environment 50 illustrating vision-assisted user clustering. The wireless communication environment 50 includes a classroom having a radio network node 12 and associated imaging device 16. The radio network node 12 is configured to service a plurality of wireless communication devices 20, each associated with a particular user within the classroom. In an embodiment, the radio network node 12 receives an image captured by the imaging device 16 depicting the wireless communication devices 20 and/or the associated users. The radio network node 12 may further receive user location information or other channel information associated with one or more of the users when available. The radio network node 12 generates a plurality of images in which each image depicts a single user, and performs a best beam prediction for each user based upon one or more of the images and user location information. The radio network 12 node performs NOMA clustering of the wireless communication devices 20 into one or more groups based upon the best beam prediction for each user, and assigns a beam to each group for communication of the wireless communication devices 20 with the radio network node 12.

Fig. 6 is a logic flow diagram of one embodiment of a method 600 for vision assisted user clustering in a wireless communication environment performed by a base station or other radio network node. In an embodiment, the base station is associated with at least one of a non- orthogonal multiple access (NOMA) communication network or a millimeter wave (mmWave) communication network. The method 600 includes receiving (Block 602) image data of a wireless communication environment from at least one imaging device associated with the base station. In an example embodiment, the image data includes one or more RGB images. The method 600 further includes detecting (Block 604) each of two or more users within the image data, each of the users being associated with a wireless communication device. In an embodiment, the image data includes at least one first image having a plurality of users depicted therein. In an embodiment, detecting each of two or more users within the image data further comprises detecting two or more of the plurality of users within the at least one first image uses an object detection procedure; and generating a plurality of second images, each of the second images including one of the detected users, the predicting of the beam direction for each of the two or more users being based upon the plurality of second images.

The method 600 further includes predicting (Block 606) a beam direction for each of the two or more users based upon the image data. In an embodiment, predicting the beam direction for each of the two or more users utilizes a neural network. In particular embodiment, the neural network is a CNN or DNN. In still other embodiments, a suitable technique for predicting beam direction is used without requiring use of a neural network. The method 600 further includes clustering the two or more users into one or more groups based upon the predicted beam direction associated with each user. In a particular embodiment, the clustering is performed, for at least one of the users, without regard to channel information for that user. The method 600 further includes assigning (Block 610) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group.

In an embodiment, the method 600 may further include receiving channel information associated with at least one of the users, wherein the predicting of the beam direction for each user is further based upon the channel information associated with the at least one user. In a particular example, the channel information includes location information associated with each user. In a particular embodiment, the predicted beam direction for each user is further based on a weighted combination of the image data and the channel information. In a particular example, an algorithm may weight the image data and channel information by different amounts when predicting the beam direction for each user. For example, the image data may be given greater weight than the channel information, or vice versa.

In an embodiment, clustering the one or more users into one or more groups is based upon a clustering metric. In a particular example, the clustering metric is calculated based on a score assigned to each user, where the score is based upon which of at least two categories applies to a beam serving a cluster to which the user is assigned. In a particular example, the at least two categories include (a) the best beam for the user, given the predicted beam direction, and (b) a beam spatially adjacent to the best beam for the user, given the predicted beam direction. In a particular example, the at least two categories further include (c) any beam other than the beast beam for the user and a beam spatially adjacent to the best beam for the user, given the predicted beam direction.

In an embodiment, the method 600 may further include determining the score assigned to each user by: assigning a first score to a particular user based upon a determination that the assigned beam for the particular user is a best beam for the particular user from the set of candidate beams; assigning a second score to a particular user based upon a determination that the predicted beam for the particular user is adjacent to the best beam for the particular user; and assigning a third score to a particular user based upon a determination that the predicted beam for the particular user is any other beam that is not the best beam or adjacent to the best beam for the particular user. In a particular example, the first score is greater than the second score, and tiie second score is greater than the third score.

Fig. 7 is a flow diagram of another embodiment of a method 700 for vision assisted user clustering in a wireless communication environment, performed by a base station or other radio network node. The method 700 includes receiving (Block 702) image data of a wireless communication environment from at least one imaging device associated with the base station. The method 700 further includes detecting (Block 704) each of two or more users within the image data, each of the users being associated with a wireless communication device. The method 700 further determining (Block 706) whether ungrouped users remain among the two or more users. If ungrouped users are remaining, the method 700 further includes, for each particular user of the two or more users, determining (Block 708), for the particular user, whether the particular user is detected within the image data, determining (Block 710), for the particular user, whether channel information associated with the particular user is available, and predicting (Block 712) a beam direction for the particular user based upon at least one of the image data and the channel information.

If no ungrouped users are remaining, the method 700 further includes clustering (Block 714) the two or more users into one or more groups based upon the predicted beam direction associated with each user. The method 700 further includes assigning (Block 716) a beam from a set of candidate beams to each of the one or more groups and using each assigned beam for communication with users in the respective group.

In an embodiment, the method 700 further includes, responsive to determining that the particular user is detected within the image data and channel information associated with the particular user is not available, predicting the beam direction for the particular user based upon tiie image data. In another embodiment, the method 700 further includes, responsive to determining that the particular user is not detected within the image data and channel information associated with the particular user is available, predicting the beam direction for the particular user based upon the channel information. In still another embodiment, the method 700 further includes, responsive to determining that the particular user is detected within the image data and channel information associated with the particular user is available, predicting the beam direction for the particular user based upon the image data and the channel information.

In an embodiment, the predicated beam direction for the particular user is further based on a weighted combination of the image data and the channel information. In an embodiment, the channel information includes location information associated with each user.

In an embodiment, clustering the one or more users into one or more groups is based upon a clustering metric. In a particular embodiment, the clustering metric is calculated based on a score assigned to each user, where the score is based upon which of at least two categories applies to a beam serving a cluster to which the user is assigned. In a particular embodiment, the at least two categories include (a) the best beam for the user, given the predicted beam direction, and (b) a beam spatially adjacent to the best beam for the user, given the predicted beam direction. In another particular embodiment, the at least two categories further include (c) any other beams, i.e., any beam other than the beast beam for the user and a beam spatially adjacent to the best beam for the user, given the predicted beam direction.

In an embodiment, the method 700 further includes determining the score assigned to each user by: assigning a first score to a particular user based upon a determination that the assigned beam for the particular user is a best beam for the particular user from the set of candidate beams; assigning a second score to a particular user based upon a determination that the predicted beam for the particular user is adjacent to the best beam for the particular user; and assigning a third score to a particular user based upon a determination that the predicted beam for the particular user is any other beam that is not the best beam or adjacent to the best beam for the particular user. In an embodiment, the first score is greater than the second score, and the second score is greater than the third score.

Fig. 8 is a diagram illustrating simulation results 800 of an example vision-assisted user clustering procedure. In simulations, the vision-assisted user clustering procedure as described herein utilizing a ResNet-18 CNN clustering technique was evaluated against the following: 1) an optimal solution using channel information; and 2) K-means clustering. The optimal solution is obtained using the channel information from the dataset, which allows determining of the best beam for each user. All users having the same beast beam were clustered together. The example ResNet-18 CNN clustering technique described herein performs clustering based of predictions made by the neural network from the RGB images captured by the base station instead of the channel information. For the K-means case, the same location information available for each user is fed into a standard K-means algorithm and the clustering result is obtained.

Fig. 8 shows that the ResNet-18 CNN clustering technique performs close to the optimal solution, without requiring the overhead of channel information for each user. The x-axis is labeled with the number of users and the y-axis is label with a custom clustering metric. The optimal solution is a function y = 2x, since each user always gets a score of 2. Hence, the slope for the optimal solution is 2. The CNN-based clustering technique using image data described herein performs close to that of the optimal solution where it reflects close to 90% prediction success. Even when the ResNet-18 CNN makes a wrong prediction, if it places the user in an adjacent cluster to its best beam, it still receives a score of 1. Finally, the results are compared against a traditional K-means clustering algorithm, which performs the clustering based on a distance metric using the location of the users. While K-means is capturing the closeness of the users, what is desired is the spatial similarity in angle. Accordingly, the simulation results illustrate that the performance of the ResNet-18 CNN-based clustering is close to that of the optimal solution and significantly better than K-means clustering for a large number of users. As the number of users in the system increases, and the quality of the clustering decreases the ResNet-18 CNN-based clustering scheme performs much better than K-means clustering and close to the optimal clustering.

Fig. 9 is a diagram of one embodiment of a radio network node 12 configured for vision assisted user clustering. The radio network node 12 is in communication with an imaging device 16 for receiving one or more images therefrom. The example radio network node 12 comprises communication interface circuitry 910, processing circuitry 902, and storage 904. The communication interface circuitry 910 comprises communication interface circuitry configured for communicating with one or more wireless communication devices 20. Such circuitry includes radiofrequency transmitter circuitry 920 and receiver circuitry 922. Further, the communication interface circuitry 910 may include other interface circuitry not explicitly shown, such as a network communication interface, e.g., an SI interface, for communicating one or more nodes in a “core network,” and an inter-base-station communication interface, e.g., an X2 interface, for communication with other radio network nodes 12.

The processing circuitry 902 comprises fixed circuitry, programmed circuitry, or a mix of fixed and programmed circuitry. In an example embodiment, the processing circuitry 902 comprises one or more microprocessor-based circuits or one or more DSP-based, FPGA-based, or ASIC-based circuits, or any mix thereof. In a particular example, the processing circuitry 902 is specially adapted or otherwise configured to operate according to the radio network node method(s) of operation herein, via the execution of computer program instructions comprising a computer program or computer instructions 906. The processing circuitry 902 may further use and/or store various items of configuration data 908 associated with such operation, via the storage 904.

The storage 904 comprises any one or more of solid-state storage, disk storage, etc., and may provide both volatile, working memory and non-volatile, program and data storage. The storage 904, therefore, may include a mix of memory or storage circuit or device types. Non- limiting examples include SRAM or DRAM, FLASH, EEPROM, and Solid State Disk (SSD) storage. In any case, it shall be understood that in one or more embodiments the storage 904 includes a non-transitory computer-readable medium storing a computer program 906, the execution of which by processing circuitry in the radio network node 12 configures the processing circuitry 902 according to the teachings herein. Non-transitory, as used here, does not necessarily mean permanent or unchanging, but does denote storage of at least some persistence.

In an example embodiment, the transmitter circuitry 920 and/or receiver 922 is configured to transmit or receive signals in a plurality of directional beams via an associated antenna array. Each directional beam has a respective coverage area, which is defined by the direction and shape or size of the directional beam Correspondingly, the processing circuitry 902, is configured to perform the operations associated with vision assisted user clustering described herein. While the methods 600 and 700 may be implemented in the radio network node 12 seen in the example embodiment of Fig. 9, other architectures or implementation details may be used. In more general terms, a radio network node 12 includes processing circuitry that is adapted programmatically or otherwise to implement functions or modules that operate according to the method operations set forth herein.

Fig. 10 is a block diagram of one embodiment of a radio network node 12 comprising one or more operational or functional modules. In the arrangement of Fig. 10, the radio network node 12 includes a transceiver module 1000 that is configured to communicate with one or more wireless communication devices via a plurality of directional beams. The radio network node 12 further includes a detecting module 1002 configured to detect two or more uses within received image data, and a beam prediction module configured to predict a beam direction for each of the two or more users. The radio network node 12 further includes a clustering module 1006 configured to cluster the two or more users into one or more groups based upon the predicted beam direction associated with each user. The radio network node 12 further includes a beam assignment module 1008 configured to assign a beam from a set of candidate beams to each of the one or more groups, wherein wireless communication devices within a group use the assigned beam for communication.

Notably, modifications and other embodiments of the disclosed invention(s) will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the invention(s) is/are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of this disclosure. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.