Title:
APPARATUS, METHOD AND COMPUTER PROGRAM FOR ACCELERATING GRID-OF-BEAMS OPTIMIZATION WITH TRANSFER LEARNING
Document Type and Number:
WIPO Patent Application WO/2021/244912
Kind Code:
A3
Abstract:
A deep transfer reinforcement learning (DTRL) method based on transfer learning within a deep reinforcement learning (DRL) framework is provided to accelerate the GoB optimization decisions when experiencing environment changes in the same source radio network agent or when being applied from a source radio network agent to a target radio network agent. The transferability of the knowledge embedded in a pre-trained neural network model as a Q-approximator is exploited, and a mechanism to transfer parameters from a source agent to a target agent is provided, where the transferability criterion is based on the similarity measure between the source and target domain.
Inventors:
LIAO QI (DE)
SYED MUHAMMAD FAHAD (FR)
CAPDEVIELLE VERONIQUE (FR)
FEKI AFEF (FR)
KALYANASUNDARAM SURESH (IN)
MALANCHINI ILARIA (DE)
SYED MUHAMMAD FAHAD (FR)
CAPDEVIELLE VERONIQUE (FR)
FEKI AFEF (FR)
KALYANASUNDARAM SURESH (IN)
MALANCHINI ILARIA (DE)
Application Number:
PCT/EP2021/064010
Publication Date:
January 13, 2022
Filing Date:
May 26, 2021
Export Citation:
Assignee:
NOKIA TECHNOLOGIES OY (FI)
International Classes:
G06N3/04; G06N3/08; H04B7/0456; H04B7/06
Domestic Patent References:
| WO2020055408A1 | 2020-03-19 |
Other References:
"Zero-touch network and Service Management (ZSM); Means of Automation", vol. ISG ZSM Zero touch network and Service Management, no. V0.4.0, 19 February 2020 (2020-02-19), pages 1 - 79, XP014366225, Retrieved from the Internet [retrieved on 20200219]
LIVNE DOR ET AL: "PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning", IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, IEEE, US, vol. 14, no. 4, 16 January 2020 (2020-01-16), pages 789 - 801, XP011805147, ISSN: 1932-4553, [retrieved on 20200810], DOI: 10.1109/JSTSP.2020.2967566
ANDREI A. RUSU ET AL: "Policy Distillation", 7 June 2016 (2016-06-07), XP055497611, Retrieved from the Internet [retrieved on 20180806]
MISMAR FARIS B ET AL: "Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination", IEEE TRANSACTIONS ON COMMUNICATIONS, IEEE SERVICE CENTER, PISCATAWAY, NJ. USA, vol. 68, no. 3, 23 December 2019 (2019-12-23), pages 1581 - 1592, XP011779494, ISSN: 0090-6778, [retrieved on 20200316], DOI: 10.1109/TCOMM.2019.2961332
WATANABE SHINJI ET AL: "Student-teacher network learning with enhanced features", 2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), IEEE, 5 March 2017 (2017-03-05), pages 5275 - 5279, XP033259417, DOI: 10.1109/ICASSP.2017.7953163
YIM JUNHO ET AL: "A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning", 2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), IEEE COMPUTER SOCIETY, US, 21 July 2017 (2017-07-21), pages 7130 - 7138, XP033250080, ISSN: 1063-6919, [retrieved on 20171106], DOI: 10.1109/CVPR.2017.754
LIVNE DOR ET AL: "PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning", IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, IEEE, US, vol. 14, no. 4, 16 January 2020 (2020-01-16), pages 789 - 801, XP011805147, ISSN: 1932-4553, [retrieved on 20200810], DOI: 10.1109/JSTSP.2020.2967566
ANDREI A. RUSU ET AL: "Policy Distillation", 7 June 2016 (2016-06-07), XP055497611, Retrieved from the Internet
MISMAR FARIS B ET AL: "Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination", IEEE TRANSACTIONS ON COMMUNICATIONS, IEEE SERVICE CENTER, PISCATAWAY, NJ. USA, vol. 68, no. 3, 23 December 2019 (2019-12-23), pages 1581 - 1592, XP011779494, ISSN: 0090-6778, [retrieved on 20200316], DOI: 10.1109/TCOMM.2019.2961332
WATANABE SHINJI ET AL: "Student-teacher network learning with enhanced features", 2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), IEEE, 5 March 2017 (2017-03-05), pages 5275 - 5279, XP033259417, DOI: 10.1109/ICASSP.2017.7953163
YIM JUNHO ET AL: "A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning", 2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), IEEE COMPUTER SOCIETY, US, 21 July 2017 (2017-07-21), pages 7130 - 7138, XP033250080, ISSN: 1063-6919, [retrieved on 20171106], DOI: 10.1109/CVPR.2017.754
Attorney, Agent or Firm:
NOKIA EPO REPRESENTATIVES (FI)
Download PDF:
Previous Patent: FRONT STRUCTURE FOR A UTILITY VEHICLE, PREFERABLY FOR A LORRY
Next Patent: UTILITY VEHICLE HAVING A PARKING BRAKE
Next Patent: UTILITY VEHICLE HAVING A PARKING BRAKE