Game-based user association and resource allocation in B5G heterogeneous networks: A prioritized experience replay-aided distributed DRL solution

Author(s)
Author(s)
Chen, Y.
Liu, Z.
Zhang, X.
Lai, P.
Xie, Y.
Date Issued
2025
Publisher
IEEE
Journal
IEEE Transactions on Green Communications and Networking
Volume
10
Start page
610
End page
622
Abstract
In large-scale beyond 5G (B5G) heterogeneous networks (HetNets), the diversity and density of base stations, coupled with limited spectrum, intensify resource competition among user equipments (UEs). Efficient user association and channel allocation are thus vital for guaranteeing high-quality services in such networks. Given the inherent uncertainty in network state transitions and the selfish behavior of UEs, we model the user association and channel selection problem as a non-cooperative stochastic game. In this game, the system state evolves randomly over time, influenced by the previous state and the actions taken by all UEs, while each user equipment makes independent decisions to maximize its own utility without considering the utilities of other UEs. Since this game problem involves a large number of inter-user interference terms and exhibits non-convexity, its solution becomes particularly challenging, especially under imperfect channel state information (CSI) conditions. Deep reinforcement learning (DRL) provides a feasible solution. However, existing DRL-based approaches typically adopt a uniform random sampling strategy, failing to fully utilize the key experiences that are very beneficial for improving the agent’s policy. To address this issue, this paper proposes a distributed algorithm based on multi-agent double deep Q-network with prioritized experience replay (MADDQN-PER) to solve the constructed game problem. The proposed algorithm operates without requiring globally precise CSI and prioritizes learning from critical experiences with large temporal-difference errors, thereby enhancing the decision quality of user association and channel selection more effectively. The simulation results show that the proposed MADDQN-PER algorithm exhibits excellent scalability in large-scale B5G HetNets and significantly outperforms existing schemes in terms of sum rate performance.
URI
DOI
SFU Affiliated Publication
Yes
Availability at SFU Library

No database links found.