Abstract
The sixth-generation (6G) has defined ultra-massive machine type communication (umMTC) as a key scenario to address advances in Internet of Things technologies. However, the dramatic increase of machine-type communication devices (MTDs)in umMTC calls for the research of efficient random access (RA) schemes. To cope with the problem of high collision rates in dense networks, a multi-dimensional RA scheme is proposed in this paper, which utilizes both power and code domain non-orthogonal multiple access (NOMA) to significantly expand the contention space. In particular, we present a novel dual decoupled Q-Learning algorithm to enable each MTD to find its unique multi-dimensional resources for transmission. Unlike the traditional Q-Learning algorithm, the proposed approach utilizes two independent Q-Learning mechanisms to reduce the scale of action sets, and make use of a novel reward function to solve the coupling relationship among multiple action sets. Simulation results indicate that the proposed scheme can notably enhance the throughput performance while effectively reducing the convergence time by compared to the existing ones.