Abstract
Non-orthogonal multiple access (NOMA) and massive multiple-input multiple-output (MIMO) systems are highly efficient. Massive MIMO systems are inherently resistant to passive attackers (eavesdroppers), thanks to transmissions directed to the users. However, active attackers can transmit a combination of legitimate user pilot signals during the channel estimation phase. This way, they can mislead the base station (BS) to rotate the transmission in their direction and allow them to eavesdrop during the downlink data transmission phase. In this paper, we analyze this vulnerability with two-user pairing strategies and investigate how physical layer security can mitigate such attacks and ensure secure (confidential) communication. We derive the secrecy outage probability (SOP) and a lower bound on the ergodic secrecy capacity using stochastic geometry tools when the number of antennas in the BSs tends to infinity (i.e., massive MIMO BS). The numerical and simulation results show that one of the strategies performs better and has a higher ergodic secrecy rate (ESR) and lower SOP. Moreover, we show that using NOMA instead of orthogonal multiple access (OMA) improves system performance significantly.