Abstract
This paper presents a detailed framework for detecting anomalies and tracking service usage within the O-RAN architecture, focusing on preventing DDoS attacks caused by unauthorized or compromised User Equipment (UE). The system consists of three main parts: the dApp, xApp-U, and xApp-S. Each part plays a role in identifying suspicious activities and monitoring UE service usage across both the RAN and near-Real-Time (RT) RIC, ensuring effective threat detection and response. We implemented and evaluated various Machine Learning (ML) algorithms, comparing them based on important metrics such as accuracy, precision, recall, False Positive Rate (FPR), and training and testing time. Our analysis shows that different ML algorithms perform better depending on the system's needs, and choosing the right one requires balancing accuracy, low delay, and fewer false positives. The dApp operates in the RAN, where decisions must be made quickly with minimal delay, while xApp-U, working in the near-RT RIC, benefits from having more data and achieves better accuracy in detecting anomalies. Finally, xApp-S focuses on tracking service usage to identify patterns that contribute to suspicious behavior. This multi-layered approach allows for flexible and accurate security measures that suit the specific needs of each part of the system.