The rapid advancement of wireless communication technologies has significantly increased energy consumption in cellular networks. To address this issue, we propose a multi-level sleep mode strategy for multi base station (BS) collaboration assisted by cell zooming and user association. First, the sleep mode allows BSs with low-traffic load to enter different sleep states to achieve energy savings, while cell zooming dynamically adjusts BS coverage, enabling collaboration through joint user association. However, since sleep mode leads to increased delay, we propose an energy-saving method with a delay constraint. In addition, optimizing the energy consumption of multiple BSs under delay constraints is a complex non-convex problem. To solve this problem, we first model the optimization process as a Markov decision process (MDP) and then optimize it using the proximal policy optimization (PPO) algorithm. Furthermore, although traffic load fluctuates dynamically, it still follows certain predictable patterns. Therefore, we propose a bidirectional long short-term memory (Bi-LSTM) traffic prediction algorithm to assist the sleep mode with cell zooming in making optimal or suboptimal decisions during the energy consumption optimization process. The simulation results demonstrate the superiority of the proposed scheme over benchmarks in terms of energy savings. Index Terms—Base station multi-level sleep strategy, cell zooming , deep reinforcement learning, bidirectional long short-term memory.