Abstract
Filter bank based multicarrier (FBMC) systems are one of the promising waveform candidates to satisfy the requirements of future wireless networks. FBMC employs prototype filters with lower side lobe and faster spectral decay, which enables it to have the advantages of reduced out-of-band energy and theoretically higher spectral efficiency (SE) compared to conventional multicarrier scheme i.e., orthogonal frequency division multiplexing (OFDM). These systems also have the ability to facilitate aggregation of non-adjacent bands to acquire higher bandwidths for data transmission. They also support asynchronous transmissions to reduce signaling overhead to meet the ever increasing demand of high data rate transmission in future wireless networks. In this work, the primary research objective is to address some of the critical challenges in FBMC systems to make it viable for practical applications. To this end, the following contributions are provided in this thesis. First of all, despite numerous advantages, FBMC systems suffers from long filter tails which may reduce the SE of the system. Filter output truncation (FOT) can reduce this overhead by discarding the filter tails but may also destroy the orthogonality in FBMC system. As a result, the signal to interference ratio (SIR) can be significantly degraded. To address this problem, we first presented a theoretical analysis on the effect of FOT in a multiple input multiple output (MIMO) FBMC system, when assuming that transmitter and receiver have the same number of antennas. We derive the matrix model of MIMO-FBMC system which is subsequently used to analyze the impact of finite filter length and FOT on the system performance. The analysis reveals that FOT can avoid the overhead in time domain but also introduces extra interference in the received symbols. To combat the interference terms, we then propose a compensation algorithm that considers odd and even overlapping factors as two separate cases, where the signals are interfered by the truncation in different ways. A general form of the compensation algorithm is then proposed to compensate all the symbols in a MIMO-FBMC block to improve the SIR values of each symbol for better detection at the receiver. Secondly, transmission of quadrature modulated symbols using FBMC systems has been an issue due to the self-interference between the transmitted symbols both in the time and frequency domain (so-called intrinsic interference). To address this issue, we propose a novel low complexity interference-free FBMC system with QAM modulation (FBMC/QAM) using filter deconvolution. The proposed method is based on inversion of the prototype filters which completely removes the intrinsic interference at the receiver and allows the use of quadrature modulated signaling. The interference terms in FBMC/QAM with and without the proposed system are analyzed and compared in terms of mean square error (MSE). It is shown with theoretical and simulation results that the proposed method cancels the intrinsic interference and improves the output signal to interference plus noise ratio (SINR) at the expense of slight enhancement of residual interferences caused by multipath channel. The complexity of the proposed system is also analyzed along with performance evaluation in an asynchronous multi-service scenario. It is shown that the proposed FBMC/QAM system with filter deconvolution outperforms the conventional OFDM system. Finally, subcarrier index modulation (SIM) a.k.a., index modulation (IM) has recently emerged as a promising concept for spectrum and energy-efficient next generation wireless communications systems due to the excellent trade-offs they offer among error performance, complexity, and SE. Although IM is well studied for OFDM, FBMC with index modulation (FBMC-IM) has not been thoroughly investigated. To address this topic, we shed light on the potential and implementation of IM technique for FBMC system. We first derived a mathematical matrix model of FBMC-IM system (FBMC/QAM-IM) along with the derivation of interference terms at the receiver due to channel distortions and the intrinsic behavior of the transceiver model. We have analytically shown that the interference power in FBMC/QAM-IM is smaller compared to that of conventional FBMC/QAM system as some subcarriers are inactive in FBMC/QAM-IM system. We then evaluated the performance of FBMC/QAM-IM in term of MSE, SIR and output SINR. The results show that combining IM with FBMC/QAM can improve the system performance since the inactive subcarriers are not contributing to the overall interference in the system. Based on the interference analysis, we then proposed an improved log-likelihood ratio (LLR) detection scheme for FBMC/QAM-IM system. At the end, BER performance of FBMC/QAM system with and without IM is presented and it can be seen that since the power from inactive subcarriers is reallocated to the active subcarriers in FBMC/QAM-IM, the system shows improved performance compared to conventional FBMC/QAM system.