$$F\_{11} = \left[ R\_{\mathbf{XX}}^{(n\text{H})}(j, 1) + R\_{\mathbf{XX}}^{(n\text{V})}(j, \mathbf{7}) \right] = \begin{cases} 0, & \text{for } j = 1 \text{ and } n = \text{V} \\ 2, & \text{for } j = 1 \text{ and } n = \text{H} \\ 1, & \text{for } j = 2, 3 \end{cases} \tag{5}$$

and

$$F\_{21} = \left[ R\_{\mathbf{XX}}^{(nV)}(j, 1) + R\_{\mathbf{XX}}^{(n\mathbf{H})}(j, \mathbf{\tilde{1}}) \right] = \begin{cases} 2, & \text{for } j = 1 \text{ and } n = \mathbf{V} \\ 0, & \text{for } j = 1 \text{ and } n = \mathbf{H} \\ 1, & \text{for } j = 2, 3 \end{cases} \tag{6}$$

where *R*(*n*V) *XX j*, 1 and *R*(*n*H) *XX* (*j*, 1) are the optical signals at the upper and lower input ports of the upper coupler, respectively. The expressions *R*(*n*V) *XX* (*j*, 1) and *R*(*n*H) *XX j*, 1 are the optical signals at the upper and lower input ports of the lower coupler, respectively. These optical signals arrive at the input ports of the BPD for correlation subtraction and MAI removal according to the results of Equation (4) in Section 2 and are then converted into electrical signals. Therefore, the output of the BPD can be converted into model (*F*), as follows, when the codeword of X*j* is received:

$$F = F\_{21} - F\_{11} = \begin{cases} 2, \text{for } j = 1 \text{ and } n = \text{V} \\ -2, \text{for } j = 1 \text{ and } n = \text{H} \\ 0, \text{otherwise} \end{cases} \tag{7}$$

Finally, a decision current (*I*) is used to determine the data bit of the desired user from the wireless optical channel.

#### **3. Experimental Setup and Results**

On the basis of the structure illustrated in Figure 3 with *N* = 3, the feasibility of the proposed FSO communication system was verified through several experiments by using a model with the following specifications: (1) for the light source, the NXTAR SLD-2000 was adopted. (2) Couplers (Fiber Optic Communications, Inc., Hsinchu, Taiwan) were used as 1 × 2 splitters and 2 × 1 couplers. (3) A Pirelli 10-Gbps integrated optic intensity modulator—which used two EOMs—was used to modulate the signal of the pattern output. (4) An Agilent 81130A pulse pattern generator was used to generate desired patterns for transmission. (5) A left-handed plastic circular polarizer (CP42HE) 12.5 mm in diameter—which used two polarizers (0◦ and 90◦)—was used to assign the light signal to the specific polarization state. (6) A 25-mm nonpolarizing cube BS with a wavelength range of 1100–1620 nm was used to combine optical signals from different paths. (7) A 5-mm VIS polarizing cube BS was used to split two polarization states (0◦ and 90◦) from the input optical signal. (8) A single-mode circulator (1550 nm and 500 mW; FCIR-1550-3-3-A-0-1-2-1-2) was used as the optical circulator. (9) The attenuator range was adjusted from 0 to −30 dB to alleviate the influences of noise and the lower branch signal in the front of PD2. (10) The BPD (Model-1817, New Focus Inc., USA) was used for optical signal subtraction and the conversion of the results into electrical signals. (11) A Tektronix oscilloscope (OSC, model TDS2102B) was used to monitor BPD outputs. (12) A limited power supply of 15 V was provided to the BPD. (13) An Anritsu MS9710C optical spectrum analyzer was employed to assess the accuracy of the spectral output acquired from a codec.

Assume that the first user is the desired user. First, the FBG resonance wavelengths used for the FBG codec are 1543, 1546, and 1549 nm for λ1, λ2, and λ3, respectively. Therefore, user 1 is assigned the codeword *X*1 = [λ1 0 λ3].

Figure 4 presents the measured reflected spectra (λ1, λ3) for user 1 with corresponding central wavelengths of 1543 and 1549 nm and light intensities of −22.96 and −23.02 dBm, which appeared at the circulator port in Encoder 1. After the reflected spectra passed through the EDFA and 1 × 2 splitter, they were modulated by EOMs 1 and 2 according to the normal (D) and complementary (D) outputs of the pattern generator, and the spectra were then entered as inputs that were parallel to the input ports of the two polarizers (90◦ and 0◦) to determine suitable polarization states.

**Figure 3.** Proposed free-space optical communication system using bipolar optical code division multiple access and dual electro-optical modulator schemes.

**Figure 4.** Reflected spectra (λ1, λ3) used for user 1.

Figure 5 presents the spectra obtained at the output ports of the two EOMs operating with different data bits. Figure 5A presents the output spectrum obtained at the output port of EOM 1 when the data bit (D) of user 1 is "1". The central wavelengths of λ1 and λ3 are 1543 and 1549 nm, respectively, and the corresponding light intensities are −8.82 and −8.77 dBm, respectively. Figure 5B presents the output spectrum obtained at the output port of EOM 2 when the data bit (D) of user 1 is "0". The central wavelengths of λ1 and λ3 are 1543 and 1549 nm, respectively, and the corresponding light intensities are −10.39 and −10.55 dBm, respectively. Figure 6 indicates that a signal frequency of 500 Hz was acquired from pattern generation and entered as an input into EOMs 1 and 2 of Encoder 1, where the input signals of the two EOMs complemented each other. The output signals of the EOM 1 and EOM 2 were assigned polarization states of 0◦ and 90◦, respectively, and they were then combined into a free-space channel passing through the BS and collimator.

**Figure 5.** Output spectrum acquired at the output ports of the two electro-optical modulators (EOMs) for different data bits. (**A**) Optical spectra of EOM 1 for a data bit of "1". (**B**) Optical spectra of EOM 2 for a data bit of "0".

**Figure 6.** Input signals of the two electro-optical modulators (EOMs) when the signals of Channels 1 and 2 were entered as inputs into EOM 1 and EOM 2, respectively.

Figure 7 presents the depolarized spectra obtained at the output ports of the PBS in Decoder 1 when the encoded signal with different data bits was received from Encoder 1. Figure 7A,B presents the depolarized spectra acquired at the output ports of the PBS in Decoder 1 when a data bit (D) of "1" is transmitted. In the depolarized spectra (λ1, λ3) with the 90◦polarization state at the first output port of the PBS, the corresponding central wavelengths were 1543 and 1549 nm, and the light intensities were −25.31 and −24.68 dBm, respectively (Figure 7A). Figure 7B indicates that no signal appeared at the second output port of the PBS when the data bit (D) of user 1 was "1". Figure 7C,D presents the depolarized spectra acquired at the output ports of the PBS in Decoder 1 when a data bit (**D**) of "0" is transmitted. Figure 7C indicates that no signal appeared at the first output port of the PBS when the data bit (D) of user 1 was "0". In the depolarized spectra (λ1, λ3) with the 0◦ polarization state at the second output port of the PBS, the corresponding central wavelengths were 1543 and 1549 nm, and the light intensities were −24.06 and −23.9 dBm, respectively (Figure 7D). Therefore, in the experiment, the degree of light intensity demonstrated an approximate 13.35–16.5-dB loss from the EOM output in Encoder 1 to the PBS output in Decoder 1 through the wireless optical channel.

**Figure 7.** Optical spectrum acquired at the output ports of the polarization beam splitter (PBS) in Decoder 1 when different data bits (D) are sent from Encoder 1. Depolarized spectra acquired at (**A**) Output Port 1 and (**B**) Output Port 2 of the PBS when a data bit of "1" is sent. Depolarized spectra acquired at (**C**) Output Port 1 and (**D**) Output Port 2 of the PBS when a data bit of "0" is sent.

Figure 8 presents the measured spectra before the optical signals entered the upper and lower couplers when a data bit (D) of "1" is sent from Encoder 1. Figure 8A presents the spectra measured using the upper optical circulator and FBG Decoder 1 before input was entered in Port 1 of the upper coupler when the a data bit (D) of user 1 was "1". The corresponding central wavelengths were 1543 and 1549 nm, and the light intensities were −34.29 and −35.64 dBm, respectively. Figure 8B indicates that no signal appeared at Port 2 of the upper coupler for user 1's data bit (D) of "1" when the spectra passed through the horizontal (0◦ component) path; the corresponding central wavelengths were 1543, 1546, and 1549 nm, and all light intensities were less than −53 dBm. Figure 8C presents the decoded spectra that passed through the upper optical circulator and FBG Decoder 1 before it was entered as an input into Port 1 of the lower coupler when the data bit (D) of user 1 was "1". The corresponding central wavelengths were 1543 and 1549 nm, and the light intensities were −28.24 and −27.57 dBm, respectively. Figure 8D indicates that no signal appeared at Port 2 of the lower coupler for user 1's data bit (D) of "1" when the spectra passed through the horizontal (0◦ component) path; the corresponding central wavelengths were 1543, 1546, and 1549 nm, and all light intensities were less than −53 dBm. Some depolarized spectral outputs were generated because of the imperfect reflection obtained from FBG Decoder 1 (Figure 8A).

**Figure 8.** Optical spectrum before the optical signals entered upper and lower couplers when a data bit (D) of "1" was sent from Encoder 1. (**A**) Optical spectra obtained at Port 1 of the upper coupler when the optical signals passed through fiber Bragg grating (FBG) Decoder 1. (**B**) Optical spectra obtained at Port 2 of the upper coupler when the optical signals passed through the 0◦ component path. (**C**) Optical spectra obtained at Port 1 of the lower coupler when the optical signals passed through FBG Decoder 1. (**D**) Optical spectra obtained at Port 2 of the lower coupler when the optical signals passed through the 0◦ component path.

Figure 9 presents the spectra measured before the optical signals entered the upper and lower couplers when a data bit (D) of "0" was sent from Encoder 1. Figure 9A indicates that no signal appeared at Port 1 of the upper coupler for user 1's data bit (D) of "0" when the spectra passed through the vertical (90◦ component) path; the corresponding central wavelengths were 1543, 1546, and 1549 nm, and all light intensities were less than −53 dBm. Figure 9B presents the spectra decoded using the lower optical circulator and FBG Complement Decoder 1 before they were input into Port 2 of the upper coupler for user 1's data bit (D) of "0". The corresponding central wavelengths were 1543 and 1549 nm, and the light intensities were −28.97 and −28.56 dBm, respectively. Figure 9C indicates that no signal appeared at Port 1 of the lower coupler for user 1's data bit (D) of "0" when the spectra passed through the vertical (90◦ component) path; the corresponding central wavelengths were 1543, 1546, and 1549 nm, and all light intensities were less than −53 dBm. Figure 9D presents the optical spectra obtained through the lower optical circulator and FBG Complement Decoder 1 before they were input into Port 2 of the lower coupler for user 1's data bit (D) of "0". The corresponding central wavelengths were 1543, 1546, and 1549 nm, and all light intensities were less than −47 dBm. Some unwanted spectral outputs were produced because of the imperfect upper coupler connection and lower circulator (Figure 9D). However, the decoded output was unaffected by leakage intensities (Figure 9D).

**Figure 9.** Output spectrum before optical signals entered the upper and lower couplers when a data bit (D) of "0" was sent from Encoder 1. (**A**) Optical spectra acquired at Port 1 of the upper coupler when the signal passed through the 90◦ component path. (**B**) Optical spectra acquired at Port 2 of the upper coupler when the signal passed through fiber Bragg grating (FBG) Complement Decoder 1. (**C**) Optical spectra acquired at Port 1 of the lower coupler when the signal passed through the 90◦ component path. (**D**) Optical spectra acquired at Port 2 of the lower coupler when the signal passed through FBG Complement Decoder 1.

Figure 10 presents the output spectrum obtained at the output ports of the upper and lower couplers in Decoder 1 when the different data bits (D) of user 1 were sent. Figure 10A,B presents the optical spectra appearing at the output ports of the upper and lower couplers in Decoder 1, respectively, and Figure 10C,D presents those appearing for data bits (D) of "1" and "0" entered as inputs into Encoder 1. Subsequently, the BPD converted the decoded spectra corresponding to its input ports into electrical signals.

Figure 11 presents the decoding results of changing frequencies when data were transmitted from Encoder 1. A digital OSC was used to access the transmitted signal from Encoder 1. In Figure 11A–D, input frequencies of 0.5, 50, 5000, and 10,000 kHz were used as inputs for FBG Encoder 1. Compared with previous systems [21], the novel FSO communication system with the proposed Bi-OCDMA scheme was implemented successfully and further enhanced the overall transmission rate.

**Figure 10.** Decoded spectra appearing at the output ports of the upper and lower couplers when different data bits (D) of user 1 were sent. Optical spectra obtained at the output port of the (**A**) upper coupler and (**B**) lower coupler for a data bit (D) of "1" for user 1. Optical spectra obtained at the output port of the (**C**) upper coupler and (**D**) lower coupler for a data bit (D) of "0" for user 1.

**Figure 11.** Decoding results for Decoder 1 acquired by the balanced photodetector at signal frequencies of (**A**) 500 Hz, (**B**) 50 kHz, (**C**) 5 MHz, and (**D**) 10 MHz input to Encoder 1.

#### **4. Conclusions**

In this study, the use of Bi-OCDMA with a dual EOM scheme implemented in WOC environments was proposed and successfully demonstrated at normal atmospheric temperatures. FBGs were employed as primary devices for developing the codec. The measurement results of the signal transmission rate revealed that switching limitations of previous systems using an optical switch [21] can be improved through use of the proposed design with a dual EOM structure.

The proposed Bi-OCDMA method is based on original SAC OCDMA techniques, which theoretically alleviate the MAI effect and reduce crosstalk from other FBG encoders. When deployed, the proposed FSO system exhibited excellent properties in terms of its light weight, cost effectiveness, moderate security, and EMI resistance. These properties may further enhance the overall transmission rates of WOC applications in the near future.

Future work can apply the proposed Bi-OCDMA technique to multiuser and long-distance WOC scenarios that involve MAI mitigation and performance measurement by using parameters such as the bit error rate, Q-factor, and eye diagrams.

**Author Contributions:** Conceptualization, S.-P.T. and H.-C.C.; methodology, S.-P.T. and H.-C.C.; validation, S.-P.T. and H.-C.C.; resources, H.-C.C.; data curation, P.-H.L.; writing—original draft preparation, E.W.; writing—review and editing, S.-P.T. and H.-C.C. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the Ministry of Science and Technology, Taiwan, under Grant MOST 108-2221-E-150-041 and Grant MOST 108-2221-E-239-003.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Letter* **Compressed Nonlinear Equalizers for 112-Gbps Optical Interconnects: Efficiency and Stability**

#### **Wenjia Zhang \*, Ling Ge, Yanci Zhang, Chenyu Liang and Zuyuan He**

State Key Laboratory of Advanced Optical Communication Systems and Networks, Shanghai Jiao Tong University, Shanghai 200240, China<EMAIL_ADDRESS>(L.G<EMAIL_ADDRESS>(Y.Z<EMAIL_ADDRESS>(C.L<EMAIL_ADDRESS>(Z.H.) **\*** Correspondence<EMAIL_ADDRESS>Tel.: +86-21-34204316

Received: 12 May 2020; Accepted: 17 August 2020; Published: 19 August 2020

**Abstract:** Low-complexity nonlinear equalization is critical for reliable high-speed short-reach optical interconnects. In this paper, we compare the complexity, efficiency and stability performance of pruned Volterra series-based equalization (VE) and neural network-based equalization (NNE) for 112 Gbps vertical cavity surface emitting laser (VCSEL) enabled optical interconnects. The design space of nonlinear equalizers and their pruning algorithms are carefully investigated to reveal fundamental reasons of powerful nonlinear compensation capability and restriction factors of efficiency and stability. The experimental results show that NNE has more than one order of magnitude bit error rate (BER) advantage over VE at the same computation complexity and pruned NNE has around 50% lower computation complexity compared to VE at the same BER level. Moreover, VE shows serious performance instability due to its intricate structure when communication channel conditions become tough. Moreover, pruned VE presents more consistent equalization performance within varying bias values than NNE.

**Keywords:** VCSEL; neural network-based equalization; Volterra series-based equalization

#### **1. Introduction**

Recent decades have witnessed the explosion of data traffic especially in regional spaces like data centers and supercomputers. The hardware infrastructure for supporting such massive connectivity has been turning to optical components and fibers even over a distance less than 100 m. The data rate of optical interconnects have exceeded 100 Gbps per lambda in commercialized products and will soon upgrade to 200 Gbps per lambda for 800 Gbps or 1 Tbps optical module [1]. Vertical cavity surface emitting laser (VCSEL)-based optical interconnect is a typical and competitive candidate because of interesting features of low cost and power consumption [2]. To meet capacity, cost and power consumption requirements, intensity modulation and direct detection (IM-DD) with more advanced modulations like four- or eight-level pulse amplitude modulation (PAM4/8) [3], carrier-less amplitude phase modulation (CAP) [4] and discrete multi-tone modulation (DMT) [5] has been proposed for 100/200 Gbps per lambda optical interconnects, in which PAM4 has become a widely accepted modulation format for 400G products. It is of necessity rather than an option to employ advanced modulations for improving spectral efficiency due to severe bandwidth limitation and noise accumulation of devices and channels. However, advanced modulations, though mitigating the system requirement on frequency response of critical devices, will also bring serious modulation non-linearity [6,7], reduced signal–noise ratio (SNR), and level-dependent noise accumulation [8]. In particular, VCSEL-based interconnect solution confronts large challenges to realize reliable 100/200 Gbps per lambda communications with advanced modulations mostly because of bandwidth limitation [9] and complex impairment combination of relative intensity noise (RIN) [10], mode

partition noise (MPN) [11] and fiber mode dispersion. The complicated signal distortions in time, amplitude and frequency domain need advanced nonlinear equalization for compensation and adaption [3,6,8,11–14].

Advanced digital signal processing (DSP) technologies-based nonlinear equalization, such as Volterra series-based equalization (VE) [3,15–18] and neural network-based equalization (NNE) [12,19–22], have been proposed to realize powerful nonlinear compensation capability in the VCSEL enabled optical interconnect. However, high computation complexity of these nonlinear equalization hinders their practical implementation in the cost and power-sensitive module. Fortunately, it has been proved that there are various design spaces for nonlinear equalization to reduce computation complexity without sacrificing communication quality [3,12]. Therefore, only by marrying strong algorithms with limited computing and storage resources of underlying hardware, nonlinear equalization can find their values in the next wave of hardware evolution for a high-end optical module. There are various low-complexity designs of VE and NNE in the literature. In [23], coefficients in Volterra series that are not on the diagonal have been intentionally abandoned to construct a memory polynomial Volterra equalizer. However, this approach is not able to adapt to various transmission conditions since that simplification process runs before training. Moreover, in [12,17,24], the authors employ an approach of weight-pruning and retraining to reach a sparse but optimized equalizer structure for VE or NNE. Additionally, more aggressive methods [3,16,18] by introducing -0 and -1-regularization penalty term into original cost function for pruning Volterra nonlinear equalizer are proposed to force large amount of coefficients to zeros, demonstrating a over 93% complexity reduction in the 112 Gbps VCSEL-based optical interconnects. However, equalization architectures for Volterra series and neural network are fundamentally different in the nonlinear construction. In addition, there is tremendous difference in the growth pattern of computation complexity with construction parameters. So, pruning strategies should be carefully designed for two equalization architectures for exploiting best potentials. Questions are raised on which nonlinear equalization prevails over others of realizing lowest complexity equalization while keeping the reliable transmission performance [25]. More importantly, it is still unclear the window of key parameter variation for a working channel that equalization performance gain can keep consistent.

In this paper, we compare the efficiency and stability of a pruned three-layer neural network-based equalizer and a pruned three-order Volterra series-based equalizer for VCSEL enabled 112 Gbps optical interconnects. A threshold-based pruning and retraining algorithm, which is proposed in our previous publications [3,12], is used to compare the pruning efficiency of nonlinear equalizers. In this comparison study, we extend the conference paper [25] by elaborating design space of nonlinear equalizers, including engineering parameters on memory length, neural network layer and activation function, in order to reveal fundamental reasons of powerful nonlinear compensation capability and restriction factors of efficiency and stability. In addition, signaling performance under different bias voltages for a VCSEL is evaluated by using pruned VE and pruned NNE, respectively.

#### **2. Principle of Pruned Nonlinear Equalizers**

#### *2.1. Volterra Series and Neural Network-Based Equalizer*

The mathematical expression of *P*-order VE with memory length of *Mr* can be expressed as Equation (1),

$$\mathcal{Y}(k) = \mathcal{W}\_{dc} + \sum\_{r=1}^{P} \sum\_{k\_1=0}^{M\_r - 1} \cdots \sum\_{k\_r=k\_{r-1}}^{M\_r - 1} \mathcal{W}\_r(k\_1, k\_2, \dots, k\_r) \cdot \mathbf{x}(k - k\_1) \cdots \mathbf{x}(k - k\_r). \tag{1}$$

where *x*(*k*) is *kth* sampled data from received signals and *y*(*k*) is output data through equalization. *Wr* is the *rth*-order Volterra kernel. *Wdc* is responsible for *DC* component, which is not included in the final model of an AC-coupled circuit. This model presented in Equation (1) can approximate any

nonlinear system in theory. However, it is physically impossible in many cases due to high computation complexity. As low-order part of Volterra series model indicates most features of nonlinear system, approximation approaches are often used to truncate theoretically infinitely long expressions.

It can be seen from Equation (1), the number of coefficients, indicating computation burden of multiplication and addition, will grow very fast with increase of *P* and memory length *Mr*. It has been known that multiplication contributes most of computation resources compared to addition and will be main reason of complexity effect of pruning algorithms [18]. Therefore, computation complexity for a *P*-order VE with memory length of *Mr* can be defined as the number of multiplication operations [18] and expressed as Equation (2),

$$\text{Complexity} = M\_1 + M\_2(M\_2 + 1) + M\_3(M\_3 + 1)(M\_3 + 2)/2 + \dotsb \tag{2}$$

According to [3,26], a three-order structure is sufficient for short-reach optical interconnects. By expanding Equation (1), a three-order VE, named by *VE*(*M*1, *M*2, *M*3) where *M*1, *M*2, *M*3 are memory lengths of first, second and third order of VE, is described by

$$\begin{aligned} y(k) &= \sum\_{k\_1=0}^{M\_1-1} w\_1(k\_1) \mathbf{x}(k - k\_1) \\ &+ \sum\_{k\_1=0}^{M\_2-1} \sum\_{k\_2=k\_1}^{M\_2-1} w\_2(k\_1, k\_2) \mathbf{x}(k - k\_1) \mathbf{x}(k - k\_2) \\ &+ \sum\_{k\_1=0}^{M\_3-1} \sum\_{k\_2=k\_1}^{M\_3-1} \sum\_{k\_3=k\_2}^{M\_3-1} w\_3(k\_1, k\_2, k\_3) \mathbf{x}(k - k\_1) \mathbf{x}(k - k\_2) \mathbf{x}(k - k\_3) .\end{aligned} \tag{3}$$

A basic *VE*(2, 2, 2) structure with 9 coefficients is shown in Figure 1a. In this work, a three-order VE (*P* = 3) is applied for further experimental investigation.

**Figure 1.** The structure of (**a**) three-order *VE*(2, 2, 2) and (**b**) three-layer *NNE*.

Artificial neural networks have been widely deployed in various areas such as image classification and natural language processing, showing much better performance than traditional algorithms. Recently, it has been intensively researched in the field of optical communication, in which nonlinear equalization is one of hottest topic [13]. To equalize PAM-N communication signals, the received analog signals are taken to a neural network with N possible outputs, representing N levels. The signal stream is first delayed through a delay array, and then input into an input layer with designed neuron number. The numbers of input neuron and layer number are also of great significance to balance the equalization performance and computation complexity. According to universal approximation theorem, a three-layer neural network will provide strong nonlinear equalization capability for the communication systems.

The mathematical formula of NNE process can be expressed as Equation (4),

$$y = \arg\max \left[ \text{softmax} \left( f \left( \mathbf{x}^T (k) \times \mathcal{W}\_{\text{i}\text{h}} + B\_{\text{h}} \right) \times \mathcal{W}\_{\text{ho}} + B\_{\text{o}} \right) \right] \tag{4}$$

where *x(k)* is a sampled signal sequence, *Wih* and *Who* are weight matrices of input layer to hidden layer and hidden layer to output layer respectively. *Bh* and *Bo* are bias vectors of hidden layer and output layer. *f* means activation function of hidden layer. Rectified linear unit (ReLU), tangent hyperbolic (Tanh) and sigmoid functions are considered in this work. Function *so f tmax*(·) is used to convert the results of output layer to probability distribution for each class. Finally, *argmax*[·] is decision function, which returns an index of the maximum value of output probability vector. *y* is final result indicating a specific symbol of information. Loss functions of ANN, including mean square error (MSE) loss function, logarithmic loss function and cross-entropy loss function, are important for training model and final performance. The MSE loss function usually shows slow convergence and is seldom used in classification task. In this work, back propagation (BP) algorithm combined with cross-entropy loss function are used to train NNE to achieve better classification performance.

The structure of the 3-layer NNE is shown in Figure 1b, where *NNE*(*N*1, *N*2, *N*3) represents a 3-layer NNE with *N*1, *N*2, *N*3 as neuron number of input, hidden and output layer. Similar to VE, computation complexity for NNE is defined as number of multiplication operation as calculated in Equation (5),

$$\text{Complexity} = N\_1 \times N\_2 + N\_2 \times N\_3 + N\_3 \times N\_4 + \dotsb \tag{5}$$

#### *2.2. Pruning Algorithm*

It has been proved that most of multiplication operations in the nonlinear equalizers are redundant [23] as either weight value is small or connections in network level contribute very little to final results. Therefore, there is possibility of best pruning algorithm that can realize a lightest equalizer without sacrificing transmission performance. The choice of strategy to prune the complex networks as shown in Figure 1 become very important to reduce computation complexity for each architecture while maintaining equalization performance. We proposed a threshold-based pruning and retaining approach for equalizers, where coefficients after initial training are intentionally discarded based on a threshold and damages caused by pruning process are recovered as much as possible through retraining [3,12]. The pruning process can be easily expressed as Equation (6), where *S*(·) represents weight setting and *T* is threshold. With weight value setting to zero, connections relating to this value is cut in these complex networks. Due to different features of two architectures, there also needs special design considerations for each equalizer. For VE, second- and third-order coefficients are pruned because we find linear terms only occupy a small amount of computation complexity while contribute most on the performance gain due to bandwidth limitation of the experimental system. For NNE, all weights both from input layer to hidden layer and hidden layer to output layer are run for pruning based on the threshold. An iterative pruning process is developed to achieve a more efficient network structure.

$$S(W) = 0, \quad \text{when} \quad S(\cdot) < T. \tag{6}$$

#### **3. Experiment and Results**

#### *3.1. Experiment Setup*

Figure 2 shows experimental setup for 112 Gbps optical interconnects. In the transmitter, 112 Gbps pseudo-random binary sequence (PRBS)-11 PAM-4 signals are generated by an arbitrary waveform generator (AWG), which is Keysight M8195A with 64-GSa/s sampling rate and 25-GHz 3dB bandwidth. Please noted that the waveform generator is bandwidth limited for generating 112 Gbps PAM4 signals, but this inherent bandwidth limitation can be mitigated by pre-distortion and equalization. PRBS-11 is used due to memory limitation of our AWG. High-speed data-streaming combined with DC bias are applied on an 850 nm multimode VCSEL bare die through a high-speed probe. Light-current-voltage

(LIV) curve and 3-dB bandwidth of VCSEL can be found in the inset of the Figure 2. Lights from VCSEL are coupled into a 100 m OM3 multimode fiber through a three-dimension coupling station. Through a variable optical attenuator (VOA), lights are detected by a photodetector (PD, New Focus 1484-A-50) integrated with a transimpedance amplifier (TIA). At the receiver, detected signals are sampled and captured by a real-time digital storage oscilloscope (DSO, Keysight DSOZ592A) with 160-GSa/s sampling rate and 59-GHz analog bandwidth for offline digital signal processing. The captured data are re-sampled to one sample per symbol, followed by nonlinear equalization and bit error rate (BER) calculation.

**Figure 2.** Experimental setup.

To avoid the risk of performance over-estimation from nonlinear equalization, we randomly disrupt the signals to create a sequence without periodicity before the received signals are fed into equalizer and conducted mini-batch training [12]. After such process, a disrupted sequence, with an ultra-long unrepeated pattern, is used to train equalizer, which enables equalizer to learn system characteristics instead of generation rules of PRBS.

#### *3.2. Efficiency Comparison*

First, memory length should be carefully tuned for VE and NNE to better understand the efficiency of pruning algorithms. If these parameters are intentionally set with large numbers, complexity reduction ratio will be exaggerated through pruning due to expanded redundancy, which is not fair for comparison. Therefore, we have tuned memory length of first order in VE by setting high-order memory length to zeros. Figure 3a shows that BER keeps stable when memory length of this order is from 51 to 251 for 100 Gbps PAM4 transmission, which is similar for back-to-back (B2B) and 100 m transmission with various received optical powers. Therefore, 51 is set for the first-order memory length. For nonlinear terms selection, second- and third-order memory lengths are increased simultaneously while trying to make their coefficient number in a similar level in order to balance performance gain introduced by second- and third-order term. For NNE, we fix the number of hidden layer neurons as 51 and only change the number of input symbols from 9 to 51. Figure 3b shows BER is reduced with increasing input symbol. The smooth BER change, without large-scale jump between any adjacent two points, also indicates that NNE in our experiment does not obtain over-estimation gain thanks to the use of randomization process, since that over-estimation problem for NNE will lead to stair-like curve of BER versus input symbol number [27]. Moreover, design space for NNE also includes activation function and layer number, which influence equalization performance. As shown in Figure 4, ReLU and Tanh activation function show better BER performance than Sigmoid. Tanh has similar BER with ReLU when optical power is smaller than 2-dBm while ReLU will achieve better performance when received optical power is more than 2-dBm. We can also learn from Figure 4 that BER performance can be slightly improved by extending the layer number of NNE from three to four with the same activation function of ReLU through comparing a three-layer *NNE*(51, 51, 4) with a

four-layer *NNE*(51, 51, 51, 4). However, more BER performance improvement can be obtained by increasing input neuron number to 71 by comparing three-layer *NNE*(51, 51, 4) with *NNE*(71, 71, 4).

**Figure 3.** (**a**) BER versus memory length of first-order VE with received optical power of 1 dBm and 0 dBm (B2B) and 0 dBm (100 m); (**b**) BER versus input symbol number with 51 hidden neurons and three layers and received optical power of 1 dBm.

**Figure 4.** Impact of activation function and layer number of NNE on equalization performance.

Second, 100 Gbps PAM4 signaling at a typical 1-dBm received optical power is used for comparing efficiency of VE and NNE and their pruning version. Figure 5 shows the experimental results at B2B and 100 m MMF cases. We carefully set the initial equalizer configurations of *VE*(51, 31, 15) and *NNE*(51, 56, 4) with the initial complexity 3083 for VE and 3080 for NNE in order to balance the initial complexity of VE and NNE for fair comparison. As we can see from Figure 5, NNE presents more than one order of magnitude BER advantage over VE for both B2B and 100 m MMF cases at the same computation complexity. By using pruning algorithms, lower computation complexity could be achieved for both VE and NNE. At 7% HD FEC limit of 100 m MMF case, complexity values are 1144 and 603 for general VE and NNE and is reduced to 743 and 386 with pruning algorithms, in which pruned NNE shows 48% complexity performance improvement than pruned VE.

Third, in order to better understand behavior of VE and NNE, we increase the data rate to 112 Gbps and eliminate the pre-distortion process for pressuring more on the bandwidth limitation as indicated in the inset of Figure 1. Figure 6 illustrates experimental results with received optical power of 0-dBm. Noted that the computation complexity of equalizers without pruning process is changing with memory lengths of equalization architecture. From the Figure 6, VE shows serious numerical instability due to its intricate structure. With memory length increasing, BER performance powered by VE shows a slow reduce and even starts to increase until misconvergence with BER of 0.5. On the other hand, NNE is still able to achieve remarkable equalization performance for B2B and 100 m

cases. Therefore, NNE has significantly more robust performance than VE in a harsh link situation. With pruning algorithms, NNE attains around 50% complexity reduction compared to conventional NNE at the FEC limit of BER with 3.8 × 10−3. In addition, at the same complexity, equalizer without pruning algorithms shows worse BER performance due to reduced memory length. From Figure 6, we also learn that VE outperforms NNE when complexity is lower than 800 since that VE can always maintain the least linear equalizer. However, the performance of NNE, on the other hand, will decline rapidly after large-scale pruning, as NNE fails to have the property of separate order like VE.

**Figure 5.** The efficiency comparison for (**a**) B2B case and (**b**) 100 m MMF case at 100 Gbps PAM-4 with pre-distortion.

**Figure 6.** The efficiency comparison at 112 Gbps PAM-4 without pre-distortion.

#### *3.3. Stability Comparison*

Performance stability for equalization is of great significance in the real application where output optical power or 3-dB bandwidth of critical devices in the optical interconnect will change within a life-cycle because of component aging or ambient temperature fluctuation. It is likely that performance stability will become worse with pruning algorithms since that the redundancy of connections in the equalization will be useful in changing scenarios. To verify the stability of pruning algorithm with changing link conditions, we choose DC bias of VCSEL as a key parameter for further evaluation because DC bias variation of VCSEL will not only change the output power but also bandwidth of directly modulated laser. The transmission experiment is run with a 100 Gbps PAM4 pre-distorted link. We set *VE*(51, 23, 11) with complexity of 1461 and *NNE*(31, 41, 4) with complexity of 1435, respectively. BER performance under different bias is shown in Figure 7. The BER bathtub curve indicates that equalization performance variation, instead of bandwidth variation with bias current, is the main reason of BER deviation. First, BER is calculated as function of bias for NNE and VE with B2B and 100 m condition. Figure 7a shows stable BER level with bias from 4 mA to 10 mA. 6 mA, the optimal

operating point of VCSEL, is chosen as a benchmark point. Second, pruning algorithms for VE and NNE are running and optimized at the bias of 6 mA. The same pruned equalizers are then applied to data received with a different bias current for VCSEL. As we can see from Figure 7b, the pruned VE optimized for 6 mA bias also shows excellent tolerance for bias from 7 mA to 9 mA. This means that sparse VE structure has consistent equalization performance within a bias range of 2.5 mA in this experiment. However, this is not case for NNE. Pruned NNE, optimized using data of 6 mA bias, seems less effective and more fluctuating for cases with a different bias. The experimental results in Figure 7b of fluctuation BER with pruned NNE shows that the iterative pruning and retraining approach has broken the equalization resilience due to reduced connections between layers.

**Figure 7.** BER versus VCSEL DC bias for (**a**) VE and NNE (**b**) Pruned VE and NNE.

#### **4. Conclusions**

In this paper, we compare the efficiency and stability performance of pruned Volterra series-based equalization and neural network-based equalization for 112 Gbps VCSEL enabled optical interconnects. From the 112 Gbps PAM4 transmission experiment, we can conclude that NNE has more than one order of magnitude BER advantage over VE at the same computation complexity. By using pruning algorithms, NNE has around 50% lower computation complexity compared to VE at the same BER level. Moreover, when communication channel condition become tough, VE shows serious performance instability due to its intricate structure. Through performance stability evaluation by changing bias current of a VCSEL, pruned VE shows more consistent equalization performance than NNE.

**Author Contributions:** Conceptualization, W.Z. and Z.H.; methodology, L.G.; validation, Y.Z. and C.L.; writing–original draft preparation, W.Z.; writing–review and editing, Z.H. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by National Key Research and Development Program of China (2019YFB1802903) and Open Projects Foundation of Yangtze Optical Fiber and Cable Joint Stock Limited Company (YOFC) (grant No.SKLD1903).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Abbreviations**

The following abbreviations are used in this manuscript:

#### **References**

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

#### *Article*

## **Impact of Equalization-Enhanced Phase Noise on Digital Nonlinearity Compensation in High-Capacity Optical Communication Systems**

**Jiazheng Ding 1, Tianhua Xu 1,2,3,\* , Cenqin Jin 2, Ziyihui Wang 1, Jian Zhao 1,\* and Tiegen Liu 1**

Received: 23 June 2020; Accepted: 24 July 2020; Published: 26 July 2020

**Abstract:** Equalization-enhanced phase noise (EEPN) can severely degrade the performance of long-haul optical fiber transmission systems. In this paper, the impact of EEPN in Nyquist-spaced dual-polarization quadrature phase shift keying (DP-QPSK), dual-polarization 16-ary quadrature amplitude modulation (DP-16QAM), and DP-64QAM optical transmission systemsisinvestigated considering the use of electrical dispersion compensation (EDC) and multi-channel digital backpropagation (MC-DBP). Our results demonstrate that full-field DBP (FF-DBP) is more susceptible to EEPN compared to single-channel and partial-bandwidth DBP. EEPN-induced distortions become more significant with the increase of the local oscillator (LO) laser linewidth, and this results in degradations in bit-error-rates (BERs), achievable information rates (AIRs), and AIR-distance products in optical communication systems. Transmission systems using higher-order modulation formats can enhance information rates and spectral efficiencies, but will be more seriously degraded by EEPN. It is found that degradations on AIRs, for the investigated FF-DBP schemes, in the DP-QPSK, the DP-16QAM, and the DP-64QAM systems are 0.07 Tbit/s, 0.11 Tbit/s, and 0.57 Tbit/s, respectively, due to the EEPN with an LO laser linewidth of 1 MHz. It is also seen that the selection of a higher-quality LO laser can significantly reduce the bandwidth requirement and the computational complexity in the MC-DBP.

**Keywords:** optical fiber communication; electrical dispersion compensation; multi-channel digital backpropagation; equalization-enhanced phase noise; achievable information rates

#### **1. Introduction**

With the explosive growth of the 5G and cloud services, the demand for high-capacity optical fiber communication infrastructure has become unprecedentedly urgent. Researchers have utilized dense wavelength division multiplexing (DWDM) and high-order modulation formats to enhance the spectral efficiency (SE) and the capacity of optical transmission systems. Moreover, Nyquist-spaced superchannel transmission systems, composed of closely-spaced WDM channels spaced at, or close to, the symbol rate, have been applied to further increase the spectral efficiencies [1–5]. However, the chromatic dispersion (CD), the polarization mode dispersion (PMD), the laser phase noise (PN), as well as Kerr fiber nonlinearities (NLs) severely degrade the performance of optical communication systems, leading to constraints on achievable information rates (AIRs), a measure of achievable capacity of communication systems [6,7]. Advancements in powerful digital signal processing (DSP) enhanced the reaches of optical communication systems and achieved the compensation of transmission

impairments via the use of electronic dispersion compensation (EDC), digital backpropagation (DBP), and carrier phase estimation (CPE) [8–13].

However, in DSP-based coherent systems, the interplay between the laser phase noise and the electronic dispersion compensation can introduce an effect of equalization-enhanced phase noise (EEPN), leading to a significant degradation to the performance of system. A few studies regarding EEPN have been reported for linear optical communication systems, where fiber nonlinearities were neglected. Shieh and Ho have conducted the theoretical derivation and analysis of the generation process of EEPN [14]. Fatadin and Savory have studied the impact of EEPN on the quadrature phase shift keying (QPSK), the 16-ary quadrature amplitude modulation (16-QAM), and the 64-QAM coherent systems using a time-domain dispersion equalizer [15]. Kakkar and Schatz have investigated the effect of the local oscillator (LO) laser linewidth and the symbol rate on the bit-error-rate (BER) of the transmission system [16]. Yoshida and Sugihara simulated the Q-factor of the system under the influence of EEPN by varying the laser linewidth from 100 kHz to 1 MHz [9]. It has been demonstrated that the impact of EEPN scales with the accumulated dispersion, the laser linewidth, the symbol rate, and the modulation format in the communication system.

To compensate for the fiber nonlinearities in optical communication systems, multi-channel digital backpropagation (MC-DBP) has been applied at different bandwidths to mitigate intra-channel and inter-channel fiber nonlinearities, such as self-phase modulation (SPM), cross-phase modulation (XPM), and four-wave mixing (FWM) [13,17–19]. The effect of the number of steps per span, the PMD, the backpropagated signal bandwidth, and the LO phase synchronization have been studied in the optimization of MC-DBP [20–22]. However, the influence of EEPN on the performance of MC-DBP in optical transmission systems was rarely reported and the impact of EEPN has never been quantified [13]. Therefore, it is of great significance to investigate the effect of EEPN on the MC-DBP-based nonlinear compensation in closely-spaced WDM communication systems.

In this paper, the impact of EEPN on the behaviors of EDC and MC-DBP in 9-channel 32-Gbaud dual-polarization (DP) Nyquist-spaced WDM transmission systems has been investigated, where DP-QPSK, DP-16QAM, and DP-64QAM have been applied. The nonlinearity compensation (NLC) performance using the single-channel DBP, the partial-bandwidth DBP, and the full-field DBP (FF-DBP) were numerically simulated with and without the application of EEPN. The EEPN degradations on AIRs in high-speed optical communication systems have also been studied. It is found that EEPN significantly deteriorates the performance of the communication systems, especially in the case of FF-DBP. Systems using higher-order modulation formats can enhance the AIRs and spectral efficiencies, but will be more seriously deteriorated by EEPN. It is also found that the selection of a better-quality LO laser can greatly reduce the bandwidth requirement and the computational complexity in the MC-DBP.

This paper is organized as follows. In Section 2, the numerical transmission system, the principle of EEPN, the mutual information (MI), and the modulation error ratio (MER) are introduced. In Section 3, simulations and results under different scenarios are analyzed. In Section 4, additional insights and discussions are provided. The conclusions are summarized in Section 5.

#### **2. Transmission Setup and Principles**

#### *2.1. Transmission Setup*

Figure 1 depicts the 9-channel 32-Gbaud Nyquist-spaced superchannel transmission system using the following modulation formats: DP-QPSK, DP-16QAM, DP-64QAM. At the transmitter (Tx), a 9-line 32-GHz spaced laser comb with a central wavelength of 1550 nm is employed as the phase-locked optical carrier for each sub-channel. Each optical carrier is split into two branches using a polarization beam splitter (PBS) and is then fed into two in-phase and quadrature (I-Q) modulators, respectively. The transmitted data sequences in all channels and polarizations are mutually independent based on the use of pseudo-random binary codes (PRBS) with a pattern length of 216. The Nyquist pulse shaping (NPS) is performed using an ideal root-raised cosine (RRC) filter with a roll-off coefficient of 0.1%. After the modulation, two orthogonally polarized signal sequences are combined using a polarization beam combiner (PBC) and are then injected into the standard single-mode fiber (SSMF). Numerical simulation of the SSMF is implemented using the split-step Fourier method based on the nonlinear Schrödinger equation (NLSE) with a step size of 0.1 km. Simulation parameters of SSMF include an attenuation coefficient of 0.2 dB/km, CD coefficient of 17 ps/nm/km, and a nonlinear coefficient of 1.2 /w/km, which represents a typical set of SSMF according to the Corning SMF-28 datasheet. An erbium-doped fiber amplifier (EDFA) is used to compensate for the loss of fiber in each span during the transmission. The noise figure of EDFA is 5.0 dB, which is a typical value for standard EDFA [1]. The fiber length of each span is 80 km. At the receiver (Rx), the received signal is mixed with the LO laser to achieve an ideal coherent detection of I-Q components in each polarization state. The signals are sampled using analog-to-digital converters (ADCs) and are then processed by DSP modules which include a dispersion compensation module using EDC, nonlinearity compensation module using MC-DBP, matched filter, CPE, and symbol demodulation. In our simulation, the frequency domain equalizer (FDE) is used for CD compensation [23], and the MC-DBP is used for NLC [24]. The MC-DBP algorithm is implemented based on the reverse split-step Fourier solution of the NLSE and the compensation bandwidth is set as 32 GHz (for 1-channel DBP), 160 GHz (for 5-channel DBP), and 288 GHz (for 9-channel DBP, FF-DBP), respectively. Before the MC-DBP module, an ideal RRC filter with a roll-off of 0.1% is applied to select the compensation bandwidth and to remove the unwanted amplifier spontaneous emission noise (ASE). An ideal CPE is used for the compensation of phase noise, achieved using the conjugate multiplication between the received signal and the extracted intrinsic laser phase noise [13,25]. This is to isolate the impact from EEPN, where no amplitude noise mitigation effect is employed in the CPE module [15,26]. BER and MI estimation are carried out to evaluate the transmission performance. The PMD and the frequency offset are neglected in all simulations.

**Figure 1.** Schematic of the 9-channel 32-Gbaud Nyquist-spaced superchannel transmission system. (NPS: Nyquist pulse shaping, PBS: polarization beam splitter, PBC: polarization beam combiner, LO: local oscillator, CPE: carrier phase estimation, MI: mutual information).

#### *2.2. Principle of Equalization Enhanced Phase Noise*

As illustrated in Figure 1, the EDC is used to compensate for dispersion, and the ideal CPE is applied for the CPE in the coherent optical transmission system. The dispersion compensation module (either EDC or MC-DBP) is applied prior to the CPE module [13]. It can be clearly seen that the phase noise from the transmitter passes through the transmission fiber and the dispersion compensation module, so the net dispersion experienced by the phase noise from the transmitter is close to zero. However, the phase noise from the LO laser only goes through the dispersion compensation module. Due to the non-commutability of the phase noise and fiber dispersion, the phase noise of the LO laser will interact with the DSP-based dispersion compensation module. Consequently, the electronic equalizers (EDC or MC-DBP) will enhance the distortions from the LO laser phase noise, and the resulting EEPN will seriously reduce the performance of the large-capacity transmission system.

It has been reported in [11,14,27] that the variance of EEPN has a linear relationship with the accumulated chromatic dispersion, the linewidth of LO laser, and the symbol rate in the system. The noise variance generated by EEPN can be expressed as [14]

$$
\sigma\_{\rm EEPN}^2 = \frac{\pi \lambda^2 D \cdot L \cdot \Delta f\_{\rm LO}}{2cT\_S} \tag{1}
$$

where λ is the center wavelength of the carrier laser, *D* is the dispersion coefficient of SSMF, *L* is the total length of fiber link, Δ*fLO* is the 3-dB linewidth of the LO laser, *c* is the speed of light in vacuum, and *TS* is the symbol period of the transmission system.

Therefore, the total noise variance of coherent transmission system considering the impairment of EEPN is expressed as follows:

$$
\sigma\_{\text{Total}}^2 = \sigma\_{\text{Tx}}^2 + \sigma\_{LO}^2 + \sigma\_{EEPN}^2 + 2\rho\sigma\_{LO}\sigma\_{EEPN} \tag{2}
$$

$$
\sigma\_{T\chi}^2 = 2\pi\Delta f\_{T\chi} T\_S \tag{3}
$$

$$
\sigma\_{LO}^2 = 2\pi \Delta f\_{LO} T\_S \tag{4}
$$

where σ2 *Tx* and σ2 *LO* are the variances of the phase noise from the transmitter and receiver, respectively, ρ is the correlation coefficient of the LO phase noise and the EEPN. It has been proved that ρ ≈ 0 when the length of the optical fiber link exceeds 80 km [11]. Therefore, in a high-speed and long-distance superchannel optical communication system, the total noise variance can be approximately expressed as

$$
\sigma\_{\text{Total}}^2 = \sigma\_{\text{Tx}}^2 + \sigma\_{\text{LO}}^2 + \sigma\_{\text{EEPN}}^2 \tag{5}
$$

It is noted that when the ideal CPE is applied, the intrinsic phase noise σ2 *Tx* and σ2 *LO* will be fully removed.

#### *2.3. Mutual Information and Achievable Information Rate Estimation*

Due to the property of the binary symmetric channel, the hard-decision forward error correction (HD-FEC) based code rate of the QPSK and m-QAM signal transmission over an additive white Gaussian noise (AWGN) channel can be expressed as [28,29]

$$R\subset = 1 + BER \cdot \log\_2(BER) + (1 - BER) \cdot \log\_2(1 - BER) \tag{6}$$

In communication systems, mutual information is an indicator of the achievable channel capacity under specific modulation schemes. In dual-polarization coherent optical transmission systems, the *MI* can be expressed as

$$MI = 2 \cdot \log\_2(m) \cdot R\_{\mathbb{C}} \tag{7}$$

where *m* is the order of the modulation format. The *AIR* is also an important indicator, which refers to the net data rate that can be achieved using FEC decoders [30]. In a dual-polarization Nyquist-spaced optical transmission system, AIRs are expressed by MI as

$$AIR = N\_{\rm ch} \cdot R\_{\rm S} \cdot MI \tag{8}$$

where *N*ch is the number of WDM channels in the system, and *RS* is the symbol rate of the transmitted signal.

#### *2.4. Modulation Error Ratio*

The *MER* is also an important indicator to evaluate the quality of a communication system, and it can well reflect the ability of digital receivers in restoring transmitted data [31]. The *MER* is measured using the ratio between the average signal power and the average error power [32], and is defined in *dB* as follows

$$MER(dB) = 10\log\_{10}(\frac{P\_{signal}}{P\_{error}}) \tag{9}$$

where *Perror* is the root mean square (RMS) power of the error vector and *Psignal* is the RMS power of the ideal transmitted signal. It can be found that the MER provides a similar evaluation on the quality of the received signal, compared to the concept of the signal-to-noise-ratio (SNR) [4].

#### **3. Simulation Results**

In this section, numerical simulations based on the above analyses will be conducted. Firstly, the back-to-back (BtB) performance is investigated to evaluate the benchmark quality of considered transmission systems. Figure 2 shows the BtB performance of the measured BER versus the optical signal-to-noise-ratio (OSNR) for the 9-channel 32-Gbaud Nyquist-spaced DP-QPSK, DP-16QAM, and DP-64QAM systems. It is found that, to achieve a BER target of 10−3, the required OSNRs in the DP-QPSK, the DP-16QAM, and the DP-64QAM transmission systems are 14.0 dB, 20.6 dB, and 27.0 dB, respectively.

**Figure 2.** Back-to-back (BtB) performance of 9-channel 32-GbaudNyquist-spaced dual-polarization quadrature phase shift keying (DP-QPSK), dual-polarization 16-ary quadrature amplitude modulation (DP-16QAM), and dual-polarization 64-ary quadrature amplitude modulation (DP-64QAM) transmission systems.

With and without taking EEPN into account, the BERs and AIRs of the 9-channel Nyquist-spaced transmission systems using EDC and MC-DBP are studied. Modulation formats of DP-QPSK, DP-16QAM, and DP-64QAM were applied. The linewidths of the transmitter laser and the LO laser were both set as 0 Hz to remove the EEPN. To include the EEPN, both the transmitter and LO laser linewidths were set to be equal, and two sets of values were considered: 100 kHz (for common ECL lasers) and 1 MHz (for common DFB lasers), respectively. Single-channel, partial-bandwidth, and full-field DBP were applied to perform nonlinearity compensation in the DSP-based receiver. The case of CD compensation only was also analyzed as reference results. In all following simulations, the MC-DBP was operated with 800 steps per span and a nonlinear coefficient of 1.2 /W/km to ensure its optimal performance.

Simulation results of the 9-channel 32-Gbaud Nyquist-spaced DP-QPSK transmission system with and without the application of EEPN are shown in Figure 3, where the total distance of fiber link is 8000 km. The results are obtained under the use of EDC or MC-DBP with different back-propagated bandwidths. Figure 3a shows the BER versus optical launch power in DP-QPSK transmission system, the system BER using MC-DBP is obviously reduced compared with the EDC-only scheme under different LO linewidths. The compensation effect of FF-DBP is the most obvious because the intra-channel and inter-channel nonlinear interactions in the transmission channels were fully mitigated. When there is no EEPN, the optimum launch powers of −2.0 dBm, −2.0 dBm, −1.0 dBm, and 1.0 dBm with the best BERs of 3.43 × 10−3, 2.10 × 10−3, 1.09 × 10−3, and 1.70 × 10−3 are achieved for the scenarios of EDC, single-channel, 5-channel, and full-field DBP, respectively. However, when EEPN is applied with an LO laser linewidth of 100 kHz, the optimum launch powers for EDC, signal-channel, and 5-channel DBP are still −2.0 dBm, −2.0 dBm, and −1.0 dBm, respectively. However, the optimum launch powers for full-field DBP is changed to 0.0 dBm. Corresponding BERs rise to 4.25 × 10−3, 2.54 × 10−3, 1.45 × 10−3, and 1.46 × 10−3 compared to the scenario of no EEPN. When the LO laser linewidth rises to 1 MHz, the distortions from EEPN will become greater according to Equation (1). Simulation results in Figure 3a show that the optimum launch powers for EDC, signal-channel, and 5-channel DBP remain constant, respectively, but for FF-DBP, it drops to −1.0 dBm. The system BERs increase further to 1.41 × 10−2, 1.07 × 10−2, 8.75 × 10−3, and 6.95 × 10−3 for EDC, signal-channel, 5-channel, and full-field DBP, respectively. It can be clearly seen that the performance of FF-DBP is the most susceptible to EEPN compared to EDC, single-channel, and 5-channel DBP. Due to the EEPN (with an LO laser linewidth of 1 MHz), the best BER increases by two and a half orders of magnitude (from 1.70 × 10−5 to 6.95 × 10−3) in the case of FF-DBP, while it only increases by less than an order of magnitude (from 3.43 × 10−3 to 1.41 × 10−2) in the case of EDC.

The relationship between the AIRs and optical launch powers in the 9-channel 32-Gbaud Nyquist-spaced DP-QPSK optical communication system was investigated, as described in Figure 3b. When there is no EEPN, the highest AIRs for the system are 1.11 Tbit/s, 1.13 Tbit/s, 1.14 Tbit/s, and 1.15 Tbit/s for the cases of EDC, single-channel, 5-channel, and full-field DBP, respectively. When the EEPN is applied with an LO linewidth of 100 kHz, it can be seen from Figure 3b that the AIRs of the transmission system are not significantly reduced in all compensation schemes for such an amount of EEPN. However, the degradation on AIRs becomes significant when the LO laser linewidth increases to 1 MHz. The best AIRs now become 1.03 Tbit/s, 1.05 Tbit/s, 1.07 Tbit/s, and 1.08 Tbit/s for the cases of EDC, single-channel, 5-channel, and full-field DBP, respectively. It is noted that, for all launch powers smaller than 0.5 dBm, the AIRs in the cases of single-channel and 5-channel DBP with the LO laser linewidth of 100 kHz are higher than the AIRs in the case of FF-DBP with the LO laser linewidth of 1 MHz. This indicates that a better-quality LO laser, which has a narrower linewidth and leads to a smaller EEPN, can significantly save the bandwidth and the computational effort in the operation of MC-DBP.

**Figure 3.** Simulation results of the DP-QPSK system with (w/) and without (w/o) equalization-enhanced phase noise (EEPN) at different multi-channel digital backpropagation (MC-DBP) bandwidths. (**a**) Bit-error-rate (BER) versus optical launch power, (**b**) achievable information rate (AIR) versus optical launch power.

Figure 4 shows the simulation results of the 9-channel 32-Gbaud Nyquist-spaced DP-16QAM system at different scenarios of MC-DBP with and without the application of EEPN. The transmission distance is 2000 km. It can be seen from Figure 4a that the optimum launch powers for the use of EDC, single-channel, partial-channel, and full-field DBP are −2.0 dBm, −2.0 dBm, −1.0 dBm, 3.0 dBm, respectively, when there is no EEPN. Corresponding optimal BERs are 6.56 × 10−3, 5.11 × 10−3, 2.35 × 10−3, and 2.36 × 10−3, respectively. When the EEPN is applied with an LO laser linewidth of 1 MHz, the optimum launch powers keep the same for the cases of EDC, single-channel, and 5-channel DBP, but decrease to 1.0 dBm for the case of FF-DBP. Meanwhile, the optimal BERs rise to 1.67 × 10−2, 1.51 × 10−2, 1.51 × 10−2, and 5.51 × 10−3, respectively. Due to the EEPN with an LO laser linewidth of 1 MHz, the best BER for the FF-DBP increases by two and a half orders of magnitude, similar as

the DP-QPSK system. Apparently, the DP-16QAM system is more sensitive to the EEPN than the DP-QPSK system. Because the transmission distance in the DP-16QAM system is only a quarter of that in the DP-QPSK system, this will lead to the reduction of EEPN variance (to a quarter) according to Equation (1).

**Figure 4.** Simulation results of the DP-16QAM system with (w/) and without (w/o) EEPN at different MC-DBP bandwidths. (**a**) BER versus optical launch power, (**b**) AIR versus optical launch power.

The AIRs versus optical launch powers in the 9-channel 32-Gbaud Nyquist-spaced DP-16QAM transmission system are shown in Figure 4b. It is found from Figure 4b that, similar to the DP-QPSK system, the EEPN with an LO laser linewidth of 100 kHz poses marginal degradation on AIRs in all EDC and MC-DBP schemes compared to the scenario without any EEPN. However, the EEPN with an LO laser linewidth of 1 MHz will cause a significant degradation to the AIRs. For the FF-DBP, the best AIR drops from 2.30 Tbit/s to 2.19 Tbit/s and the optimum launch power decreases from 3.0 dBm to 1.0 dBm compared to the scenario without any EEPN. It is noted again that, for launch powers smaller than 1.0 dBm, the AIRs in the case of single-channel DBP with the LO laser linewidth of 100 kHz are higher than the AIRs in the case of 5-channel DBP with the LO laser linewidth of 1 MHz, and the AIRs in the case of 5-channel DBP with the LO laser linewidth of 100 kHz are higher than the AIRs in the case of FF-DBP with the LO laser linewidth of 1 MHz. This again demonstrates that the choice of a better-quality LO laser can significantly reduce the bandwidth requirement and the computational complexity in the MC-DBP.

Simulation results of the 9-channel 32-Gbaud Nyquist-spaced DP-64QAM transmission system using EDC and MC-DBP with and without the application of EEPN are illustrated in Figure 5, where the transmission distance is 1200 km. Figure 5a describes that BERs versus optical launch powers in the DP-64QAM superchannel transmission, which shows a similar trend as the BERs the DP-QPSK and the DP-16QAM systems. Due to the EEPN with an LO laser linewidth of 1 MHz, the optimum launch power for the FF-DBP drops from 3.5 dBm to 2.0 dBm, and the best BER increases from 4.63 × 10−3 to 3.23 × 10−2 compared to the scenario without any EEPN. This is similar as the BER degradation in the DP-QPSK and the DP-16AM system. This also indicates that the DP-64QAM system is more vulnerable to the EEPN than the DP-QPSK and the DP-16QAM systems, since the transmission distance in the DP-64QAM system is only 3/20 of the transmission distance in the DP-QPSK system and 3/5 of the transmission distance in the DP-16QAM system. This will lead to a significant reduction of EEPN variance compared to that in the DP-QPSK and the DP-16QAM systems according to Equation (1). Therefore, the EEPN will pose greater distortions on the systems with higher-order modulation formats, resulting in a stricter requirement on laser linewidths, although they can improve the transmission spectral efficiencies.

Simulation results of AIRs versus optical launch powers for the 9-channel 32-Gbaud Nyquistspaced DP-64QAM system at different MC-DBP bandwidths, with and without the application of EEPN, are shown in Figure 5b. Again, similar to the DP-QPSK and the DP-16QAM systems, the EEPN with an LO laser linewidth of 100 kHz puts marginal degradation on AIRs in all schemes of EDC and MC-DBP, compared to the scenario without any EEPN. The EEPN with an LO laser linewidth of 1 MHz will pose a significant degradation to the AIRs of the DP-64QAM system. For the FF-DBP, the best AIR decreases from 3.31 Tbit/s to 2.74 Tbit/s. Meanwhile, a similar effect can be found that, for launch powers smaller than 0.5 dBm, the AIRs in the use of single-channel DBP with the LO laser linewidth of 100 kHz are higher than the AIRs in the use of 5-channel DBP with the LO laser linewidth of 1 MHz, and the AIRs in the use of 5-channel DBP with the LO laser linewidth of 100 kHz are higher than the AIRs in the use of FF-DBP with the LO laser linewidth of 1 MHz. This again implies that the use of a better-quality LO laser can significantly reduce the bandwidth requirement and the computational complexity in the MC-DBP. It can also be found in the AIR results that the DP-64QAM system is more susceptible to the EEPN compared to the DP-QPSK and the DP-16QAM systems, since the AIR degradation (~0.57 Tbit/s) for the case of FF-DBP in the DP-64QAM system due to the EEPN with the LO laser linewidth of 1 MHz is more significant than those (~0.11 Tbit/s and ~0.07 Tbit/s, respectively) in the DP-QPSK and the DP-16QAM systems, although the DP-64QAM system has the shortest transmission distance. In addition, the increase in EEPN due to the increase of the LO laser linewidth shows a smaller effect on the AIR degradations in single-channel DBP and partial-bandwidth DBP than in FF-DBP.

**Figure 5.** Simulation results of the DP-64QAM system with (w/) and without (w/o) EEPN at different MC-DBP bandwidths. (**a**) BER versus optical launch power, (**b**) AIR versus optical launch power.

#### **4. Discussions**

The MER reflects the ability of digital receivers in restoring transmitted data. Considering the EEPN, the relationship between the MER and the optical launch power in the DP-64QAM communication system using EDC and MC-DBP is investigated in Figure 6, where the transmission distance is 1200 km. It is found that, without EEPN, the maximum MER is 20.81 dB in the case of FF-DBP when the optical launch power is 3 dBm. However, the MER will be reduced to 17.06 dB when the LO laser linewidth is increased to 1 MHz, and the optimal launch power will be reduced to 2 dBm accordingly. It can be found that the EEPN degradations on MERs are also more serious in the case of FF-DBP than those in the scenarios of the EDC, the single-channel, and the partial-bandwidth DBP. This shows a consistent behavior compared to previous analyses.

**Figure 6.** Modulation error ratio (MER) versus optical launch power in the DP-64QAM system with (w/) and without (w/o) EEPN at different MC-DBP bandwidths.

To provide a more practical study on the impact of EEPN in optical fiber communication systems, numerical simulations have been carried out based on an 820 km optical fiber field link installed in the eastern coastal area of Sweden. Figure 7 illustrates the block diagram of the 820 km field link, which interconnects seven nodes at Kista, Råsunda, Norrtälje, Östhammar, Gävle, Söderhamn, and Hudiksvall [33]. The transmitter, the receiver, and the DSP setups are the same as those employed in Figure 1. The impact of EEPN on the performance of EDC and MC-DBP in the 820 km transmission system is investigated, as shown in Figure 8, where the modulation format of DP-64QAM is applied. Similar to previous analyses, it can be found that the EEPN also poses significant degradations on the performance of the MC-DBP for such a practical transmission link. For the FF-DBP and the EEPN with an LO laser linewidth of 1 MHz, the optimum launch power drops from 4.0 dBm to 3.0 dBm, and the best BER increases from 4.15 × 10−3 to 2.39 × 10−2, and the best AIR decreases from 3.32 Tbit/s to 2.89 Tbit/s, compared to the scenario without any EEPN.

**Figure 7.** Block diagram of 820 km installed fiber field link in the eastern coast of Sweden.

**Figure 8.** Simulation results of the DP-64QAM system with (w/) and without (w/o) EEPN at different MC-DBP bandwidths in the 820 km field link. (**a**) BER versus optical launch power, (**b**) AIR versus optical launch power.

From the above analyses, the intrinsic phase noise from the transmitter and LO lasers in the transmission system was completely compensated by the ideal CPE in the DSP, so the degradations in the system AIRs come from EEPN and the fiber NLs. From the aforementioned results, FF-DBP is the most susceptible to EEPN compared to the single-channel and the partial-bandwidth DBP. This is because the phase change in the DBP (actually the dispersion compensation) filter increases quadratically with the increase of the signal frequency [13,14]. Consequently, EEPN will pose a more serious degradation in the case of FF-DBP. The distortions from EEPN also become more significant for a larger LO linewidth, resulting in a larger increase of BER and a larger reduction in the optimum launch power of FF-DBP. Therefore, in high-capacity optical communication systems, EEPN put strict

requirements on the linewidths of the LO lasers, and the restrictions are stronger for the systems using higher-order modulation formats.

The use of higher-order modulation formats can improve the spectral efficiency in optical communication systems. However, it can be seen from the above simulation results that the systems using higher-order modulation formats are more seriously distorted by EEPN. Figure 9 illustrates the maximum AIRs versus transmission distances for the DP-QPSK, the DP-16QAM, the DP-64QAM, and the field-link Nyquist-spaced transmission systems with and without the application of EEPN. Here, the product between AIR and transmission distance *AIR* · √ *L* is employed as an indicator to evaluate the transmission quality of optical communication systems, the three hyperbolas in Figure 9 are provided using the formula of *AIR* · √ *L* = *C* based on the 9-channel 32-Gbaud Nyquist-spaced DP-QPSK system under the transmission distance of 8000 km and three LO laser linewidths (0 Hz, 100 kHz, 1 MHz).

**Figure 9.** Maximum AIRs versus transmission distance for the DP-QPSK, the DP-16QAM, the DP-64QAM, and the 820 km field-link Nyquist-spaced transmission systems with and without EEPN. The three hyperbolas are obtained by *AIR* · √ *L* = *C* based on the 9-channel 32-Gbaud Nyquist-spaced DP-QPSK system under the transmission distance of 8000 km and three LO laser linewidths (0 Hz, 100 kHz, 1 MHz).

The AIR-distance product can reach 103.00 Tbit s−1km1/2 and 102.82 Tbit s−1km1/2 for the DP-QPSK system when the LO linewidths are 0 Hz and 100 kHz, respectively. However, it can only achieve 96.88 Tbit s−1km1/2 when the LO linewidth is 1 MHz. It is very interesting that the AIR-distance products of the 9-channel 32-Gbaud Nyquist-spaced DP-16QAM system are basically equal to the AIR-distance products in the DP-QPSK system for all different values of EEPN. For the 9-channel 32-Gbaud Nyquist-spaced DP-64QAM system, the AIR-distance products are 114.63 Tbit s−1km1/2 and 111.68 Tbit s−1km1/2 for the LO linewidths of 0 Hz and 100 kHz, respectively. This shows a better performance (based on the indicator of the AIR-distance product) compared to the DP-QPSK and the DP-16QAM systems. However, the product drops to 95.05 Tbit s−1km1/2 when the LO linewidth rises to 1 MHz, which is similar to the indicators in the DP-QPSK and the DP-16QAM systems. For the 820 km field-link transmission, the system shows a significant performance degradation compared to all above three systems in terms of the AIR-distance product. It can only reach 82.88 Tbit s−1km1/2 in the case of FF-BDP when the LO laser linewidth is 1 MHz.

In this research, the impact of EEPN is investigated in optical communication systems based on the use of MC-DBP, which is one of the most promising approaches for nonlinearity compensation in next-generation optical fiber networks, due to its flexible implementation and optimization in the hardware. Meanwhile, the optical phase conjugation (OPC) is another promising technique to compensate for fiber nonlinearities in WDM optical transmission systems [34,35]. In such systems, the phase fluctuation from the high-power pump laser, which is used for realizing the FWM-based OPC, will be transferred into the phase of output signals [13]. Therefore, the EEPN in OPC-based systems arises from the interactions between the phase noise from the high-power pump laser and the fiber dispersion experienced by the phase-conjugated optical signals.

#### **5. Conclusions**

In this paper, the impact of EEPN on both BER performance and AIRs of Nyquist-spaced superchannel transmission system have been investigated based on the use of EDC and MC-DBP. Different modulation formats including DP-QPSK, DP-16QAM, DP-64QAM have been applied. Distortions from EEPN are more significant in the transmission system when the LO linewidth increases. This results in a reduction in both the AIRs and the AIR-distance products. It is shown that the FF-DBP is more susceptible to EEPN compared to the scenarios of EDC, single-channel, and partial-bandwidth DBP. For the investigated FF-DBP schemes, AIRs are reduced by ~0.07 Tbit/s, ~0.11 Tbit/s, and ~0.57 Tbit/s in the DP-QPSK, the DP-16QAM, and the DP-64QAM systems, respectively, due to the EEPN with an LO laser linewidth of 1 MHz. Systems using higher-order modulation formats will improve the AIRs and spectral efficiencies, but will be more susceptible to EEPN. It is also found that the choice of a better-quality LO laser can significantly reduce the bandwidth requirement and the computational complexity in the MC-DBP. Our research provides a novel insight in the design of long-haul high-capacity optical communication systems using multi-channel nonlinearity compensation techniques.

**Author Contributions:** T.X. and T.L. proposed the concept and initiated the study. J.D., T.X., and C.J. carried out numerical simulations. J.D., T.X., and J.Z. conducted analytical calculations. T.X., J.Z., and T.L. supervised the studies. The paper was written by J.D., T.X., Z.W., J.Z., and T.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Time-Domain Blind ICI Compensation in Coherent Optical FBMC**/**OQAM System**

**Binqi Wu 1, Jin Lu 1,\*, Mingyi Gao 2, Hongliang Ren 1, Zichun Le 3 , Yali Qin 1, Shuqin Guo 1 and Weisheng Hu 4**

Received: 29 August 2020; Accepted: 3 November 2020; Published: 9 November 2020

**Abstract:** A blind discrete-cosine-transform-based phase noise compensation (BD-PNC) is proposed to compensate the inter-carrier-interference (ICI) in the coherent optical offset-quadrature amplitude modulation (OQAM)-based filter-bank multicarrier (CO-FBMC/OQAM) transmission system. Since the phase noise sample can be approximated by an expansion of the discrete cosine transform (DCT) in the time-domain, a time-domain compensation model is built for the transmission system. According to the model, phase noise compensation (PNC) depends only on its DCT coefficients. The common phase error (CPE) compensation is firstly performed for the received signal. After that, a pre-decision is made on a part of compensated signals with low decision error probability, and the pre-decision results are used as the estimated values of transmitted signals to calculate the DCT coefficients. Such a partial pre-decision process reduces not only decision error but also the complexity of the BD-PNC method while keeping almost the same performance as in the case of the pre-decision of all compensated signals. Numerical simulations are performed to evaluate the performance of the proposed scheme for a 30 GBaud CO-FBMC/OQAM system. The simulation results show that its bit error rate (BER) performance is improved by more than one order of magnitude through the mitigation of the ICI in comparison with the traditional blind PNC scheme only aiming for CPE compensation.

**Keywords:** coherent optical communication; offset-quadrature amplitude modulation-based filter-bank multicarrier (FBMC/OQAM); blind phase noise compensation; inter-carrier-interference (ICI); discrete cosine transform (DCT)

#### **1. Introduction**

A coherent optical offset-quadrature amplitude modulation (OQAM)-based filter-bank multicarrier (CO-FBMC/OQAM) system has recently become a promising candidate for high-speed long-haul optical fiber transmission due to its higher spectral efficiency (SE) and its robustness to channel impairments [1–5]. As a multicarrier modulation (MCM) technique, the FBMC is more robust against pulse dispersion and has less out-of-band leakage than orthogonal frequency division multiplexing (OFDM) [6–14]. Meanwhile, the FBMC/OQAM scheme does not need cyclic prefix and, therefore, its SE can be increased remarkably [15,16].

The performance of optical fiber transmission system can be degraded severely by the phase noise (PN) [17], and the MCM technique is especially vulnerable to PN like OFDM. In CO-FBMC/OQAM systems, the PN generally includes common phase error (CPE) and inter-carrier interference (ICI) [18], similar to that in coherent optical OFDM (CO-OFDM) systems. Nevertheless, the original PN compensation (PNC) algorithms in CO-OFDM cannot be simply transplanted in CO-FBMC/OQAM systems. This is because the OQAM is employed between two immediate adjacent subcarriers, and the orthogonality between subchannels is ensured only in real plane. Due to the nonorthogonality in imaginary plane, the PNC and channel estimation techniques are severely restricted by the so-called intrinsic imaginary interference (IMI) in CO-FBMC/OQAM systems [19–24]. Consequently, many studies on the PNC method focus on the IMI cancellation [25–27]. In [26], a pilot-based PNC method is used to suppress the IMI at the expense of SE, where a pilot structure consists of several additional data symbols surrounding a pilot symbol. Another alternative approach is a coding scheme, and data symbols surrounding a pilot are deliberately coded to eliminate the IMI in that pilot [26,27]. Although the complexity of two mainly pilot-based PNC methods is moderate, their performance is limited by only CPE compensation.

At present, the PNC methods for CO-FBMC/OQAM systems can be classified into two categories: blind and pilot-aided PNC. As mentioned above, to cancel the IMI, the pilot-aided PNC methods have a low complexity and avoid the effect of cycle slips at the cost of the SE loss [25–27]. Compared with the pilot-aided methods, the blind methods can achieve higher SE at the price of higher computational complexity (CC) [28–36]. As a de-facto standard in coherent optical communication systems, the blind phase search (BPS) method can achieve a relatively good PNC effect and a high laser linewidth tolerance. Its main disadvantage is high CC resulting from a large number of test phases. The modified BPS (M-BPS) simplifies the distance calculation in the complex plane as the operations in the real plane [28], and thereby reduces the number of multiplications significantly. However, both blind and pilot-aided methods mainly emphasize on the CPE compensation in frequency domain, and the ICI is not considered to be mitigated and is assumed to be additive noise in these studies. Because the MCM system with a large number of subcarriers can increase its fiber chromatic dispersion (CD) tolerance [6], only CPE compensation is obviously not enough in this case due to serious negative impact of the ICI on the system performance [25–28].

Recently, time-domain ICI compensation has been proposed based on pilot symbols in either wireless or optical FBMC/OQAM systems [18,37]. An orthogonal-basis-expansion (OBE)-based PNC method has been proposed in a polarization-division-multiplexed CO-FBMC/OQAM system [18]. With the aid of pilot symbols, the PN in time domain can be mitigated by estimating the corresponding coefficients of the OBE. In wireless FBMC/OQAM systems, by using the discrete cosine transform (DCT) method, the samples of the PN in time-domain can be described as sums of products of a group of DCT bases and their corresponding coefficients of the DCT [37]. Using pilot symbols, the ICI effect can be tracked by calculating these coefficients of the DCT. It is obvious that the pilot overhead loss is very high for two time-domain pilot-based methods. Several blind ICI compensation schemes have been presented for CO-OFDM systems [38–40], but they could not be used in CO-FBMC/OQAM systems.

In this paper, we propose a blind time-domain ICI compensation scheme for CO-FBMC/OQAM systems by combining the M-BPS method and DCT approximation of PN. The M-BPS method is used to obtain the pre-estimated values of the transmitted signals, with which the DCT coefficients can be obtained from the time-domain compensation model. To improve its performance and reduce its CC, the pre-decision error and the overlapped symbol structure are considered to develop the proposed algorithm. The proposed method is numerically validated in a 30 GBaud back to back (BTB) CO-FBMC/OQAM system. The numerical results show that the proposed algorithm achieves a significantly better performance compared to the M-BPS scheme while having a limited increase in CC.

The remainder of this paper is organized as follows. A theoretical model of CO-FBMC/OQAM system with laser phase noise is derived in Section 2. Section 3 introduces the principle of the proposed PNC method. In Section 4, the overlapped symbol structure in the CO-FBMC/OQAM system is implemented to improve the effectiveness of the algorithm. We numerically investigate the performance of our method and propose the complexity analysis in Section 5. Finally, Section 6 summarizes the paper.

#### **2. CO-FBMC**/**OQAM System with Laser Phase Noise**

In this section, the influence of PN is studied in CO-FBMC/OQAM systems, and the analytical expressions of the PN induced interference are derived. As shown in Figure 1, a single polarization CO-FBMC/OQAM BTB system is considered to investigate the PN induced interference and the performance of the proposed PNC algorithm. The single polarization BTB system is used to concentrate on the impact of laser PN in this paper.

**Figure 1.** Block diagram of CO-FBMC/OQAM BTB system.

At the transmitter, the process of OQAM pre-processing transforms the complex-valued QAM symbols into real-valued symbols, which means that the real-valued symbols are formed with the real and imaginary parts of QAM symbols destaggered by half a QAM symbol period. These real-valued symbols are modulated (i.e., pulse amplitude modulation (PAM)) in such a way that the FBMC system maintains the same data rate as the OFDM system without cyclic-prefix. Then, the PAM symbols are mapped onto a set of orthogonal subcarriers generated by an inverse fast Fourier transform (IFFT). The following operation is a cost-efficient poly-phase network (PPN) for filter bank synthesis implementation. With the digital signal processing (DSP) at the transmitter, its discrete time-domain transmitted signal, *s*[*k*], can be expressed as,

$$s[k] = \sum\_{n=0}^{N\_S - 1} \sum\_{m=0}^{M-1} a\_{n,m} g\_{n,m}[k] \tag{1}$$

where *an,m* and *gn,m*[*k*] are the PAM symbol and the synthesis filter at the (*n*,*m*)th time-frequency index, respectively; *NS* is the total number of CO-FBMC/OQAM blocks; and *M* is the total number of subcarriers in each CO-FBMC/OQAM block. The (*n*,*m*)th synthesis filter is described as,

$$\log\_{n,m}[k] = \lg[k - n\frac{M}{2}] \cdot e^{j\frac{2\pi}{M}mk} e^{j\psi\_{n,m}} \tag{2}$$

$$\mathbf{g}[k] = \frac{(4\alpha k/T)\cos[\pi(1+\alpha)k/T] + \sin[\pi(1-\alpha)k/T]}{\left(\pi k/T\right)\left[1 - \left(4\alpha k/T\right)^2\right]}\tag{3}$$

where the square root raised cosine function *g*[*k*] is employed as the prototype filter with length *Lg* [41], ψ*n*,*m* = (*n* + *m*)/2 is phase modulation factor, α denotes the roll-off coefficient, and *T* represents the duration of the QAM complex symbols.

In the BTB CO-FBMC/OQAM system, under the impact of laser PN and additive Gaussian white noise (AWGN), the received signal *r*[*k*] can be written as,

$$\begin{split} r[k] &= e^{j\eta[k]}s[k] + w[k] \\ &= e^{j\eta[k]} \Big{[} \sum\_{n=0}^{N\_S-1} \sum\_{m=0}^{M-1} a\_{l,m} \mathbf{g}[k - n\frac{M}{2}] \cdot e^{j\frac{2\pi}{M}mk} e^{j\frac{\pi(m+n)}{2}} \Big{)} + w[k] \\ &= e^{j\eta[k]} \Big{[} a\_{l\eta,m\_0} \mathbf{g}[k - n\frac{M}{2}] e^{j\frac{2\pi}{M}m\phi k} e^{j\frac{\pi(m\_0+n\_0)}{2}} + \sum\_{\substack{\{n,m\} \neq \{n\_0,m\_0\}}} a\_{l,m} \mathbf{g}[k - n\frac{M}{2}] e^{j\frac{2\pi}{M}mk} e^{j\frac{\pi(m+n)}{2}} \Big{)} + w[k] \end{split} \tag{4}$$

Here, ϕ[*k*] denotes the PN in the time-domain, and the notation *w*[*k*] is the AWGN. The laser PN originates from either the external cavity laser at the transmitter or the local oscillator at the receiver. It is modeled as a random walk Wiener process, and the PN change follows a Gaussian distribution with zero mean and a specific variance 2π · Δν · *TS*, where Δν is the combined laser linewidth, and *TS* is the normalized symbol period. Here, *TS* = (1/*Br*)/*M*, and *Br* is the baud rate of QAM symbols. The combined laser linewidth is defined as the sum of linewidths of the transmitter and receiver lasers. For the sake of simplicity, in the BTB system, the transmitter laser has the same linewidth as the received laser, and the combined laser linewidth is twice the linewidth of the single laser.

At the receiver side, the incoming optical wave signal is first coherently detected in a 90◦ optical hybrid. Subsequently, the obtained digital data are sent to the DSP modules to recover the original data. After a time synchronization is performed perfectly, a time-frequency translation can be completed by a hardware-efficient structure of PPN for filter bank analysis followed by a fast Fourier transform (FFT). Eventually, the complex-valued received demodulated signal at the (*n0*,*m0*)th time-frequency index can be written as,

$$\begin{array}{rcl}D\_{n\_0,m\_0} &=& \sum\_{k=0}^{L\_{\mathfrak{g}}-1} r[k]g^\*\_{n\_0,m\_0}[k] \\ &=& \sum\_{k=0}^{L\_{\mathfrak{g}}-1} r[k]g[k-m\_0\frac{M}{2}] \cdot e^{-j\frac{2\pi}{M}m\_0k}e^{-j\frac{\pi(m\_0+n\_0)}{2}} \end{array} \tag{5}$$

here, the upper script "∗" denotes conjugate operation. By substituting Equation (4) into Equation (5), *Dn*0,*m*0 can be further expressed as [18],

$$\begin{split} \, \_{\mathcal{D}\_{\text{H}\_{0},\text{H}\_{0}}} &= \sum\_{k=0}^{L\_{\text{g}}-1} r[k] \, \_{\mathcal{S}} [k - n\_{0} \, \_{\mathcal{D}} \mathbf{M}] \cdot e^{-j\frac{2\pi}{M} m\_{0} k} e^{-j\frac{\pi (m\_{0} + n\_{0})}{2}} \\ &= \eta\_{0,0} a\_{n0,m0} + \sum\_{\substack{(p,q) \neq (0,0)}} a\_{n0+p,m0+q} \eta\_{p,q} + \mathcal{N}\_{n0,m0} \end{split} \tag{6}$$

here, *Nn*0 ,*m*0 denotes the filter processed additive noise, and η*p*,*q* is defined as

$$\begin{array}{rcl}\eta\_{p\mathcal{A}} & = \sum\_{k} \varepsilon^{-j\frac{\pi(p+q)}{2}} \varepsilon^{-j\frac{2\pi}{M}qk} \mathcal{g}\left[k - n\_0 \frac{M}{2}\right] \\\\ \cdot \mathcal{g}^\*\left[k - \frac{M}{2}(n\_0 + p)\right] \left[\frac{1}{M} \sum\_{s=0}^{M-1} \varepsilon^{-j\frac{2\pi}{M}qs} \mathcal{e}^{j\nu\_{00}\left[s\right]}\right] \end{array} \tag{7}$$

at (*p*, *q*)=(0, 0), η0,0 is described by the following expression.

$$\begin{array}{ll} \eta\_{0,0} &= \sum\_{k} \lg\left[k - n\_0 \frac{M}{2}\right] \cdot \lg^\*[k - n\_0 \frac{M}{2}] \left| \frac{1}{M} \sum\_{s=0}^{M-1} e^{j\varphi\_{n\_0}[s]}\right| \\ &= \frac{1}{M} \sum\_{s=0}^{M-1} e^{j\varphi\_{n\_0}[s]} \end{array} \tag{8}$$

In Equation (6), η0,0 is defined as the CPE for the information symbol *an*0,*m*0 , and it leads to phase rotation of the received constellation diagram. The second item of Equation (6) is defined as the ICI, which causes divergence of the received constellation diagram. The ICI in CO-FBMC/OQAM systems is a convolution between the coefficient η*p*,*q* and PAM symbols from different FBMC/OQAM blocks, while the ICI in CO-OFDM systems is a convolution between the ICI coefficients and different sub-carriers in one OFDM symbol. Hence, it is more difficult to compensate the ICI in CO-FBMC/OQAM systems.

In many previous studies, assuming that the PN time-domain samples are the same in one FBMC/OQAM symbol period, the complex-valued received signal *Dn*0,*m*0 can be approximately rewritten as [25–29],

$$\begin{split} D\_{n0,m0} &\quad \approx \eta\_{0,0} a\_{n0,m0} + \eta\_{0,0} \sum\_{k=0}^{L\_{\mathcal{T}}-1} \sum\_{\substack{(p,q)\neq(0,0)}} a\_{n0+p,m0+q} g\_{n0+p,m0+q} [k] \mathcal{g}\_{n0,m0}^{\star} [k] + N\_{n0,m0} \\ &= \eta\_{0,0} (a\_{n0,m0} + j u\_{n0,m0}) + N\_{n0,m0} \end{split} \tag{9}$$

where *un*0,*m*0 is called the IMI in CO-FBMC/OQAM systems, and it is the interference term which results from neighboring PAM symbols on the interested (*n0*,*m0*)th position. Based on the approximation expression above, various studies have focused on the cancellation of the IMI by using pilot symbols [25–27]. However, only CPE compensation effect can be improved through the complete removal of the IMI.  So in the rest of the paper, the blind ICI compensation is proposed for the CO-FBMC/OQAM systems.

#### **3. Principle of the Phase Noise Compensation Method**

In this section, a blind PNC scheme is proposed to effectively compensate both CPE and ICI in CO-FBMC/OQAM systems. Figure 2 shows a block diagram of the proposed blind PNC scheme. In this method, a time-domain PNC model is firstly built by using the DCT approximation method, where each PN sample can be expanded as a linear combination of DCT coefficients and a group of DCT bases. To calculate these DCT coefficients in the time-domain model, it is necessary to pre-estimate the transmitted frequency-domain signals. As a result, the CPE is firstly compensated by using the M-BPS algorithm [28], and then the pre-decision process is performed on a part of the compensated signals to obtain the pre-estimated values of the transmitted signals.

**Figure 2.** Block diagram of the proposed blind phase noise compensation scheme.

After implementing PPN for filter bank analysis and FFT, the M-BPS method is firstly used to compensate the CPE. In the M-BPS method, the signal is firstly rotated by several test phases. The values of test phases are defined as φ*n*,*y* = (*y*/*Y*)· π − π/2, where *y* = 1, 2, ... , *Y*, and *Y* is the total number of test phases. The rotated version of *Dn*,*m* can be given by

$$
\hat{D}\_{n,m,y} = D\_{n,m} \cdot e^{-j\phi\_{n,y}} \tag{10}
$$

The estimated value of the CPE is obtained by minimizing the sum of distances between the hard decision of the received samples after test phase compensation and their projections on the real axis,

$$\hat{\phi}\_{n,y} = \operatorname\*{argmin}\_{\phi\_{n,y}} \sum\_{m=0}^{M-1} \left| \text{Re}(\hat{D}\_{n,m,y}) - DD(\text{Re}(\hat{D}\_{n,m,y})) \right| \tag{11}$$

where Re(·) denotes the real projection operator, and *DD*(·) is the direct pre-decision operator. Subsequently, the conventional unwrap operation is performed to partly solve the problem of phase ambiguity [42]. Then, the received PAM symbol is obtained as follows:

$$\hbar'\_{n,\mathfrak{m}} = \mathrm{Re}(\hat{D}\_{n,\mathfrak{m}}) = \mathrm{Re}(D\_{n,\mathfrak{m}} \cdot e^{-\hat{j}\hat{\phi}\_n}) \tag{12}$$

Next, the estimated values *a*ˆ*n*,*m* of the transmitted signal are recovered via the pre-decisions upon the signal *a*ˆ *n*,*m*. Since the performance of the proposed method strongly depends on the pre-decisions, the negative influence of the decision errors should be considered. Here, an effective approach is taken to reduce the decision errors. As shown in Figure 3, taking the 16-OQAM as an example, the constellation points are classified into 4 classes. When the constellation point *D*ˆ *n*,*m* falls in the border regions between two neighbouring classes, the decision errors often occur. Then, three shadow border regions are regarded as the high decision error probability regions, where the parameter δ is defined as is the width of a shadow rectangle, and it decides the selected range of the high decision error probability regions. Therefore, if *D*ˆ *n*,*m* falls in these high decision error probability regions, these signals are thrown away and not used to perform pre-decisions; while *D*ˆ *n*,*m* does not fall in the shadow regions, a hard pre-decision is made normally. However, if excessive signals are removed from the pre-decision process, this can lead to the performance degradation of the proposed method. Thus, the pre-decision parameter δ needs to be optimized to achieve good performance.

 **Figure 3.** Taking the 16-OQAM as an example, the signals with high decision error probability in three shadow regions (*a*ˆ *n*,*m* ∈ [−2 − δ/2, −2 + δ/2] ∪ [−δ/2, δ/2] ∪ [2 − δ/2, 2 + δ/2]) not being used to perform the pre-decision.

Secondly, a time-domain PNC model is built to estimate the PN with the aid of DCT. The compensated signal *r*ˆ*n*[*i*] can be obtained by multiplying the received signal *rn*[*i*] by the complex conjugate of the estimate of PN,

$$
\hat{r}\_n[i] = r\_n[i] e^{-j\phi\_n[i]} \tag{13}
$$

where ϕ*n*[*i*] denotes the *i*th time-domain PN sample in the *n*th CO-FBMC/OQAM block. The high frequency components in the PN can be neglected, and the complex conjugate of PN can be expressed as the linear combination of a group of DCT basis and DCT coefficients [37],

$$
\Phi\_n \approx \text{tr}\mathbf{C}\_n \tag{14}
$$

where **Φ***n* = [*e*−*j*ϕˆ*n*[0] ,*e*−*j*ϕˆ*n*[2] , ... ,*e*−*j*ϕˆ*n*[*KM*−1] ] *T* and *Cn* = [*Cn*(0),*Cn*(1), ... ,*Cn*(*L* − 1)]*T* are *L* × 1 unknown DCT coefficient vector. Here, [·] *T* is the transpose operator, and *L* refers to the length of DCT coefficients. The *Lg* × *L* matrix τ of DCT basis is given by

$$
\mathbf{T} = \begin{bmatrix}
\frac{\sqrt{2}}{2} & \cos\left(\frac{\pi(1+0.5)}{KM}\right) & \cos\left(\frac{2\pi(1+0.5)}{KM}\right) & \cdots & \cos\left(\frac{(L-1)\pi(1+0.5)}{KM}\right) \\
\frac{\sqrt{2}}{2} & \cos\left(\frac{\pi(2+0.5)}{KM}\right) & \cos\left(\frac{2\pi(2+0.5)}{KM}\right) & \cdots & \cos\left(\frac{(L-1)\pi(2+0.5)}{KM}\right) \\
\frac{\sqrt{2}}{2} & \cos\left(\frac{\pi(3+0.5)}{KM}\right) & \cos\left(\frac{2\pi(3+0.5)}{KM}\right) & \cdots & \cos\left(\frac{(L-1)\pi(3+0.5)}{KM}\right) \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
\frac{\sqrt{2}}{2} & \cos\left(\frac{\pi(KM+0.5)}{KM}\right) & \cos\left(\frac{2\pi(KM+0.5)}{KM}\right) & \cdots & \cos\left(\frac{(L-1)\pi(KM+0.5)}{KM}\right) \\
\end{bmatrix} \tag{15}
$$

substituting Equation (14) into Equation (13), the compensated temporal signal *r*ˆ*n*[*i*] can be rewritten as,

$$\mathfrak{r}\_n[i] = \sum\_{l=0}^{L-1} r\_n[i] \pi\_{i,l} \mathbb{C}\_n(l) \tag{16}$$

hence, the frequency domain compenstated symbol after analysis processing can be expressed as,

$$\begin{array}{lcl}\mathcal{R}\_{n,m} &=& \sum\_{\substack{i=0\\L\_{1}-1\\i\neq 0}}^{L\_{1}-1} \hat{r}\_{n}[i]g[i-n\frac{\mathsf{M}}{2}] \cdot e^{-j\frac{2\pi}{M}mi}e^{-j\frac{\pi(m+n)}{2}}\\ &=& \sum\_{\substack{i=0\\L\_{1}-1\\i=0}}^{L\_{1}-1} e^{-j\phi\_{n}[i]}r\_{n}[i] \, \mathcal{S}[i-n\frac{\mathsf{M}}{2}] \cdot e^{-j\frac{2\pi}{M}mi}e^{-j\frac{\pi(m+n)}{2}}\\ &=& \sum\_{i=0}^{L\_{1}-1} e^{-j\phi\_{n}[i]} \Big(e^{j\psi\_{n}[i]}s\_{n}[i] + wn\_{1}[i]\Big) \xi[i-n\frac{\mathsf{M}}{2}] \cdot e^{-j\frac{2\pi}{M}mi}e^{-j\frac{\pi(m+n)}{2}}\\ &\approx& \sum\_{i=0}^{L\_{1}-1} s\_{n}[i]g[i-n\frac{\mathsf{M}}{2}] \cdot e^{-j\frac{2\pi}{M}mi}e^{-j\frac{\pi(m+n)}{2}}\\ &=& \underbrace{\displaystyle}\_{A\_{n,m}} \end{array} \tag{17}$$

where *An,m* is the received symbol when the PN is perfectly compensated — i.e., the transmitted PAM symbol *an,m* can be recovered by taking real part of *An,m* — and ξ*n*,*m* is the noise term. By replacing *r*ˆ*n*[*i*] from Equation (16) into Equation (17), the symbol *R*ˆ *n*,*m* is rewritten into

$$\begin{array}{rcl} \mathcal{R}\_{n,m} & = & \sum\_{i=0}^{L\_{\mathcal{S}}-1} e^{-j\phi\_{n}[i]} r\_{n}[i] \mathcal{R}[i-n\frac{M}{2}] \cdot e^{-j\frac{2\pi}{M}mi} e^{-j\frac{\pi(m+n)}{2}} \\ & = & \sum\_{i=0}^{L\_{\mathcal{S}}-1} \sum\_{l=0}^{L-1} \mathcal{C}\_{n}(l) \tau\_{i} r\_{l}[i] \mathcal{g}[i-n\frac{M}{2}] \cdot e^{-j\frac{2\pi}{M}mi} e^{-j\frac{\pi(m+n)}{2}} \\ & = & \sum\_{l=0}^{L-1} \mathcal{C}\_{n}(l) \sum\_{i=0}^{L\_{\mathcal{S}}-1} r\_{n}[i] \mathcal{g}[i-n\frac{M}{2}] \tau\_{i} \mu e^{-j\frac{2\pi}{M}mi} e^{-j\frac{\pi(m+n)}{2}} \\ & = & \sum\_{l=0}^{L-1} \mathcal{C}\_{n}(l) V\_{n,m}^{l} \end{array} \tag{18}$$

where the symbol *Vl n*,*m* satisfies

$$V\_{n,m}^{l} = \sum\_{i=0}^{L\_{\mathcal{S}}-1} r\_n[i] g[i-n\frac{M}{2}] \pi\_{i,l} e^{-j\frac{2\pi}{M}mi} e^{-j\frac{n(m+n)}{2}} \tag{19}$$

By combining Equation (17) and Equation (18), the received symbol *An,m* can be expressed as

$$A\_{\mathfrak{n},\mathfrak{m}} \approx \sum\_{l=0}^{L-1} \mathbb{C}\_{\mathfrak{n}}(l) V\_{\mathfrak{n},\mathfrak{m}}^{l} - \underline{\xi}\_{\mathfrak{n},\mathfrak{m}} \tag{20}$$

While ignoring the noise term ξ*n*,*m*, the corresponding estimated vector *A*ˆ *n* can be approximated as,

$$
\hat{A}\_n = \mathbf{V}\_n \mathbf{C}\_n \tag{21}
$$

where *A*ˆ *n* = [*A*ˆ *n*,0, *A*ˆ *n*,1, ... , *A*ˆ *n*,*M*−1] *T* and **V***n* = [*V*0 *n*, *V*1 *n*, ... , *VL*−1 *n* ] (*m* ∈ [0, *M* − 1]). *A*ˆ*n*,*m* is the estimated value of *An,m*, and the *M* × 1 vector *Vl n* = [*Vl n*,0, *Vl n*,1, ... , *Vl n*,*M*−1] *T* is a column of the matrix **V***n*.

Taking the real part of Equation (21) in the left and right sides, the equation is changed into

$$
\hat{a}\_{ll} = \mathbb{P}\_n \mathbb{Q}\_n \tag{22}
$$

where *a*ˆ *n* = [*a*ˆ*n*,0, *a*ˆ*n*,1, ... , *a*ˆ*n*,*M*−1], **P***n* = [Re(**V***n*) − Im(**V***n*)], *Qn* = [Re(*CT n* ) Im(*CT n* )]*T* . *a*ˆ*n*,*m* is the estimated value of the transmitted PAM symbol, and Im(·) denotes the imaginary projection operator. As above-mentioned, in order to reduce the decision errors, only partially estimated values of the transmitted signal can be obtained by making a pre-decision on the the constellation points in the low decision error probability regions. When the optimized parameter δ is selected by balancing performance and complexity of the proposed algorithm, a *Zn* × 1 vector is picked out to perform the pre-decision, where *Zn* is the total number of the pre-estimated transmitted signals in the *n*th FBMC/OQAM block. The pre-estimated transmitted signal vector can be given by **S***na*ˆ *n*, where the permutation matrix **S***n* = [λ*t*1 , λ*t*2 , ... , λ*tZn* ] *T* is a *Zn* × *M* matrix, *tz* (*z* = 1, 2, ... , *Zn*) represents the subcarrier index of *z*th pre-estimated transmitted signal, and λ*tz* is a *M* × 1 vector [0, ... , 0 "--#\$--, 1, 0, ... , 0 "--#\$--] *T*.

$$\stackrel{\smile}{t\_x - 1} \qquad \qquad \stackrel{\smile}{M - t\_x}$$

Subsequently, Equation (22) can be rewritten as

$$\mathbf{S}\_n \hat{\mathbf{a}}\_n = \mathbf{S}\_n \mathbf{P}\_n \mathbf{Q}\_n \tag{23}$$

Finally, the least square (LS) solution of the unknown DCT coefficient vector can be easily found as follows:

$$\mathbf{Q}\_n = \left( (\mathbf{S}\_n \mathbf{P}\_n)^T (\mathbf{S}\_n \mathbf{P}\_n) \right)^{-1} (\mathbf{S}\_n \mathbf{P}\_n)^T (\mathbf{S}\_n \mathbf{a}\_n) \tag{24}$$

Once the DCT coefficient vector *Qn* is obtained, the PN, including the CPE and ICI, can be compensated via Equation (22).

#### **4. Modified Algorithm Based on Overlapped Symbol Structure**

Different from OFDM symbols, neighboring temporal FBMC/OQAM blocks are overlapped with each other. Figure 4 shows the time-domain FBMC/OQAM transmitted blocks with overlapped structure. The *i*th sample of *n*th received FBMC/OQAM block can be given by

$$r\_n[i] = r[i + (n-1)\frac{M}{2}], \; i \in [1, KM] \tag{25}$$

#### **Time-domain FBMC/OQAM signal sequence**

**Figure 4.** Time-domain FBMC/OQAM transmitted blocks with overlapped structure.

Then, the last *KM-M*/*2* samples of *n*th FBMC/OQAM block are the same as the first *KM-M*/*2* samples of (*n* + 1)th FBMC/OQAM block—i.e., *rn*[*j* + *M*/2] = *rn*+1[*j*], *j* ∈ [1,*KM* − *M*/2]. As a consequence, the PN sample also satisfies that ϕ*n*[*j* + *M*/2] = ϕ*n*+1[*j*], where ϕ*n*+1[*j*] is the *j*th sample of the (*n* + 1)th PN vector ϕ*n*+1. Let τ*pre* and τ*last* be the first and the last *KM-M*/*2* rows of DCT basis τ, repectively, and we can obtain that τ*lastCn* = τ*preCn*+1. Using LS estimation, the DCT coefficient vector *Cn* for *n*th CO-FBMC/OQAM block can be obtained as follows,

$$\mathbf{C}\_{n} = \left(\boldsymbol{\pi}\_{\text{last}}^{T}\boldsymbol{\pi}\_{\text{last}}\right)^{-1}\boldsymbol{\pi}\_{\text{last}}^{T}\boldsymbol{\pi}\_{\text{prc}}\mathbf{C}\_{n+1} \tag{26}$$

 If the first DCT coefficient vector *C*1 is known, the other DCT coefficient vectors can be calculated according to the recursive expression in Equation (26). It is obviously unreasonable in such a way to calculate the other DCT coefficient vectors, because the updating between the PN vectors of ϕ*n* and ϕ*n*+1 is not completely neglected. Moreover, two truncated matrices of τ*pre* and τ*last* have very small singular values owing to the circularity of the DCT basis matrix τ. This leads to the significant degradation of the accuracy in the LS estimation. Therefore, an approximate expression between the DCT coefficient vectors of *Cn* and *Cn*+1 is derived to achieve high estimate accuracy.

Although the DCT basis matrix is not an orthogonal matrix in the proposed algorithm, the following condition is still satisfied:

$$
\boldsymbol{\pi}^T \boldsymbol{\pi} = \boldsymbol{b} \mathbf{I} \tag{27}
$$

where *b* is a constant, and **I** is a *L* × *L* identity matrix. When the difference between the first and the last *M*/2 samples of ϕ*n* and ϕ*n*+1 are ignored, ϕ*n*+1 can be approximated as the *M*/2 circular shift of ϕ*n*. Hence, the terms satisfy that

$$\begin{cases} \mathsf{ \texttt{\texttt{\texttt{\texttt{C}}}}} = q\mathsf{\texttt{\texttt{\texttt{z}}}} \approx \mathfrak{\mu}\_{-1}q\mathsf{\texttt{\texttt{\texttt{z}}}+1} = \mathfrak{\mu}\_{-1}\mathsf{\texttt{\texttt{\texttt{\texttt{\texttt{\texttt{\texttt{z}}}}}}}\mathsf{\texttt{\texttt{\texttt{\texttt{\texttt{z}}}}}}}{\texttt{\texttt{\texttt{\texttt{\texttt{z}}}}}} \mathbf{\texttt{\texttt{\texttt{\texttt{z}}}}} \end{cases} \tag{28}$$

where μ−1 and μ+1 are two *Lg* × *Lg* circular shift matrices,

$$\begin{cases} \begin{array}{cc} \mathsf{u}\_{-1} = \begin{bmatrix} \, 0\_{\frac{\mathsf{d}}{2} \times (L\_{\mathfrak{x}} - \frac{\mathsf{d}}{2})} & \mathbf{I}\_{\frac{\mathsf{d}}{2}} \\ \mathbf{I}\_{L\_{\mathfrak{x}} - \frac{\mathsf{d}}{2}} & \mathbf{O}\_{(L\_{\mathfrak{x}} - \frac{\mathsf{d}}{2}) \times \frac{\mathsf{d}}{2}} \\ \mathbf{0}\_{(L\_{\mathfrak{x}} - \frac{\mathsf{d}}{2}) \times \frac{\mathsf{d}}{2}} & \mathbf{I}\_{L\_{\mathfrak{x}} - \frac{\mathsf{d}}{2}} \\ \mathbf{I}\_{\frac{\mathsf{d}}{2}} & \mathbf{O}\_{\frac{\mathsf{d}}{2} \times (L\_{\mathfrak{x}} - \frac{\mathsf{d}}{2})} \end{bmatrix} \end{array} \tag{29}$$

here, **0***m* × *n* denotes a *m* × *n* zero matrix, and **I***n* is a *n* × *n* identity matrix. In combination with Equation (27), Equation (28) can be further expressed as

$$\begin{cases} \mathbf{C}\_{n} = \mathbf{y}\_{-1} \mathbf{C}\_{n+1} \\ \mathbf{C}\_{n+1} = \mathbf{y}\_{+1} \mathbf{C}\_{n} \end{cases} \tag{30}$$

$$\begin{cases} \mathbf{y}\_{-1} = \frac{1}{b} \mathbf{\bar{\pi}}^T \boldsymbol{\mu}\_{-1} \mathbf{\bar{\pi}}\\ \mathbf{y}\_{+1} = \frac{1}{b} \mathbf{\bar{\pi}}^T \boldsymbol{\mu}\_{+1} \mathbf{\bar{\pi}} \end{cases} \tag{31}$$

Assuming that three neighboring FBMC/OQAM blocks participate in the estimation process, Equation (21) can be expanded as

$$
\begin{bmatrix}
\hat{A}\_{n-1} \\
\hat{A}\_n \\
\hat{A}\_{n+1}
\end{bmatrix} = \begin{bmatrix}
\mathbf{V}\_{n-1}\mathbf{y}\_{-1} \\
\mathbf{V}\_n \\
\mathbf{V}\_{n+1}\mathbf{y}\_{+1}
\end{bmatrix} \mathbf{C}\_n \tag{32}
$$

The matrices of *an* and **P***n* are separately defined as *an* = [*a*ˆ*T n*−1, ˆ*aT n* , ˆ*aT n*+1] *T* and

$$
\overline{\mathbf{P}}\_{\rm ll} = \begin{bmatrix}
\operatorname{Re}(\mathbf{V}\_{n-1}\mathbf{y}\_{-1}) & -\operatorname{Im}(\mathbf{V}\_{n-1}\mathbf{y}\_{-1}) \\
\operatorname{Re}(\mathbf{V}\_{\rm n}) & -\operatorname{Im}(\mathbf{V}\_{\rm n}) \\
\operatorname{Re}(\mathbf{V}\_{n+1}\mathbf{y}\_{+1}) & -\operatorname{Im}(\mathbf{V}\_{n+1}\mathbf{y}\_{+1})
\end{bmatrix}.
\tag{33}
$$

Similar to Equations (22)–(24), the LS solution of the unknown DCT coefficient vector can be obtained from the following expression,

$$Q\_n = \left( \left( \overline{\mathbf{S}}\_n \overline{\mathbf{P}}\_n \right)^T \left( \overline{\mathbf{S}}\_n \overline{\mathbf{P}}\_n \right) \right)^{-1} \left( \overline{\mathbf{S}}\_n \overline{\mathbf{P}}\_n \right)^T \left( \overline{\mathbf{S}}\_n \overline{a}\_{\text{il}} \right) \tag{34}$$

where **S***n* = [**S***T n*−1, **S***T n* , **S***T n*+1] *T* .

In the modified proposed algorithm, we throw away the first *M*/2 samples of the (*n* − 1)th PN vector ϕ*n*−1 and the last *M*/2 samples of the (*n* + 1)th PN vector ϕ*n*+1; in spite of this, the relatively small power is on both ends of impulse response of the prototype filter so that the approximation affects rarely its performance, as the vector *Cn* is obtained in such a way, and the vectors of *Cn* − 1 and *Cn* + 1 are easily calculated using the recurrence relations in Equations (30)–(31). Therefore, only one inverse operation in Equation (34) is required for every three neighboring FBMC/OQAM blocks, and the complexity can be decreased by the simplified operation. The detailed complexity analysis will be described in the next section. The acronyms used in this article are summarized in the following Table 1.

#### **5. Simulation Results and Discussion**

A 30 GBaud BTB CO-FBMC/OQAM transmission system is simulated to evaluate the performance of the proposed blind PNC scheme, which is built by Optisystem 17.0 and MATLAB. Firstly, a pseudo-random bit sequence with a word length of 217 is mapped into m-QAM and the symbol rate is set to 30 GBaud/s. All the complex-valued QAM symbols are converted into the real-valued PAM symbols through QAM to PAM module. The base-band time-domain FBMC/OQAM symbols are generated after all PAM data sequences through IFFT and FBS-PPN. The pulse length of the square root raised cosine filter is chosen as *Lg* = 4*M*, and its roll-off coefficent α = 1.

Then, the base-band FBMC/OQAM signals are modulated onto an optical carrier at 1550 nm using a Mach–Zehnder modulator with a pair of independently controllable branches. At the receiver, the optical signal is coherently detected in a 90◦ optical hybrid, and the received analog signals are converted into digital signals by analog-to-digital converters with two samples per symbol. Subsequently, the obtained digital symbols are fed into the offline DSP module. With time synchronization being perfectly implemented, all the time-domain symbols are converted to the frequency domain by the PPN for filter bank analysis and FFT. In this paper, perfect channel equalization is considered, and specifically, we suppose that the channel response is flat and time-invariant on each carrier [27]. Next, the blind discrete-cosine-transform-based phase noise compensation (BD-PNC) is implemented, including CPE and ICI compensation. In the related M-BPS algorithm, the total numbers of test phases for 4/16/64-QAM are chosen as 16/32/64, respectively [28]. The original binary bits can be recovered after m-OQAM and m-QAM de-mapping, and the bit error rate (BER) is estimated by comparing them with the transmitted bits.

The effects of two parameters of δ and *L* on the performance of the algorithm are investigated in Figure 5a,b. δ is the width of shadow rectangle in a high decision error probability region, which determines how many signals after CPE compensation join the pre-decision process. As mentioned in Section 3, δ = 0 means that all the data are used to perform the pre-decision while δ = 2 implies that only the leftmost and rightmost data in the I-Q plane participate in the process in Figure 3. The decision error increases with the participation of more data after the CPE compensation. In Figure 5a, δ is optimized at the length of DCT basis *L* = 2 for the CO-FBMC 4/16/64-QAM BTB system with 512 subcarriers. When the value of δ is changed from 0 to 1.8, it can be seen that the BER performance does not degrade significantly due to the reduction in decision error probability. Moreover, with the increase in the value of δ, the participation of a low number of data benefits greatly to complexity reduction in the proposed algorithm. Hence, δ = 1.8 is adopted in the remainder of this paper. *L* denotes the length of the DCT basis. Generally, under a large value of *L*, we can obtain a more accurate approximation of the PN by the DCT in Equation (14). Figure 5b displays the BER performance as a function of *L* at the linewidth and symbol duration product Δν · *TS* = 10−2 with 12 dB optical signal-to-noise ratio (OSNR) and 256/512/1024 subcarriers when using the proposed BD-PNC algorithm and its modified version. The modified algorithm is developed by considering the overlapped symbol structure, which is called the M-BD-PNC scheme. For the BD-PNC, with the increase in *L*, the BER performance makes a small improvement, and is even deteriorated for the 256/512 subcarrier FBMC/OQAM systems. While using the M-BD-PNC method, there is a significant improvement for its BER performance as the value of *L* is increased. The PN estimation precision depends on the accuracy of the DCT coefficients estimated by the LS method. In Equation (24) or Equation (34), if the matrix **S***n***P***n* or **S***n***P***n* does not meet the column full-rank condition, its LS solution could produce a large error. With the increase in the number of subcarriers, more rows in the matrix allow it to meet the column full-rank condition more easily. Hence, when *L* is increased, the proposed BD-PNC algorithm achieves better performance under a large number of subcarriers; its performance becomes worse in the system with a low number of subcarriers, while in the M-BD-PNC method, **P***n* is the extended version of **P***n* so that it is easier for the matrix **S***n***P***n* to satisfy a column full-rank term. In this case, the PN estimation precision is almost not negatively influenced by the LS method, and a large value of *L* contributes to the improvement of the PN estimation precision. Therefore, for the M-BD-PNC, the BER performance exhibits a better result at a large *L*. The optimized values of *L* in the BD-PNC and M-BD-PNC are selected as 2 and 10, respectively.

The CO-FBMC/OQAM system with a large number of subcarriers is robust against fiber CD [12]. However, the PN becomes strong with the increase in the number of subcarriers. It is always a challenge for the existing PNC algorithm to achieve good performance under a large number of subcarriers [18,25–29]. So the performance of the proposed algorithm is evaluated in the BTB system with a large number of subcarriers. In Figure 5c, the influence of the number of subcarriers on the performance of the method is investigated at *L* = 2 and δ = 1.8 using M-BPS and BD-PNC for the CO-FBMC 4/16/64 QAM systems with several different values of Δν · *TS* and OSNR. When the number of subcarriers becomes larger, on one hand, the effect of AWGN is better averaged so that their performance is improved rapidly; on the other hand, the FBMC/OQAM symbol duration increases, and the PN has a larger variance and becomes stronger. As a result, it can be seen that the BER using M-BPS starts to deteriorate at *M* = 1024. Note that the performance using BD-PNC remains almost unchanged when the number of subcarriers is increased from 128 to 1024 for 4/16/64-QAM modulation order. The simulation results prove that its performance is not sensitive to the number of subcarriers in the CO-FBMC/OQAM systems. Under the large number of subcarriers, the proposed algorithm is far superior to the M-BPS. For the 4-QAM modulation at *M* = 1024 with Δν · *TS* = 5 × 10−2 and 11 dB OSNR, the BER performance of BD-PNC is improved by more than one order of magnitude compared to the M-BPS.

**Figure 5.** (**a**) BER versus the width of shadow rectangle in a high decision error probability region, δ. (**b**) BER versus the length of DCT coefficient, *L*. (**c**) BER versus the number of subcarriers, *M*.

Figure 6a depicts one realization of the real PN and its estimation within six consecutive FBMC/OQAM blocks at the linewidth and symbol duration product Δν · *TS* = 1.5 × 10−2 and an OSNR of 20 dB for 1024 sub-carrier 16-QAM CO-FBMC systems using the BD-PNC and M-BPS. For these time-domain overlapped symbols, the CPE estimated by the M-BPS is only a rough estimate of the PN, which is far from the real PN. The proposed method shows a better phase tracking capability due to the ICI compensation in time-domain. Figure 6b shows the illustration of ICI influence and the necessity of PN estimation in time domain for CO-FBMC/OQAM systems. The constellation diagrams before and after PNC are separately displayed using M-BPS and BD-PNC with an OSNR of 27 dB and combined laser linewidth of 150 kHz. The PN in time domain, including CPE and ICI, is compensated by using BD-PNC, while only CPE is compensated by using M-BPS. It is clear that the PN in time domain is compensated by the proposed BD-PNC method resulting in an improvement of about 125 times in terms of BER over the M-BPS. Moreover, the BD-PNC achieves a root-mean-square error vector magnitude (*EVMRMS*) of 4.7%, which is much lower than the *EVMRMS* of 7% by the M-BPS.

Supposing that the pre-forward error correction BER should be lower than 3.8 × 10−3 for hard-decision forward error correction (HD-FEC) [43], the performance of these PNC algorithms is investigated at the BER target limit. Figure 7a–c show the OSNR penalty as a function of the linewidth and symbol duration product Δν · *TS* using M-BPS and the proposed algorithms for CO-FBMC/OQAM BTB transmission system with 256, 512, and 1024 subcarriers, respectively. The optimal numbers of test phases in the related M-BPS scheme are chosen as 16/32/64 for 4/16/64-QAM modulation order, respectively [28]. When the number of subcarriers is 256 or 512, the length *L* of DCT basis is set to 2. The length of DCT basis *L* is selected as 10 with total number of subcarriers *M* = 1024. The parameter δ is set to 1.8 in this paper. The OSNR penalty is defined as the difference between the required OSNR for a BER of 3.8 × 10−3 at a current laser linewidth and zero laser linewidth in CO-FBMC/OQAM systems, where the PNC algorithm is used in the former, and none of the PNC methods is applied in the PN free system in the latter. When no PN exists in the system, the reference OSNRs for 4/16/64-QAM

30 Gbaud signal at the HD-FEC threshold are ∼4.4, ∼10, and∼16.8 dB, respectively. For comparison, an acceptable maximum of 1 dB OSNR penalty is regarded as a threshold value [27].

On the whole, the proposed algorithm provides significantly better performance than M-BPS. In Figure 7a, at 1 dB OSNR penalty threshold, with the number of subcarriers *M* = 256, the M-BPS achieves a Δν · *TS* tolerance of 3.5 × 10−2, 7 × 10−3, and 10−3 for 4/16/64-QAM 30 Gbaud signal, respectively. While using the proposed BD-PNC method, under 1 dB OSNR penalty constraint, the maximum Δν · *TS* that can be achieved for 4/16/64-QAM are 5 × 10−2, 10−2, and 1.8 × 10−3, respectively. With the increase in the QAM modulation order, the difference between the performance of the two methods is more remarkable. At higher order modulation format, the ICI effect becomes stronger, and the BD-PNC scheme achieves more significant results than M-BPS. In Figure 7b, at *M* = 512, M-BPS and BD-PNC achieve a Δν · *TS* tolerance of 5.6 × 10−4 and 1.4 × 10−3 for 64-QAM 30 Gbaud signal, respectively.

The studies focus on the investigation of the proposed BD-PNC algorithm's performance with the number of subcarriers *M* = 1024. As a comparison, the performance of the M-BD-PNC algorithm is also evaluated in the 30 GBaud BTB CO-FBMC/OQAM systems with 1024 subcarriers. In Figure 7c, the M-BD-PNC achieves a better Δν · *TS* tolerance than the BD-PNC. This is because the column full-rank condition is easily satisfied in the M-BD-PNC scheme, and its performance benefits from the enhanced estimation accuracy of DCT coefficients by the LS method. Moreover, the M-BD-PNC has a lower CC than the corresponding BD-PNC algorithm, and a detailed analysis of their CC will be described at the end of the section. With a large number of subcarriers, the ICI components in PN have a huge negative impact on the performance. With the ICI effect mitigated, the proposed algorithms offer a significantly better performance compared to the M-BPS under the same QAM modulation order. In particular, the BD-PNC and M-BD-PNC achieve tolerated linewidth and symbol duration products of 2.2 × 10−2 and 2.6 × 10−3 for 64-QAM 30 Gbaud signal, while the M-BPS can only achieve a Δν · *TS* tolerance of 10−3.

Finally, we compare the CC of M-BPS, the proposed BD-PNC, and M-BD-PNC algorithms in terms of the number of real multiplications in one FBMC/OQAM block. Since the M-BD-PNC method involves several neighboring FBMC/OQAM blocks, its CC is evaluated as the average CC per block. As shown in Table 2, the CC of the proposed BD-PNC and M-BD-PNC algorithms are compared. According to the block diagram in Figure 2, the CC of the proposed BD-PNC involves the CC of its four DSP steps, including M-BPS, time-domain PNC, calculation of DCT coefficients, and final compensation. The CC of the M-BPS method is given by *O*(2*MY*) [28]. The time-domain PNC model is described by Equation (19), and its CC is *O*(*LM*log2*M*+*LKM*). For the BD-PNC, the DCT coefficients are calculated in Equation (24), and its CC is given by *O*(8*L*3 + 8*ZnL*2 + 2*ZnL*), where *Zn* is the total number of pre-estimated transmitted signals in the *n*th FBMC/OQAM block, while in the M-BD-PNC, the DCT coefficients are obtained by Equations (30) and (34), and real multiplications needed in these equations are *O*(8*L*3/3 + 8*ZnL*2 + 2*ZnL* + 2*L*2/3). For BD-PNC and M-BD-PNC, the final PNC is completed by using Equation (22), and its CC is *O*(2*ML*). In order to compare the CC of these algorithms directly, several simulations are carried out to investigate the average value of *Zn* at a BER of 3.8 × 10−3. In this case, the average value of *Zn* is shown in Table 3. Consequently, the required real multiplication numbers per block are counted by type, as shown in Table 4. As a comparison, when all the data after CPE compensation participate in the pre-decision (δ = 0), the required real multiplication numbers of BD-PNC or M-BD-PNC is provided in parentheses. It is observed that the optimization of the parameter δ in the pre-decision process results in a remarkable decrease in complexity. Furthermore, it is clear that the CC of proposed BD-PNC or M-BD-PNC is not significantly higher than M-BPS, and their CC is less than four times higher than that of M-BPS for 1024 subcarriers 64-QAM 30 Gbaud signals. Nevertheless, the proposed algorithms have a great advantage over M-BPS in terms of the PNC performance. The CC of M-BD-PNC is slightly lower than that of BD-PNC, and its performance is even superior to the BD-PNC algorithm. Therefore, at *M* = 1024, the proposed M-BD-PNC should be the first choice.

**Figure 6.** (**a**) One realization of the real phase noise and its estimations after the M-BPS and BD-PNC methods at Δν · *TS* = 1.5 × 10−2 and OSNR = 20 dB for 30 Gbaud CO-FBMC 16-QAM BTB transmission systems with 1024 subcarriers. (**b**) Illustration of constellations before and after PNC employing M-BPS and BD-PNC at Δν · *TS* = 5 × 10−3 and OSNR = 27 dB for 30 Gbaud CO-FBMC 64-QAM BTB systems with 1024 subcarriers.

**Figure 7.** OSNR penalty versus Δν · *TS* at a BER of 3.8 × 10−3 using M-BPS, the proposed schemes for CO-FBMC 4/16/64-QAM systems with 256 subcarriers (**a**), 512 subcarriers (**b**), and 1024 subcarriers (**c**).

**Table 2.** The CC of the proposed BD-PNC and M-BD-PNC algorithms.

Note: *M*: number of subcarriers. *Y*: number of test phases. *L*: length of DCT coefficient. *K*: overlap factor. *Zn*: total number of pre-estimated transmitted signals.

**Table 3.** The average value of *Zn*.

**Table 4.** The required real multiplication numbers per block using several algorithms.

#### **6. Conclusions**

In this paper, we have presented a blind time-domain PNC scheme for the CO-FBMC/OQAM transmission, where the PN sample is approximately expanded by DCT. In the proposed algorithms, the PN in time-domain can be compensated by obtaining their corresponding DCT coefficients. The pre-decision process is firstly performed over the selected signals after CPE compensation by M-BPS, and the decision results are used to calculate these DCT coefficients. This selective decision process reduces the CC largely. Considering the overlapped symbol structure, the modified version of the proposed algorithm is reported with its performance improvement and complexity decrease. Since the proposed algorithms are devoted to removing the ICI effect, they achieve significant performance improvement compared to M-BPS for a simulated 30 GBaud CO-FBMC/OQAM system, especially with a large number of subcarriers and high order QAM modulation format. The algorithm complexity analysis shows that its CC is slightly higher than M-BPS.