Quaternion MLP Neural Networks Based on the Maximum Correntropy
  Criterion
We propose a gradient ascent algorithm for quaternion multilayer perceptron
(MLP) networks based on the cost function of the maximum correntropy criterion
(MCC). In the algorithm, we use the split quaternion activation function based
on the generalized Hamilton-real quaternion gradient. By introducing a new
quaternion operator, we first rewrite the early quaternion single layer
perceptron algorithm. Secondly, we propose a gradient descent algorithm for
quaternion multilayer perceptron based on the cost function of the mean square
error (MSE). Finally, the MSE algorithm is extended to the MCC algorithm.
Simulations show the feasibility of the proposed method.
Gang Wang, Xinyu Tian, Zuxuan Zhang
# Quaternion MLP Neural Networks Based on the Maximum Correntropy Criterion

###### Abstract

We propose a gradient ascent algorithm for quaternion multilayer perceptron (MLP) networks based on the cost function of the maximum correntropy criterion (MCC). In the algorithm, we use the split quaternion activation function based on the generalized Hamilton-real quaternion gradient. By introducing a new quaternion operator, we first rewrite the early quaternion single layer perceptron algorithm. Secondly, we propose a gradient descent algorithm for quaternion multilayer perceptron based on the cost function of the mean square error (MSE). Finally, the MSE algorithm is extended to the MCC algorithm. Simulations show the feasibility of the proposed method.

 Generalized Hamilton-real-least mean square-quaternion gradient-Wirtinger calculus

## I Introduction

Tultilayer (MLP) neural networks are the most popular neural networks. The connection weights can be adjusted to represent the input-output relations. Recently MLP neural network models have been explored based on quaternions in [1, 2, 3, 4, 5, 6], because the 4-D signals in the quaternion domain can incorporate mutual information to reduce complexity. Applications of quaternion neural networks include image classification [7, 8, 9], signal classification [10, 11], image retrieval [12], the control problem [13, 14], the prediction of time series [15, 16, 17], forecasting three-dimensional wind signals [35], pose estimation [18, 19] and convolutional neural networks [20, 21, 22]. The adjusting process of quaternion connection weights is also called learning. Quaternion learning algorithms can be loosely divided into not-gradient-based methods, such as extreme learning machines [23, 24, 25], and gradient-based methods, such as least-mean-square (LMS) algorithms [26]. This paper is focused on gradient-based methods. When the LMS algorithm, or the back-propagation algorithm, of MLP neural networks was generalized from the real domain to the complex domain, the analyticity of the activation functions had to be studied [27, 28, 29]. If an activation function is analytic, then it is a constant function. As an alternative, a split complex activation function was proposed for the MLP. Following the split idea in the complex domain, a split quaternion activation function was proposed in and used in a nonlinear filter, or single layer perceptron(SLP). In addition to the split function, in and, another activation function was introduced using a local analyticity condition for a quaternion nonlinear filter and echo state networks, respectively. However, the quaternion gradient in misused the non-commutative property, and it prevented the extension of the quaternion SLP to the MLP. From 2014 to 2019, the correct quaternion gradient was proposed using three approaches: generalized Hamilton-real (GHR) calculus [30, 31, 32], the quaternion product [33], and the quaternion involutions [34]. Hitzer detected the gradient mistake in, and obtained corrections based on Clifford calculus. The LMS algorithm is based on the cost function of the mean square error (MSE), and works well in Gaussian additive noise, whose higher statistics are constants. In real scenarios, non-Gaussian problems arise [36]. The signals are often contaminated by non-Gaussian noise or impulsive noise. The performance of the above LMS algorithm may be poor in non-Gaussian noise. Recently, information theoretic learning was proposed for non-Gaussian noise. The proposed cost functions include [12] the maximum correntropy criterion (MCC), improved least sum of exponentials, and least mean kurtosis. The MCC is robust to large outliers or impulsive noise, and has been applied in quaternion filters. This motivates us to study quaternion MLP networks based on the MCC [18, 19]. In this paper, we revisit the split quaternion activation function based on the GHR quaternion gradient, and rewrite the quaternion nonlinear filter algorithm by introducing a new quaternion operator. Then we propose two LMS algorithms for the quaternion MLP based on the MSE and MCC. Simulations show the feasibility of the proposed method [37, 38].

## II Preliminaries

### _Quaternion Algebra_

The quaternion domain is a non-commutative extension of the complex domain. A quaternion variable consists of a real part and three imaginary components and can be expressed as

\[\mathrm{q}=\mathrm{q}_{\mathrm{a}}+\iota\mathrm{q}_{\mathrm{b}}+J\mathrm{q}_{ \mathrm{c}}+\kappa\mathrm{q}_{\mathrm{d}}\]

The quaternion product is non-commutative, that is, but Given we have

\[\left\{\begin{array}{l}\iota J=\kappa,J\kappa=\iota,\kappa\iota=J,\\ \iota J\kappa=\iota^{2}=J^{2}=\kappa^{2}=-1.\end{array}\right.\]

\[\mathrm{q}_{1}\mathrm{q}_{2}\neq\mathrm{q}_{2}\mathrm{q}_{1}.\]

Three perpendicular quaternion involutions are given by

\[\left\{\begin{array}{l}\mathrm{q}^{t}=-\iota\mathrm{q}_{t}=\mathrm{q}_{ \mathrm{a}}+\iota\mathrm{q}_{\mathrm{b}}-J\mathrm{q}_{\mathrm{c}}-\kappa \mathrm{q}_{\mathrm{d}},\\ \mathrm{q}^{J}=-J\mathrm{q}_{\mathrm{a}}-\iota\mathrm{q}_{\mathrm{b}}+J \mathrm{q}_{\mathrm{c}}-\kappa\mathrm{q}_{\mathrm{d}},\\ \mathrm{q}^{\kappa}=-\kappa\mathrm{q}_{\mathrm{c}}=\mathrm{q}_{\mathrm{a}}- \iota\mathrm{q}_{\mathrm{b}}-J\mathrm{q}_{\mathrm{c}}+\kappa\mathrm{q}_{\mathrm{ d}}.\end{array}\right.\]
Then the quaternion conjugate operation can be expressed as

\[\mathrm{q}^{*}=\mathrm{q}_{\mathrm{a}}-\iota\mathrm{q}_{\mathrm{b}}-J\mathrm{q}_{ \mathrm{c}}-\kappa\mathrm{q}_{\mathrm{d}}=\frac{1}{2}\left(\mathrm{q}^{\iota}+ \mathrm{q}^{J}+\mathrm{q}^{\kappa}-\mathrm{q}\right).\]

The norm of a quaternion is defined as

\[\left\|\mathrm{q}\right\|_{2}^{2}=\mathrm{q}\mathrm{q}^{*}=\mathrm{q}_{\mathrm{ a}}^{2}+\mathrm{q}_{\mathrm{b}}^{2}+\mathrm{q}_{\mathrm{c}}^{2}+\mathrm{q}_{ \mathrm{d}}^{2}.\]

\(HR\) calculus is an extension of Wirtinger calculus, which is also called calculus. calculus provides a simple and straightforward approach to calculating derivatives with respect to complex parameters. HR calculus comprises the HR derivatives

\[\frac{\partial J}{\partial\mathrm{q}^{*}}=\frac{1}{4}\left(\frac{\partial J}{ \partial\mathrm{q}_{\mathrm{a}}}+\frac{\partial J}{\partial\mathrm{q}_{b}} \iota+\frac{\partial J}{\partial\mathrm{q}_{c}}J+\frac{\partial J}{\partial \mathrm{q}_{d}}\kappa\right).\]

### Quaternion Split Product

Definition 1. (Quaternion Split Product ) For two quaternions, \(x,y\), the quaternion split product is defined as

\[x\odot y=y\odot x=x_{\mathrm{a}}y_{\mathrm{a}}+\iota\mathrm{x}_{\mathrm{b}}y_ {\mathrm{b}}+Jx_{\mathrm{c}}y_{\mathrm{c}}+\kappa x_{\mathrm{d}}y_{\mathrm{d}} \in H^{\mathrm{m}},\]

where represents the dot product. For two quaternions,, the quaternion split product is defined as

\[\mathrm{x}\odot y=y\odot\mathrm{x}=\mathrm{x}_{\mathrm{a}}y_{\mathrm{a}}+ \iota\mathrm{x}_{\mathrm{b}}y_{\mathrm{b}}+J\mathrm{x}_{\mathrm{c}}y_{\mathrm{ c}}+\kappa\mathrm{x}_{\mathrm{d}}y_{\mathrm{d}}\in H^{\mathrm{m}}\]

The quaternion split product has the following property,

\[\left(x\odot y\right)^{*}=x\odot y^{*}=x^{*}\odot y=y^{*}\odot x=y\odot x^{*}.\]

### Nonlinear Filtering or SLP

When the quaternion nonlinear filtering problem is considered in the discrete time domain i, there is an input quaternion vector \(\mathbf{u}\left(i\right)\in H^{n}\) with the unknown original quaternion parameter \(\mathbf{w}^{\nu}\in H^{n}\) and the desired response \(d\left(i\right)\in H^{1}\) as follows:

\[\left\{\begin{array}{l}\mathrm{x}=\mathbf{w}^{H}u\left(\mathrm{i}\right),\\ \mathrm{x}=\mathrm{x}_{\mathrm{a}}+\iota\mathrm{x}_{\mathrm{b}}+J\mathrm{x}_{ \mathrm{c}}+\kappa\mathrm{x}_{\mathrm{d}}\in H^{1},\\ \mathbf{w}=\mathbf{w}_{\mathrm{a}}+\iota\mathbf{w}_{\mathrm{b}}+J\mathbf{w}_{ \mathrm{c}}+\kappa\mathbf{w}_{\mathrm{d}}\in H^{n},\\ u\left(\mathrm{i}\right)=\mathrm{u}_{\mathrm{a}}+u\mathrm{b}_{\mathrm{b}}+Ju_{ \mathrm{c}}+\kappa u_{\mathrm{d}}\in H^{n}.\end{array}\right.\]

The error signal for the quaternion linear filter is defined as

\[e\left(i\right)=d\left(i\right)-\Phi\left[\mathbf{w}^{H}\mathbf{u}\left(i \right)\right].\]

The quaternion nonlinear function or the activation function is defined as

\[\Phi\left(x\right)=\phi\left(x_{a}\right)+\phi\left(x_{b}\right)\iota+\phi \left(x_{c}\right)J+\phi\left(x_{d}\right)\kappa,\]

where

\[\left\{\begin{array}{l}\phi\left(a\right)=\tanh\left(a\right),\\ \frac{\partial\phi\left(a\right)}{\partial\mathrm{a}}=\mathrm{sech}^{2}\left( a\right).\end{array}\right.\]

The gradient of (13) is

\[\begin{array}{l}4\frac{\partial\Phi}{\partial x^{*}}=\mathrm{sech}^{2}\left( x_{a}\right)+\mathrm{sech}^{2}\left(x_{b}\right)\iota\\ +\mathrm{sech}^{2}\left(x_{c}\right)J+\mathrm{sech}^{2}\left(x_{d}\right)\kappa.\end{array}\]

The cost function of the MSE is defined as to estimate the parameter

\[J(\mathbf{w})=e\left(i\right)\left(d^{*}\left(i\right)-\Phi^{*}\left[x\right] \right)=\left(d\left(i\right)-\Phi\left[x\right]\right)e^{*}\left(i\right).\]

The corresponding gradient is defined as

\[\frac{\partial J}{\partial\mathbf{w}^{*}}=\frac{1}{4}\left(\frac{\partial J}{ \partial\mathbf{w}_{a}}+\frac{\partial J}{\partial\mathbf{w}_{b}}\iota+\frac{ \partial J}{\partial\mathbf{w}_{c}}J+\frac{\partial J}{\partial\mathbf{w}_{d} }\kappa\right).\]

The quaternion gradient descent algorithm was proposed in [35]

\[\begin{array}{l}\mathbf{w}\left(i{+}1\right)=\mathbf{w}\left(i\right)+\eta \mathbf{u}\left(i\right)e^{*}\left(i\right)=\mathbf{w}\left(i\right)+\\ \eta\mathbf{u}\left[\mathrm{sech}^{2}\left(x_{a}\right)e_{a}-\mathrm{sech}^{2} \left(x_{b}\right)e_{b}\iota-\mathrm{sech}^{2}\left(x_{c}\right)e_{c}J-\mathrm{ sech}^{2}\left(x_{d}\right)e_{d}\kappa\right],\end{array}\]

where is the step size. Using the definition of the quaternion split product (8), we rewrite it as

\[\mathbf{w}\left(i{+}1\right)=\mathbf{w}\left(i\right)+\eta\mathbf{u}\left(i \right)\left[\frac{\partial\Phi}{\partial x^{*}}\odot e^{*}\left(i\right)\right],\]

which has a similar expression to that in the complex domain.

## 3 Quaternion MLP Based on The MSE

Fig. 1 shows the schematic diagram of the MLP. The following expression is used:

\[\left\{\begin{array}{l}y=W\mathrm{H}x+p,\\ \Psi\left(y\right)=\Psi\left(W^{\mathrm{H}}x+p\right),\\ \mathrm{z}=v^{H}\Psi\left(y\right)+\mathrm{q},\\ h_{W,v}\left(x\right)=\Phi\left(z\right)=\Phi\left[v^{H}\Psi\left(y\right)+ \mathrm{q}\right],\\ =\Phi\left[v^{H}\Psi\left(W^{\mathrm{H}}x+p\right)+\mathrm{q}\right],\end{array}\right.\]

where

\[\left\{\begin{array}{l}y=\left[\begin{array}{cccc}\mathrm{y}=\left[\begin{array}{ cccc}\mathrm{y}_{1}&\mathrm{y}_{2}&\cdots&\mathrm{y}_{m}\\ W=\left[\begin{array}{cccc}w_{1}&w_{2}&\cdots&w_{n}\\ \end{array}\right]\in H^{\mathrm{m}\times\mathrm{n}},\\ p=\left[\begin{array}{cccc}\mathrm{p}_{1}&\mathrm{p}_{2}&\cdots&\mathrm{p}_{m}\\ v=\left[\begin{array}{cccc}\mathrm{v}_{1}&\mathrm{v}_{2}&\cdots&\mathrm{v}_{m} \\ \end{array}\right]\in H^{\mathrm{n}},\\ \mathrm{y}_{\mathrm{i}}=w_{\mathrm{i}}^{H}x+\mathrm{p}_{\mathrm{i}}\mathrm{i}=1,2, \cdots\mathrm{n},\\ \mathrm{z}_{\mathrm{d}}\mathrm{,q}\Phi\left(z\right)\in H^{1}\mathrm{,x}\in H^{ \mathrm{m}}.\end{array}\right.\end{array}\right.\]

The activation functions have the same expressions:

\[\left\{\begin{array}{l}\Psi\left(\mathbf{y}\right)=\phi\left(\mathbf{y}_{a} \right)+\phi\left(\mathbf{y}_{b}\right)\iota+\phi\left(\mathbf{y}_{c}\right)J+ \phi\left(\mathbf{y}_{d}\right)\kappa,\\ \Phi\left(z\right)=\phi\left(z_{a}\right)+\phi\left(z_{b}\right)\iota+\phi\left(z _{c}\right)J+\phi\left(z_{d}\right)\kappa.\end{array}\right.\end{array}\right.\]

Figure 1: Schematic diagram of the MLP
The cost function is defined as to estimate the parameters W, V, p and q:

\[\left\{\begin{array}{l}J\left(\mathbf{W},\mathbf{V},\mathbf{p},q\right)=ee^{*}=e \left(d^{*}-\Phi^{*}\left[z\right]\right)\mathbf{=}\left(d-\Phi\left[z\right] \right)e^{*},\\ e=d-\Phi\left[z\right].\end{array}\right.\]

### Quaternion Gradient for V and q

The gradients of v and q are similar to the gradient of w in the above nonlinear filter, and can be obtained by (18) directly as follows:

\[-4\frac{\partial J}{\partial\mathbf{v}^{*}}=2\left[\mathrm{sech}^{2}\left(z_ {a}\right)e_{a}+\mathrm{sech}^{2}\left(z_{b}\right)e_{b}t\right.\]

\[\left.\qquad+\mathrm{sech}^{2}\left(z_{c}\right)e_{c}J-\mathrm{sech}^{2}\left(z _{d}\right)e_{a}\kappa\right]\]

\[=2\frac{\partial\Phi\left(z\right)}{\partial z^{*}}\odot e,\]

\[-4\frac{\partial J}{\partial\mathbf{v}^{*}}=2\Psi\left(\mathbf{y}\right)\left[ \mathrm{sech}^{2}\left(z_{a}\right)e_{a}-\mathrm{sech}^{2}\left(z_{b}\right)e _{b}t\right.\]

\[\left.\qquad-\mathrm{sech}^{2}\left(z_{c}\right)e_{c}J-\mathrm{sech}^{2}\left( z_{d}\right)e_{a}\kappa\right]\]

\[=2\Psi\left(\mathbf{y}\right)\left[\frac{\partial\Phi\left(z\right)}{\partial z ^{*}}\odot e^{*}\right].\]

Those two equations above have similar expressions to those in the complex domain.

### Quaternion Gradient for p

We rewrite it as follows to use the chain rule:

\[\left\{\begin{array}{l}\left[\begin{array}{c}\mathbf{y}_{a}\\ \mathbf{y}_{b}\\ \mathbf{y}_{c}\\ \mathbf{y}_{d}\end{array}\right]=\left[\begin{array}{c}\mathbf{W}_{a}^{T}& \mathbf{W}_{b}^{T}&\mathbf{W}_{c}^{T}&\mathbf{W}_{d}^{T}\\ -\mathbf{W}_{d}^{T}&\mathbf{W}_{d}^{T}&\mathbf{W}_{d}^{T}\\ -\mathbf{W}_{d}^{T}&-\mathbf{W}_{d}^{T}&\mathbf{W}_{d}^{T}&\mathbf{W}_{d}^{T}\\ -\mathbf{W}_{d}^{T}&\mathbf{W}_{d}^{T}&-\mathbf{W}_{b}^{T}&\mathbf{W}_{d}^{T} \\ \end{array}\right]\left[\begin{array}{c}\mathbf{x}_{a}\\ \mathbf{x}_{b}\\ \mathbf{x}_{c}\\ \mathbf{x}_{d}\end{array}\right]\]

\[\left.\qquad\qquad+\left[\begin{array}{c}\mathbf{p}_{b}\\ \mathbf{p}_{c}\\ \mathbf{p}_{d}\end{array}\right],\]

\[\left.\qquad\qquad+\left[\begin{array}{c}q_{a}\\ q_{b}\\ q_{c}\\ q_{d}\end{array}\right],\]

\[\Phi\left[z\right]=\phi\left(z_{a}\right)+\phi\left(z_{b}\right)\iota+\phi \left(z_{c}\right)J+\phi\left(z_{d}\right)\kappa.\]

The corresponding gradient of p is defined as

\[\frac{\partial J}{\partial\mathbf{p}^{*}}=\frac{1}{4}\left(\frac{\partial J}{ \partial\mathbf{p}_{a}}+\frac{\partial J}{\partial\mathbf{p}_{b}}\iota+\frac {\partial J}{\partial\mathbf{p}_{c}}J+\frac{\partial J}{\partial\mathbf{p}_{d}} \kappa\right),\]

where

\[\left\{\begin{array}{l}\frac{\partial J}{\partial\mathbf{p}_{a}}=-\frac{ \partial e\Phi^{*}[z]}{\partial\mathbf{p}_{b}}-\frac{\partial\Phi[z]e^{*}}{ \partial\mathbf{p}_{b}},\\ \frac{\partial J}{\partial\mathbf{p}_{b}}\iota=-\frac{\partial e\Phi^{*}[z]}{ \partial\mathbf{p}_{b}}\iota-\frac{\partial\Phi[z]e^{*}}{\partial\mathbf{p}_{b }}\iota,\\ \frac{\partial J}{\partial\mathbf{p}_{c}}J=-\frac{\partial e\Phi^{*}[z]}{ \partial\mathbf{p}_{c}}J-\frac{\partial e\Phi^{*}[z]}{\partial\mathbf{p}_{c}} J,\\ \frac{\partial J}{\partial\mathbf{p}_{d}}\kappa=-\frac{\partial e\Phi^{*}[z]}{ \partial\mathbf{p}_{d}}\kappa-\frac{\partial\Phi[z]}{\partial\mathbf{p}_{d}} \kappa.\end{array}\right.\]

Note that

\[\frac{\partial e\Phi^{*}\left[z\right]}{\partial\mathbf{p}_{\delta}}=\left( \frac{\partial\Phi\left[z\right]e^{*}}{\partial\mathbf{p}_{\delta}}\right)^{*},\delta\in\left\{a,b,c,d\right\}\cdot\left(\begin{array}{cc}1&0\\ 0&1\end{array}\right)\]

We only provide the details of \(\partial e\Phi^{*}\left[z\right]/\partial\mathbf{p}_{a}\), because of the limitation of space:

\[\frac{\partial e\Phi^{*}\left[z\right]}{\partial\mathbf{p}_{a}}=e \left[\frac{\partial\partial\phi\left(z_{a}\right)}{\partial\mathbf{p}_{a}}- \frac{\partial\phi\left(z_{b}\right)}{\partial\mathbf{p}_{a}}t-\frac{\partial \phi\left(z_{b}\right)}{\partial\mathbf{p}_{a}}J-\frac{\partial\phi\left(z_{d }\right)}{\partial\mathbf{p}_{a}}\kappa\right]\] \[=e\left[\frac{\partial\phi\left(z_{a}\right)}{\partial z_{a}} \frac{\partial\phi}{\partial z_{a}}\frac{\partial\phi}{\partial z_{a}}\frac{ \partial\phi}{\partial z_{a}}\frac{\partial\phi}{\partial z_{a}}\frac{\partial \phi}{\partial z_{a}}-\frac{\partial\phi\left(z_{b}\right)}{\partial z_{a}} \frac{\partial\phi}{\partial z_{b}}\frac{\partial\phi}{\partial z_{a}}\frac{ \partial\phi}{\partial z_{a}}\frac{\partial\phi}{\partial z_{a}}\frac{\partial \phi}{\partial z_{a}}\right]\] \[-\frac{\partial\phi\left(z_{c}\right)}{\partial z_{a}}\frac{ \partial\phi}{\partial z_{a}}\frac{\partial\phi}{\partial z_{a}}\frac{\partial \phi}{\partial z_{a}}\frac{\partial\phi}{\partial z_{a}}\frac{\partial\phi}{ \partial z_{a}}\frac{\partial\phi}{\partial z_{a}}\kappa\right]\]

\[=\mathrm{sech}^{2}\left(z_{a}\right)\left[\mathbf{v}_{\mathrm{s}}\mathrm{sech }^{2}\left(\mathbf{y}_{a}\right)\right]e+\mathrm{sech}^{2}\left(z_{b}\right) \left[\mathbf{v}_{\mathrm{s}}\mathrm{sech}^{2}\left(\mathbf{y}_{a}\right) \right]e\iota\]

\[+\mathrm{sech}^{2}\left(z_{c}\right)\left[\mathbf{v}_{\mathrm{s}}\mathrm{sech }^{2}\left(\mathbf{y}_{a}\right)\right]e\kappa\]

\[\frac{\partial e\Phi^{*}[z]}{\partial\mathbf{p}_{b}}J=\mathrm{sech}^{2} \left(z_{a}\right)\left[\mathbf{v}_{\mathrm{s}}\mathrm{sech}^{2}\left(\mathbf{y}_{c }\right)\right]eJ\] \[-\mathrm{sech}^{2}\left(z_{b}\right)\left[\mathbf{v}_{\mathrm{s}} \mathrm{sech}^{2}\left(\mathbf{y}_{c}\right)\right]e\kappa\] \[\frac{\partial e\Phi^{*}[z]}{\partial\mathbf{p}_{c}}J=\mathrm{sech}^{2} \left(z_{a}\right)\left[\mathbf{v}_{\mathrm{s}}\mathrm{sech}^{2}\left( \mathbf{y}_{c}\right)\right]eJ\] \[-\mathrm{sech}^{2}\left(z_{b}\right)\left[\mathbf{v}_{\mathrm{s}} \mathrm{sech}^{2}\left(\mathbf{y}_{c}\right)\right]e\iota\] \[+\mathrm{sech}^{2}\left(z_{c}\right)\left[\mathbf{v}_{\mathrm{s}} \mathrm{sech}^{2}\left(\mathbf{y}_{c}\right)\right]e\] \[-\mathrm{sech}^{2}\left(z_{d}\right)\left[\mathbf{v}_{\mathrm{s}} \mathrm{sech}^{2}\left(\mathbf{y}_{c}\right)\right]e\iota,\]

\[\frac{\partial e\Phi^{*}[z]}{\partial\mathbf{p}_{d}}\kappa=\mathrm{sech}^{2} \left(z_{d}\right)\left[\mathbf{v}_{\mathrm{s}}\mathrm{sech}^{2}\left( \mathbf{y}_{d}\right)\right]e\kappa\]

\[-\mathrm{sech}^{2}\left(z_{b}\right)\left[\mathbf{v}_{\mathrm{s}} \mathrm{sech}^{2}\left(\mathbf{y}_{d}\right)\right]e\iota\]

\[-\mathrm{sech}^{2}\left(z_{c}\right)\left[\mathbf{v}_{\mathrm{s}} \mathrm{sech}^{2}\left(\mathbf{y}_{d}\right)\right]e\iota\]

\[-4\frac{\partial J}{\partial\mathbf{p}_{b}}=-\left(\frac{\
Note that

\[\frac{\partial e\Phi^{*}\left[z\right]}{\partial\mathbf{W}_{\delta}}=\left(\frac{ \partial\Phi\left[z\right]e^{*}}{\partial\mathbf{W}_{\delta}}\right)^{*}, \delta\in\left\{a,b,c,d\right\}.\]

Because of the limitation of space, we only provide the details of \(\partial e\Phi^{*}\left[z\right]/\partial\mathbf{W}_{a}\) :

\[\frac{\partial e\Phi[z]^{*}}{\partial\mathbf{W}_{a}}=e\left[\frac{\partial \phi\left(z_{a}\right)}{\partial\mathbf{W}_{a}}-\frac{\partial\phi\left(z_{b} \right)}{\partial\mathbf{W}_{a}}{}_{t}-\frac{\partial\phi\left(z_{c}\right)}{ \partial\mathbf{W}_{a}}J-\frac{\partial\phi\left(z_{d}\right)}{\partial \mathbf{W}_{a}}\kappa\right],\]

where

\[\frac{\partial\phi\left(z_{a}\right)}{\partial\mathbf{W}_{a}} =\frac{\partial\phi\left(z_{a}\right)}{\partial z_{a}}\frac{\partial \phi_{a}}{\partial z_{a}}\frac{\partial\phi_{a}}{\partial z_{a}}\frac{\partial \psi_{a}}{\partial\mathbf{W}_{a}}\frac{\partial\mathbf{y}_{a}}{\partial \mathbf{W}_{a}}\frac{\partial\mathbf{y}_{a}}{\partial\mathbf{W}_{a}}\frac{ \partial\mathbf{y}_{a}}{\partial\mathbf{W}_{a}}\] \[+\frac{\partial\phi\left(z_{a}\right)}{\partial z_{a}}\frac{ \partial\phi_{a}}{\partial z_{a}}\frac{\partial\phi_{a}}{\partial z_{a}}\frac{ \partial\phi_{a}}{\partial z_{a}}\frac{\partial\psi_{a}}{\partial\mathbf{W}_{ a}}\frac{\partial\psi_{a}}{\partial\mathbf{W}_{a}}\frac{\partial\psi_{a}}{ \partial\mathbf{W}_{a}}\] \[=\mathrm{sech}^{2}\left(z_{a}\right)\left(\lambda_{a}\big{[} \mathbf{v}_{a}\mathrm{sech}^{2}\left(\mathbf{y}_{a}\right)\big{]}^{T}+ \mathbf{x}_{b}\big{[}\mathbf{v}_{b}\mathrm{sech}^{2}\left(\mathbf{y}_{b} \right)\big{]}^{T}\] \[+\mathbf{x}_{c}\big{[}\mathbf{v}_{a}\mathrm{sech}^{2}\left( \mathbf{y}_{c}\right)\big{]}^{T}+\mathbf{x}_{d}\big{[}\mathbf{v}_{d}\mathrm{ sech}^{2}\left(\mathbf{y}_{d}\right)\big{]}^{T}\right),\]

\[\frac{\partial\phi\left(z_{a}\right)}{\partial\mathbf{W}_{a}} =\mathrm{sech}^{2}\left(z_{b}\right)\left(\mathbf{x}_{a}\big{[} \mathbf{-v}_{b}\mathrm{sech}^{2}\left(\mathbf{y}_{a}\right)\big{]}^{T}+ \mathbf{x}_{b}\big{[}\mathbf{v}_{a}\mathrm{sech}^{2}\left(\mathbf{y}_{b} \right)\big{]}^{T}+\mathbf{x}_{c}\big{[}\mathbf{v}_{d}\mathrm{sech}^{2}\left( \mathbf{y}_{d}\right)\big{]}^{T}\right),\]

\[\frac{\partial\phi\left(z_{a}\right)}{\partial\mathbf{W}_{a}} =\mathrm{sech}^{2}\left(z_{c}\right)\left(\mathbf{x}_{a}\big{[} \mathbf{-v}_{c}\mathrm{sech}^{2}\left(\mathbf{y}_{a}\right)\big{]}^{T}+ \mathbf{x}_{d}\big{[}\mathbf{v}_{a}\mathrm{sech}^{2}\left(\mathbf{y}_{d} \right)\big{]}^{T}\right),\]

\[\frac{\partial\phi\left(z_{d}\right)}{\partial\mathbf{W}_{a}} =\mathrm{sech}^{2}\left(z_{d}\right)\left(\mathbf{x}_{a}\big{[} \mathbf{-v}_{d}\mathrm{sech}^{2}\left(\mathbf{y}_{a}\right)\big{]}^{T}+ \mathbf{x}_{b}\big{[}\mathbf{v}_{c}\mathrm{sech}^{2}\left(\mathbf{y}_{b} \right)\big{]}^{T}\right.\] \[\left.+\mathbf{x}_{c}\big{[}\mathbf{v}_{d}\mathrm{sech}^{2}\left( \mathbf{y}_{c}\right)\big{]}^{T}+\mathbf{x}_{d}\big{[}\mathbf{v}_{a}\mathrm{ sech}^{2}\left(\mathbf{y}_{d}\right)\big{]}^{T}\right).\]

Others in can be obtained similarly. Then the gradient of W can be obtained as follows:

\[-4\,\frac{\partial J}{\partial\mathbf{W}^{*}}=\left(\frac{ \partial J}{\partial\mathbf{W}_{a}}+\frac{\partial J}{\partial\mathbf{W}_{b}} \iota+\frac{\partial J}{\partial\mathbf{W}_{a}}J+\frac{\partial J}{\partial \mathbf{W}_{a}}\kappa\right)\] \[=2\mathbf{x}\Big{(}\Big{(}v\left[\frac{\partial\Phi\left(z\right) }{\partial z^{*}}\odot e\right]\Big{)}\odot\left(\frac{\partial\Phi\left[ \mathbf{y}\right]}{\partial\mathbf{y}^{*}}\right)\Big{)}^{H}.\]

Finally, the quaternion LMS algorithm for the MLP (19) with the split quaternion activation function is summarized as

\[q\left(i{+}1\right)=q\left(i\right)+\eta_{q}\frac{\partial\Phi\left(z\right) }{\partial z^{*}}\odot e\left(i\right),\]

\[\mathbf{V}\left(i{+}1\right)=\mathbf{V}\left(i\right)+\eta_{v}\Psi\left( \mathbf{y}\right)\left[\frac{\partial\Phi\left(z\right)}{\partial z^{*}} \odot e^{*}\left(i\right)\right],\]

\[\mathbf{p}\left(i{+}1\right)=\mathbf{p}\left(i\right)+\eta_{p}\left(\mathbf{v }\left[\frac{\partial\Phi\left(z\right)}{\partial z^{*}}\odot e\right] \right)\odot\frac{\partial\Psi\left(\mathbf{y}\right)}{\partial\mathbf{y}^{*}},\]

\[\mathbf{W}\left(i{+}1\right)=\mathbf{W}\left(i\right){+}\eta_{w}\mathbf{x} \bigg{[}\bigg{(}v\left[\frac{\partial\Phi\left(z\right)}{\partial z^{*}} \odot e\right]\bigg{)}\odot\left(\frac{\partial\Psi\left(\mathbf{y}\right)}{ \partial\mathbf{y}^{*}}\right)\bigg{]}^{H}.\]

The obtained algorithm (38a-d) has the similar expression to that in the real and complex domains.

## IV Quaternion MLP Based on The MCC

The cost function of the MCC is defined as

\[J_{MCC}\left(\mathbf{w}\right)=\exp\left(-\frac{e(i)e^{*}(i)}{2\sigma^{2}} \right),\]

to estimate the parameter \(\mathbf{w}\in H^{n}\). Then, the corresponding quaternion stochastic gradient ascent algorithm for the MLP based on the MCC can be derived from (38a-d) as

\[q\left(i{+}1\right)=q\left(i\right)+\eta_{q}J_{MCC}\left(\mathbf{w}\right) \frac{\partial\Phi\left(z\right)}{\partial z^{*}}\odot e\left(i\right),\]

\[\mathbf{V}\left(i{+}1\right)=\mathbf{V}\left(i\right)+\eta_{v}J_{MCC}\left( \mathbf{w}\right)\Psi\left(\mathbf{y}\right)\left[\frac{\partial\Phi\left(z \right)}{\partial z^{*}}\odot e^{*}\left(i\right)\right],\]

\[\mathbf{p}\left(i{+}1\right)=\mathbf{p}\left(i\right){+}\eta_{v}J_{MCC}\left( \mathbf{w}\right)\left(\mathbf{v}\left[\frac{\partial\Phi\left(z\right)}{ \partial z^{*}}\odot e\right]\right)\odot\frac{\partial\Psi\left(\mathbf{y} \right)}{\partial\mathbf{y}^{*}},\]

\[\mathbf{p}\left(i{+}1\right)=\mathbf{p}\left(i\right){+}\eta_{v}J_{MCC}\left( \mathbf{w}\right)\left(\mathbf{v}\left[\frac{\partial\Phi\left(z\right)}{ \partial z^{*}}\odot e\right]\right)\odot\frac{\partial\Psi\left(\mathbf{y} \right)}{\partial\mathbf{y}^{*}},\]

## V Simulation

In the simulations, we use two proposed quaternion MLP algorithms to predict the chaotic time-series, which were generated using the following Mackey-Glass time-delay differential equation.

\[\dot{x}\left(t\right)=\frac{0.2x\left(t-\tau\right)}{1+x^{10}\left(t-\tau\right)}-0.1x\left(t\right),\]

where \(\tau=17\), \(x\left(0\right){=}0.12\), and for all \(t<0\). In this prediction task, we used the previous five historical data to predict the next data in the future. Thus, the sizes of the inputs and outputs were and, the parameter in the hidden layer had dimensions of, and the size of was.

### _Prediction without Noises_

First, we simulated the one-step prediction of Mackey-Glass data without any noise. We set the step size. Fig. 1 shows that the MSE algorithm and MCC algorithm obtained similar results, so we only present the good prediction results of the MSE algorithm in Fig. 2. The prediction results include one real part and three imaginary components.

### _Prediction under Noise_

Additionally, we assumed that the Mackey-Glass data are noisy. To simulate the representative noise scenario, two types of noise signals were generated. The first type of noise was Gaussian. We set the step size ; the simulations of the MCC and MSE algorithms are shown in Fig. 3. From the comparison, we found that the MCC algorithm converged slower than the MSE algorithm while the steady errors of MCC performs better than MSE.The difference between the steady-state errors of the two algorithms is more obvious in Figure 3 (b) in
## VI Conclusion

In this paper, we have proposed two algorithms for quaternion MLP networks based on the cost functions of the MSE

Fig. 4: Error comparison under Gaussian noise

Fig. 5: Error comparison under Gaussian noise

Fig. 3: One–step prediction results for the quaternion MSE without noise.

Fig. 6: Error comparison under impulsive noise

Fig. 7: Error comparison under impulsive noise

Fig. 2: Error comparison between the MCC and MSE without noise
and MCC, respectively. In the algorithms, we utilized the GHR quaternion gradient. The two algorithms performed similarly under noiseless and Gaussian noise circumstances. The MCC algorithm performed better than the MSE algorithm under impulsive noise.
