e^{-2.40i}&0.0000\cdot e^{-2.40i}&0.7622\cdot e^{0.72i}&0.0000\cdot
e^{3.00i}&0.0000&0.0000\cdot e^{-0.33i}&0.0000\cdot e^{-2.30i}&0.6467\cdot
e^{0.72i}\\\ 0.0000\cdot e^{3.10i}&-0.0000&-0.0000&0.0002\cdot
e^{-0.03i}&0.0290&0.0025&0.0009\cdot e^{-3.10i}&0.0005&0.0002\cdot
e^{-0.03i}\\\ 0.0000\cdot e^{-3.00i}&0.0000\cdot e^{-3.10i}&0.0000\cdot
e^{3.10i}&0.0000\cdot e^{1.00i}&0.0002\cdot
e^{-3.10i}&0.0029&0.0000&0.0006&0.0000\cdot e^{1.00i}\\\ 0.0000\cdot
e^{2.10i}&0.0000\cdot e^{2.10i}&0.0000\cdot e^{2.10i}&0.0227\cdot
e^{-1.00i}&0.0001\cdot e^{-3.10i}&0.0028\cdot e^{-1.00i}&0.0000\cdot
e^{-0.73i}&0.0005\cdot e^{-1.00i}&0.0193\cdot e^{-1.00i}\\\ 0.0000\cdot
e^{-3.10i}&0.0000\cdot e^{-3.10i}&0.0004\cdot e^{-3.10i}&0.0001\cdot
e^{-3.10i}&0.0008\cdot e^{0.01i}&0.9815\cdot e^{0.03i}&0.0034\cdot
e^{0.03i}&0.1893\cdot e^{0.03i}&0.0000\cdot e^{-0.26i}\\\ \end{pmatrix},$
(B.20)
$\displaystyle\left(U^{n}_{R}\right)^{\dagger}P_{\overline{6}}U^{e}_{R}=$
(B.21) $\displaystyle\begin{pmatrix}0.0000\cdot e^{-0.64i}&0.0000\cdot
e^{2.50i}&0.0000\cdot e^{-0.70i}&0.0000\cdot e^{-0.64i}&0.0000\cdot
e^{1.80i}&0.0000\cdot e^{2.80i}&0.0000\cdot e^{1.80i}&0.0000\cdot
e^{-0.29i}&0.0000\cdot e^{-0.64i}\\\ 0.0000\cdot e^{-1.50i}&0.0000\cdot
e^{1.60i}&0.0000\cdot e^{-1.50i}&0.0000\cdot e^{-1.50i}&0.0000\cdot
e^{-1.70i}&0.0000\cdot e^{-2.10i}&0.0000\cdot e^{-1.70i}&0.0000\cdot
e^{1.10i}&0.0000\cdot e^{-1.50i}\\\ 0.0000\cdot e^{1.70i}&0.0000\cdot
e^{-1.50i}&0.0000\cdot e^{1.60i}&0.0000\cdot e^{1.70i}&0.0000\cdot
e^{-1.40i}&0.0000\cdot e^{1.80i}&0.0000\cdot e^{-1.40i}&0.0000\cdot
e^{-1.30i}&0.0000\cdot e^{1.70i}\\\ 0.0000&-0.0000&0.0000&0.3825&0.0046\cdot
e^{2.80i}&0.0209\cdot e^{0.02i}&0.0002\cdot e^{2.80i}&0.0021\cdot
e^{-3.10i}&0.3324\\\ 0.0000\cdot e^{-1.60i}&0.0000\cdot e^{1.60i}&0.0000\cdot
e^{-1.60i}&0.3825\cdot e^{-1.60i}&0.0046\cdot e^{1.20i}&0.0209\cdot
e^{-1.60i}&0.0002\cdot e^{1.20i}&0.0021\cdot e^{1.60i}&0.3324\cdot
e^{-1.60i}\\\ 0.0000\cdot e^{0.02i}&0.0000\cdot e^{-3.10i}&0.0000&0.0185\cdot
e^{0.03i}&0.3424&-0.5214&0.0111&0.0509&0.0161\cdot e^{0.03i}\\\ 0.0000\cdot
e^{-1.60i}&0.0000\cdot e^{1.60i}&0.0000\cdot e^{-1.60i}&0.0185\cdot
e^{-1.50i}&0.3473\cdot e^{-1.60i}&0.5201\cdot e^{1.60i}&0.0113\cdot
e^{-1.60i}&0.0508\cdot e^{-1.60i}&0.0161\cdot e^{-1.50i}\\\ 0.0000\cdot
e^{-0.13i}&0.0000\cdot e^{3.00i}&0.0000&0.0066\cdot e^{-0.27i}&0.6148\cdot
e^{-3.10i}&0.3247\cdot e^{-3.10i}&0.0200\cdot e^{-3.10i}&0.0317&0.0057\cdot
e^{-0.27i}\\\ 0.0000\cdot e^{-1.70i}&0.0000\cdot e^{1.50i}&0.0000\cdot
e^{-1.60i}&0.0067\cdot e^{-1.80i}&0.6120\cdot e^{1.60i}&0.3294\cdot
e^{1.60i}&0.0199\cdot e^{1.60i}&0.0322\cdot e^{-1.60i}&0.0058\cdot
e^{-1.80i}\\\ 0.0000\cdot e^{-0.06i}&0.0000\cdot e^{3.10i}&0.0000&0.0006\cdot
e^{-0.10i}&-0.0192&-0.0203&-0.0006&0.0020&0.0005\cdot e^{-0.10i}\\\
0.0000\cdot e^{-1.70i}&0.0000\cdot e^{1.40i}&0.0000\cdot
e^{-1.60i}&0.0002\cdot e^{-1.90i}&0.0205\cdot e^{1.60i}&0.0091\cdot
e^{1.60i}&0.0007\cdot e^{1.60i}&0.0009\cdot e^{-1.60i}&0.0002\cdot
e^{-1.90i}\\\
0.0000&-0.0000&0.0000&0.0112&0.0624&-0.3418&0.0020&0.0334&0.0097\\\
0.0000\cdot e^{-1.60i}&0.0000\cdot e^{1.60i}&0.0000\cdot
e^{-1.60i}&0.0111\cdot e^{-1.60i}&0.0619\cdot e^{-1.60i}&0.3397\cdot
e^{1.60i}&0.0020\cdot e^{-1.60i}&0.0332\cdot e^{-1.60i}&0.0096\cdot
e^{-1.60i}\\\ -0.0000&0.0000&-0.0000&0.3716\cdot e^{-3.10i}&0.0044\cdot
e^{-0.38i}&0.0202\cdot e^{-3.10i}&0.0001\cdot e^{-0.38i}&0.0020\cdot
e^{0.02i}&0.3228\cdot e^{-3.10i}\\\ 0.0000\cdot e^{1.60i}&0.0000\cdot
e^{-1.60i}&0.0000\cdot e^{1.60i}&0.3716\cdot e^{1.60i}&0.0044\cdot
e^{-2.00i}&0.0202\cdot e^{1.60i}&0.0001\cdot e^{-1.90i}&0.0020\cdot
e^{-1.60i}&0.3228\cdot e^{1.60i}\\\ 0.0000\cdot e^{-0.34i}&0.0000\cdot
e^{2.80i}&0.0000\cdot e^{-0.34i}&0.0000\cdot e^{-0.34i}&0.0000\cdot
e^{1.50i}&0.0000\cdot e^{-0.69i}&0.0000\cdot e^{1.50i}&0.0000\cdot
e^{2.50i}&0.0000\cdot e^{-0.34i}\\\ 0.0000\cdot e^{-0.22i}&0.0000\cdot
e^{2.90i}&0.0000\cdot e^{-0.22i}&0.0000\cdot e^{-0.22i}&0.0000\cdot
e^{3.00i}&0.0000\cdot e^{-0.15i}&0.0000\cdot e^{3.00i}&0.0000\cdot
e^{3.00i}&0.0000\cdot e^{-0.22i}\\\ 0.0000&-0.0000&0.0000&0.0000&0.0000\cdot
e^{2.70i}&0.0000\cdot e^{0.02i}&0.0000\cdot e^{2.70i}&0.0000\cdot
e^{-3.10i}&0.0000\\\ \end{pmatrix}.$

The upper-left $3\times 3$ block of $U^{\dagger}_{u_{L}}P_{6}U_{d_{L}}$ and
$U^{\dagger}_{n_{L}}P_{6}U_{e_{L}}$ correspond to the CKM matrix and hermitian
conjugate of the PMNS matrix, respectively. We also note that the W-boson
coupling to the SM fermions in the right-handed current are negligible. The
leptoquark couplings with $g_{4}=1$ are given by

$\displaystyle\hat{g}^{X}_{d_{L}}=$ (B.22)
$\displaystyle\begin{pmatrix}0.0000&0.0000&0.0000&0.0000&0.7067&0.0000&0.0230&0.0000\cdot
e^{-3.10i}&0.0000\\\ 0.0000&0.0283&0.0001&0.0211&0.0000\cdot
e^{-3.10i}&0.7026&0.0000&0.0686\cdot e^{-3.10i}&0.0184\\\ 0.0000\cdot
e^{-3.10i}&0.7065&0.0012&0.0055\cdot e^{-3.10i}&0.0000\cdot
e^{-3.10i}&0.0279\cdot e^{-3.10i}&0.0000\cdot e^{-3.10i}&0.0027&0.0052\cdot
e^{-3.10i}\\\
0.0000&0.0007&-0.5391&0.3009&0.0000&-0.0000&-0.0000&-0.0001&-0.3447\\\
-0.7068&-0.0000&0.0000&0.0001&-0.0006&0.0000&0.0211&0.0000&0.0001\\\
-0.0002&-0.0064&-0.0012&-0.5235&0.0001&0.0146&-0.0027&-0.1360&-0.4551\\\
0.0211&-0.0000&-0.0000&-0.0018&-0.0230&-0.0001&0.7064&-0.0015&-0.0016\\\
0.0000&-0.0012&-0.0002&-0.1009&-0.0000&0.0728&0.0010&0.6905&-0.0879\\\
-0.0000&0.0010&-0.4576&-0.3531&0.0000&0.0000&0.0000&0.0001&0.4074\\\
\end{pmatrix},$ $\displaystyle\hat{g}^{X}_{d_{R}}=$ (B.23)
$\displaystyle\begin{pmatrix}0.0000\cdot e^{-3.10i}&0.0000\cdot
e^{-3.10i}&0.0000\cdot e^{-3.10i}&0.0000&0.0230\cdot e^{-3.10i}&0.0000\cdot
e^{-3.10i}&0.7067&0.0001\cdot e^{-3.10i}&0.0000\cdot e^{-3.10i}\\\ 0.0000\cdot
e^{-3.10i}&0.0000\cdot e^{-3.10i}&0.0000\cdot
e^{-3.10i}&0.0000&0.0000&0.0687\cdot e^{-3.10i}&0.0001\cdot
e^{-3.10i}&0.7038\cdot e^{-3.10i}&0.0001\cdot e^{-3.10i}\\\ 0.0000\cdot
e^{-3.10i}&0.0000\cdot e^{-3.10i}&0.0000\cdot
e^{-3.10i}&0.4638&0.0000&0.0003&0.0001\cdot e^{-3.10i}&0.0000&0.5338\cdot
e^{-3.10i}\\\
0.0000&-0.0001&0.4575&0.4070&-0.0000&0.0006&0.0000&-0.0001&0.3536\\\
0.0211&0.0003&0.0000&0.0000&-0.7064&0.0002&-0.0230&-0.0000&-0.0000\\\
-0.0046&0.1339&0.0001&0.0007&-0.0003&-0.6910&-0.0000&0.0675&0.0002\\\
0.7067&0.0116&0.0000&0.0000&0.0211&-0.0024&0.0007&0.0002&-0.0000\\\
0.0109&-0.6942&-0.0001&0.0001&0.0000&-0.1333&-0.0000&0.0131&0.0001\\\
-0.0000&0.0001&-0.5392&0.3453&0.0000&0.0004&-0.0000&-0.0000&0.3001\\\
\end{pmatrix},$ $\displaystyle\left(\hat{g}^{X}_{u_{L}}\right)^{\dagger}=$
(B.24) $\displaystyle\begin{pmatrix}0.0029\cdot e^{-2.00i}&0.0221\cdot
e^{-2.40i}&0.2727\cdot e^{-2.40i}&0.5584\cdot e^{-2.30i}&0.3358\cdot
e^{2.80i}&0.0163\cdot e^{-2.30i}&0.0001\cdot e^{1.10i}&0.0171\cdot
e^{-1.60i}&0.0013\cdot e^{0.84i}\\\ 0.0044\cdot e^{-1.40i}&0.0329\cdot
e^{-1.70i}&0.4043\cdot e^{-1.70i}&0.4204\cdot e^{1.80i}&0.3975\cdot
e^{-2.20i}&0.0121\cdot e^{1.80i}&0.0001\cdot e^{-1.60i}&0.0227\cdot
e^{-0.56i}&0.0032\cdot e^{1.40i}\\\ 0.0055\cdot e^{1.90i}&0.0414\cdot
e^{1.50i}&0.5083\cdot e^{1.50i}&0.1048\cdot e^{3.00i}&0.4784\cdot
e^{-2.30i}&0.0030\cdot e^{3.10i}&0.0000\cdot e^{-1.90i}&0.0006\cdot
e^{-0.25i}&0.0051\cdot e^{-1.60i}\\\ 0.0039\cdot
e^{0.32i}&0.0291&0.0042&0.0004\cdot e^{-0.12i}&0.0111\cdot
e^{-0.72i}&0.0010\cdot e^{-3.10i}&0.0011\cdot e^{-3.10i}&0.3539\cdot
e^{-2.10i}&0.3518\cdot e^{3.10i}\\\ 0.0039\cdot e^{-1.30i}&0.0291\cdot
e^{-1.60i}&0.0042\cdot e^{-1.60i}&0.0004\cdot e^{-1.70i}&0.0111\cdot
e^{-2.30i}&0.0010\cdot e^{1.60i}&0.0011\cdot e^{1.60i}&0.3539\cdot
e^{2.60i}&0.3518\cdot e^{1.50i}\\\ 0.1532&-0.4131&0.0318\cdot
e^{0.01i}&0.0000&0.0000\cdot e^{-0.99i}&0.0010&0.2319&0.0002\cdot
e^{-2.10i}&0.0328\cdot e^{3.10i}\\\ 0.1567\cdot e^{-1.60i}&0.4130\cdot
e^{1.60i}&0.0317\cdot e^{-1.60i}&0.0001\cdot e^{1.60i}&0.0000\cdot
e^{-2.60i}&0.0046\cdot e^{-1.60i}&0.2297\cdot e^{-1.60i}&0.0002\cdot
e^{2.60i}&0.0327\cdot e^{1.60i}\\\ -0.4755&-0.1260&0.0151\cdot
e^{-0.11i}&0.0005&0.0000\cdot e^{3.00i}&0.0166\cdot
e^{-3.10i}&0.0855&0.0001\cdot e^{-2.30i}&0.0154\cdot e^{3.00i}\\\ 0.4743\cdot
e^{1.60i}&0.1297\cdot e^{1.60i}&0.0154\cdot e^{-1.70i}&0.0004\cdot
e^{-1.60i}&0.0000\cdot e^{1.50i}&0.0147\cdot e^{1.60i}&0.0865\cdot
e^{-1.60i}&0.0001\cdot e^{2.40i}&0.0157\cdot e^{1.40i}\\\
-0.0165&-0.0109&0.0010\cdot e^{-0.06i}&0.0145\cdot e^{-3.10i}&0.0001\cdot
e^{2.40i}&0.4993&0.0113\cdot e^{-3.10i}&0.0001\cdot e^{-2.20i}&0.0024\cdot
e^{3.10i}\\\ 0.0156\cdot e^{1.60i}&0.0030\cdot e^{1.60i}&0.0004\cdot
e^{-1.70i}&0.0145\cdot e^{1.60i}&0.0001\cdot e^{0.82i}&0.4995\cdot
e^{-1.60i}&0.0051\cdot e^{1.60i}&0.0001\cdot e^{2.50i}&0.0017\cdot
e^{1.50i}\\\ 0.0113\cdot e^{3.10i}&-0.2451&0.0199&0.0004&0.0000\cdot
e^{-0.17i}&0.0157\cdot e^{-3.10i}&0.4345\cdot e^{-3.10i}&0.0000\cdot
e^{0.97i}&0.0188\cdot e^{3.10i}\\\ 0.0114\cdot e^{1.60i}&0.2436\cdot
e^{1.60i}&0.0198\cdot e^{-1.60i}&0.0001\cdot e^{-1.60i}&0.0000\cdot
e^{-1.70i}&0.0064\cdot e^{1.60i}&0.4356\cdot e^{1.60i}&0.0000\cdot
e^{-0.60i}&0.0187\cdot e^{1.50i}\\\ 0.0037\cdot
e^{-2.80i}&-0.0275&0.0148&0.0004\cdot e^{-3.10i}&0.0100\cdot
e^{-0.72i}&0.0009&0.0009&0.3526\cdot e^{-2.10i}&0.3530\cdot e^{-0.03i}\\\
0.0037\cdot e^{1.90i}&0.0275\cdot e^{1.60i}&0.0148\cdot e^{-1.60i}&0.0004\cdot
e^{1.70i}&0.0100\cdot e^{-2.30i}&0.0009\cdot e^{-1.60i}&0.0009\cdot
e^{-1.60i}&0.3526\cdot e^{2.60i}&0.3529\cdot e^{-1.60i}\\\ 0.0000\cdot
e^{0.36i}&0.0000\cdot e^{0.34i}&0.0000\cdot e^{0.21i}&0.0000\cdot
e^{2.00i}&0.0000\cdot e^{3.00i}&0.0000\cdot e^{3.00i}&0.0000\cdot
e^{0.39i}&0.0000\cdot e^{-1.40i}&0.0000\cdot e^{-2.90i}\\\ 0.0000\cdot
e^{0.53i}&0.0000\cdot e^{0.24i}&0.0000\cdot e^{0.40i}&0.0000\cdot
e^{0.48i}&0.0000\cdot e^{2.60i}&0.0000\cdot e^{3.10i}&0.0000\cdot
e^{0.21i}&0.0000\cdot e^{-2.00i}&0.0000\cdot e^{-2.90i}\\\ 0.0000\cdot
e^{0.32i}&0.0000&-0.0000&-0.0000&0.0000\cdot e^{2.40i}&0.0006&0.0006\cdot
e^{-3.10i}&0.0006\cdot e^{1.00i}&0.0000\cdot e^{3.10i}\\\ \end{pmatrix},$
$\displaystyle\left(\hat{g}^{X}_{u_{L}}\right)^{\dagger}=$ (B.25)
$\displaystyle\begin{pmatrix}0.0000\cdot e^{2.50i}&0.0000\cdot
e^{2.50i}&0.0000\cdot e^{2.50i}&0.0000\cdot e^{-0.92i}&0.0000\cdot
e^{1.30i}&0.0000\cdot e^{2.20i}&0.0000\cdot e^{1.50i}&0.0000\cdot
e^{-0.07i}&0.0000\cdot e^{-1.10i}\\\ 0.0000\cdot e^{1.60i}&0.0000\cdot
e^{1.60i}&0.0000\cdot e^{1.60i}&0.0000\cdot e^{-1.40i}&0.0000\cdot
e^{0.83i}&0.0000\cdot e^{1.70i}&0.0000\cdot e^{-1.60i}&0.0000\cdot
e^{-0.59i}&0.0000\cdot e^{1.50i}\\\ 0.0000\cdot e^{-1.60i}&0.0000\cdot
e^{-1.60i}&0.0000\cdot e^{-1.60i}&0.0000\cdot e^{1.60i}&0.0000\cdot
e^{0.85i}&0.0000\cdot e^{-1.60i}&0.0000\cdot e^{1.60i}&0.0000\cdot
e^{-0.57i}&0.0000\cdot e^{1.50i}\\\ 0.0000\cdot
e^{-0.01i}&0.0000&0.3486&0.0033\cdot e^{-0.38i}&0.3582\cdot
e^{-0.72i}&0.0003\cdot e^{-3.00i}&0.0005\cdot e^{-0.04i}&0.0101\cdot
e^{1.00i}&0.0051\cdot e^{-3.10i}\\\ 0.0000\cdot e^{-1.60i}&0.0000\cdot
e^{-1.60i}&0.3486\cdot e^{-1.60i}&0.0032\cdot e^{-2.00i}&0.3582\cdot
e^{-2.30i}&0.0000\cdot e^{2.90i}&0.0002\cdot e^{-1.70i}&0.0105\cdot
e^{-0.57i}&0.0051\cdot e^{1.60i}\\\ 0.0011\cdot
e^{-3.10i}&0.2320&-0.0100&0.2425\cdot e^{-3.10i}&0.0170\cdot
e^{-0.69i}&0.0064\cdot e^{-3.10i}&0.0015&0.0012\cdot e^{0.99i}&0.3700\cdot
e^{-0.03i}\\\ 0.0047\cdot e^{1.60i}&0.2298\cdot e^{-1.60i}&0.0099\cdot
e^{1.60i}&0.2460\cdot e^{1.60i}&0.0170\cdot e^{-2.30i}&0.0063\cdot
e^{1.60i}&0.0013\cdot e^{-1.60i}&0.0014\cdot e^{-0.58i}&0.3691\cdot
e^{-1.60i}\\\ 0.0166&0.0855&-0.0062&0.4345&0.0059\cdot
e^{-1.00i}&0.0130&0.0008&0.0006\cdot e^{1.00i}&0.2310\cdot e^{-0.03i}\\\
0.0147\cdot e^{-1.60i}&0.0865\cdot e^{-1.60i}&0.0063\cdot
e^{1.60i}&0.4326\cdot e^{-1.60i}&0.0061\cdot e^{-2.60i}&0.0131\cdot
e^{-1.60i}&0.0007\cdot e^{-1.60i}&0.0007\cdot e^{-0.57i}&0.2344\cdot
e^{-1.60i}\\\ -0.4995&-0.0112&-0.0004&0.0136&0.0005\cdot
e^{-0.83i}&0.0012&0.0007\cdot e^{-3.10i}&0.0008\cdot e^{1.00i}&0.0144\cdot
e^{-0.03i}\\\ 0.4997\cdot e^{1.60i}&0.0051\cdot e^{1.60i}&0.0002\cdot
e^{1.60i}&0.0145\cdot e^{-1.60i}&0.0002\cdot e^{-2.60i}&0.0007\cdot
e^{-1.60i}&0.0003\cdot e^{1.60i}&0.0003\cdot e^{-0.57i}&0.0065\cdot
e^{-1.60i}\\\ 0.0156&-0.4344&-0.0065&0.0444\cdot e^{-3.10i}&0.0102\cdot
e^{-0.72i}&0.0000\cdot e^{-3.10i}&0.0001&0.0016\cdot e^{1.00i}&0.2427\cdot
e^{-0.03i}\\\ 0.0063\cdot e^{-1.60i}&0.4355\cdot e^{1.60i}&0.0065\cdot
e^{1.60i}&0.0440\cdot e^{1.60i}&0.0101\cdot e^{-2.30i}&0.0004\cdot
e^{1.60i}&0.0005\cdot e^{-1.60i}&0.0012\cdot e^{-0.57i}&0.2412\cdot
e^{-1.60i}\\\ -0.0000&-0.0000&0.3581&0.0031\cdot e^{2.80i}&0.3479\cdot
e^{2.40i}&0.0007\cdot e^{3.10i}&0.0006\cdot e^{0.03i}&0.0109\cdot
e^{-2.10i}&0.0237\cdot e^{-0.02i}\\\ 0.0000\cdot e^{1.60i}&0.0000\cdot
e^{1.60i}&0.3582\cdot e^{-1.60i}&0.0031\cdot e^{1.20i}&0.3479\cdot
e^{0.85i}&0.0003\cdot e^{1.50i}&0.0003\cdot e^{-1.50i}&0.0105\cdot
e^{2.60i}&0.0237\cdot e^{-1.60i}\\\ 0.0000\cdot e^{-0.27i}&0.0000\cdot
e^{-0.45i}&0.0000\cdot e^{-2.70i}&-0.0160&0.0090\cdot
e^{-1.20i}&0.5537&0.3132\cdot e^{0.47i}&0.3081\cdot e^{-2.60i}&0.0017\cdot
e^{-2.70i}\\\ 0.0000\cdot e^{-0.32i}&0.0000\cdot e^{-0.19i}&0.0000\cdot
e^{0.17i}&0.0047\cdot e^{1.60i}&0.0145\cdot e^{-0.88i}&0.1627\cdot
e^{-1.50i}&0.4850\cdot e^{-3.00i}&0.4879\cdot e^{-2.30i}&0.0028\cdot
e^{0.12i}\\\ -0.0012&-0.0012&-0.0012&0.0119&0.0123\cdot e^{-0.72i}&0.4080\cdot
e^{-3.10i}&0.4083&0.4081\cdot e^{-2.10i}&0.0010\cdot e^{-0.03i}\\\
\end{pmatrix},$

The Yukawa couplings $h_{qq^{\prime}}$ defined in Eq. (3.38) are given by

$\displaystyle h_{dd}=0.00000183\cdot e^{-0.01i},\quad h_{uu}=0.00000182\cdot
e^{-0.01i},\quad h_{du}=0.00000183\cdot e^{-0.01i},$ (B.26)

where the Yukawa coupling in the gauge basis, $h$ is set to be identity
matrix.

Finally we summarize the prediction of observables at this benchmark point,
lower and upper limits in Table 4. We take $\tan\beta=50$ and $m_{H}=3.0$ TeV
to calculate the neutral meson mixing. The benchmark point explains the $b\to
s\mu\mu$ anomaly. On the other hand it is hard to explain the $b\to c\tau\nu$
anomaly. The all experimental constraints listed in the table are satisfied
and the tuning level defined in Eq. (2.41) is about $0.1\%$.

Table 4:  The values of predictions at this benchmark point, lower and upper limit are shown. The value of the fine-tuning parameter is $\Delta_{\mathrm{FT}}^{-1}=1.39\times 10^{3}$.  observable | prediction | lower limit | upper limit | Ref.
---|---|---|---|---
$\mathrm{Re}\Delta C_{9}$ ($b\to s\mu\mu$ anomaly) | -0.505 | -0.59 | -0.41 | [84]
$\Delta C_{V}$ ($b\to c\tau\nu$ anomaly) | $2.75\times 10^{-8}$ | $5.2\times 10^{-2}$ | 0.124 | [85]
$\mathrm{BR}(B\to e\nu)$ | $8.74\times 10^{-12}$ | 0 | $9.8\times 10^{-7}$ | [90]
$\mathrm{BR}(B\to\mu\nu)$ | $3.88\times 10^{-7}$ | $2.90\times 10^{-7}$ | $1.07\times 10^{-6}$ | [90]
$\mathrm{BR}(B\to\tau\nu)$ | $8.82\times 10^{-5}$ | $8.5\times 10^{-5}$ | $1.33\times 10^{-4}$ | [90]
$\mathrm{BR}(B_{c}\to e\nu)$ | $2.20\times 10^{-9}$ | 0 | 0.6 | [111]
$\mathrm{BR}(B_{c}\to\mu\nu)$ | $9.79\times 10^{-5}$ | 0 | 0.6 | [111]
$\mathrm{BR}(B_{c}\to\tau\nu)$ | $2.39\times 10^{-2}$ | 0 | 0.6 | [111]
$\mathrm{BR}(K_{L}\to e\mu)$ | $3.04\times 10^{-13}$ | 0 | $4.7\times 10^{-12}$ | [90]
$\mathrm{BR}(B_{d}\to e\tau)$ | $1.27\times 10^{-18}$ | 0 | $2.8\times 10^{-5}$ | [90]
$\mathrm{BR}(B_{d}\to\mu\tau)$ | $4.70\times 10^{-13}$ | 0 | $2.2\times 10^{-5}$ | [90]
$\mathrm{BR}(B_{d}\to\mu e)$ | $5.46\times 10^{-13}$ | 0 | $2.8\times 10^{-9}$ | [90]
$\mathrm{BR}(B_{d}\to ee)$ | $2.47\times 10^{-15}$ | 0 | $8.3\times 10^{-8}$ | [90]
$\mathrm{BR}(B_{d}\to\mu\mu)$ | $1.00\times 10^{-10}$ | $0$ | $3.9\times 10^{-10}$ | [90]
$\mathrm{BR}(B_{d}\to\tau\tau)$ | $2.34\times 10^{-8}$ | 0 | $2.1\times 10^{-3}$ | [90]
$\mathrm{BR}(B_{s}\to e\tau)$ | $2.83\times 10^{-18}$ | - | - | [90]
$\mathrm{BR}(B_{s}\to\mu\tau)$ | $7.51\times 10^{-13}$ | 0 | $4.2\times 10^{-5}$ | [90]
$\mathrm{BR}(B_{s}\to\mu e)$ | $7.99\times 10^{-13}$ | 0 | $5.4\times 10^{-9}$ | [90]
$\mathrm{BR}(B_{s}\to ee)$ | $7.48\times 10^{-14}$ | 0 | $2.8\times 10^{-7}$ | [90]
$\mathrm{BR}(B_{s}\to\mu\mu)$ | $3.81\times 10^{-9}$ | $2.2\times 10^{-9}$ | $3.8\times 10^{-9}$ | [90]
$\mathrm{BR}(B_{s}\to\tau\tau)$ | $7.16\times 10^{-7}$ | 0 | $6.8\times 10^{-3}$ | [90]
$\mathrm{BR}(\mu\to e)^{\mathrm{Al}}$ | $7.67\times 10^{-16}$ | 0 | $6\times 10^{-17}$ | [91]
$\mathrm{BR}(\mu\to e)^{\mathrm{Au}}$ | $9.37\times 10^{-16}$ | 0 | $3\times 10^{-13}$ |  [93](prospect)
$|C_{B_{d}}|$ | 0.94 | 0.83 | 1.27 | [100, 101]
$\mathrm{Arg}C_{B_{d}}$ | -0.75 | -5.6 | 1.6 | [100, 101]
$|C_{B_{s}}|$ | 1.03 | 0.93 | 1.29 | [100, 101]
$\mathrm{Arg}C_{B_{s}}$ | $-1.71\times 10^{-2}$ | -1.36 | 2.2 | [100, 101]
$\mathrm{Im}C_{K}$ | 0.92 | 0.88 | 1.36 | [100, 101]
$\mathrm{BR}(\mu\to e\gamma)$ | $3.69\times 10^{-14}$ | 0 | $4.2\times 10^{-13}$ | [96]
$\mathrm{BR}(\tau\to e\gamma)$ | $4.38\times 10^{-15}$ | 0 | $3.3\times 10^{-8}$ | [90]
$\mathrm{BR}(\tau\to\mu\gamma)$ | $5.09\times 10^{-9}$ | 0 | $4.4\times 10^{-8}$ | [90]

## References

  * [1] W. Buchmuller, R. Ruckl, and D. Wyler, Leptoquarks in Lepton - Quark Collisions, Phys. Lett. B 191 (1987) 442–448. [Erratum: Phys.Lett.B 448, 320–320 (1999)].
  * [2] Y. Sakaki, M. Tanaka, A. Tayduganov, and R. Watanabe, Testing leptoquark models in $\bar{B}\to D^{(*)}\tau\bar{\nu}$, Phys. Rev. D 88 (2013), no. 9 094012, [arXiv:1309.0301].
  * [3] M. Bauer and M. Neubert, Minimal Leptoquark Explanation for the R${}_{D^{(*)}}$ , RK , and $(g-2)_{g}$ Anomalies, Phys. Rev. Lett. 116 (2016), no. 14 141802, [arXiv:1511.01900].
  * [4] S. Fajfer and N. Košnik, Vector leptoquark resolution of $R_{K}$ and $R_{D^{(*)}}$ puzzles, Phys. Lett. B 755 (2016) 270–274, [arXiv:1511.06024].
  * [5] D. Bečirević, S. Fajfer, N. Košnik, and O. Sumensari, Leptoquark model to explain the $B$-physics anomalies, $R_{K}$ and $R_{D}$, Phys. Rev. D 94 (2016), no. 11 115021, [arXiv:1608.08501].
  * [6] B. Bhattacharya, A. Datta, J.-P. Guévin, D. London, and R. Watanabe, Simultaneous Explanation of the $R_{K}$ and $R_{D^{(*)}}$ Puzzles: a Model Analysis, JHEP 01 (2017) 015, [arXiv:1609.09078].
  * [7] X.-Q. Li, Y.-D. Yang, and X. Zhang, Revisiting the one leptoquark solution to the R(D(∗)) anomalies and its phenomenological implications, JHEP 08 (2016) 054, [arXiv:1605.09308].
  * [8] R. Barbieri, C. W. Murphy, and F. Senia, B-decay Anomalies in a Composite Leptoquark Model, Eur. Phys. J. C 77 (2017), no. 1 8, [arXiv:1611.04930].
  * [9] S. Sahoo, R. Mohanta, and A. K. Giri, Explaining the $R_{K}$ and $R_{D^{(*)}}$ anomalies with vector leptoquarks, Phys. Rev. D 95 (2017), no. 3 035027, [arXiv:1609.04367].
  * [10] N. Assad, B. Fornal, and B. Grinstein, Baryon Number and Lepton Universality Violation in Leptoquark and Diquark Models, Phys. Lett. B 777 (2018) 324–331, [arXiv:1708.06350].
  * [11] D. Buttazzo, A. Greljo, G. Isidori, and D. Marzocca, B-physics anomalies: a guide to combined explanations, JHEP 11 (2017) 044, [arXiv:1706.07808].
  * [12] L. Di Luzio, A. Greljo, and M. Nardecchia, Gauge leptoquark as the origin of B-physics anomalies, Phys. Rev. D 96 (2017), no. 11 115011, [arXiv:1708.08450].
  * [13] D. Bečirević and O. Sumensari, A leptoquark model to accommodate $R_{K}^{\mathrm{exp}}<R_{K}^{\mathrm{SM}}$ and $R_{K^{\ast}}^{\mathrm{exp}}<R_{K^{\ast}}^{\mathrm{SM}}$, JHEP 08 (2017) 104, [arXiv:1704.05835].
  * [14] A. Angelescu, D. Bečirević, D. Faroughy, and O. Sumensari, Closing the window on single leptoquark solutions to the $B$-physics anomalies, JHEP 10 (2018) 183, [arXiv:1808.08179].
  * [15] S. Iguro, T. Kitahara, Y. Omura, R. Watanabe, and K. Yamamoto, D∗ polarization vs. ${R}_{D^{\left(\ast\right)}}$ anomalies in the leptoquark models, JHEP 02 (2019) 194, [arXiv:1811.08899].
  * [16] C. Cornella, J. Fuentes-Martin, and G. Isidori, Revisiting the vector leptoquark explanation of the B-physics anomalies, JHEP 07 (2019) 168, [arXiv:1903.11517].
  * [17] J. C. Pati and A. Salam, Lepton Number as the Fourth Color, Phys. Rev. D 10 (1974) 275–289. [Erratum: Phys.Rev.D 11, 703–703 (1975)].
  * [18] Z. Poh, S. Raby, and Z.-z. Wang, Pati-Salam SUSY GUT with Yukawa unification, Phys. Rev. D 95 (2017), no. 11 115025, [arXiv:1703.09309].
  * [19] L. Calibbi, A. Crivellin, and T. Li, Model of vector leptoquarks in view of the $B$-physics anomalies, Phys. Rev. D 98 (2018), no. 11 115002, [arXiv:1709.00692].
  * [20] M. J. Dolan, T. P. Dutka, and R. R. Volkas, Lowering the scale of Pati-Salam breaking through seesaw mixing, arXiv:2012.05976.
  * [21] LHCb Collaboration, R. Aaij et al., Measurement of Form-Factor-Independent Observables in the Decay $B^{0}\to K^{*0}\mu^{+}\mu^{-}$, Phys. Rev. Lett. 111 (2013) 191801, [arXiv:1308.1707].
  * [22] LHCb Collaboration, R. Aaij et al., Angular analysis of the $B^{0}\to K^{*0}\mu^{+}\mu^{-}$ decay using 3 fb-1 of integrated luminosity, JHEP 02 (2016) 104, [arXiv:1512.04442].
  * [23] LHCb Collaboration, R. Aaij et al., Test of lepton universality with $B^{0}\rightarrow K^{*0}\ell^{+}\ell^{-}$ decays, JHEP 08 (2017) 055, [arXiv:1705.05802].
  * [24] LHCb Collaboration, R. Aaij et al., Test of lepton universality using $B^{+}\rightarrow K^{+}\ell^{+}\ell^{-}$ decays, Phys. Rev. Lett. 113 (2014) 151601, [arXiv:1406.6482].
  * [25] LHCb Collaboration, R. Aaij et al., Search for lepton-universality violation in $B^{+}\to K^{+}\ell^{+}\ell^{-}$ decays, Phys. Rev. Lett. 122 (2019), no. 19 191801, [arXiv:1903.09252].
  * [26] Belle Collaboration, A. Abdesselam et al., Test of lepton flavor universality in ${B\to K^{\ast}\ell^{+}\ell^{-}}$ decays at Belle, arXiv:1904.02440.
  * [27] Belle Collaboration, A. Abdesselam et al., Measurement of $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ with a semileptonic tagging method, arXiv:1904.08794.
  * [28] LHCb Collaboration, R. Aaij et al., Measurement of $CP$-Averaged Observables in the $B^{0}\rightarrow K^{*0}\mu^{+}\mu^{-}$ Decay, Phys. Rev. Lett. 125 (2020), no. 1 011802, [arXiv:2003.04831].
  * [29] BaBar Collaboration, J. Lees et al., Evidence for an excess of $\bar{B}\to D^{(*)}\tau^{-}\bar{\nu}_{\tau}$ decays, Phys. Rev. Lett. 109 (2012) 101802, [arXiv:1205.5442].
  * [30] BaBar Collaboration, J. Lees et al., Measurement of an Excess of $\bar{B}\to D^{(*)}\tau^{-}\bar{\nu}_{\tau}$ Decays and Implications for Charged Higgs Bosons, Phys. Rev. D 88 (2013), no. 7 072012, [arXiv:1303.0571].
  * [31] LHCb Collaboration, R. Aaij et al., Measurement of the ratio of branching fractions $\mathcal{B}(\bar{B}^{0}\to D^{*+}\tau^{-}\bar{\nu}_{\tau})/\mathcal{B}(\bar{B}^{0}\to D^{*+}\mu^{-}\bar{\nu}_{\mu})$, Phys. Rev. Lett. 115 (2015), no. 11 111803, [arXiv:1506.08614]. [Erratum: Phys.Rev.Lett. 115, 159901 (2015)].
  * [32] Belle Collaboration, M. Huschle et al., Measurement of the branching ratio of $\bar{B}\to D^{(\ast)}\tau^{-}\bar{\nu}_{\tau}$ relative to $\bar{B}\to D^{(\ast)}\ell^{-}\bar{\nu}_{\ell}$ decays with hadronic tagging at Belle, Phys. Rev. D 92 (2015), no. 7 072014, [arXiv:1507.03233].
  * [33] Belle Collaboration, Y. Sato et al., Measurement of the branching ratio of $\bar{B}^{0}\rightarrow D^{*+}\tau^{-}\bar{\nu}_{\tau}$ relative to $\bar{B}^{0}\rightarrow D^{*+}\ell^{-}\bar{\nu}_{\ell}$ decays with a semileptonic tagging method, Phys. Rev. D 94 (2016), no. 7 072007, [arXiv:1607.07923].
  * [34] Belle Collaboration, S. Hirose et al., Measurement of the $\tau$ lepton polarization and $R(D^{*})$ in the decay $\bar{B}\to D^{*}\tau^{-}\bar{\nu}_{\tau}$, Phys. Rev. Lett. 118 (2017), no. 21 211801, [arXiv:1612.00529].
  * [35] LHCb Collaboration, R. Aaij et al., Test of Lepton Flavor Universality by the measurement of the $B^{0}\to D^{*-}\tau^{+}\nu_{\tau}$ branching fraction using three-prong $\tau$ decays, Phys. Rev. D 97 (2018), no. 7 072013, [arXiv:1711.02505].
  * [36] LHCb Collaboration, R. Aaij et al., Angular analysis of the $B^{+}\rightarrow K^{\ast+}\mu^{+}\mu^{-}$ decay, arXiv:2012.13241.
  * [37] F. U. Bernlochner, Z. Ligeti, M. Papucci, and D. J. Robinson, Combined analysis of semileptonic $B$ decays to $D$ and $D^{*}$: $R(D^{(*)})$, $|V_{cb}|$, and new physics, Phys. Rev. D 95 (2017), no. 11 115008, [arXiv:1703.05330]. [Erratum: Phys.Rev.D 97, 059902 (2018)].
  * [38] D. Bigi, P. Gambino, and S. Schacht, $R(D^{*})$, $|V_{cb}|$, and the Heavy Quark Symmetry relations between form factors, JHEP 11 (2017) 061, [arXiv:1707.09509].
  * [39] M. Bordone, M. Jung, and D. van Dyk, Theory determination of $\bar{B}\to D^{(*)}\ell^{-}\bar{\nu}$ form factors at $\mathcal{O}(1/m_{c}^{2})$, Eur. Phys. J. C 80 (2020), no. 2 74, [arXiv:1908.09398].
  * [40] HFLAV Collaboration, Y. S. Amhis et al., Averages of $b$-hadron, $c$-hadron, and $\tau$-lepton properties as of 2018, arXiv:1909.12524.
  * [41] S. Iguro and R. Watanabe, Bayesian fit analysis to full distribution data of $\overline{\mathrm{B}}\to{\mathrm{D}}^{\left(\ast\right)}\mathrm{\ell}\overline{\nu}:\left|{\mathrm{V}}_{\mathrm{cb}}\right|$ determination and new physics constraints, JHEP 08 (2020), no. 08 006, [arXiv:2004.10208].
  * [42] B. Bhattacharya, A. Datta, D. London, and S. Shivashankara, Simultaneous Explanation of the $R_{K}$ and $R(D^{(*)})$ Puzzles, Phys. Lett. B 742 (2015) 370–374, [arXiv:1412.7164].
  * [43] R. Barbieri, G. Isidori, A. Pattori, and F. Senia, Anomalies in $B$-decays and $U(2)$ flavour symmetry, Eur. Phys. J. C 76 (2016), no. 2 67, [arXiv:1512.01560].
  * [44] D. Das, C. Hati, G. Kumar, and N. Mahajan, Towards a unified explanation of $R_{D^{(\ast)}}$, $R_{K}$ and $(g-2)_{\mu}$ anomalies in a left-right model with leptoquarks, Phys. Rev. D 94 (2016) 055034, [arXiv:1605.06313].
  * [45] A. Greljo and B. A. Stefanek, Third family quark–lepton unification at the TeV scale, Phys. Lett. B 782 (2018) 131–138, [arXiv:1802.04274].
  * [46] L. Di Luzio, J. Fuentes-Martin, A. Greljo, M. Nardecchia, and S. Renner, Maximal Flavour Violation: a Cabibbo mechanism for leptoquarks, JHEP 11 (2018) 081, [arXiv:1808.00942].
  * [47] M. Bordone, C. Cornella, J. Fuentes-Martin, and G. Isidori, A three-site gauge model for flavor hierarchies and flavor anomalies, Phys. Lett. B 779 (2018) 317–323, [arXiv:1712.01368].
  * [48] M. Bordone, C. Cornella, J. Fuentes-Martín, and G. Isidori, Low-energy signatures of the $\mathrm{PS}^{3}$ model: from $B$-physics anomalies to LFV, JHEP 10 (2018) 148, [arXiv:1805.09328].
  * [49] J. Fuentes-Martín and P. Stangl, Third-family quark-lepton unification with a fundamental composite Higgs, Phys. Lett. B 811 (2020) 135953, [arXiv:2004.11376].
  * [50] J. Fuentes-Martín, G. Isidori, M. König, and N. Selimović, Vector Leptoquarks Beyond Tree Level III: Vector-like Fermions and Flavor-Changing Transitions, Phys. Rev. D 102 (2020) 115015, [arXiv:2009.11296].
  * [51] D. Guadagnoli, M. Reboud, and P. Stangl, The Dark Side of 4321, JHEP 10 (2020) 084, [arXiv:2005.10117].
  * [52] W. Altmannshofer, S. Gori, M. Pospelov, and I. Yavin, Quark flavor transitions in $L_{\mu}-L_{\tau}$ models, Phys. Rev. D89 (2014) 095033, [arXiv:1403.1269].
  * [53] A. Crivellin, G. D’Ambrosio, and J. Heeck, Explaining $h\to\mu^{\pm}\tau^{\mp}$, $B\to K^{*}\mu^{+}\mu^{-}$ and $B\to K\mu^{+}\mu^{-}/B\to Ke^{+}e^{-}$ in a two-Higgs-doublet model with gauged $L_{\mu}-L_{\tau}$, Phys. Rev. Lett. 114 (2015) 151801, [arXiv:1501.00993].
  * [54] S. Raby and A. Trautner, Vectorlike chiral fourth family to explain muon anomalies, Phys. Rev. D97 (2018), no. 9 095006, [arXiv:1712.09360].
  * [55] S. F. King, Flavourful $Z^{\prime}$ models for ${R}_{K^{\left(\ast\right)}}$, JHEP 08 (2017) 019, [arXiv:1706.06100].
  * [56] A. Falkowski, S. F. King, E. Perdomo, and M. Pierre, Flavourful $Z^{\prime}$ portal for vector-like neutrino Dark Matter and $R_{K^{(*)}}$, JHEP 08 (2018) 061, [arXiv:1803.04430].
  * [57] J. Kawamura, S. Raby, and A. Trautner, Complete vectorlike fourth family and new U(1)’ for muon anomalies, Phys. Rev. D 100 (2019), no. 5 055030, [arXiv:1906.11297].
  * [58] J. Kawamura, S. Raby, and A. Trautner, Complete vectorlike fourth family with U(1)’ : A global analysis, Phys. Rev. D 101 (2020), no. 3 035026, [arXiv:1911.11075].
  * [59] B. Gripaios, M. Nardecchia, and S. A. Renner, Linear flavour violation and anomalies in B physics, JHEP 06 (2016) 083, [arXiv:1509.05020].
  * [60] P. Arnan, L. Hofer, F. Mescia, and A. Crivellin, Loop effects of heavy new scalars and fermions in $b\to s\mu^{+}\mu^{-}$, JHEP 04 (2017) 043, [arXiv:1608.07832].
  * [61] B. Grinstein, S. Pokorski, and G. G. Ross, Lepton non-universality in $B$ decays and fermion mass structure, JHEP 12 (2018) 079, [arXiv:1809.01766].
  * [62] P. Arnan, A. Crivellin, M. Fedele, and F. Mescia, Generic Loop Effects of New Scalars and Fermions in $b\to s\ell^{+}\ell^{-}$ and a Vector-like $4^{\rm th}$ Generation, arXiv:1904.05890.
  * [63] C.-W. Chiang and H. Okada, A simple model for explaining muon-related anomalies and dark matter, arXiv:1711.07365.
  * [64] J. M. Cline and J. M. Cornell, $R({K^{(*)}})$ from dark matter exchange, Phys. Lett. B782 (2018) 232–237, [arXiv:1711.10770].
  * [65] J. Kawamura, S. Okawa, and Y. Omura, Interplay between the b$\to s\ell\ell$ anomalies and dark matter physics, Phys. Rev. D96 (2017), no. 7 075041, [arXiv:1706.04344].
  * [66] B. Barman, D. Borah, L. Mukherjee, and S. Nandi, Correlating the anomalous results in $b\to s$ decays with inert Higgs doublet dark matter and muon $(g-2)$, arXiv:1808.06639.
  * [67] D. G. Cerdeño, A. Cheek, P. Martín-Ramiro, and J. M. Moreno, B anomalies and dark matter: a complex connection, Eur. Phys. J. C 79 (2019), no. 6 517, [arXiv:1902.01789].
  * [68] G. Arcadi, L. Calibbi, M. Fedele, and F. Mescia, Systematic approach to $B$-physics anomalies and $t$-channel dark matter, arXiv:2103.09835.
  * [69] A. Crivellin, C. Greub, and A. Kokulu, Explaining $B\to D\tau\nu$, $B\to D^{*}\tau\nu$ and $B\to\tau\nu$ in a 2HDM of type III, Phys. Rev. D 86 (2012) 054014, [arXiv:1206.2634].
  * [70] M. Tanaka and R. Watanabe, New physics in the weak interaction of $\bar{B}\to D^{(*)}\tau\bar{\nu}$, Phys. Rev. D 87 (2013), no. 3 034028, [arXiv:1212.1878].
  * [71] A. Celis, M. Jung, X.-Q. Li, and A. Pich, Sensitivity to charged scalars in $\bm{B\to D^{(*)}\tau\nu_{\tau}}$ and $\bm{B\to\tau\nu_{\tau}}$ decays, JHEP 01 (2013) 054, [arXiv:1210.8443].
  * [72] A. Crivellin, A. Kokulu, and C. Greub, Flavor-phenomenology of two-Higgs-doublet models with generic Yukawa structure, Phys. Rev. D 87 (2013), no. 9 094031, [arXiv:1303.5877].
  * [73] A. Crivellin, J. Heeck, and P. Stoffer, A perturbed lepton-specific two-Higgs-doublet model facing experimental hints for physics beyond the Standard Model, Phys. Rev. Lett. 116 (2016), no. 8 081801, [arXiv:1507.07567].
  * [74] C.-H. Chen and T. Nomura, Charged-Higgs on $R_{D^{(*)}}$, $\tau$ polarization, and FBA, Eur. Phys. J. C 77 (2017), no. 9 631, [arXiv:1703.03646].
  * [75] S. Iguro and K. Tobe, $R(D^{(*)})$ in a general two Higgs doublet model, Nucl. Phys. B 925 (2017) 560–606, [arXiv:1708.06176].
  * [76] S. Iguro and Y. Omura, Status of the semileptonic $B$ decays and muon g-2 in general 2HDMs with right-handed neutrinos, JHEP 05 (2018) 173, [arXiv:1802.01732].
  * [77] S. Iguro, Y. Omura, and M. Takeuchi, Test of the $R(D^{(*)})$ anomaly at the LHC, Phys. Rev. D 99 (2019), no. 7 075013, [arXiv:1810.05843].
  * [78] R. Alonso, B. Grinstein, and J. Martin Camalich, Lifetime of $B_{c}^{-}$ Constrains Explanations for Anomalies in $B\to D^{(*)}\tau\nu$, Phys. Rev. Lett. 118 (2017), no. 8 081802, [arXiv:1611.06676].
  * [79] A. Akeroyd and C.-H. Chen, Constraint on the branching ratio of $B_{c}\to\tau\bar{\nu}$ from LEP1 and consequences for $R(D^{(*)})$ anomaly, Phys. Rev. D 96 (2017), no. 7 075011, [arXiv:1708.04072].
  * [80] P. Hung, A. Buras, and J. Bjorken, Petite Unification of Quarks and Leptons, Phys. Rev. D 25 (1982) 805.
  * [81] G. Valencia and S. Willenbrock, Quark - lepton unification and rare meson decays, Phys. Rev. D 50 (1994) 6843–6848, [hep-ph/9409201].
  * [82] M. Borsato, V. V. Gligorov, D. Guadagnoli, D. Martinez Santos, and O. Sumensari, Effective-field-theory arguments for pursuing lepton-flavor-violating K decays at LHCb, Phys. Rev. D 99 (2019), no. 5 055017, [arXiv:1808.02006].
  * [83] M. J. Baker, J. Fuentes-Martín, G. Isidori, and M. König, High- $p_{T}$ signatures in vector–leptoquark models, Eur. Phys. J. C 79 (2019), no. 4 334, [arXiv:1901.10480].
  * [84] M. Algueró, B. Capdevila, A. Crivellin, S. Descotes-Genon, P. Masjuan, J. Matias, M. Novoa Brunet, and J. Virto, Emerging patterns of New Physics with and without Lepton Flavour Universal contributions, Eur. Phys. J. C 79 (2019), no. 8 714, [arXiv:1903.09578]. [Addendum: Eur.Phys.J.C 80, 511 (2020)].
  * [85] S. Iguro, M. Takeuchi, and R. Watanabe, Testing Leptoquark/EFT in $\bar{B}\to D^{(*)}l\bar{\nu}$ at the LHC, arXiv:2011.02486.
  * [86] S. Antusch and V. Maurer, Running quark and lepton parameters at various scales, JHEP 11 (2013) 115, [arXiv:1306.6879].
  * [87] ATLAS Collaboration, G. Aad et al., Search for high-mass dilepton resonances using 139 fb-1 of $pp$ collision data collected at $\sqrt{s}=$13 TeV with the ATLAS detector, Phys. Lett. B 796 (2019) 68–87, [arXiv:1903.06248].
  * [88] ATLAS Collaboration, M. Aaboud et al., Combination of the searches for pair-produced vector-like partners of the third-generation quarks at $\sqrt{s}=$ 13 TeV with the ATLAS detector, Phys. Rev. Lett. 121 (2018), no. 21 211801, [arXiv:1808.02343].
  * [89] J. Heeck and D. Teresi, Pati-Salam explanations of the B-meson anomalies, JHEP 12 (2018) 103, [arXiv:1808.07492].
  * [90] Particle Data Group Collaboration, P. Zyla et al., Review of Particle Physics, PTEP 2020 (2020), no. 8 083C01.
  * [91] SINDRUM II Collaboration, W. H. Bertl et al., A Search for muon to electron conversion in muonic gold, Eur. Phys. J. C 47 (2006) 337–346.
  * [92] DeeMe Collaboration, H. Natori, DeeMe experiment - An experimental search for a mu-e conversion reaction at J-PARC MLF, Nucl. Phys. B Proc. Suppl. 248-250 (2014) 52–57.
  * [93] COMET Collaboration, Y. Kuno, A search for muon-to-electron conversion at J-PARC: The COMET experiment, PTEP 2013 (2013) 022C01.
  * [94] Mu2e Collaboration, R. J. Abrams et al., Mu2e Conceptual Design Report, arXiv:1211.7019.
  * [95] L. Lavoura, General formulae for $f(1)\to f(2)\gamma$, Eur. Phys. J. C29 (2003) 191–195, [hep-ph/0302221].
  * [96] MEG Collaboration, A. Baldini et al., Search for the lepton flavour violating decay $\mu^{+}\rightarrow\mathrm{e}^{+}\gamma$ with the full dataset of the MEG experiment, Eur. Phys. J. C 76 (2016), no. 8 434, [arXiv:1605.05081].
  * [97] MEG II Collaboration, A. M. Baldini et al., The design of the MEG II experiment, Eur. Phys. J. C78 (2018), no. 5 380, [arXiv:1801.04688].
  * [98] A. J. Buras, S. Jager, and J. Urban, Master formulae for Delta F=2 NLO QCD factors in the standard model and beyond, Nucl. Phys. B605 (2001) 600–624, [hep-ph/0102316].
  * [99] A. J. Buras, Weak Hamiltonian, CP violation and rare decays, in Probing the standard model of particle interactions. Proceedings, Summer School in Theoretical Physics, NATO Advanced Study Institute, 68th session, Les Houches, France, July 28-September 5, 1997. Pt. 1, 2, pp. 281–539, 1998\.  hep-ph/9806471.
  * [100] UTfit Collaboration, M. Bona et al., The 2004 UTfit collaboration report on the status of the unitarity triangle in the standard model, JHEP 07 (2005) 028, [hep-ph/0501199].
  * [101] UTfit Collaboration, M. Bona et al., Model-independent constraints on $\Delta F=2$ operators and the scale of new physics, JHEP 03 (2008) 049, [arXiv:0707.0636].
  * [102] L. Di Luzio, M. Kirk, A. Lenz, and T. Rauh, $\Delta M_{s}$ theory precision confronts flavour anomalies, JHEP 12 (2019) 009, [arXiv:1909.11087].
  * [103] R. N. Mohapatra and R. E. Marshak, Local B-L Symmetry of Electroweak Interactions, Majorana Neutrinos and Neutron Oscillations, Phys. Rev. Lett. 44 (1980) 1316–1319. [Erratum: Phys.Rev.Lett. 44, 1643 (1980)].
  * [104] D. G. Phillips, II et al., Neutron-Antineutron Oscillations: Theoretical Status and Experimental Prospects, Phys. Rept. 612 (2016) 1–45, [arXiv:1410.1100].
  * [105] Super-Kamiokande Collaboration, K. Abe et al., The Search for $n-\bar{n}$ oscillation in Super-Kamiokande I, Phys. Rev. D 91 (2015) 072006, [arXiv:1109.4227].
  * [106] Flavour Lattice Averaging Group Collaboration, S. Aoki et al., FLAG Review 2019, arXiv:1902.08191.
  * [107] V. Cirigliano, R. Kitano, Y. Okada, and P. Tuzon, On the model discriminating power of mu —$>$ e conversion in nuclei, Phys. Rev. D 80 (2009) 013002, [arXiv:0904.0957].
  * [108] P. Junnarkar and A. Walker-Loud, Scalar strange content of the nucleon from lattice QCD, Phys. Rev. D 87 (2013) 114510, [arXiv:1301.1114].
  * [109] R. Kitano, M. Koike, and Y. Okada, Detailed calculation of lepton flavor violating muon electron conversion rate for various nuclei, Phys. Rev. D 66 (2002) 096002, [hep-ph/0203110]. [Erratum: Phys.Rev.D 76, 059902 (2007)].
  * [110] T. Suzuki, D. F. Measday, and J. Roalsvig, Total Nuclear Capture Rates for Negative Muons, Phys. Rev. C 35 (1987) 2212.
  * [111] M. Blanke, A. Crivellin, S. de Boer, T. Kitahara, M. Moscati, U. Nierste, and I. Nišandžić, Impact of polarization observables and $B_{c}\to\tau\nu$ on new physics explanations of the $b\to c\tau\nu$ anomaly, Phys. Rev. D 99 (2019), no. 7 075006, [arXiv:1811.09603].


# The Evolutionary Computation Methods No One Should Use

Jakub Kůdela 0000-0002-4372-2105 Institute of Automation and Computer Science,
Brno University of Technology
<EMAIL_ADDRESS>
###### Abstract

The center-bias (or zero-bias) operator has recently been identified as one of
the problems plaguing the benchmarking of evolutionary computation methods.
This operator lets the methods that utilize it easily optimize functions that
have their respective optima in the center of the feasible set. In this paper,
we describe a simple procedure that can be used to identify methods that
incorporate a center-bias operator and use it to investigate 90 evolutionary
computation methods that were published between 1987 and 2022. We show that
more than half (47 out of the 90) of the considered methods have the center-
bias problem. We also show that the center-bias is a relatively new phenomenon
(with the first identified method being from 2012), but its inclusion has
become extremely prevalent in the last few years. Lastly, we briefly discuss
the possible root causes of this issue.

###### keywords:

Evolutionary Computation , Benchmarking , Metaheuristics , Center-bias , Zero-
bias

††journal: …

##  1 Introduction

Imagine the following situation. Encountering a challenging optimization task,
you decide to find the most recently developed algorithm for optimization
published in some of the most prestigious journals. The analysis of the method
performed on several standard benchmarks clearly shows that it is superior to
all the other old methods. The paper also contains a link to a repository with
the code. So, you give it a try. And it fails. The best results it provides
are hardly better (or worse) than the ones you got from a simple
implementation of a method that is more than two decades old. Maybe the
problem you tried to solve is too challenging? Perhaps a bit of hyperparameter
optimization could help the method perform as advertised? Or, maybe the method
is not as good as it presented itself.

Through inspiration from natural behaviors, the field of evolutionary
computation (EC) produced over its long history a great number of important
metaheuristic algorithms, such as Evolutionary Strategy, Genetic Algorithms,
Particle Swarm Optimization, or Differential Evolution. Such methods found
applications in complex systems where the use of exact algorithms was either
inadequate or computationally too prohibitive. However, over the past few
years we have witnessed an explosion of ”novel” methods that are based on
natural/evolutionary principles. The bestiary of EC111Campelo, F., Aranha, C.
Evolutionary computation bestiary. https://github.com/fcampelo/EC-Bestiary,
which tries to catalog of a portion of these nature-based methods, now
contains over 250 methods that claim their inspiration in natural processes.
And new methods are emerging at an ever-increasing rate. It is also becoming
clearer that there is more creativity being spent at naming these ”novel”
methods, than in making sure they contain anything new computation-wise. After
many of these methods have been found to conceal their lack of novelty behind
a methaphor-rich jargon [1, 2, 3, 4, 5], a call was made from within the EC
community [6]. In the letter, the collective of authors and signatories
identified four main issues with the high-volume inflow of new methods:
useless metaphors, lack of novelty, poor experimental validation and
comparison, and publishing these methods in off-topic journals.

In this text, we will focus on the poor experimental validation of some of the
EC methods. Most of the reasoning about the viability of metaheuristics is
done through benchmarking [7]. If a new method performs well on a universally
accepted set of benchmark problems, it is likely to be seen as valid. There
have been several benchmark functions/sets proposed over the years, but the
most widely recognized ones came from special sessions (competitions) on
black-box optimization at two conferences: the IEEE Congress on Evolutionary
Computation (CEC), and the Genetic and Evolutionary Computation Conference
(GECCO), where the Black-Box Optimization Benchmarking (BBOB) workshop was
held.

There is, however, another quite widely used benchmark set that contains some
of the most well-known functions such as Griewank, Ackley, Rastrigin,
Rosenbrock, and Schwefel. It was recently uncovered [8] that this set contains
a serious design flaw, as a large portion of the functions in the set have
their respective optimum at a zero vector (or in the centre of the feasible
set). This would be fine, if it were not for the methods that utilize this
flaw to appear competitive. These methods incorporate a “check-the-middle”
routine or have a centre-bias (or zero-bias) operator that draws them towards
the center of the feasible set. One would expect that such methods do not get
published very often, are easily spotted, or at the very least do not appear
in high-profile journals.

In this paper, we describe a simple methodology that we use to uncover whether
or not a given evolutionary computation method utilizes a center-bias
operator. We then investigate 90 evolutionary computation methods from the
mealpy library222N. V. Thieu, “A collection of the state-of-the-art meta-
heuristics algorithms in python: Mealpy,” Available:
https://doi.org/10.5281/zenodo.3711948 and Mathworks code
repositories333https://www.mathworks.com/ for the inclusion of the center-
bias.

Table 1: The 13 benchmark functions, dimension 30. U -– unimodal, M –-
multimodal, S –- separable, N –- non-separable, $f^{*}$ – the optimal function
value, $f(0)$ – function value at the zero vector, $x^{*}$ – optimal solution.

ID | name | type | range | $f^{*}$ | $f(0)$ | $x*$
---|---|---|---|---|---|---
F01 | Sphere | U, S | [-100,100] | 0 | 0 | [0,0,…]
F02 | Schwefel 2.22 | U, N | [-100,100] | 0 | 0 | [0,0,…]
F03 | Schwefel 1.2 | U, N | [-100,100] | 0 | 0 | [0,0,…]
F04 | Schwefel 2.21 | U, S | [-100,100] | 0 | 0 | [0,0,…]
F05 | Rosenbrock | U, N | [-30,30] | 0 | 2.90E+01 | [1,1,…]
F06 | Step | U, S | [-100,100] | 0 | 7.50E+00 | [-0.5,-0.5,…]
F07 | Quartic with noise | U, S | [-1.28,1.28] | 0 | 0 | [0,0,…]
F08 | Schwefel 2.26 | M, S | [-500,500] | -1.25E+04 | 0 | [420.9, 420.9,…]
F09 | Rastrigin | M, S | [-5.12,5.12] | 0 | 0 | [0,0,…]
F10 | Ackley | M, N | [-32,32] | 0 | 0 | [0,0,…]
F11 | Griewank | M, N | [-600,600] | 0 | 0 | [0,0,…]
F12 | Penalized1 | M, N | [-50,50] | 0 | 1.67E+00 | [-1,-1,…]
F13 | Penalized2 | M, S | [-50,50] | 0 | 3.00E+00 | [1,1,…]

##  2 Methodology

We utilize the same methodology that was used to uncover the center-bias
problem in [9] and [8]. The 13 benchmark function used for our test (and
optimization ranges) are shown in Table 1. One can easily see that many of
these functions, apart from F08, have the corresponding optima either at the
zero vector or very close to it. The problem F08 is quite different from the
rest, as its optimum is far away from the center.

For the evaluation we set the dimension of the problems to 30 and allow for at
most 50,000 function evaluations. We also chose a simple performance measure -
the mean error (as the difference between the optimal function value and best
function value found) over 20 independent runs. Here, we also treat any value
smaller than 1e–08 as identical to 1e–08, as the problem is essentially solved
and additional precision is not needed (we could treat it as a 0 as well, but
we will shortly use fractions of these numbers, which would bring unwanted
hassle). We refer to the results of the computation as the “unshifted” ones.
Afterwards, we introduce a shift operation, that “moves” the benchmark
function by a predetermined vector $s$, meaning that function $f(x)$ becomes
$f(x+s)$. One expects that a “small” value of $s$ should not result in a large
deviation in the behaviour of the optimization method, as the two problems are
very similar. We chose the shift vector as 10% of the range - e.g., for F01,
$s=[20,20,\dots]$. We use the same computational framework (i.e., dimension
30, at maximum 50,000 function evaluations, and 20 independent runs) and refer
to the results of these computations as the “shifted” ones.

What we are interested in is the “ratio” between the “shifted” and “unshifted”
results for the individual benchmark functions, i.e., how many times are the
results on the shifted problem worse than on the unshifted one. For the
methods that do not incorporate a center-bias, one expects this number to be
close to 1 (as the ushifted and shifted problems are similar), while for the
methods that include a center-bias, this ratio should be much bigger than 1.
Naturally, the value of this ratio will fluctuate depending on the given
benchmark function, as well as on the number of independent runs of the
algorithms. As a simple indicator of the center-bias, we look at the geometric
mean of the ratios for the different benchmarks – if this value is bigger than
1E+01 (meaning that the method performs roughly at least on order of magnitude
better on unshifted problems), we take it as a confirmation of the presence of
the center-bias operator.

A small example of this computation is shown in Table 2, where we investigate
five EC methods – Artificial Bee Colony (ABC) [10], Differential Evolution
(DE) [11], LSHADE [12], Satin Bowerbird Optimizer (SBO) [13], and Runge Kutta
Optimizer (RKO) [14]. The first two methods (ABC and DE) can be thought of as
the “standard” ones, LSADE is among the state-of-the ones (as it served as a
basis of many of the best methods for recent CEC competitions), and the last
two (SBO and RKO) are the “new” ones. One can quite easily see that for the
first three methods (ABC, DE, and LSHADE) the geometric mean of the ratios is
roughly 1, meaning that no center-bias was detected. For SBO, the situation is
a bit more complicated, as on many benchmark functions the ratio is relatively
low (roughly between 1 and 2), but is very large (almost 5E+04) on F2. This
could be a fluke. Fortunately, the nature of the geometric mean will suppress
some of the individual flukes - the value for SBO is 3.95E+00 (i.e.,
$<$1E+01), so we do not label it as a method with a center-bias. The same
cannot be said about RKO. Here, many of the ratios are extremely big
($>$1E+06), and the value of the geometric mean is 7.36E+04. We can
confidently say that RKO incorporates a center bias operator.

An interesting observation can be made regarding the benchmark function F08.
For all five methods, the ratio between the shifted and unshifted results on
F08 is very close to 1. Recall that F08 is the only function in the benchmark
set that has the optimum quite far away from the center of the feasible set,
and its function value at the zero-vector is also quite far away from the
optimal value. Although it is arguably not surprising that the methods have a
ratio around 1 on this function, it is still valuable to have it confirmed –
the function F08 serves as a sanity check in the benchmark set.

Table 2: The results of proposed methodology demonstrated on five methods.

|  | ABC |  |  | DE |  |  | LSHADE |  |  | SBO |  |  | RKO |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
| unshifted | shifted | ratio | unshifted | shifted | ratio | unshifted | shifted | ratio | unshifted | shifted | ratio | unshifted | shifted | ratio
F1 | 5.19E+03 | 4.19E+03 | 8.08E$-$01 | 3.82E$-$02 | 2.61E$-$02 | 6.83E$-$01 | 1.00E$-$08 | 1.00E$-$08 | 1.00E+00 | 5.85E+00 | 1.20E+01 | 2.05E+00 | 1.00E$-$08 | 1.30E$-$06 | 1.30E+02
F2 | 3.08E+03 | 5.95E+02 | 1.93E$-$01 | 2.79E+00 | 3.06E+00 | 1.10E+00 | 6.37E$-$06 | 9.55E$-$06 | 1.50E+00 | 1.25E+15 | 6.19E+19 | 4.95E+04 | 1.00E$-$08 | 2.05E+00 | 2.05E+08
F3 | 8.16E+04 | 8.04E+04 | 9.86E$-$01 | 1.89E+04 | 1.64E+04 | 8.68E$-$01 | 4.13E$-$04 | 4.16E$-$04 | 1.01E+00 | 1.48E+04 | 9.08E+04 | 6.14E+00 | 1.00E$-$08 | 5.03E$-$02 | 5.03E+06
F4 | 5.81E+01 | 5.82E+01 | 1.00E+00 | 1.35E+01 | 1.31E+01 | 9.70E$-$01 | 5.08E$-$03 | 3.75E$-$03 | 7.38E$-$01 | 1.28E+01 | 1.98E+01 | 1.55E+00 | 1.00E$-$08 | 2.54E+00 | 2.54E+08
F5 | 1.40E+03 | 1.38E+04 | 9.86E+00 | 6.09E+02 | 5.88E+02 | 9.66E$-$01 | 2.07E+01 | 2.12E+01 | 1.02E+00 | 5.63E+02 | 1.17E+03 | 2.07E+00 | 2.53E+01 | 3.99E+01 | 1.58E+00
F6 | 5.36E+03 | 6.12E+03 | 1.14E+00 | 6.68E+00 | 7.10E+00 | 1.06E+00 | 1.00E$-$08 | 1.00E$-$08 | 1.00E+00 | 1.35E+01 | 2.42E+01 | 1.79E+00 | 1.00E$-$08 | 1.74E$-$06 | 1.74E+02
F7 | 1.45E+00 | 2.26E+00 | 1.55E+00 | 8.56E$-$02 | 8.18E$-$02 | 9.56E$-$01 | 3.19E$-$03 | 3.33E$-$03 | 1.04E+00 | 4.37E$-$01 | 8.21E$-$01 | 1.88E+00 | 9.26E$-$05 | 1.64E$-$02 | 1.76E+02
F8 | 5.07E+03 | 5.79E+03 | 1.14E+00 | 1.43E+04 | 1.50E+04 | 1.05E+00 | 1.64E+02 | 1.80E+02 | 1.10E+00 | 6.99E+03 | 5.99E+03 | 8.58E$-$01 | 4.19E+03 | 4.90E+03 | 1.17E+00
F9 | 2.49E+01 | 3.03E+01 | 1.22E+00 | 3.98E+02 | 3.98E+02 | 1.00E+00 | 1.08E+01 | 1.10E+01 | 1.02E+00 | 3.66E+01 | 5.16E+01 | 1.41E+00 | 1.00E$-$08 | 2.99E+01 | 2.99E+09
F10 | 8.69E+00 | 7.57E+00 | 8.70E$-$01 | 1.53E+00 | 1.44E+00 | 9.41E$-$01 | 2.38E$-$07 | 2.57E$-$07 | 1.08E+00 | 4.72E+00 | 5.48E+00 | 1.16E+00 | 1.00E$-$08 | 2.86E+00 | 2.86E+08
F11 | 3.16E+02 | 3.35E+02 | 1.06E+00 | 1.07E+00 | 1.06E+00 | 9.91E$-$01 | 1.00E$-$08 | 1.00E$-$08 | 1.00E+00 | 1.06E+00 | 1.10E+00 | 1.04E+00 | 1.00E$-$08 | 1.25E$-$02 | 1.25E+06
F12 | 7.71E+05 | 1.09E+06 | 1.41E+00 | 1.13E+00 | 1.27E+00 | 1.12E+00 | 1.00E$-$08 | 1.00E$-$08 | 1.00E+00 | 6.81E+00 | 1.29E+01 | 1.89E+00 | 1.00E$-$08 | 3.37E+00 | 3.37E+08
F13 | 9.98E+05 | 5.45E+06 | 5.46E+00 | 6.36E+00 | 5.94E+00 | 9.34E$-$01 | 1.00E$-$08 | 1.00E$-$08 | 1.00E+00 | 1.51E+00 | 4.63E+00 | 3.07E+00 | 4.40E$-$03 | 1.18E$-$02 | 2.68E+00
geomean | - | - | 1.29E+00 | - | - | 9.66E$-$01 | - | - | 1.03E+00 | - | - | 3.95E+00 | - | - | 7.36E+04

##  3 Results and Discussion

In this section we report the results of using the methodology described in
the previous section on 90 selected EC methods. The selected methods, the year
of the publication that describes them, and the geometric mean of the ratios
are shown (in alphabetical order) in Table 3, with the ones with a confirmed
center-bias (i.e., values $>$1E+01) highlighted in red. These results are
extremely worrying, as more than a half (47 out of the 90) methods have a
confirmed center-bias. And they become even worse when we take a look at the
number of methods with center-bias that were proposed recently, as shown in
Figure 1.

Figure 1: Number of papers proposing methods with/without center-bias in time.

Table 3: Considered algorithms and results.

.

Abbreviation | Method name | Year | Geomean | Abbreviation | Method name | Year | geomean
---|---|---|---|---|---|---|---
ABC [10] | Artificial Bee Colony | 2008 | 1.29E+00 | HC [15] | Hill Climbing | 1993 | 1.13E+00
ACOR [16] | Ant Colony Optimization Continuous | 2008 | 7.40E-01 | HGS [17] | Hunger Games Search | 2021 | 3.69E+06
AEO [18] | Artificial Ecosystem-based Optimization | 2020 | 1.01E+07 | HGSO [19] | Henry Gas Solubility Optimization | 2019 | 8.07E+03
ALO [20] | Ant Lion Optimizer | 2015 | 1.44E+00 | HHO [21] | Harris Hawks Optimization | 2019 | 1.62E+05
AO [22] | Aquila Optimization | 2021 | 2.26E+05 | HS [23] | Harmony Search | 2001 | 9.97E-01
AOA [24] | Arithmetic Optimization Algorithm | 2021 | 1.01E+10 | IWO [25] | Invasive Weed Optimization | 2006 | 1.88E+00
ArchOA [26] | Archimedes Optimization Algorithm | 2021 | 3.75E+07 | JA [27] | Jaya Algorithm | 2016 | 1.19E+01
ASO [28] | Atom Search Optimization | 2019 | 8.71E-01 | KMA [29] | Komodo Mlipir Algorithm | 2022 | 1.84E+05
BA [30] | Bat-inspired Algorithm | 2010 | 1.44E+00 | LCO [31] | Life Choice-based Optimization | 2020 | 8.31E+07
BBO [32] | Biogeography-Based Optimization | 2008 | 6.43E-01 | MA [33] | Memetic Algorithm | 1989 | 1.68E-03
BeesA [34] | Bees Algorithm | 2006 | 1.16E+00 | MFO [35] | Moth-Flame Optimization | 2015 | 1.73E-01
BES [36] | Bald Eagle Search | 2020 | 2.62E+08 | MGO [37] | Mountain Gazelle Optimizer | 2022 | 1.28E+01
BFO [38] | Bacterial Foraging Optimization | 2002 | 9.66E-01 | MPA [39] | Marine Predators Algorithm | 2020 | 1.06E+02
BOA [40] | Butterfly Optimization Algorithm | 2019 | 9.57E+05 | MRFO [41] | Manta Ray Foraging Optimization | 2020 | 6.40E+07
BRO [42] | Battle Royale Optimization | 2021 | 2.59E+09 | MSA [43] | Moth Search Algorithm | 2018 | 8.37E+00
BSA [44] | Bird Swarm Algorithm | 2016 | 1.09E+01 | MVO [45] | Multi-Verse Optimizer | 2016 | 1.75E+00
BSO [46] | Brain Storm Optimization | 2011 | 7.85E+00 | NMRA [47] | Naked Mole-Rat Algorithm | 2019 | 5.65E+08
CA [48] | Culture Algorithm | 2009 | 7.18E-01 | NRO [49] | Nuclear Reaction Optimization | 2019 | 2.30E+06
CEM [50] | Cross-Entropy Method | 2005 | 1.33E+00 | PFA [51] | Pathfinder Algorithm | 2019 | 3.11E+08
CGO [52] | Chaos Game Optimization | 2021 | 2.14E+07 | PSO [53] | Particle Swarm Optimization | 1995 | 9.70E-01
ChOA [54] | Chimp optimization algorithm | 2020 | 3.89E+03 | PSS [55] | Pareto-like Sequential Sampling | 2021 | 1.77E+03
COA [56] | Coyote Optimization Algorithm | 2018 | 4.00E+06 | QSA [57] | Queuing Search Algorithm | 2021 | 7.91E-01
CRO [58] | Coral Reefs Optimization | 2014 | 9.69E-01 | RKO [14] | Runge Kutta Optimizer | 2021 | 7.36E+04
CSA [59] | Cuckoo Search Algorithm | 2009 | 1.10E+00 | SA [60] | Simulated Annealing | 1987 | 8.95E-01
CSO [61] | Cat Swarm Optimization | 2006 | 9.58E-01 | SARO [62] | Search And Rescue Optimization | 2019 | 2.27E+00
DE [11] | Differential Evolution | 1997 | 9.66E-01 | SBO [13] | Satin Bowerbird Optimizer | 2017 | 3.95E+00
DandO [63] | Dandelion Optimizer | 2022 | 3.59E+02 | SCA [64] | Sine Cosine Algorithm | 2016 | 1.18E+04
DO [65] | Dragonfly Optimization | 2016 | 6.62E+02 | SFO [66] | SailFish Optimizer | 2019 | 2.57E+07
EFO [67] | Electromagnetic Field Optimization | 2016 | 6.78E-01 | SHO [68] | Spotted Hyena Optimizer | 2017 | 1.31E+00
EHO [69] | Elephant Herding Optimization | 2015 | 3.99E+03 | SLO [70] | Sea Lion Optimization Algorithm | 2019 | 2.83E+00
EO [71] | Equilibrium Optimizer | 2020 | 4.65E+03 | SMA [72] | Slime Mould Algorithm | 2020 | 4.54E+06
EOA [73] | Earthworm Optimisation Algorithm | 2018 | 2.55E+05 | SRSR [74] | Swarm Robotics Search And Rescue | 2017 | 2.03E+00
EP [75] | Evolutionary Programming | 1999 | 1.43E+00 | SSA [76] | Sparrow Search Algorithm | 2020 | 2.61E+06
ES [77] | Evolution Strategies | 2002 | 1.14E+00 | SSDO [78] | Social Ski-Driver Optimization | 2020 | 5.40E+08
FA [79] | Fireworks Algorithm | 2010 | 1.34E+00 | SSO [80] | Salp Swarm Optimization | 2017 | 2.28E+01
FBIO [81] | Forensic-Based Investigation Optimization | 2020 | 5.07E+06 | SSpiderA [82] | Social Spider Algorithm | 2015 | 1.12E+00
FFA [83] | Firefly Algorithm | 2011 | 1.18E+00 | STOA [84] | Sooty Tern Optimization Algorithm | 2019 | 6.78E+04
FOA [85] | Fruit-fly Optimization Algorithm | 2012 | 4.01E+00 | TLO [86] | Teaching Learning-based Optimization | 2012 | 3.19E+04
FPA [87] | Flower Pollination Algorithm | 2012 | 9.74E-01 | TPO [88] | Tree Physiology Optimization | 2019 | 2.20E+01
GA [89] | Genetic Algorithm | 1994 | 1.02E+00 | TSA [90] | Tunicate Swarm Algorithm | 2020 | 6.25E+06
GBO [91] | Gradient-Based Optimizer | 2020 | 7.17E+07 | TWO [92] | Tug of War Optimization | 2017 | 9.69E-01
GCO [93] | Germinal Center Optimization | 2018 | 1.02E+00 | VCS [94] | Virus Colony Search | 2016 | 2.90E+04
GOA [95] | Grasshopper Optimization Algorithm | 2017 | 3.39E+00 | WDO [96] | Wind Driven Optimization | 2013 | 4.86E+01
GSKA [97] | Gaining Sharing Knowledge-based Algorithm | 2020 | 4.51E-01 | WHO [98] | Wildebeest Herd Optimization | 2019 | 8.63E+02
GWO [99] | Grey Wolf Optimizer | 2014 | 8.89E+05 | WOA [100] | Whale Optimization Algorithm | 2016 | 1.87E+03

We can find that while the number newly proposed methods that do not have the
center-bias problem increased only slightly over the last three decades, the
number of methods that we have identified as having a center-bias problem is
growing extremely fast, especially in the last five years. It has gotten so
bad that an overwhelming majority of newly proposed methods have the center-
bias problem. An important thing to remark is that we only considered the
“baseline” (or original) versions of the methods, and not any of the
“improved” or “enhanced” variants that are also being published at an ever-
increasing rate. If these were considered as well, we suspect that the graph
would look even worse.

We can also see that the first method that we have found to incorporate the
center-bias was Teaching Learning-based Optimization (TLO) in 2012, followed
by Wind Driven Optimization (WDO) in 2013, and Grey Wolf Optimizer (GWO) in
2014. From these three, TLO and GWO have become extremely influential
(gathering thousand of citations) and spawned a large number of variants and
modifications. Our failure to quickly identify that they are defective is one
of the root causes of the mess we have to deal with now. Although the defect
of the GWO was uncovered in 2019 [101], GWO is still used in numerical
comparisons (even on problems that are susceptible to the center-bias
operator). Similar defects have been also found for the Salp Swarm
Optimization (SSO), Sooty Tern Optimization Algorithm (STOA), Tunicate Swarm
Algorithm (TSA), Harris Hawks Optimization (HHO), Butterfly Optimization
Algorithm (BOA), Slime Mould Algorithm (SMA), Gradient-Based Optimizer (GBO),
Marine Predators Algorithm (MPA), and Komodo Mlipir Algorithm (KMA), all in
2022 [102, 9, 8].

For the most part, the methods that incorporate a center-bias procedure have
been developed by a diverse groups of authors (i.e., most authors have only
one or two such methods). There is, however, one very notable exception. The
group of S. Mirjalili, A. H. Gandomi, and A. A. Heidari is collectively
responsible for 20 of the 47 methods that contain center-bias (and S.
Mirjalili is also one of the authors of GWO).

Another interesing point to make is that some of methods that display the
worst center-bias properties (i.e., the largest values of the geometric mean
of the ratios) are the ones which were supposedly based on “mathematical”
processes – Arithmetic Optimization Algorithm (AOA), Gradient-Based Optimizer
(GBO), Runge Kutta Optimizer (RKO), and Sine Cosine Algorithm (SCA). The
following are the first few sentences from the abstract of the paper
describing RKO [14]:

“The optimization field suffers from the metaphor-based “pseudo-novel” or
“fancy” optimizers. Most of these cliché methods mimic animals’ searching
trends and possess a small contribution to the optimization process itself.
Most of these cliché methods suffer from the locally efficient performance,
biased verification methods on easy problems, and high similarity between
their components’ interactions. This study attempts to go beyond the traps of
metaphors and introduce a novel metaphor-free population-based optimization
method based on the mathematical foundations and ideas of the Runge Kutta (RK)
method widely well-known in mathematics.” The irony is rich.

##  4 Conclusion

The center-bias problem is right now one of the major issues plaguing the
field of evolutionary computation. In this paper, we have described a simple
procedure for identifying methods with center-bias and used it to investigate
90 methods that were proposed in the last three decades. We have found that 47
of the 90 methods utilize center-bias. We have also shown that the utilization
of center-bias is a relatively new phenomenon, with first instances from
2012-2014. However, the number of methods that use it grew extremely fast in
the last five years.

We should note that there is an additional problem that plagues the field
right now, which is the equivalence of some of the methods that is hidden
under a metaphore-rich jargon. Some of the methods that we have identified as
not having a center bias, such as Harmony Search (HS), Cockoo Search Algorith
(CSA), Firefly Algorithm (FA), Moth-Flame Optimization (MFO), Ant Lion
Optimizer (ALO) should also not be used, as they have been found to be either
extremely similar (or identical) to other methods [1, 4, 103].

Further utilization, development and improvement of the methods that contain a
center-bias is an exercise in futility, as by their very nature they cannot be
considered as efficient algorithms. Enough computational and human resources
were already wasted in writing, testing, comparing, and reviewing these
methods. The field of evolutionary computation needs a spring cleaning. The
sooner the better.

## Acknowledgment

This work was supported by IGA BUT: FSI-S-20-6538.

## References

  * [1] D. Weyland, A rigorous analysis of the harmony search algorithm: How the research community can be misled by a “novel” methodology, International Journal of Applied Metaheuristic Computing (IJAMC) 1 (2) (2010) 50–60.
  * [2] C. L. Camacho-Villalón, M. Dorigo, T. Stützle, The intelligent water drops algorithm: why it cannot be considered a novel algorithm, Swarm Intelligence 13 (3) (2019) 173–192.
  * [3] C. L. Camacho Villalón, T. Stützle, M. Dorigo, Grey wolf, firefly and bat algorithms: Three widespread algorithms that do not contain any novelty, in: International conference on swarm intelligence, Springer, 2020, pp. 121–133.
  * [4] C. Villalón, T. Stützle, M. Dorigo, Cuckoo search$\equiv$($\mu$\+ $\lambda$)–evolution strategy, in: IRIDIA–Technical Report Series, IRIDIA, 2021\.
  * [5] A. P. Piotrowski, J. J. Napiorkowski, P. M. Rowinski, How novel is the “novel” black hole optimization approach?, Information Sciences 267 (2014) 191–200.
  * [6] C. Aranha, C. L. Camacho Villalón, F. Campelo, M. Dorigo, R. Ruiz, M. Sevaux, K. Sörensen, T. Stützle, Metaphor-based metaheuristics, a call for action: the elephant in the room, Swarm Intelligence 16 (1) (2022) 1–6.
  * [7] M. Hellwig, H.-G. Beyer, Benchmarking evolutionary algorithms for single objective real-valued constrained optimization–a critical review, Swarm and evolutionary computation 44 (2019) 927–944.
  * [8] J. Kudela, A critical problem in benchmarking and analysis of evolutionary computation methods, Nature Machine Intelligence 4 (2022) 1238–1245.
  * [9] J. Kudela, Commentary on:“stoa: A bio-inspired based optimization algorithm for industrial engineering problems”[eaai, 82 (2019), 148–174] and “tunicate swarm algorithm: A new bio-inspired based metaheuristic paradigm for global optimization”[eaai, 90 (2020), no. 103541], Engineering Applications of Artificial Intelligence 113 (2022) 104930.
  * [10] D. Karaboga, B. Basturk, On the performance of artificial bee colony (abc) algorithm, Applied soft computing 8 (1) (2008) 687–697.
  * [11] R. Storn, K. Price, Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces, Journal of global optimization 11 (4) (1997) 341–359.
  * [12] R. Tanabe, A. S. Fukunaga, Improving the search performance of shade using linear population size reduction, in: 2014 IEEE congress on evolutionary computation (CEC), IEEE, 2014, pp. 1658–1665.
  * [13] S. H. S. Moosavi, V. K. Bardsiri, Satin bowerbird optimizer: A new optimization algorithm to optimize anfis for software development effort estimation, Engineering Applications of Artificial Intelligence 60 (2017) 1–15.
  * [14] I. Ahmadianfar, A. A. Heidari, A. H. Gandomi, X. Chu, H. Chen, Run beyond the metaphor: An efficient optimization algorithm based on runge kutta method, Expert Systems with Applications 181 (2021) 115079.
  * [15] M. Mitchell, J. Holland, S. Forrest, When will a genetic algorithm outperform hill climbing, Advances in neural information processing systems 6 (1993).
  * [16] K. Socha, M. Dorigo, Ant colony optimization for continuous domains, European journal of operational research 185 (3) (2008) 1155–1173.
  * [17] Y. Yang, H. Chen, A. A. Heidari, A. H. Gandomi, Hunger games search: Visions, conception, implementation, deep analysis, perspectives, and towards performance shifts, Expert Systems with Applications 177 (2021) 114864.
  * [18] W. Zhao, L. Wang, Z. Zhang, Artificial ecosystem-based optimization: a novel nature-inspired meta-heuristic algorithm, Neural Computing and Applications 32 (13) (2020) 9383–9425.
  * [19] F. A. Hashim, E. H. Houssein, M. S. Mabrouk, W. Al-Atabany, S. Mirjalili, Henry gas solubility optimization: A novel physics-based algorithm, Future Generation Computer Systems 101 (2019) 646–667.
  * [20] S. Mirjalili, The ant lion optimizer, Advances in engineering software 83 (2015) 80–98.
  * [21] A. A. Heidari, S. Mirjalili, H. Faris, I. Aljarah, M. Mafarja, H. Chen, Harris hawks optimization: Algorithm and applications, Future generation computer systems 97 (2019) 849–872.
  * [22] L. Abualigah, D. Yousri, M. Abd Elaziz, A. A. Ewees, M. A. Al-Qaness, A. H. Gandomi, Aquila optimizer: a novel meta-heuristic optimization algorithm, Computers & Industrial Engineering 157 (2021) 107250.
  * [23] Z. W. Geem, J. H. Kim, G. V. Loganathan, A new heuristic optimization algorithm: harmony search, simulation 76 (2) (2001) 60–68.
  * [24] L. Abualigah, A. Diabat, S. Mirjalili, M. Abd Elaziz, A. H. Gandomi, The arithmetic optimization algorithm, Computer methods in applied mechanics and engineering 376 (2021) 113609.
  * [25] A. R. Mehrabian, C. Lucas, A novel numerical optimization algorithm inspired from weed colonization, Ecological informatics 1 (4) (2006) 355–366.
  * [26] F. A. Hashim, K. Hussain, E. H. Houssein, M. S. Mabrouk, W. Al-Atabany, Archimedes optimization algorithm: a new metaheuristic algorithm for solving optimization problems, Applied Intelligence 51 (3) (2021) 1531–1551.
  * [27] R. Rao, Jaya: A simple and new optimization algorithm for solving constrained and unconstrained optimization problems, International Journal of Industrial Engineering Computations 7 (1) (2016) 19–34.
  * [28] W. Zhao, L. Wang, Z. Zhang, Atom search optimization and its application to solve a hydrogeologic parameter estimation problem, Knowledge-Based Systems 163 (2019) 283–304.
  * [29] S. Suyanto, A. A. Ariyanto, A. F. Ariyanto, Komodo mlipir algorithm, Applied Soft Computing 114 (2022) 108043.
  * [30] X.-S. Yang, A new metaheuristic bat-inspired algorithm, in: Nature inspired cooperative strategies for optimization (NICSO 2010), Springer, 2010, pp. 65–74.
  * [31] A. Khatri, A. Gaba, K. Rana, V. Kumar, A novel life choice-based optimizer, Soft computing 24 (12) (2020) 9121–9141.
  * [32] D. Simon, Biogeography-based optimization, IEEE transactions on evolutionary computation 12 (6) (2008) 702–713.
  * [33] P. Moscato, et al., On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms, Caltech concurrent computation program, C3P Report 826 (1989) 1989.
  * [34] D. T. Pham, A. Ghanbarzadeh, E. Koç, S. Otri, S. Rahim, M. Zaidi, The bees algorithm—a novel tool for complex optimisation problems, in: Intelligent production machines and systems, Elsevier, 2006, pp. 454–459.
  * [35] S. Mirjalili, Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm, Knowledge-based systems 89 (2015) 228–249.
  * [36] H. A. Alsattar, A. Zaidan, B. Zaidan, Novel meta-heuristic bald eagle search optimisation algorithm, Artificial Intelligence Review 53 (3) (2020) 2237–2264.
  * [37] B. Abdollahzadeh, F. S. Gharehchopogh, N. Khodadadi, S. Mirjalili, Mountain gazelle optimizer: A new nature-inspired metaheuristic algorithm for global optimization problems, Advances in Engineering Software 174 (2022) 103282.
  * [38] K. M. Passino, Biomimicry of bacterial foraging for distributed optimization and control, IEEE control systems magazine 22 (3) (2002) 52–67.
  * [39] A. Faramarzi, M. Heidarinejad, S. Mirjalili, A. H. Gandomi, Marine predators algorithm: A nature-inspired metaheuristic, Expert systems with applications 152 (2020) 113377.
  * [40] S. Arora, S. Singh, Butterfly optimization algorithm: a novel approach for global optimization, Soft Computing 23 (3) (2019) 715–734.
  * [41] W. Zhao, Z. Zhang, L. Wang, Manta ray foraging optimization: An effective bio-inspired optimizer for engineering applications, Engineering Applications of Artificial Intelligence 87 (2020) 103300.
  * [42] T. Rahkar Farshi, Battle royale optimization algorithm, Neural Computing and Applications 33 (4) (2021) 1139–1157.
  * [43] G.-G. Wang, Moth search algorithm: a bio-inspired metaheuristic algorithm for global optimization problems, Memetic Computing 10 (2) (2018) 151–164.
  * [44] X.-B. Meng, X. Z. Gao, L. Lu, Y. Liu, H. Zhang, A new bio-inspired optimisation algorithm: Bird swarm algorithm, Journal of Experimental & Theoretical Artificial Intelligence 28 (4) (2016) 673–687.
  * [45] S. Mirjalili, S. M. Mirjalili, A. Hatamlou, Multi-verse optimizer: a nature-inspired algorithm for global optimization, Neural Computing and Applications 27 (2) (2016) 495–513.
  * [46] Y. Shi, Brain storm optimization algorithm, in: International conference in swarm intelligence, Springer, 2011, pp. 303–309.
  * [47] R. Salgotra, U. Singh, The naked mole-rat algorithm, Neural Computing and Applications 31 (12) (2019) 8837–8857.
  * [48] B. Chen, L. Zhao, J. H. Lu, Wind power forecast using rbf network and culture algorithm, in: 2009 international conference on sustainable power generation and supply, IEEE, 2009, pp. 1–6.
  * [49] Z. Wei, C. Huang, X. Wang, T. Han, Y. Li, Nuclear reaction optimization: A novel and powerful physics-based algorithm for global optimization, IEEE Access 7 (2019) 66084–66109.
  * [50] P.-T. De Boer, D. P. Kroese, S. Mannor, R. Y. Rubinstein, A tutorial on the cross-entropy method, Annals of operations research 134 (1) (2005) 19–67.
  * [51] H. Yapici, N. Cetinkaya, A new meta-heuristic optimizer: Pathfinder algorithm, Applied soft computing 78 (2019) 545–568.
  * [52] S. Talatahari, M. Azizi, Chaos game optimization: a novel metaheuristic algorithm, Artificial Intelligence Review 54 (2) (2021) 917–1004.
  * [53] J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of ICNN’95-international conference on neural networks, Vol. 4, IEEE, 1995, pp. 1942–1948.
  * [54] M. Khishe, M. R. Mosavi, Chimp optimization algorithm, Expert systems with applications 149 (2020) 113338.
  * [55] M. Shaqfa, K. Beyer, Pareto-like sequential sampling heuristic for global optimisation, Soft Computing 25 (14) (2021) 9077–9096.
  * [56] J. Pierezan, L. D. S. Coelho, Coyote optimization algorithm: a new metaheuristic for global optimization problems, in: 2018 IEEE congress on evolutionary computation (CEC), IEEE, 2018, pp. 1–8.
  * [57] B. M. Nguyen, B. Hoang, T. Nguyen, G. Nguyen, nqsv-net: a novel queuing search variant for global space search and workload modeling, Journal of Ambient Intelligence and Humanized Computing 12 (1) (2021) 27–46.
  * [58] S. Salcedo-Sanz, J. Del Ser, I. Landa-Torres, S. Gil-López, J. Portilla-Figueras, The coral reefs optimization algorithm: a novel metaheuristic for efficiently solving optimization problems, The Scientific World Journal 2014 (2014).
  * [59] X.-S. Yang, S. Deb, Cuckoo search via lévy flights, in: 2009 World congress on nature & biologically inspired computing (NaBIC), Ieee, 2009, pp. 210–214.
  * [60] P. J. Van Laarhoven, E. H. Aarts, Simulated annealing, in: Simulated annealing: Theory and applications, Springer, 1987, pp. 7–15.
  * [61] S.-C. Chu, P.-W. Tsai, J.-S. Pan, Cat swarm optimization, in: Pacific Rim international conference on artificial intelligence, Springer, 2006, pp. 854–858.
  * [62] A. Shabani, B. Asgarian, S. A. Gharebaghi, M. A. Salido, A. Giret, A new optimization algorithm based on search and rescue operations, Mathematical Problems in Engineering 2019 (2019).
  * [63] S. Zhao, T. Zhang, S. Ma, M. Chen, Dandelion optimizer: A nature-inspired metaheuristic algorithm for engineering applications, Engineering Applications of Artificial Intelligence 114 (2022) 105075.
  * [64] S. Mirjalili, Sca: a sine cosine algorithm for solving optimization problems, Knowledge-based systems 96 (2016) 120–133.
  * [65] S. Mirjalili, Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems, Neural computing and applications 27 (4) (2016) 1053–1073.
  * [66] S. Shadravan, H. R. Naji, V. K. Bardsiri, The sailfish optimizer: A novel nature-inspired metaheuristic algorithm for solving constrained engineering optimization problems, Engineering Applications of Artificial Intelligence 80 (2019) 20–34.
  * [67] H. Abedinpourshotorban, S. M. Shamsuddin, Z. Beheshti, D. N. Jawawi, Electromagnetic field optimization: a physics-inspired metaheuristic optimization algorithm, Swarm and Evolutionary Computation 26 (2016) 8–22.
  * [68] G. Dhiman, V. Kumar, Spotted hyena optimizer: a novel bio-inspired based metaheuristic technique for engineering applications, Advances in Engineering Software 114 (2017) 48–70.
  * [69] G.-G. Wang, S. Deb, L. d. S. Coelho, Elephant herding optimization, in: 2015 3rd international symposium on computational and business intelligence (ISCBI), IEEE, 2015, pp. 1–5.
  * [70] R. Masadeh, B. A. Mahafzah, A. Sharieh, Sea lion optimization algorithm, International Journal of Advanced Computer Science and Applications 10 (5) (2019).
  * [71] A. Faramarzi, M. Heidarinejad, B. Stephens, S. Mirjalili, Equilibrium optimizer: A novel optimization algorithm, Knowledge-Based Systems 191 (2020) 105190\.
  * [72] S. Li, H. Chen, M. Wang, A. A. Heidari, S. Mirjalili, Slime mould algorithm: A new method for stochastic optimization, Future Generation Computer Systems 111 (2020) 300–323.
  * [73] G.-G. Wang, S. Deb, L. D. S. Coelho, Earthworm optimisation algorithm: a bio-inspired metaheuristic algorithm for global optimisation problems, International journal of bio-inspired computation 12 (1) (2018) 1–22.
  * [74] M. Bakhshipour, M. J. Ghadi, F. Namdari, Swarm robotics search & rescue: A novel artificial intelligence-inspired optimization approach, Applied Soft Computing 57 (2017) 708–726.
  * [75] X. Yao, Y. Liu, G. Lin, Evolutionary programming made faster, IEEE Transactions on Evolutionary computation 3 (2) (1999) 82–102.
  * [76] J. Xue, B. Shen, A novel swarm intelligence optimization approach: sparrow search algorithm, Systems Science & Control Engineering 8 (1) (2020) 22–34.
  * [77] H.-G. Beyer, H.-P. Schwefel, Evolution strategies–a comprehensive introduction, Natural computing 1 (1) (2002) 3–52.
  * [78] A. Tharwat, T. Gabel, Parameters optimization of support vector machines for imbalanced data using social ski driver algorithm, Neural computing and applications 32 (11) (2020) 6925–6938.
  * [79] Y. Tan, Y. Zhu, Fireworks algorithm for optimization, in: International conference in swarm intelligence, Springer, 2010, pp. 355–364.
  * [80] S. Mirjalili, A. H. Gandomi, S. Z. Mirjalili, S. Saremi, H. Faris, S. M. Mirjalili, Salp swarm algorithm: A bio-inspired optimizer for engineering design problems, Advances in engineering software 114 (2017) 163–191.
  * [81] J.-S. Chou, N.-M. Nguyen, Fbi inspired meta-optimization, Applied Soft Computing 93 (2020) 106339.
  * [82] J. James, V. O. Li, A social spider algorithm for global optimization, Applied soft computing 30 (2015) 614–627.
  * [83] A. H. Gandomi, X.-S. Yang, A. H. Alavi, Mixed variable structural optimization using firefly algorithm, Computers & Structures 89 (23-24) (2011) 2325–2336.
  * [84] G. Dhiman, A. Kaur, Stoa: a bio-inspired based optimization algorithm for industrial engineering problems, Engineering Applications of Artificial Intelligence 82 (2019) 148–174.
  * [85] W.-T. Pan, A new fruit fly optimization algorithm: taking the financial distress model as an example, Knowledge-Based Systems 26 (2012) 69–74.
  * [86] R. Rao, V. Patel, An elitist teaching-learning-based optimization algorithm for solving complex constrained optimization problems, international journal of industrial engineering computations 3 (4) (2012) 535–560.
  * [87] X.-S. Yang, Flower pollination algorithm for global optimization, in: International conference on unconventional computing and natural computation, Springer, 2012, pp. 240–249.
  * [88] A. H. Halim, I. Ismail, Tree physiology optimization in benchmark function and traveling salesman problem, Journal of Intelligent Systems 28 (5) (2019) 849–871.
  * [89] D. Whitley, A genetic algorithm tutorial, Statistics and computing 4 (2) (1994) 65–85.
  * [90] S. Kaur, L. K. Awasthi, A. Sangal, G. Dhiman, Tunicate swarm algorithm: A new bio-inspired based metaheuristic paradigm for global optimization, Engineering Applications of Artificial Intelligence 90 (2020) 103541.
  * [91] I. Ahmadianfar, O. Bozorg-Haddad, X. Chu, Gradient-based optimizer: A new metaheuristic optimization algorithm, Information Sciences 540 (2020) 131–159.
  * [92] A. Kaveh, Tug of war optimization, in: Advances in metaheuristic algorithms for optimal design of structures, Springer, 2017, pp. 451–487.
  * [93] C. Villaseñor, N. Arana-Daniel, A. Y. Alanis, C. López-Franco, E. A. Hernandez-Vargas, Germinal center optimization algorithm, International Journal of Computational Intelligence Systems 12 (1) (2018) 13.
  * [94] M. D. Li, H. Zhao, X. W. Weng, T. Han, A novel nature-inspired algorithm for optimization: Virus colony search, Advances in engineering software 92 (2016) 65–88.
  * [95] S. Saremi, S. Mirjalili, A. Lewis, Grasshopper optimisation algorithm: theory and application, Advances in engineering software 105 (2017) 30–47.
  * [96] Z. Bayraktar, M. Komurcu, J. A. Bossard, D. H. Werner, The wind driven optimization technique and its application in electromagnetics, IEEE transactions on antennas and propagation 61 (5) (2013) 2745–2757.
  * [97] A. W. Mohamed, A. A. Hadi, A. K. Mohamed, Gaining-sharing knowledge based algorithm for solving optimization problems: a novel nature-inspired algorithm, International Journal of Machine Learning and Cybernetics 11 (7) (2020) 1501–1529.
  * [98] D. Amali, M. Dinakaran, Wildebeest herd optimization: a new global optimization algorithm inspired by wildebeest herding behaviour, Journal of Intelligent & Fuzzy Systems 37 (6) (2019) 8063–8076.
  * [99] S. Mirjalili, S. M. Mirjalili, A. Lewis, Grey wolf optimizer, Advances in engineering software 69 (2014) 46–61.
  * [100] S. Mirjalili, A. Lewis, The whale optimization algorithm, Advances in engineering software 95 (2016) 51–67.
  * [101] P. Niu, S. Niu, L. Chang, et al., The defect of the grey wolf optimization algorithm and its verification method, Knowledge-Based Systems 171 (2019) 37–43.
  * [102] M. Castelli, L. Manzoni, L. Mariot, M. S. Nobile, A. Tangherloni, Salp swarm optimization: A critical review, Expert Systems with Applications 189 (2022) 116029\.
  * [103] C. L. Camacho-Villalón, M. Dorigo, T. Stützle, Exposing the grey wolf, moth-flame, whale, firefly, bat, and antlion algorithms: six misleading optimization techniques inspired by bestial metaphors, International Transactions in Operational Research (2022).