# Copper plasmonics with excitons

David Ziemkiewicz<EMAIL_ADDRESS>Sylwia Zielińska-Raczyńska
Institute of Mathematics and Physics, Technical University of Bydgoszcz,
Al. Prof. S. Kaliskiego 7, 85-789 Bydgoszcz, Poland

###### Abstract

We investigate the propagation of surface plasmons (SPPs) in a thin layer of
copper surrounded by copper oxide Cu2O. It is shown that particularly strong
excitons in Cu2O can have considerable impact on plasmon propagation,
providing many opportunities for plasmon-exciton and plasmon-plasmon
interactions. It is demonstrated that by the use of sufficiently thin metal
layer, one can excite the so-called long range plasmons (LRSPPs) which can
overcome the inherently high ohmic losses of Cu as compared to usual plasmonic
metals such as silver. Analytical results are confirmed by numerical
calculations.

##  I Introduction

The surface plasmon-polaritons (SPPs) are bound, nonradiative electromagnetic
excitations associated with charge density waves propagating along the
metal/dielectric interface. It is well known that surface plasmons can be used
to confine the field of electromagnetic wave into subwavelength areas, greatly
increasing field intensity and as a result the intensity of electromagnetic
field is enhanced. Plasmonic devices offer many significant advantages in
solid state device applications; electromagnetic wave bound to a metallic
surface can be not only focused, but also easily guided or slowed down. All
these properties are very useful for providing a favourable environment for
light - exciton interaction. Development of applications where both energy
propagation and field confinement are desired requires detailed analysis of
the trade-off between the two. This interchange is a direct consequence of a
field confinement and plasmon propagation being reciprocal to each other, i.e.
propagating SPPs assume localized character in frequency region of surface
plasmon while as confinement gets stronger, the electric field penetrates
deeper into the metal (surface), leading to increased energy losses. While
noble metals are the most popular among plasmonic materials, copper-based
plasmonic devices are a recent, dynamically expanding area of research, with
promising applications in nanoantennas [1], photovoltaics [2, 3], waveguides
[4].

A nontrivial aspect of design copper-based devices is the surface protection
from oxidation. Various protective layers were proposed, including graphene
[5]. Without the oxide layer, Cu nanostructures exhibit a narrow plasmonic
resonance comparable to the one found in silver and gold nanostructures [6].
On the other hand, a controlled layer of oxide with specific thickness might
enhance the plasmonic resonance instead of perturbing it [7, 8]. The ongoing
research towards reduction of ohmic losses in copper nanostructures leads to
geometries that can match the performance of the usual plasmonic materials
such as gold and silver [9].

An exciton in a semiconductor is a bound state of a conduction band electron
and a valence band hole, which are attracted to each other by the Coulomb
interaction; they form an electrically neutral quasiparticle, transferring the
energy without transporting net electric charge. The ability to couple
plasmons with excitons, creating so-called plexcitons [10, 11] leads to a
variety of plasmonic devices that can be tuned on demand [12, 13, 14].
Exciton-plasmon coupling has been demonstrated in the case of Wannier excitons
in semiconductors [15], but the use of copper oxide, in particular Cu - Cu2O
structures is a novel idea. Excitonic states in Cu2O are a solid-state
analogue of hydrogen atoms and states with principal quantum number up to
$n=$25 for dipole-allowed P-type envelope wavefunctions, called Rydberg
excitons (REs), have been observed [16]. Rydberg excitons are becoming one of
the most versatile, scalable and tunable platform for quantum computing
technologies. The field is developing very rapidly, with first theoretical
studies [17, 18] and experiments [19, 20] exploring nonlinear properties of RE
in low-dimensional systems being performed right now. The fabrication
techniques are just entering the stage where consistent synthesis of high-
quality Cu2O nanostructures becomes possible [21, 22]. Furthermore, from a
more general point of view, Rydberg excitons are one of a kind structures that
may offer new insight into fundamental physics; their huge dimension make them
an ideal candidate for performing experiments with spatially engineered light
fields and micrometer size plasmonic systems. The prominent idea of this paper
is to take advantage of their unusual properties coupling these structures
with plasmons

The paper is organized as follows: in the next section the basic properties of
Rydberg excitons important to the propagation of plasmons are discussed and
some general limitations of the system are outlined. Next, the dispersive
properties of plasmons excited on a thin metal layer are investigated focusing
on the specific case of copper surrounded by copper oxide. Section IV contains
calculation results. Finally, the details of employed numerical method are
given in the appendix and the last section presents the summary of our
results.

##  II Excitons and Rydberg blockade

An exciton, being an excited electronic state of the crystal is an electron-
hole air, which is weakly bound and can extend over many thousands of lattice
unit cells, thus its interaction is screened by static permittivity of the
semiconductor. The first observation of higher excited excitons, called
Rydberg excitons, first in a cuprous oxide bulk [16] and then in
nanostructures [19] revealed a lot of their astonishing features, such as
extraordinary large dimension scaling as $n^{2}$, long life-times $\sim
n^{3}$, reaching nanoseconds. Their extraordinary vulnerability to
interactions with external fields is due to huge polarizability scaling as
$n^{7}$.

Another characteristic property of excitons is the so-called Rydberg blockade.
For a given exciton with principal quantum number $n$, another exciton cannot
be created in immediate vicinity due to the exciton-exciton interaction that
shifts the energy levels. The space where another exciton cannot be created is
described by a so-called blockade volume [16]

$V_{B}=3\cdot 10^{-7}n^{7}~{}\mu m^{3}.$ (1)

The crucial feature is the extremely fast $n^{7}$ scaling of the blockade;
highly excited states very quickly reach the saturation level when the medium
is completely filled with excitons. As a result, the light propagating through
the medium is not absorbed to create new excitons. This is so-called optical
bleaching. On the other hand, the lowest excitonic states can reach
considerable density. Specifically, for $n=2$, taking into account a semi-
random distribution of blockade volumes as opposed to ideal sphere packing
[23] one obtains an upper density limit on the order of $10^{17}$ $cm^{-3}$,
with some sources reporting even $10^{19}$ $cm^{-3}$ densities [24]. The issue
of maximum possible density is important to the effect of excitonic
transitions on surface plasmons; specifically, the propagation properties of
SPP are dependent on the Cu2O permittivity $\epsilon=7.5+\chi$, where the
changes of susceptibility $\chi$ caused by excitons are directly proportional
to exciton density. It is beneficial to obtain conditions where these
susceptibility changes are not negligible when compared with the ,,static”
part $\epsilon_{b}=7.5$. This can be only achieved with the lowest excitonic
states. Furthermore, another reason to consider only the lowest P-excitonic
states ($n$=2,3) is the interaction of excitons with metallic surfaces. As
mentioned above, excitons with high principal quantum number are very
sensitive to external electric fields, including electrostatic fields at
metal-dielectric interface [25], leading to ionisation of excitons that are
separated from the metal surface by a distance comparable to exciton radius.
Thus, for example, one can expect that $n=2$ exciton in Cu2O is strongly
affected by the metallic environment when located closer than approximately 6
nm from the metal-dielectric interface.

##  III Propagation of surface plasmons

Surface plasmon is a localized electromagnetic excitation on an interface
between two media with exhibit opposite signs of dielectric permittivity at
some frequency $\omega$. Typically, the material with negative permittivity
$\epsilon_{1}(\omega)$ is a metal and the other one characterized by
$\epsilon_{2}(\omega)>0$ is a dielectric. In such a system, with the help of
Maxwell’s equations, one can derive the wave vector of a electromagnetic wave
mode that propagates along the metal-dielectric interface [26]

$\kappa(\omega)=\kappa^{\prime}+i\kappa^{\prime\prime}=\frac{\omega}{c}\sqrt{\frac{\epsilon_{1}\epsilon_{2}}{\epsilon_{1}+\epsilon_{2}}},$
(2)

where $\kappa^{\prime}$ and $\kappa^{\prime\prime}$ are the real and imaginary
parts of the wave vector component, respectively and $c$ is the speed of light
in vacuum. For real values of $\epsilon_{1}$ and $\epsilon_{2}$, one obtains
real wave vector when $\epsilon_{1}\epsilon_{2}<0$ and
$\epsilon_{1}+\epsilon_{2}<0$ and in such a case propagating SPPs can be
excited.

In this paper, we will consider a metal layer of finite thickness $d$. In such
a case, one obtains modified boundary conditions which result in two solutions
[27, 28]

$\displaystyle\frac{\kappa_{1}\tanh(\kappa_{1}\frac{d}{2})}{\epsilon_{1}}=\frac{-\kappa_{2}}{\epsilon_{2}},$
$\displaystyle\frac{\kappa_{1}\coth(\kappa_{1}\frac{d}{2})}{\epsilon_{1}}=\frac{-\kappa_{2}}{\epsilon_{2}},$
(3)

where $\kappa_{1,2}=\sqrt{k^{2}-\epsilon_{1,2}\frac{\omega^{2}}{c^{2}}}$. In
the limit of $d\rightarrow\infty$, Eqs. (III) reduce to the form given by Eq.
(2). The two above solutions are the so-called short-range and long-range
plasmons (SRSPP, LRSPP accordingly). They are characterized by a small or
large group velocity

$V_{g}=\frac{\partial\omega}{\partial\kappa^{\prime}}=\frac{c}{n_{eff}+\omega\frac{\partial
n_{eff}}{\partial\omega}},$ (4)

where $n_{eff}=\sqrt{\epsilon}$ is the effective refraction index. The crucial
property of long-range plasmons is reduced absorption and elevated group
velocity, which allows them to propagate over distances on the order of tens
to hundreds of $\mu$m [29, 30, 31, 32]. One can take an advantage of this to
produce relatively long-living plasmons in copper. As mentioned before, in
some cases the ohmic losses in Cu-based systems can approach the values
typical to silver and gold nanostructures, provided that the detrimental
effects of oxide layer is mitigated. Here, we propose a system where the Cu2O
is an inherent part of the structure necessary for its operation.
Particularly, we consider a thin layer of Cu surrounded by semi-infinite (much
thicker than wavelength) layers of Cu2O. The system is shown on the Fig. 1 a).
An external light source (point source of radiation in Finite Difference Time
Domain (FDTD) calculations, described in Appendix A) generates both long- and
short range excitons; they can be seen on the Fig. 1 b), where the field
distribution is symmetric or antisymmetric for LRSPP and SRSPP, accordingly.

a) b)

Figure 1: a) Schematic representation of the system considered in the paper;
surface plasmon (yellow) propagates along the metal surface at some group
velocity $V_{g}$; b) Spatial distribution of normalized electric field (color)
obtained in FDTD simulation.

In order to derive the optical properties of the proposed system, one needs a
suitable model of the materials. In the considered spectral range of $2-2.2$
eV, the permittivity of the oxide is very close to a constant
$\epsilon_{2}=7.5$, with the additional impact of excitons as outlined in the
next section. For the properties of copper, we refer to [33]. The real and
imaginary part of permittivity and the fitted model used in FDTD calculations
are shown on the Fig. 2.

Figure 2: Permittivity data from [33] (dots) and the fitted model (lines).
Dashed black lines mark the part of the spectrum occupied by Rydberg excitons
in Cu2O, indicating energy of 2P exciton and gap energy.

The model is fitted to provide a good match in the spectral region of Rydberg
excitons, marked by dashed lines; specifically, the maximum difference between
experimentally measured susceptibility and model is less than 2$\%$ in a 100
meV range around the 2P exciton energy, which is considerably more than the
spectral width of simulated plasmons. Two main features of the susceptibility
relation $\epsilon_{1}(\omega)$ are apparent; there is a considerable increase
of absorption (proportional to the imaginary part of permittivity), which is
the result of inter-band transitions in Cu [33]. This increase starts just
above the gap energy of Cu2O, so that the excitons, which have energies
smaller than the band gap, still remaining in the low absorption part of the
spectrum. Furthermore, the real part of permittivity is negative and in fact
in the spectral region of excitons, it provides a close match to the oxide
permittivity, e. g. $\epsilon_{1}+\epsilon_{2}\approx 0$. This is a crucial
condition for excitation of strong plasmonic resonance; these findings are
consistent with [34], where a strong plasmonic resonance in the region of 2 eV
has been demonstrated in a thin copper film.

##  IV Tunable long range plasmons

Let’s consider a long-range plasmon propagating along a thin layer of Cu
surrounded by Cu2O. By using the above-discussed model of metal permittivity
$\epsilon_{1}$ and the oxide permittivity $\epsilon_{2}$ containing a
numerically calculated excitonic spectrum [35], we can calculate the exciton
wave vector from Eq (III). The results are shown on the Fig. 3.

Figure 3: Dispersion relation of surface plasmon calculated for a case of
thick and thin layer of Cu.

The left panel shows the Cu2O susceptibility
$\chi(\omega)=\epsilon_{2}(\omega)-7.5$ exhibiting the characteristic spectrum
of Rydberg excitons with multiple states corresponding to principal quantum
number $n=1,2...20$. These resonances have a noticeable impact on the
dispersion curve $\kappa(\omega)$ (right panel, inset). While the absolute
value of the changes of $\epsilon_{2}$ caused by excitons is rather small,
these changes occur in a very narrow spectral range of excitonic resonances.
This means that the group velocity of plasmon, which is connected with the
slope of the function $\kappa(\omega)$ according to Eq. (4), is significantly
affected by the excitons. This is a key component of our proposal of tunable
plasmons. Due to the change of the plasmon group velocity, the time needed to
propagate some certain distance is changed. This, in turn, affects the amount
of plasmon energy absorbed by the metal due to the ohmic losses. As a result,
we can significantly change the transmission coefficient of the system despite
the fact that direct absorption by Cu2O is negligible when compared to Cu.
Another crucial component of our proposal is the use of a thin Cu layer. One
can see that the dispersion relation on the Fig. 3 calculated for a thick
layer changes from a normal dispersion below $2.14$ eV to anomalous dispersion
above (e.g. the value of $\kappa$ is decreasing with energy). However, by
using a sufficiently thin layer, one can increase the energy where transition
to anomalous dispersion occurs; the same technique has been used in [36] to
facilitate propagation of UV plasmons. In the case here, it allows for
excitation of plasmons in the $2.1-2.2$ eV range, fitted to the spectral
region of Rydberg excitons. Specifically, Fig. 3 inset shows the dispersion
relation of such plasmons. One can see that the general slope of the function
$\omega(\kappa)$ is steeper than in the thick layer case, indicating increased
group velocity of LRSPPs. At the same time, the slope of $\omega(\kappa)$ near
excitonic resonances is considerably smaller, corresponding to greatly reduced
group velocity. As discussed in [37], group velocities on the order of 103 m/s
are possible for a light pulse tuned to $n>10$ exciton propagating in Cu2O. In
the case of low $n$ excitonic states and plasmons considered here, the
slowdown is less considerable; since the plasmon dispersion relation depends
on the permittivity of both dielectric and metal, the effect of excitonic
resonances is reduced. Furthermore, the spectrum of the plasmon is much wider
than the width of excitonic resonance, so that only selected spectral
components of SPP are affected by the exciton. This is discussed further
below. Finally, from practical point of view, apart from potential
irregularity of oxidation film (Cu2O), the close separation of higher
excitonic resonances prevents them from being selectively excited by a
relatively short-living, wide spectrum plasmon.

The FDTD results, where the transmission coefficient of LRSPP propagating
through a distance of $L=4$ $\mu$m, on a $d=8$ nm thick Cu layer was
calculated, is shown on the Fig. 4.

Figure 4: Transmission coefficient of long range plasmons on a thin Cu layer.

Notably, transmission coefficient on the order of 5$\%$ is obtained, which is
considerably bigger than can be achieved with bulk plasmons, even with noble
metals [32]. The two transmission coefficients shown on the Fig. 4 correspond
to the entire plasmon and its central frequency. One can see a noticeable drop
of transmission in the region of the strongest 2P exciton. This minimum is
more pronounced when only the central frequency is considered. This is caused
by the fact that the spectrum of propagating plasmon is considerably wider
that the excitonic resonance. Specifically, in this particular case the
plasmon lifetime is on the order of $10^{-13}$ s, corresponding to the
spectral width $\Delta E\sim 30$ meV. This value is comparable to the distance
between $n=2$ and $n=3$ excitonic states (14 meV) and approximately 6 times
larger than the widest resonance of $n=2$ exciton (5 meV) [16].
Correspondingly, the exciton lifetime is several times longer than plasmonic
lifetime, being equal to $\tau\sim 1$ ps for $n=2$. Finally, while the low-n
excitons are considerably smaller than the space occupied by propagating
plasmon (2P exciton radius is approx. $r_{E}=1.5$ nm, blockade radius
$r_{B}=19$ nm), they can easily exceed the plasmon size for larger n. This
particular set of size and lifetime proportions has several important
consequences:

  * •

The local reduction of the group velocity is applied only to some wave modes
close to the central frequency of excitonic resonance and not entire plasmon
spectrum; this means that the effectiveness of slowdown of a surface plasmon
as a whole is reduced and in general, a single group velocity cannot be
attributed to the plasmon (e.g. there is considerable group velocity
dispersion).

  * •

By fine-tuning the frequency of the light exciting the plasmon, one can
effectively aim at $n=2$ and $n=3$ excitonic states; higher states are located
too closely to each others to be targeted selectively in such a system.
However, this restriction does not apply when a second, non-plasmon field is
used to excite some specific state, which is then used in exciton-plasmon
interaction.

  * •

The locally enhanced absorption of specific modes produces a narrow dip in the
plasmon spectrum, similar to the so-called spectral hole-burning [38].

To confirm the above findings and further investigate the plasmon dynamics and
its interaction with excitons, one can analyse the group velocity of excited
plasmonic modes by considering the electric field amplitude as a function of
time and position, as shown on the Fig.5.

Figure 5: Normalized field amplitude (color) of excited plasmonic modes as a
function of time and position along the metal layer.

The figure is dominated by two strong lines representing LRSPP and SRSPP
modes; their slope differs considerably, indicating group velocities on the
order of $0.3$ c and $0.04$ c, accordingly. One can also see that the long-
range plasmon maintains non-negligible amplitude after reaching the end of
computation domain at L=4 $\mu m$, and then it is reflected. At the point of
reflection, boths long-range and short-range modes are created (more
horizontal and vertical lines, accordingly). Finally, one can notice weaker
traces of several modes with group velocities that differ considerably from
the LRSPP. These modes correspond to the spectral range of 2P exciton,
specifically in the regions of increased normal dispersion around the
excitonic resonance. As mentioned before, due to the reduction of group
velocity, these modes are more strongly absorbed by the metal. Moreover, since
the group velocity change caused by excitonic resonance occurs in a very
narrow part of the spectrum, being only a fraction of the plasmon spectrum,
the slow modes carry relatively little energy and thus are barely visible on
the Fig. 5. A somewhat explicit picture can be obtained by considering the
image of the plasmon in frequency domain.

Figure 6: Frequency spectra of the propagating LRSPP calculated at various
propagation lengths.

The spectrum of LRSPP is shown on the Fig. 6. One can see how the plasmon
spectrum evolves with the propagation length; initially, it is approximately a
Gaussian with the width on the order of 100 meV. However, as the plasmon
propagates along the metal surface, a noticeable dip is formed at the energy
of 2147 meV, which corresponds to 2P exciton resonance. Another, weaker local
minimum is present above the band gap energy $E_{g}=$2172 meV. This means that
increased absorption of Cu2O in the region above the band gap is a relatively
minor contribution to absorption as compared to the slowdown of $V_{g}$ and
subsequent absorption by Cu. After the propagation distance of $L=3\mu$m, the
local minimum in the spectrum is quite substantial, with approximately 30$\%$
lower amplitude. This is consistent with the results in the Fig. 4, where a
local reduction of transmission coefficient from $6\%$ to $4\%$ is shown. It
should be stressed that the LRSPP is not an ideal Gaussian but a collection of
modes that propagate at various group velocities that can differ considerably;
this complicates calculation of the spectrum and is the cause of other local
minima visible on the Fig. 6. However, in contrast to the minimum
corresponding to the 2P exciton, these apparent minima do not appear
consistently in each spectrum, at the same energy. Finally, from the Fig. 1 b)
one can see that the electric field of plasmons extends into Cu2O up to a
distance of $\sim 300$ nm from the metal surface; therefore, the plasmon field
can easily reach excitons that are not strongly affected by the immediate
vicinity of metal-oxide interface and the use of unperturbed energy of 2P
exciton state $E_{2P}=2147$ meV in calculations is justified.

As mentioned before, the strength of the optical response of Cu2O is directly
proportional to the exciton density. This, in turn, depends on the energy
provided to generate the excitons and is limited by the Rydberg blockade. As a
result, one has several options for dynamically controlling the system:

  * •

When the propagating plasmon’s frequency is tuned to the excitonic resonance,
its energy can be used to create new excitons. In such a case, the response of
the system (transmission coefficient) will be highly dependent on the power of
the plasmon.

  * •

One can use external optical field (control field) to create a given density
of excitons, which then affects the propagation of weaker probe field
(plasmon).

  * •

Due to the narrow spectrum of excitonic resonances, only a slight detuning of
plasmon frequency is needed to considerably affect its transmission.

  * •

By exciting two plasmons in a short period, one can potentially create
conditions where the excitons generated by the first plasmon affect the
propagation of the second plasmon, allowing for exciton-mediated plasmon-
plasmon interactions.

  * •

In a similar manner, one can arrange interaction between plasmons propagating
along two closely spaced metal layers.

From the point of view of practical applications, it is important to consider
the maximum thickness of Cu layer that allows for propagation of long-range
plasmons. As mentioned in [32], the thickness $d$ should be considerably
smaller than the plasmon wavelength. On the fig. 1 b), one can estimate that
this wavelength is about 200 nm (vacuum wavelength of incident light is
$\lambda=577$ nm, which indicates that effective refraction index is
$n_{eff}\approx 2.9$; this is slightly larger than the index of Cu2O
$n_{2}=2.73$). Therefore, one can assume that the metal layer thickness must
be $d<<200$ nm. This is the case on the Fig. 7; the transmission coefficient
decreases exponentially with the layer thickness,with a value of $T\sim 4\%$
for $d$=10 nm and $T\sim 0.1\%$ for $d=$35 nm.

Figure 7: Transmission coefficient over the distance $L=4$ $\mu$m as a
function of metal layer thickness $d$.

##  V FDTD method

One of the most popular tools for numerical analysis of SPP propagation is the
Finite-Difference Time-Domain (FDTD) method [39]. It is based directly on
Maxwell’s equations, which provides a high flexibility and accuracy in
modelling the propagation of electromagnetic waves. In this manuscript, we
focus on the two-dimensional variant of the method; the computational domain
is a cross-section of the system placed on $xy$ plane. The structure is
assumed to be symmetric and semi-infinite (e.g. much larger than light
wavelength) in $z$ direction. The domain is divided into rectangular grid of
discrete cells with the size $\Delta x=4$ nm. In every cell, the values of the
components of the fields $\vec{E}(x,y,t)$, $\vec{H}(x,y,t)$ are stored. By
defining a discrete time step $\Delta t$, one can use the Maxwell’s equations
to derive evolution equations that use the field values $\vec{E}(x,y,t)$,
$\vec{H}(x,y,t)$ to calculate the next ones $\vec{E}(x,y,t+\Delta t)$,
$\vec{H}(x,y,t+\Delta t)$. In the two-dimensional case, without the loss of
generality one can assume that the fields have only three non-zero components
$\vec{E}=[E_{x},E_{y},0]$, $\vec{H}=[0,0,H_{z}]$. The optical response of
materials in the simulation is described by the polarization vector
$\vec{P}(x,y,t)$. Specifically, one can use the so-called Drude-Lorentz model
that specifies the connection between electric field in polarization in the
frequency domain

$\displaystyle\vec{P}(\omega)$ $\displaystyle=$
$\displaystyle\epsilon(\omega)\vec{E}(\omega),$
$\displaystyle\epsilon(\omega)$ $\displaystyle=$
$\displaystyle\epsilon_{\infty}+\sum_{j=1}^{n}\frac{\omega_{pj}^{2}}{\omega_{0j}^{2}-\omega^{2}-i\gamma_{j}\omega}$
(5)

with a set of $n$ oscillators with the so-called plasma frequency
$\omega_{pj}$, resonance frequency $\omega_{0j}$ and dissipation constant
$\gamma_{j}$. In the case of Cu, one can use two oscillators; the first one,
with $\omega_{p1}=0$ describes the baseline Drude model of free electrons in
metal, while the second oscillator provides the correction responsible for the
increase of absorption due to the inter-band transitions (the local maximum at
$\omega=0.047$ on the Fig. 2). For Cu2O, one has $\epsilon_{\infty}=7.5$ and
the series of oscillators corresponding to excitonic resonances, with
appropriate oscillator strengths and damping constants. The medium model has
been verified by comparing the numerically obtained absorption spectrum of
Cu2O with the results in [16]. In the performed calculations, the plasmon
center frequency is tuned to $n=2$ exciton and thus an inclusion of two
excitons $n=2$ and $n=3$ provides sufficient accuracy of results. The above
medium frequency response is integrated into time domain simulation with the
use of ADE (Axillary Differential Equations) approach [40]. In this method,
one can calculate the polarization as a function of time with the following
differential equation

$\ddot{P}+\gamma_{j}\dot{P}+\omega^{2}_{0j}P=\frac{\omega^{2}_{pj}}{\epsilon_{\infty}}E$
(6)

for every oscillator $j$. The final set of equations based on Eq. 6 and
Maxwell’s equation is as follows

$\displaystyle\frac{\partial E_{y}(x,y,t)}{\partial x}-\frac{\partial
E_{x}(x,y,t)}{\partial y}=-\mu_{0}\frac{\partial H_{z}(x,y,t)}{\partial t},$
(7) $\displaystyle\frac{\partial H_{z}(x,y,t)}{\partial
y}=j_{x}(x,y,t)+\epsilon_{0}\frac{\partial E_{x}(x,y,t)}{\partial
t}-\frac{\partial P_{x}(x,y,t)}{\partial t},$ $\displaystyle-\frac{\partial
H_{z}(x,y,t)}{\partial x}=j_{y}(x,y,t)+\epsilon_{0}\frac{\partial
E_{y}(x,y,t)}{\partial t}+\frac{\partial P_{y}(x,y,t)}{\partial t}.$

Where the polarization has two components $P_{x}$, $P_{y}$ which depend on
$E_{x}$, $E_{y}$ according to Eq. (6). The $j_{x}$ and $j_{y}$ are current
densities in the corresponding directions and $\epsilon_{0}$, $\mu_{0}$ are
vacuum permittivity and permeability. By rearranging the terms in Eq. (7), one
can express the future values of the fields $E_{x}(x,y,t+1)$,
$E_{y}(x,y,t+1)$, $H_{z}(x,y,t+1)$ as a function of the current values. Such
relations allow for advancing the state of the system in discrete time steps
by calculating the new field values basing on the current ones.

##  VI Conclusions

The discovery of Rydberg excitons in Cu2O has opened many new perspectives in
semiconductor science, but also posed many theoretical problems and
challenges. While the optical properties of REs, especially in bulk media, are
already well understood, the ongoing research on Rydberg excitons is now
entering the phase where the first experiments involving confined systems are
performed. Thus, it is of high importance to provide the theoretical framework
for understanding the properties of these highly complex media. Metallic
nanostructures and exciton-plasmon interaction are one of the promising
directions of study. Due to the inherent losses in Cu, regular plasmon modes
alone are not suitable as carriers of information over the long distances.
However, as we have demonstrated, the so-called long-range plasmons can
achieve propagation length on the order of micrometres. At this scale, it is
possible to couple them with excitons in Cu2O, allowing for many interesting
options to dynamically control the plasmon propagation. It is shown that the
group velocity of propagating plasmon can be significantly affected by the
presence of excitons, which affects the system transmission. The presented
results may pave the way to compact, copper-based tunable devices.

## References

  * [1] A. Böhme, F. Sterl, E. Kath, M. Ubl, V. Manninen, and H. Giessen, ,,Electrochemistry on Inverse Copper Nanoantennas: Active Plasmonic Devices with Extraordinarily Large Resonance Shift”, ACS Photonics 6, 1863-1868 (2019).
  * [2] C. Melo, M. Jullien, Y. Battie, A. Naciri, J. Ghanbaja, F. Montaigne, J. Pierson, F. Rigoni, N. Almqvist, A. Vomiero, S. Migot, F. Mücklich, and D. Horwat, ,,Tunable Localized Surface Plasmon Resonance and Broadband Visible Photoresponse of Cu Nanoparticles/ZnO Surfaces”, CS Appl. Mater. Interfaces 10, 47, 40958-40965 (2018).
  * [3] C. Huang, G. Kumar, G. Sharma, and F. Chen, ,,Plasmonic effects of copper nanoparticles in polymer photovoltaic devices for outdoor and indoor applications”, Appl. Phys. Lett. 116, 253302 (2020).
  * [4] D. Fedyanin, D. Yakubovsky, R. Kirtaev, and V. Volkov, ,,Ultralow-Loss CMOS Copper Plasmonic Waveguides”, Nano Lett. 16, 1, 362-366 (2016).
  * [5] V. G. Kravets, R. Jalil, Y. J. Kim, D. Ansell, D. E. Aznakayeva, B. Thackray, L. Britnell, B. D. Belle, F. Withers, I. P. Radko, Z. Han, S. I. Bozhevolnyi, K. S. Novoselov, A. K. Geim, and A. N. Grigorenko, ,,Graphene-protected copper and silver plasmonics”, Scientific Reports 4, 5517 (2014).
  * [6] G. Chan, J. Zhao, E. Hicks, G. Schatz, and R. Van Duyne, ,,Plasmonic Properties of Copper Nanoparticles Fabricated by Nanosphere Lithography”, Nano Lett. 7, 7, 1947-1952 (2007).
  * [7] O. Peña-Rodríguez, U. Pal, ,,Effects of surface oxidation on the linear optical properties of Cu nanoparticles”, Journal of the Optical Society of America B 28(11) 2735-2739 (2011).
  * [8] J. Sancho-Parramon, B. Okorn, K. Salamon, V. Janicki, ,,Plasmonic resonances in copper island films”, Applied Surface Science 463, 847-853 (2019).
  * [9] V. Mkhitaryan, K. March, E. Tseng, X. Li, L. Scarabelli, L. Liz-Marzán, S. Chen, L. Tizei, O. Stéphan, J. Song, M. Kociak, F. Garcia, and A. Gloter, ,,Can Copper Nanostructures Sustain High-Quality Plasmons?”, Nano Lett. 21, 6, 2444-2452 (2021).
  * [10] N. Fofang, T. Park, O. Neumann, N. Mirin, P. Nordlander, and N. Halas, ,,Plexcitonic nanoparticles: plasmon-exciton coupling in nanoshell-J-aggregate complexes”, Nano Lett. 8(10), 3481-3487 (2008).
  * [11] E. Karademir, S. Balci, C. Kocabas, and A. Aydinli, ,,Plexcitonic crystals: a tunable platform for light-matter interactions”, Optics Express 22, 18, 21912-21920 (2014).
  * [12] H. Lee, D. Luong, M. Kim, Y. Jin, H. Kim, S. Yun, and Y. Lee, ,,Reconfigurable exciton-plasmon interconversion for nanophotonic circuits”, Nature Communications 7, 13663 (2016).
  * [13] E. Cao, W Lin, M. Sun, W. Liang, and Y. Song, ,,Exciton-plasmon coupling interactions: from principle to applications”, Nanophotonics 7(1), 145-167 (2018).
  * [14] P. Goncalves, L. Bertelsen, S. Xiao, and N. Mortensen, ,,Plasmon-exciton polaritons in two-dimensional semiconductor/metal interfaces”, Phys. Rev. B 97, 041402(R) (2018).
  * [15] J. Khurgin, ,,Pliable polaritons: Wannier exciton-plasmon coupling in metal-semiconductor structures”, Nanophotonics 8(4), 629-639 (2019).
  * [16] T. Kazimierczuk, D. Fröhlich, S. Scheel, H. Stolz, and M. Bayer, Nature 514, 344 (2014).
  * [17] A. Poddubny, M. Glazov, ,,Topological Spin Phases of Trapped Rydberg Excitons in Cu$2_{O}$”, Phys. Rev. Lett. 123, 126801 (2019).
  * [18] D. Ziemkiewicz, G. Czajkowski, K. Karpiński, S. Zielińska-Raczyńska, ,,Rydberg magnetoexcitons in Cu2O quantum wells”, Phys. Rev. B 103, 035305 (2021).
  * [19] K. Orfanakis, S. Rajendran, H. Ohadi, S. Zielińska-Raczyńska, G. Czajkowski, K. Karpiński, D. Ziemkiewicz, ,,Quantum confined Rydberg excitons in Cu2O nanoparticles”, Phys. Rev. B 103, 245426 (2021).
  * [20] A. Konzelmann, B. Frank, H. Giessen, ,,Quantum confined Rydberg excitons in reduced dimensions”, J. Phys. B: At. Mol. Opt. Phys. 53, 024001 (2020).
  * [21] S. Steinhauer, M. Versteegh, S. Gyger, A. Elshaari, B. Kunert, A. Mysyrowicz, V. Zwiller, _Rydberg excitons in Cu2O microcrystals grown on a silicon platform_ , Nature Communications Materials 1, 11, 2-7 (2020).
  * [22] M. Takahata, K. Tanaka, and N. Naka, ,,Nonlocal optical response of weakly confined excitons in Cu2O mesoscopic films”, Phys. Rev. B 97, 205305 (2018).
  * [23] C. Morin, J. Tignon, J. Mangeney, S. Dhillon, G. Czajkowski, K. Karpiński, S. Zielińska-Raczyńska, and D. Ziemkiewicz, T. Boulier, arXiv:2202.09239v1 [quant-ph].
  * [24] G.M. Kavoulakis, G. Baym and J.P. Wolfe, ,,Quantum saturation and condensation of excitons in Cu2O: a theoretical study”, Phys. Rev. B 53, 7227 (1996).
  * [25] M. Kohlhoff, ,,Interaction of Rydberg atoms with surfaces”, Eur. Phys. J. Special Topics 225, 3061-3085 (2016).
  * [26] Chubchev E., Nechepurenko I., Dorofeenko A., Vinogradov A., Lisyansky A., ,,Highly confined surface plasmon polaritons in the ultraviolet region”, Optics Express 2018, 27(7), 9050.
  * [27] H. Raether, "Surface plasmons on smooth surfaces", in _Surface plasmons on smooth and rough surfaces and on gratings_ (Springer, 1988), pp. 4-39.
  * [28] D. Ziemkiewicz, K. Słowik, S. Zielińska - Raczyńska, ,,Ultraslow long-living plasmons with electromagnetically induced transparency,” _Opt. Lett._ 43(3), 490-493 (2018).
  * [29] V. Konopsky, E. Alieva, ,,Long-Range Propagation of Plasmon Polaritons in a Thin Metal Film on a One-Dimensional Photonic Crystal Surface”, Phys. Rev. Lett. 97, 253904 (2006).
  * [30] R. Yi, L. Fang, H. Yi-Dong, D. Ohnishi, Z. Wei, and P. Jiang-De, ,,Long-range surface plasmon polaritons guided by a thin metal stripe”, Chinese Physics Letters 24, 1626 (2007).
  * [31] S. Park, J. Jin Ju, J. Tae Kim, M. Kim, S. Koo Park, J. Lee, W. Lee, and M. Lee, ,,Sub-dB/cm propagation loss in silver stripe waveguides”, Opt. Express 17, 697-702 (2009).
  * [32] P. Berini, ,,Long-range surface plasmon polaritons”, Adv. Opt. Photon. 1(3), 484-588 (2009).
  * [33] T. Hollstein, U. Kreibic, F. Lens, ,,Optical Properties of Cu and Ag”, Phys. Stat. Sol. (b) 83, 545 (1977).
  * [34] K. Takagi, S. Nair, R. Watanabe, K. Seto1, T. Kobayashi, and E. Tokunaga, ,,Surface Plasmon Polariton Resonance of Gold, Silver, and Copper Studied in the Kretschmann Geometry: Dependence on Wavelength, Angle of Incidence, and Film Thickness” J. Phys. Soc. Jpn. 86, 124721 (2017).
  * [35] S. Zielińska-Raczyńska, G. Czajkowski, and D. Ziemkiewicz, Phys. Rev. B 93, 075206 (2016).
  * [36] K. Karpiński, S. Zielińska-Raczyńska, and D. Ziemkiewicz, ,,Long-range plasmons and epsilon-near-zero modes in ultraviolet”, JOSA B 38, 1, 79-87 (2021).
  * [37] H. Stolz, F. Schöne and D. Semkat, ,,Interaction of Rydberg excitons in cuprous oxide with phonons and photons: optical linewidth and polariton effect”, New J. Phys. 20, 023019 (2018).
  * [38] U. Wahid, A. Khan, B. Amin, and A. Ullah, ,,Surface plasmon hole burning at the interface of Cesium and Gold by Kerr nonlinearity”, Optik, 202, 163651 (2020).
  * [39] K. S. Yee, ,,Numerical solution of initial boundary value problems involving Maxwell’s equations in isotropic media”, IEEE Trans. Antennas Propagat. 14, 302-307 (1966).
  * [40] M. A. Alsunaidi, A. Al-Jabr, ,,A general ADE-FDTD algorithm for the simulation of dispersive structures”, IEEE Photon. Techol. Lett. 21, 817-819 (2009).


# Chinese Idiom Paraphrasing

Jipeng Qiang, Yang Li, Chaowei Zhang, Yun Li, Yunhao Yuan, Yi Zhu, and Xindong
Wu J. Qiang, Yang. Li, C. Zhang, Yun Li, Y. Yuan, and Y Zhu are with School of
Information Engineering, Yangzhou University, Yangzhou, Jiangsu, China.
E-mail: {jpqiang, cwzhang, liyun, zhuyi<EMAIL_ADDRESS>X. Wu is with Key
Laboratory of Knowledge Engineering with Big Data (Hefei University of
Technology), Ministry of Education, Hefei, Anhui, China.
E-mail<EMAIL_ADDRESS>
###### Abstract

Idioms, are a kind of idiomatic expression in Chinese, most of which consist
of four Chinese characters. Due to the properties of non-compositionality and
metaphorical meaning, Chinese Idioms are hard to be understood by children and
non-native speakers. This study proposes a novel task, denoted as Chinese
Idiom Paraphrasing (CIP). CIP aims to rephrase idioms-included sentences to
non-idiomatic ones under the premise of preserving the original sentence’s
meaning. Since the sentences without idioms are easier handled by Chinese NLP
systems, CIP can be used to pre-process Chinese datasets, thereby facilitating
and improving the performance of Chinese NLP tasks, e.g., machine translation
system, Chinese idiom cloze, and Chinese idiom embeddings. In this study, CIP
task is treated as a special paraphrase generation task. To circumvent
difficulties in acquiring annotations, we first establish a large-scale CIP
dataset based on human and machine collaboration, which consists of 115,530
sentence pairs. We further deploy three baselines and two novel CIP approaches
to deal with CIP problems. The results show that the proposed methods have
better performances than the baselines based on the established CIP dataset.

###### Index Terms:

Chinese Idiom, Sentence Paraphrasing, Seq2Seq, mT5

##  1 Introduction

Idiom, called ”成语” (ChengYu) in Chinese, is widely used in daily
communications and various literary genres. Idioms are a kind of compact
Chinese expressions that consist of few words but imply relatively complex
social nuances. Moreover, the Chinese idioms are often used to describe
similar phenomena, events, etc, which means the idioms can not be interpreted
with their literal meanings in some cases. Thus, it has always been a
challenge for non-native speakers, even native speakers to recognize Chinese
idioms[1]. For instance, the idiom ”亡羊补牢”(MangYangBuLao) shown in Table 1,
represents ”Never be late to try”, instead of its literal meaning - ”To mend
the fence after sheep are lost”.

In real life, if some people do not understand the meaning of idioms, we have
to explain them by converting them into a set of word segments that reflect
more intuitive and understandable paraphrasing. In this study, we try to
manipulate computational approaches to automatically rephrase idiom-included
sentences into simpler sentences (a.k.a., non-idiom-included sentences) for
preserving context-based paraphrasing, and then benefit both Chinese-based
natural language processing and societal applications.

Since idioms are a kind of obstacles for many NLP tasks, CIP can be used as a
pre-processing phase that facilitates and improves the performance of machine
translation system [2, 3], Chinese idiom cloze [4, 1], and Chinese idiom
embeddings [5]. Furthermore, CIP-based applications can help specific crowds,
such as children, non-native speakers, the people with cognitive disabilities,
to improve their abilities of reading comprehension.

We propose a new task in this study, denoted as Chinese Idiom Paraphrasing
(CIP), which aims to rephrase the idiom-included sentences into fluent,
intuitive, and meaning-preserving non-idiom-included sentences. We can treat
CIP task as a special paraphrase generation task. The general paraphrase
generation task aims to rephrase a given sentence to another one that
possesses identical semantics but various lexicons or syntax [6, 7].
Similarly, CIP emphasizes rephrasing the idioms of input sentences to word
segments that reflect more intuitive and understandable paraphrasing. In
recent decades, many researchers devoted to paraphrase generation [8, 9] are
struggled due to the lack of reliable supervision dataset [10]. Inspired by
the challenge, we establish a large-scale training dataset in this work for
CIP task.

Input | 虽然你已经犯下了错误,但是亡羊补牢也为时不晚.
---|---
| (Although you have made a mistake, it’s not too late to mend it.)
Output | 虽然你已经犯下了错误,但是现在改正也为时不晚.
TABLE I: Given a Chinese idiom-included sentence, we aim to output a fluent,
intuitive, and meaning-preserving non-idiom-included sentence. In the example,
the idiom is marked in red.

Contributions. This study produces two main contributions toward the
development of CIP systems.

First, a large-scale benchmark is established for CIP task. The benchmark is
comprised of 115,530 sentence pairs, which of 8,421 idioms. A recurrent
challenge in crowdsourcing NLP-oriented datasets at scale-level is that human
writers frequently utilize repetitive patterns to fabricate examples, leading
to a lack of linguistic diversity [11]. A new large-scale CIP dataset is
created in this study by taking advantage of the collaboration between humans
and machines.

In detail, we initially divide a large-scale Chinese-English machine
translation corpus into two parts (idioms-included sub-corpus, and non-idioms-
included sub-corpus) by judging if a Chinese sentence contains idioms. Next,
we train an English-to-Chinese machine translation (MT) system using the non-
idioms-included sub-corpus. Because the training corpus for MT system does not
include any idiom, MT system will not translate input English sentences to
idiom-included Chinese sentences. Then, the MT system is deployed to translate
English sentences of idioms-included sub-corpus to non-idioms-included
sentences. A large-scale pseudo-parallel CIP dataset can be constructed by
pairing the idioms-included sentences of idioms-included sub-corpus and the
translated non-idioms-included sentences. Finally, we employ native speakers
to validate the generated sentences and modify defective sentences if
necessary.

Second, we deploy five baselines to rephrase the input idiom-included
sentence. Since the constructed dataset is used as the training dataset, we
treat CIP task as a paraphrase generation task. (i,ii,iii) We adopt three
different sequence-to-sequence (Seq2Seq) methods as baselines: LSTM-based
approach, Transformer-based approach, and mT5-based approach, where mT5 is a
massively multilingual pre-trained text-to-text Transformer [12]. (iv) People
have always used dictionaries to deal with CIP problems in the past, but the
dictionaries cannot be directly incorporated into Seq2Seq methods. We propose
a new method that allows networks to ”attach” interpretations to idioms, which
means the networks can learn the correlation capabilities between
interpretations and idioms. (v) mT5 is pre-trained aiming at span masked
language modeling, where consecutive spans of input tokens are replaced with a
mask token and the trained model can reconstruct the masked-out tokens. The
CIP problem can be dealt with by the span masked language modeling when the
idioms of the sentences are masked. Specifically, a CIP sentence pair can be
processed to produce a (corrupted) input sentence by replacing both the idioms
of the source sentence and a corresponding target extracted from the
simplified sentence. The mT5-based CIP method is fine-tuned to reconstruct the
corresponding target. Experimental results show that the baselines evaluated
on the constructed CIP dataset can output high-quality paraphrasing of
sentences that are grammatically correct and semantically appropriate.

The constructed dataset and employed baselines that are used to accelerate
this research are source-opened in Github
111https://www.github.com/jpqiang/Chinese-Idiom-Paraphrasing.

##  2 Related Works

Paraphrase Generation. Paraphrase generation aims to extract paraphrases of
given sentences. The extracted paraphrases can preserve original meanings of
the sentence, but are assembled with different words or syntactic structures
[8, 9, 13].

Most recent neural paraphrase generation methods primarily take advantage of
the sequence-to-sequence framework that can achieve inspiring performance
improvements compared with traditional approaches [13]. A long-standing issue
embraced in paraphrase generation studies is the lack of reliable supervised
datasets. The issue can be avoided by constructing manually annotated paired-
paraphrase datasets [6] or designing unsupervised paraphrase generation
methods [10]. Differ from existing paraphrase generation research, we take our
attention to Chinese idiom paraphrasing that rephrases idiom-included
sentences to non-idiom-included ones.

Chinese Idiom Understanding. Idiom is an interesting linguistic phenomenon in
the Chinese language. Compared with other types of words, most idioms are
unique in perspectives of non-compositionality and metaphorical meaning. Idiom
understanding plays an important role in the research area of Chinese language
understanding. Many types of research related to Chinese idiom understanding
have been pushed forward that can benefit a variety of related down-streaming
tasks. For example, Shao et al. [3] focused on evaluating the quality of idiom
translation of machine translation systems. Zheng et al.[1] provided a
benchmark to assess the abilities of multiple models on Chinese idiom-based
cloze tests, and evaluated how well the models can comprehend Chinese idiom-
included texts. Liu et al. [14] studied how to improve essay writing skills by
recommending Chinese idioms. Tan et al. [5] investigated the tasks on learning
and quality evaluation of Chinese idiom embeddings. In this paper, we study a
novel CIP task that is different from the above tasks. Since the proposed CIP
method can rephrase idiom-included sentences to non-idiom-included ones, it is
expected that CIP can benefit the tasks related to idiom representation and
idiom translation.

Other related tasks. Pershina et al.[15] studied a new task of English idiom
paraphrases aiming to determine if two idioms have alike or similar meanings.
They collected idioms’ definitions in dictionary and utilized word embedding
modelings to represent idioms to calculate the similarity of two idioms. Qiang
et al. [16] proposed a Chinese lexical simplification method, which focuses on
replacing complex words in given sentences with simpler and meaning-equivalent
alternatives. It is noteworthy that the substitutes in Chinese lexical
simplification are all made up of a single word, but an idiom typically cannot
be substituted by a single word to express original concepts or ideas.

##  3 Human and Machine Collaborative Dataset Construction

Figure 1: A pipelined illustration of creating a CIP dataset based on a
Chinese-English machine translation (MT) corpus. (a) The corpus is split into
an idiom-included sub-corpus and a non-idiom-included sub-corpus based on a
Chinese idiom list; (b) We train a MT system using the non-idiom-included sub-
corpus, and create a pseudo-CIP Dataset by pairing the original Chinese idiom-
included sentences and the translated non-idiom-included sentences using the
trained MT system; (c) We ask human annotators to revise the translated
Chinese sentence of pairs to strengthen the quality of the created CIP
dataset.

This section describes the process of constructing a large-scale parallel
dataset for Chinese idiom paraphrasing (CIP). A qualified CIP dataset needs to
meet the following two requirements: (1) The two sentences in a sentence pair
have to convey the same meaning; (2) A sentence pair has to contain an idiom-
included sentence and an idiom-included one. We outline a three-stage pipeline
for dataset construction, which takes advantage of both the generative
strength of machine translation (MT) methods and the evaluative strength of
human annotators. Human annotators are generally reliable in correcting
examples, but it is challenging while crafting diverse and creative examples
at scale. Therefore, we deploy a machine translator to automatically create an
initial CIP dataset, and then inquire annotators to proofread each generated
sample.

###  3.1 Pipeline

Figure 1 exhibits the details of the pipeline. Our pipeline starts with an
existing English-Chinese machine translation dataset denoted as $\mathcal{D}$.
Firstly, we refer a collect Chinese idiom list $\mathcal{I}$ to split the MT
dataset $\mathcal{D}$ into two parts: non-idiom-included sub-dataset
$\mathcal{D}1$ and idiom-included sub-dataset $\mathcal{D}2$ (Stage 1). All
the data items in both $\mathcal{D}1$ and $\mathcal{D}2$ are in forms of
sentence pairs. Then, we train a neural machine translation system
$\mathcal{M}$ using $\mathcal{D}1$, which can translate English sentences to
non-idiom-included Chinese sentences. Next, we input English sentences in
$\mathcal{D}2$ to $\mathcal{M}$ to output non-idiom-included Chinese
sentences. Afterwards, the Chinese sentences in $\mathcal{D}2$ and the
generated sentences are mated as pairs to construct a large-scale initial
parallel CIP dataset (Stage 2). Finally, the constructed dataset is reviewed
and revised by annotators for quality assurances (Stage 3).

Stage 1: Corpus Segmentation. The English-Chinese MT dataset $\mathcal{D}$ we
applied in the research are grabbed from WMT18 [17], which contains 24,752,392
sentence pairs. We collect a Chinese idiom list $\mathcal{I}$ that embraces
31,114 idioms. Since the list enables determining whether the Chinese sentence
in a pair contains idioms, $\mathcal{D}$ can be split as $\mathcal{D}1$ and
$\mathcal{D}2$. The sub-dataset $\mathcal{D}1$ is used to train a special MT
system $\mathcal{M}$ that can translate English sentences to non-idiom-
included Chinese sentences. In our experiments, only 0.2% of the translated
Chinese sentences contain idioms (see Table 6). After cleansing redundant
Chinese sentences, the number of sentence pairs in $\mathcal{D}2$ is 105,530.

Stage 2: Pseudo-CIP Dataset. Giving a sentence pair $(c_{i},e_{i})$ in
$\mathcal{D}2$, we input the English sentence $e_{i}$ into MT system
$\mathcal{M}$, and output a Chinese translation $t_{i}$. We pair Chinese
sentence $c_{i}$ and Chinese translation $t_{i}$ as a pseudo-CIP sentence
pair. Thus, a CIP dataset can be built up by pairing original Chinese
sentences and corresponding translated English-to-Chinese ones in
$\mathcal{D}2$. The pseudo-CIP dataset $\mathcal{D}2^{\prime}$ can meet the
two requirements of CIP dataset construction. On one hand, the pseudo-CIP data
is from the MT dataset, which can guarantee that the paired sentences deliver
the same meanings. On another hand, all original sentences include one or more
idioms, and all the translated sentences do not contain idioms.

Stage 3: Human Review. As the final stage of the pipeline, we recruit five
human annotators to review each sentence pair $(c_{i},t_{i})$ in the pseudo-
CIP dataset $\mathcal{D}2^{\prime}$. These annotators are all undergraduates,
and Chinese natives. Given $(c_{i},t_{i})$, annotators are asked to revise and
improve the quality of $t_{i}$. $t_{i}$ is required to be non-idiom-included
and fully meaning-preserved.

###  3.2 Corpus Statistics

|  | In-domain | Out-of-domain |
---|---|---|---|---
|  | Train | Dev | Test | Dev | Test | Total
sentence pairs | 95560 | 5000 | 5000 | 4994 | 4976 | 115530
Source sentence | token | 3382042 | 176492 | 174753 | 225850 | 221680 | 4180817
Avg. sentence length | 35 | 35 | 34 | 45 | 44 | 36
All Idioms | 103168 | 5383 | 5373 | 5808 | 5801 | 125533
Unique Idioms | 8243 | 5030 | 5038 | 5149 | 5128 | 8421
Reference sentence | tokens | 3444405 | 179232 | 177438 | 239582 | 225452 | 4266109
Avg. sentence length | 36 | 35 | 35 | 47 | 45 | 36
Avg. edit disatance | 7.82 | 7.75 | 7.67 | 6.21 | 4.82 | 7.61
TABLE II:  The statistics of CIP dataset.

The dataset $\mathcal{D}2^{\prime}$ is treated as in-domain data, which
contains 105,530 examples including 8,243 different idioms.
$\mathcal{D}2^{\prime}$ is partitioned into three parts: a training set Train,
a development set Dev, and a test set Test. We sort the examples based on the
frequencies of idioms in the corpus from high to low. We choose two samples
from each of the first 5,000 idioms, and put the samples into Dev and Test,
respectively.

We observe that both the Train and Test datasets come from a same
distribution. However, when models are deployed in real-world applications,
the inference might be performed on the data from different distributions,
i.e. out-of-domain [18]. Therefore, we additionally collected 9,970 sentences
with idioms from modern vernacular classics, including prose and fiction, as
out-of-domain data, to assess the generalization ability of CIP methods.
Unlike MT corpus, these sentences have no English sentences as their
references, we manually modify them to non-idiom-included sentences with the
helps of Chinese native speakers. The statistical details of CIP dataset are
shown in Table 2.

There are three significant differences between in-domain and out-of-domain
data. First, the average length of sentences in in-domain data is around 35
words, while is about 45 words for out-of-domain data. Second, the average
number of idioms in in-domain data is 1.07, which is lower than that of out-
of-domain data (i.e., 1.17). Third, the sentence pairs in out-of-domain data
need fewer modifications than that in in-domain data. In this case, a lack of
linguistic diversity might be taken place due to human annotators often
relying on repetitive patterns to generate sentences.

We present some examples of the idiom ”深居简出” (reclusive) in the CIP dataset,
shown in Table 3. The idiom ”深居简出” can be rephrased with different
descriptions, which displays the diversity of linguistics.

c | 约翰并不有钱,他住在海边,深居简出。
---|---
e | John is not rich, he live in a small way near the coast
t | 约翰并不有钱,他住在海边,很少出门。
c | 她除了演唱外,其余时间则人深居简出。
e | she seldom goes out at other times, except when she sings.
t | 她除了演唱外,其他时候很少出门。
c | 他们崇尚非暴力、深居简出和远离现代社会的生活方式
e | they believe in nonviolence, simple living and little contact with the
modern world.
t | 他们信仰非暴力、简单的生活,和远离现代社会的生活方式。
c | 绝大多数的时候她都是深居简出，偶尔在公众场合出现。
e | the majority of the time she lives a secluded life, only going out
occasionally.
t | 她大部分时间过着隐居的生活,偶尔在公众场合出现。
c | 我现在爱过幽静的，节俭的深居简出的生活。
e | I chose now to live retired, frugal, and within ourselves.
t | 我现在爱过幽静的,节俭的退休的生活。
TABLE III:  The examples contain the idiom ”深居简出” in CIP dataset. c and e are
a machine learning sentence pair, t is the CIP reference sentence of c
generated by collaborating machine translation and human intervention.

##  4 Methods

In this section, we introduce three baselines and two proposed CIP methods to
perform CIP task. The performances of the five methods are compared with each
other in the aspect of idioms-included sentence rephrasing.

###  4.1 Problem Formulation

The CIP task can be defined as follows. Given a source sentence
$\textbf{c}=\\{c_{1},...,c_{j},...,c_{J}\\}$ with one or more idioms,
intending to produce a target sentence
$\textbf{t}=\\{t_{1},...,t_{i},...t_{I}\\}$. More specifically, t is expected
to be non-idiom-included and meaning-preserved, where $c_{j}$ or $t_{i}$
refers to a Chinese character. In this study, we suppose to design a
supervised method to approach this monolingual machine translation task. We
adopt a Sequence-to-Sequence (Seq2Seq) framework that directly predicts the
probability of the character-sequential translation from source sentences to
target ones [19], where the probability is calculated using the following
equation 1:

$P(\textbf{t}\mid\textbf{c})=\prod_{i=1}^{I}P(t_{i}\mid t_{<i},\textbf{c})$
(1)

where $t_{<i}=t_{1},...,t_{i-1}$.

###  4.2 Seq2Seq method

In this research, we adopt three Seq2Seq methods to handle CIP task that are
LSTM-based, Transformer-based, and mT5-based models, respectively.

(1) LSTM-based Seq2Seq methods are a basic Seq2Seq method, which uses a LSTM
(Long Short-Term Memory [20]) to convert a sentence to a dense, fixed-length
vector representation. In contrast to RNN, LSTM is helpful to deal with long
sequences, but it fails to maintain the global information of the sequences.

(2) Transformer-based Seq2Seq methods[21] is a state-of-the-art Seq2Seq method
that has been widely adopted to process various NLP tasks, such as machine
translation, abstractive summarization, etc. Transformer applies a self-
attention mechanism that directly models the relationships among all words of
an input sequence regardless of words’ positions. Unlike LSTM, Transformer
handles the entire input sequence for once, rather than iterating words one by
one.

(3) mT5 is a Seq2Seq method that uses the framework of Transformer. Currently,
most downstream NLP tasks build their models by fine-tuning pre-trained
language models [22]. mT5 is a massively multilingual pre-trained language
model that is implemented in a form of unified ”text-to-text” to process
different downstream NLP problems. In this study, we fine-tune the mT5-based
approach to handle CIP task.

###  4.3  Knowledge-based CIP method

Figure 2: The knowledge-based Seq2Seq method incorporates idiom
interpretations into Seq2Seq modeling. The words marked in red are idioms, and
the words labeled with blue are the corresponding interpretations. ”$<$/s$>$”
and ”$<$SEP$>$” are special symbols.

We present an extension of Seq2Seq modeling that enables replacing idioms with
their representations at the same locations in source sentences, thereby the
network can better leverage idioms. The proposed method refers to a knowledge-
based Seq2Seq method. Figure 2 illustrates the process of encoding a sample
sentence. The model firstly extracts a dictionary of idiom interpretation from
the established CIP training dataset, and then concatenates original sentences
and all idioms’ interpretations. Finally, the concatenations are utilized for
Seq2Seq modeling.

(1) The interpretation of a given sentence pair (c,t) is extracted by
computing the edit operations. Suppose that c is ”约翰 踢 了 我 一脚 ， 所以 我 以牙还牙 .”,
and t is ”汤姆 踢 我 一脚 吧 ， 所以 我 也 踢了 他 一脚 .”, where the characters of the
sentence are split by spaces. We construct an edit sequences Diff by matching
all the words in both c and t using three edit operations (’=’,’-’,’+’), where
’=’,’-’,’+’ represent ’keep’, ’delete’ and ’add’ operations, respectively
222https://github.com/paulgb/simplediff. The output Diff is ”(’-’, ’约翰’),
(’+’, ’汤姆’), (’=’, ’踢’), (’-’, ’了’), (’=’, ’我一脚’), (’+’, ’吧’), (’=’, ’，,
所以我’), (’-’, ’以牙还牙’), (’+’, ’也踢了他一脚’), (’=’, ’.’)”. Specifically, the tuple
(’-’, ’约翰’) indicates that ’约翰’ appears in c but not in t; (’+’, ’汤姆’) denotes
that ’汤姆’ appears in t but not in c; and (’=’, ’踢’) represents that ’踢’ is in
both c and t.

We traverse the edit sequences Diff using a rule $<$’-’,’+’$>$, where the rule
means ’+’ follows by ’-’ in Diff. We get the following two matching sequence
pairs $<$’约翰’,’汤姆’$>$ and $<$’以牙还牙’,’也踢了他一脚’$>$. A sequence pair will be
ignored by the model if any idiom is included. In this example, we obtain the
sequence pair $<$’以牙还牙’,’也踢了他一脚’$>$, where ”以牙还牙” and ”也踢了他一脚” represent an
idiom and corresponding interpretation. Finally, we sort different
interpretations of each idiom based on their frequencies, and collect the top
three interpretations into the dictionary.

(2) Given a training example $\\{\textbf{c},\textbf{t}\\}$, if $\\{\textbf{c}$
contains multiple idioms, all interpretations of the idioms are collected from
the dictionary. In case of multiple interpretations of an idiom, we first
concatenate the interpretations with spaces. It is noteworthy that if a
sentence has multiple idioms, we need to concatenate the interpretations of
different idioms with symbol ”$<$SEP$>$”. Finally, the original sentence and
the collected interpretations are concatenated and processed for Seq2Seq
modeling.

We prompt the Seq2Seq modeling to learn how to make use of the concatenated
information. It’s noteworthy that the knowledge-based Seq2Seq method can be
incorporated into all three baselines. In our experiments, we adopt mT5-based
Seq2Seq modeling as the base method.

###  4.4 Infill-based CIP method

Figure 3: An example of Infill-based CIP method. The input sequence fed into
mT5-based Seq2Seq modeling comprised of an original sentence and a target
sentence, in which the idiom of the original sentence is replaced by one
special symbol ”$<$X$>$”. The interpretation of the idiom is treated as the
reference sentence, rather than the target sentence.

Given a sentence pair $\\{\textbf{c},\textbf{t}\\}$, we completely generate
the whole target sentence t from the original sentence c using the above four
CIP methods. CIP merely requires to rephrase the idioms of the sentence, which
means we only expect to generate context-based interpretations of idioms,
rather than the whole sentence. Therefore, we propose a novel CIP method,
denoted as infill-based CIP method.

As shown in Figure 3, the proposed method takes advantage of mT5 which is a
pre-trained span masked language modeling (MLM) to build a Seq2Seq model. In
contrast to MLM in BERT [23], span MLM reconstructs consecutive spans of input
tokens and mask them with a mask token $<$X$>$. With the help of mT5, the
proposed method enables reconstructing the idiom $I$’s interpretation of
sentence c via replacing the idiom with the token $<$X$>$.

To embed mT5-based Seq2Seq modeling into CIP, we additionally make two
modifications. (1) If the reconstructed c is directly fed to the span MLM, the
information of idiom $I$ is likely to be ignored. We concatenate the original
c and the reconstructed c as the input sequence. (2) When a sentence has
multiple idioms, we recursively simplify each idiom to a corresponding
representation.

##  5 Experiments

In this section, we introduce the process of conducting experiments that are
used to handle CIP problem. We firstly illuminate the system configurations
that include implementation details, the metrics used to evaluate
performances, and baseline methods. Next, the performances of different CIP
methods are assessed in Section 5.2. Then, we acquire human evaluations to
survey the capabilities of deployed CIP methods on CIP processing. In
addition, we calculate the proportions of idiom-included sentences that are
rephrased to non-idiom-included ones. Finally, a case study is demonstrated in
Section 5.5 that compares the different outputs of five methods on
representing difficult Chinese idioms.

###  5.1 Experiment Setups

Implementation Details. In this experiment design, there are five CIP methods
are deployed, including: LSTM-based Seq2Seq modeling (LSTM), Transformer-based
Seq2Seq modeling (Transformer), mT5-based Seq2Seq modeling (mT5), Knowledge-
based CIP method (Knowledge) and infill-based CIP method (Infill).

We implement LSTM and Transformer methods using _fairseq_ [24]. mT5,
Knowledge, and Infill methods are mT5-based, and are fulfilled using
Huggingface transformers [25]. Furthermore, the sentence tokenization is
accomplished using _Jieba Chinese word segmenter_ and _BPE tokenization_. The
size of the vocabulary is set to 32K. The LSTM-based Seq2Seq method adopts
Adam optimizer configured with $\beta=(0.9,0.98)$, $3e^{-4}$ learning rate,
and $0.2$ dropout rate. The Transformer-based Seq2Seq method maintains the
hyperparameters of the base Transformer [21] (base), which contains a six-
layered encoder and a six-layered decoder. The three parameters ($\beta$ of
Adam optimizer, learning rate, and dropout rate) in the Transformer-based
method are equivalent to these in the LSTM-based method. It’s noteworthy that
the learning rate is gradually increased to $3e^{-4}$ by 4k steps and
correspondingly decays according to the inverse square root schedule.
Knowledge and Infill methods are mT5-based Seq2Seq models. For mT5, Knowledge,
and Infill, we fine-tune mT5 (base) that consists of 580M parameters
333https://huggingface.co/google/mt5-base. We train the three methods with the
Adam optimizer (Kingma and Ba, 2015) and an initial learning rate of $3e^{-4}$
up to 20 epochs using early stopping on development data. The training will be
stopped when the accuracy on the development set did not improve within 5
epochs. We used a beam search with 5 beams for inference.

Metrics. As we mentioned above, the CIP task can be treated as a sentence
paraphrasing task. Therefore, We apply four metrics to evaluate sentence
paraphrasing task namely, _BLEU_ , _BERTScore_ , _ROUGE-1_ and _ROUGE-2_.
_BLEU_ is a widely used machine translation metric, which measures opposed
references to evaluate lexical overlaps with human intervention [26].
_BERTScore_ is chosen as another metric due to its high correlation with human
judgments [27]. Compared to _BLEU_ , _BERTScore_ is measured using token-wise
cosine similarity between representations produced by _BERT_. We measure
semantic overlaps between generated sentences and reference ones using ROUGE
scores [28]. _ROUGE_ is often used to evaluate text summarization. The two
metrics _ROUGE-1_ and _ROUGE-2_ refer to the overlaps of unigram and bigram
between the system and reference summaries, respectively.

Baselines. We additionally provide two unsupervised methods that can
facilitate solving the CIP problem, namely, Re-translation and BERT-CLS.

(1) Re-translation: is implemented by utilizing the back-translation
techniques of machine translation methods. We first translate an idiom-
included sentence to an English sentence using an efficient Chinese-English
translation system, and then translate the generated English sentence using
our trained English-Chinese translation system (introduced by Section 3.1) to
generate a non-idiom-included Chinese sentence. The Chinese-English
translation system can be easily accessed through the
link444data.statmt.org/wmt17_systems. The trained English-Chinese translation
system is a transformer-based Seq2Seq method.

(2) BERT-CLS: is an existing BERT-based Chinese lexical simplification method
[16]. In this task, an idiom is treated as a complex word that will be
replaced with a simpler word.

###  5.2 Performance of CIP Method

Method | Domain | BLEU | BERTS | ROU1 | ROU2
---|---|---|---|---|---
Re-translation | In | 29.71 | 80.47 | 57.25 | 35.62
Out | 15.71 | 74.63 | 46.78 | 21.93
BERT | In | 73.58 | 91.44 | 81.11 | 73.27
Out | 79.65 | 94.41 | 87.78 | 83.39
LSTM | In | 83.71 | 94.46 | 88.07 | 82.00
Out | 80.14 | 93.19 | 85.86 | 79.85
Transformer | In | 84.20 | 94.70 | 88.44 | 82.54
Out | 79.58 | 93.42 | 86.23 | 80.41
mT5 | In | 84.57 | 94.79 | 88.48 | 82.59
Out | 86.18 | 94.93 | 88.63 | 83.58
Knowledge | In | 85.25 | 94.97 | 88.94 | 83.25
Out | 86.29 | 94.88 | 88.40 | 83.46
Infill | In | 84.88 | 95.01 | 88.86 | 83.14
Out | 87.19 | 95.22 | 89.05 | 84.36
TABLE IV:  The performances of different methods evaluated using the metrics:
BLEU, BERTS, ROUGE1, and ROUGE2, where BERTS, ROU1, and ROU2 refer to
BERTScore, ROUGE-1, and ROUGE-2, respectively.

Table 4 summarizes the evaluation results on our established CIP dataset. The
proposed five supervised CIP methods are significantly better than two
unsupervised methods (Re-translation and BERT) in perspectives of the four
metrics. The results reveal that the dataset is a high-quality corpus, which
can help to benefit CIP task.

Table 4 shows that the performance of LSTM-based baseline is worsen than the
other four baselines on in-domain and out-of-domain test datasets. In general,
the three mT5-based CIP methods (mT5, Infill, and Knowledge) outperform the
other two methods (LSTM and Transformer), which suggests that CIP methods
fine-tuned on mT5 can significantly improve the CIP performance. It is
observed from Table 4 that knowledge-based baseline can yield the best results
on in-domain test set among three mT5-based baselines, because the
incorporated knowledge is extracted from the training dataset that allows
neural networks to learn how to make use of the attached knowledge. Infill
baseline outperforms mT5 and Knowledge on the out-of-domain test set, despite
using less input information than its counterpart Knowledge-based mT5 method.
It verifies that Infill is quite effective.

###  5.3 Human Evaluation

Method | Domain | Simp. | Meaning | Fluency | Avg
---|---|---|---|---|---
Reference | In | 3.94 | 4.27 | 4.15 | 4.12
Out | 3.70 | 3.70 | 3.73 | 3.71
Re-translation | In | 3.03 | 2.18 | 2.69 | 2.63
Out | 2.34 | 1.68 | 2.04 | 2.02
BERT | In | 3.13 | 2.28 | 2.74 | 2.72
Out | 2.58 | 1.95 | 2.39 | 2.31
LSTM | In | 3.82 | 3.94 | 3.90 | 3.89
Out | 3.62 | 3.54 | 3.57 | 3.58
Transformer | In | 3.86 | 3.82 | 3.83 | 3.84
Out | 3.83 | 3.74 | 3.76 | 3.78
mT5 | In | 3.77 | 3.85 | 3.83 | 3.81
Out | 3.58 | 3.51 | 3.58 | 3.56
Knowledge | In | 3.72 | 3.80 | 3.80 | 3.77
Out | 3.77 | 3.71 | 3.76 | 3.75
Infill | In | 3.83 | 4.06 | 3.98 | 3.96
Out | 3.71 | 3.79 | 3.76 | 3.76
TABLE V:  The Results of human evaluation. ”Simp.” denotes ”simplicity”, ”Avg” denotes ”average”.  Sentence | 如果我们无法找到内心的平和，去别处寻找无疑缘木求鱼。
---|---
Reference | 如果我们不能在自己身上找到和平，那么到别处去寻找是毫无意义的。
English | if we are incapable of finding peach in ourselves, it is pointless to search elsewhere.
LSTM, Transformer, mT5, Infill | 如果我们无法找到内心的平和,去别处寻找无疑是徒劳的。
Knowledge | 如果我们无法找到内心的平和,去别处寻找无疑是不能成功的。
Sentence | 老师惩罚了考试作弊的学生以儆效尤.
Reference | 老师惩罚了考试作弊的学生起威慑作用.
English | the teacher made an example of the boy who copied from another student during a test.
LSTM, Transformer, mT5, Infill | 老师惩罚了考试作弊的学生以威慑其他人.
Knowledge | 老师惩罚了考试作弊的学生起威慑作用.
Sentence | 这次入学考试他虽名落孙山，但他并不气馁.
Reference | 这次入学考试他虽没有通过，但他并不气馁.
English | the teacher made an example of the boy who copied from another student during a test.
LSTM | 这次入学考试他虽名榜,但他并不气馁.
Transformer, mT5, Knowledge, Infill | 这次入学考试他虽落榜，但他并不气馁.
Sentence | 尽管人们对这类透支讳莫如深，但它却是现实存在.
Reference | 尽管人们对这类透支从不提及，但它却是现实存在.
English | the teacher made an example of the boy who copied from another student during a test.
Transformer | 尽管人们对这类透支提及,但它却是现实存在.
LSTM, mT5, Knowledge, Infill | 尽管人们对这类透支保持沉默,但它却是现实存在.
Sentence | 我们的错误在无情地蔓延，且缺乏公正，任何国家均概莫能外。
Reference | 我们的错误在无情地蔓延，且缺乏公正，任何国家均不能例外。
English | the contagion of our mistakes shows no mercy and makes no exceptions on the basis of fair play.
LSTM | 我们的错误在无情地蔓延,且缺乏公正,任何国家均都不例外。
Transformer, Knowledge | 我们的错误在无情地蔓延,且缺乏公正,任何国家都不例外。
mT5 | 我们的错误在无情地蔓延,且缺乏公正,任何国家都不能幸免。
Infill | 我们的错误在无情地蔓延,且缺乏公正,任何国家均不例外。
TABLE VI: Output examples from different methods. The idioms of sentences are
marked in red.

For further evaluating the CIP methods, we adopt human evaluation to analyze
the deployed CIP methods. We manually select 32 infrequent and easily misused
idioms. We choose 30 sentences from in-domain test set and 32 sentences from
out-of-domain test set. We inquire four native speakers to rate each generated
sentence using three features: simplicity, meaning, and fluency. The five-
point _Likert scale_ is adopted to rate these features, and the average scores
of the features are calculated correspondingly. (1) Simplicity is responsible
for evaluating whether re-paraphrased idioms of generated sentences are easily
understandable, which means idioms in original sentences should be rewritten
with simpler and more common words. (2) Meaning assesses if generated
sentences preserve the meaning of original sentences. (3) Fluency is used to
judge if a generated sentence is fluent, and does not contain grammatical
errors.

The results of human evaluation are shown in Table 5. We calculate the scores
of annotated sentences t, denoted as Reference. We first analyze the results
of these CIP methods on in-domain test set. The infill-based mT5 method
outperforms other methods, which means Infill is an effective method on CIP
task. The human evaluation performance of Knowledge-based CIP method is worse
than that of LSTM, Transformer, and mT5. Since the knowledge-based baseline
has the best results among all the baselines while adopting automatic
metrics（see Table 4), it is inconsistent with the results yielded using human
evaluation. We also notice that the three idioms are wrongly rephrased due to
the ”attached” information. If the ”attached” information is fitful to this
example, it may bring noise into the network.

Then, we analyze the results on an out-of-domain test set. The scores of the
four CIP methods (Transformer, Infill, and Knowledge) are slightly larger than
the scores of Reference. Although it is abnormal, the sentences generated by
the CIP methods are certainly high-quality. The out-of-domain test set is
individually collected by native Chinese crowd-workers without human and
machine collaboration. The crowd-workers often take limited writing strategies
to speed up the establishment of a dataset, which is harmful to the diversity
of the dataset [29, 11]. The quality of out-of-domain test set can be further
improved.

###  5.4 Proportion of Idiom Paraphrasing

Method | In | Out
---|---|---
Re-translation | 99.72% | 99.78%
BERT | 81.48% | 74.74%
LSTM | 92.64% | 85.79%
Transformer | 94.22% | 88.32%
mT5 | 89.34% | 73.45%
Knowledge | 91.82% | 85.76%
Infill | 91.04% | 84.08%
TABLE VII:  The proportion between the times of idiom paraphrasing and the
number of all idioms.

CIP aims to rephrase an input idiom-included sentence to a meaning-preserved
and non-idiom-included sentence. In this subsection, we count the number of
idiom-included sentences that are rephrased to non-idiom-included sentences.
The results are shown in Table 7. The result shows that Re-translation
achieves the best results, which can rephrase almost all idioms to non-idiom-
included representations. That means, our idea on CIP dataset construction
using machine learning method is feasible. Theoretically, if the trained
English-Chinese machine translation method (stage 2 in the pipeline) can
output high-quality results, we do not need to ask annotators to optionally
revise Chinese translations. We observe that the proportions of these CIP
methods (LSTM, Transformer, mT5, Knowledge and Infill) are nearly 90%, which
means they have great potential on dealing with idiom paraphrasing. Moreover,
a tiny part of idioms cannot be rephrased, because some idioms are simple,
thereby are retained in the training set.

###  5.5 Case Study

We first choose five complicated Chinese idioms that are ”缘木求鱼”(climb trees to
catch fish),”以儆效尤”(warn others against following a bad example),”名落孙山”(fail in
official examinations),”讳莫如深”(carefully conceal mentioning), and ”概莫能外”(admit
of no exception whatsoever). Then, we select a sample from our CIP test set
for each idiom, and analyze the output sentences generated by the five CIP
baselines.

Table 6 shows the sentences rephrased using the five CIP methods. In general,
these methods trained on the established CIP dataset are capable to generate
meaning-preserved and non-idiom-included sentences. In these examples, all the
generated sentences are simpler than the original sentences, even if these
idioms are challenging to native speakers.

##  6 Conclusion

In this paper, we propose a novel Chinese idiom paraphrasing (CIP) task, which
aims to rephrase idiom-included sentences to non-idiom-included ones. The CIP
task can be treated as a special paraphrase generation task and is feasible to
be dealt with by adopting sequence-to-sequence(Seq2Seq) modeling. We construct
a large-scale training dataset for CIP by taking the collaborations between
humans and machines. Specifically, we first design a framework to construct a
pseudo-CIP dataset and then ask workers to revise and evaluate the dataset. In
this study, We deploy three Seq2Seq methods and propose two novel CIP methods
(Infill and Knowledge) for the CIP task. Experimental results reveal that all
the five methods trained on our dataset can yield good results. In our
subsequent research, the proposed two CIP methods will be reserved as strong
baselines, and the established dataset will also be used to accelerate the
study on this topic. In the future, we will keep exploring the abilities of
our CIP methods in solving the problems of Chinese idiom understanding and
Chinese idiom representation.

## Acknowledgement

This research is partially supported by the National Natural Science
Foundation of China under grants 62076217, 62120106008, and 61906060.

## References

  * [1] C. Zheng, M. Huang, A. Sun, ChID: A large-scale Chinese IDiom dataset for cloze test, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Florence, Italy, 2019, pp. 778–787.  doi:10.18653/v1/P19-1075.
URL https://aclanthology.org/P19-1075

  * [2] W. Y. Ho, C. Kng, S. Wang, F. Bond, Identifying idioms in chinese translations., in: LREC, 2014, pp. 716–721.
  * [3] Y. Shao, R. Sennrich, B. Webber, F. Fancellu, Evaluating machine translation performance on chinese idioms with a blacklist method, arXiv preprint arXiv:1711.07646 (2017).
  * [4] Z. Jiang, B. Zhang, L. Huang, H. Ji, Chengyu cloze test, in: Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, 2018, pp. 154–158.
  * [5] M. Tan, J. Jiang, Learning and evaluating Chinese idiom embeddings, in: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), INCOMA Ltd., Held Online, 2021, pp. 1387–1396.
URL https://aclanthology.org/2021.ranlp-main.155

  * [6] S. Kadotani, T. Kajiwara, Y. Arase, M. Onizuka, Edit distance based curriculum learning for paraphrase generation, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop, 2021, pp. 229–234.
  * [7] X. Lu, J. Qiang, Y. Li, Y. Yuan, Y. Zhu, An unsupervised method for building sentence simplification corpora in multiple languages, in: Findings of the Association for Computational Linguistics: EMNLP 2021, Association for Computational Linguistics, Punta Cana, Dominican Republic, 2021, pp. 227–237.  doi:10.18653/v1/2021.findings-emnlp.22.
URL https://aclanthology.org/2021.findings-emnlp.22

  * [8] K. R. McKeown, Paraphrasing using given and new information in a question-answer system, in: 17th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, La Jolla, California, USA, 1979, pp. 67–72.  doi:10.3115/982163.982182.
URL https://aclanthology.org/P79-1016

  * [9] M. Meteer, V. Shaked, Strategies for effective paraphrasing, in: Coling Budapest 1988 Volume 2: International Conference on Computational Linguistics, 1988, pp. 431–436.
  * [10] Y. Meng, X. Ao, Q. He, X. Sun, Q. Han, F. Wu, C. Fan, J. Li, ConRPG: Paraphrase generation using contexts as regularizer, in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 2551–2562.
  * [11] A. Liu, S. Swayamdipta, N. A. Smith, Y. Choi, Wanli: Worker and ai collaboration for natural language inference dataset creation, arXiv preprint arXiv:2201.05955 (2022).
  * [12] L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, C. Raffel, mT5: A massively multilingual pre-trained text-to-text transformer, in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Online, 2021, pp. 483–498.  doi:10.18653/v1/2021.naacl-main.41.
URL https://aclanthology.org/2021.naacl-main.41

  * [13] J. Zhou, S. Bhat, Paraphrase generation: A survey of the state of the art, in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 5075–5086.
  * [14] Y. Liu, B. Pang, B. Liu, Neural-based Chinese idiom recommendation for enhancing elegance in essay writing, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 5522–5526.
  * [15] M. Pershina, Y. He, R. Grishman, Idiom paraphrases: Seventh heaven vs cloud nine, in: Proceedings of the first workshop on linking computational models of lexical, sentential and discourse-level semantics, 2015, pp. 76–82.
  * [16] J. Qiang, X. Lu, Y. Li, Y.-H. Yuan, X. Wu, Chinese lexical simplification, IEEE/ACM Transactions on Audio, Speech, and Language Processing (2021) 1819–1828.
  * [17] O. Bojar, C. Federmann, M. Fishel, Y. Graham, B. Haddow, M. Huck, P. Koehn, C. Monz, Findings of the 2018 conference on machine translation (wmt18), in: Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers, Association for Computational Linguistics, Belgium, Brussels, 2018, pp. 272–307.
URL http://www.aclweb.org/anthology/W18-6401

  * [18] S. Desai, G. Durrett, Calibration of pre-trained transformers, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online, 2020, pp. 295–302.  doi:10.18653/v1/2020.emnlp-main.21.
URL https://aclanthology.org/2020.emnlp-main.21

  * [19] D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, ICLR (2014).
  * [20] S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural computation 9 (8) (1997) 1735–1780.
  * [21] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in neural information processing systems, 2017, pp. 5998–6008.
  * [22] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, Journal of Machine Learning Research 21 (2020) 1–67.
  * [23] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018).
  * [24] M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, M. Auli, fairseq: A fast, extensible toolkit for sequence modeling, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 48–53.
  * [25] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, A. M. Rush, Transformers: State-of-the-art natural language processing, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Association for Computational Linguistics, Online, 2020, pp. 38–45.
URL https://www.aclweb.org/anthology/2020.emnlp-demos.6

  * [26] K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, Bleu: a method for automatic evaluation of machine translation, in: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318.
  * [27] Y. Zhang, C. Yang, Z. Zhou, Z. Liu, Enhancing transformer with sememe knowledge, in: Proceedings of the 5th Workshop on Representation Learning for NLP, Association for Computational Linguistics, Online, 2020, pp. 177–184.
  * [28] C.-Y. Lin, Rouge: A package for automatic evaluation of summaries, in: Text summarization branches out, 2004, pp. 74–81.
  * [29] M. Geva, Y. Goldberg, J. Berant, Are we modeling the task or the annotator? an investigation of annotator bias in natural language understanding datasets, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 1161–1166.

|  Jipeng Qiang is currently an associate professor in the School of
Information Engineering, at Yangzhou University. He received his Ph.D. degree
in computer science and technology from Hefei University of Technology in
2016. He was a Ph.D. visiting student in the Artificial Intelligence Lab at
the University of Massachusetts Boston from 2014 to 2016. He has received two
grants from the National Natural Science Foundation of China. He has published
more than 20 papers as the first author, including AAAI, EMNLP, TKDE, TASLP,
TKDD, etc. His research interests mainly include natural language processing
and data mining.
---|---
|  Yang Li is currently working toward the M.S. degree with the School of
Information Engineering, Yangzhou University, Jiangsu, China. His research
interest is natural language processing.
---|---
|  Chaowei Zhang received the Master & Ph.D. degrees in computer science and
software engineering from the Auburn University, USA, in 2019 and 2021,
respectively. He is currently with the School of Information Engineering at
Yangzhou University. His research interests include natural language
processing, data mining, storage systems, and parallel computing.
---|---
|  Yun Li is currently a professor in the School of Information Engineering,
Yangzhou University, China. He received the M.S. degree in computer science
and technology from Hefei University of Technology, China, in 1991, and the
Ph.D. degree in control theory and control engineering from Shanghai
University, China, in 2005. He has published more than 100 scientific papers.
His research interests include data mining and cloud computing.
---|---
|  Yunhao Yuan is currently an associate professor in the School of
Information Engineering, Yangzhou University, China. He received the M. Eng.
degree in computer science and technology from Yangzhou University, China, in
2009, and the Ph.D. degree in pattern recognition and intelligence system from
Nanjing University of Science and Technology, China, in 2013. His research
interests include pattern recognition, data mining, and image processing.
---|---
|  Yi Zhu is currently an assistant professor in the School of Information
Engineering, Yangzhou University, China. He received the BS degree from Anhui
University, the MS degree from University of Science and Technology of China,
and the Ph.D. degree from Hefei University of Technology. His research
interests are in data mining and knowledge engineering. His research interests
include data mining, knowledge engineering, and recommendation systems..
---|---
|  Xindong Wu is a Yangtze River Scholar in the School of Computer Science and
Information Engineering at the Hefei University of Technology, China, and a
fellow of IEEE and AAAS. He received his B.S. and M.S. degrees in computer
science from the Hefei University of Technology, China, and his Ph.D. degree
in artificial intelligence from the University of Edinburgh, Britain. His
research interests include data mining, big data analytics, knowledge-based
systems, and Web information exploration. He is currently the steering
committee chair of the IEEE International Conference on Data Mining (ICDM),
the editor-in-chief of Knowledge and Information Systems (KAIS, by Springer),
and a series editor-in-chief of the Springer Book Series on Advanced
Information and Knowledge Processing (AI&KP). He was the editor-in-chief of
the IEEE Transactions on Knowledge and Data Engineering (TKDE, by the IEEE
Computer Society) between 2005 and 2008. He served as program committee
chair/co-chair for the 2003 IEEE International Conference on Data Mining, the
13th ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining, and the 19th ACM Conference on Information and Knowledge Management.
---|---