# Atomic Vibration Sensor

Wenxi Lai<EMAIL_ADDRESS>School of Applied Science, Beijing Information
Science and Technology University, Beijing 100192, China

###### Abstract

Previously in vibration sensors, optical glass plates, optical fibres, carbon
nanotubes, semiconductor materials, piezoelectric materials and molecules are
proved to be effective transducers for sensing vibrations. In this work, for
the first time, we will propose a model of vibration sensor using single atoms
in low temperature. In this apparatus, information of mechanical vibration
could be transferred into shaking of optical lattice through one of a cavity
mirror. Shaking lattice consequently induces Mott insulator due to quantum
interference. It is found that information of vibration is encoded in the
atomic current and it could be extracted by Fourier transformations. The
present atomic vibration sensor has wide detection range of frequency with
high precision. Our present model of sensor based on atomic system opens a new
area of studying vibration sensors.

Wide researches are carried out on vibration sensors due to their
indispensable applications in detection of civil infrastructure, seismic
activity, biochemical agents, structural health, and optimum performance of
machines. Coupling between electrochemical reaction and vibrational signal
gives rise to low frequency sensor with frequency range around $0.1$ Hz $\sim$
$10$ Hz Zhou . Optical glass plates can transform state of a mechanical
vibration into transmission strength of laser fields, they have detectable
frequency range from tens of Hz to several kHz Kamata ; Micheletto . Optical
fibres, carbon nanotubes, semiconductor materials can be applied to design
micro cantilever vibration sensors which are sensitive to frequency from
several Hz to hundreds of kHz Lu ; EHFeng ; Strus , or even a few tens of MHz
Masmanidis ; Liang . Sensors based on semiconductor piezoelectric materials
can detect vibration with frequency from tens of MHz to GHz Lu2015 ; Lu2019 .
Electron-phonon interaction assisted current in molecular junctions and
spectrum of photo-electron interactions in molecules can be used to fabricate
molecular vibration sensors with measurement range at the order from MHz to
THz Hihath ; Ritter ; Tian ; Chen ; Bian ; Liu .

Figure 1: Schematic structure of the atomic vibration sensor. It is composed
of atomic vibration transistor and Fourier transformation apparatus. The
atomic vibration transistor is an open system of cold atoms in which a double-
well optical potential is coupled to two atomic reservoirs. Mechanical wave of
the external environment is allowed to drive the optical potential through the
cavity mirror.

In this paper, we propose a model of atomic vibration sensor using quantum
transport of cold atoms in open optical lattices. It is inspired by phase
changes in shaken optical lattices. Cold atoms in shaken optical lattices feel
inertial forces which induces artificial gauge fields Struck ; Hauke ; Price .
The artificial gauge fields can induce Mott insulator Eckardt ; Zenesini ;
Liberto , fractional quantum Hall effect Sorensen ; Miao  and topological non-
trivial states Kitagawa ; Rudner ; Zheng ; Wintersperger ; Cheng ; JYZhang .
The Mott insulator is originated from coherent destruction of tunneling
Lignier ; Kierig , depending on shaking induced Peierls phases Eckardt2017 ;
Sun . Recently, sensing with quantum control of atoms and molecules is
believed to be important technology for development of fundamental physics Ye
. In the following, we will demonstrate a sensor of mechanical vibration using
the phase change in a coherently controlled atomic system.

The mirror transducer used in the earlier glass based vibration sensor Kamata
; Micheletto  is an important reference case for designing our present sensor.
Actually, oscillating mirrors have been applied to induce shaking of optical
potentials in recent experiments Ivanov ; Zenesini . Based on these
experiments, Fig. 1 shows how to transfer information of an external
mechanical wave into a cold atom system through a cavity mirror which can
oscillate about its equilibrium position. We will prove that sensing of the
mechanical wave could be achieved by taking Fourier transformations to atomic
currents which is affected by the external vibration induced lattice shaking.
Using a general equation of motion, the Mott insulator in atomic vibration
transistor is described at arbitrary shaking frequency. The transistor is
sensitive to amplitude, frequency and propagation direction of mechanical
waves with high accuracy.

Center part of the atomic vibration transistor is a double-well optical
potential which is coupled to atomic source and drain as articulated in Fig.
1. The two wells are symmetrically arranged at positions
$r_{1}=-\frac{d}{2}e_{x}$ and $r_{2}=\frac{d}{2}e_{x}$, respectively. In this
model, for simplicity, we suppose atom-atom repulsion energy is too large to
allow two or more atoms occupy a single lattice site. Then, the shaken open
system is described by the following Hamiltonian,

$\displaystyle H(t)$ $\displaystyle=$
$\displaystyle\sum_{l=1}^{2}(\varepsilon_{l}+\Omega_{l}(t))n_{l}-J(a_{1}^{{\dagger}}a_{2}+a_{2}^{{\dagger}}a_{1})+\sum_{\alpha,k}\mu_{\alpha
k}n_{\alpha
k}-g\sum_{k}(a_{1}^{{\dagger}}a_{Sk}+a_{2}^{{\dagger}}a_{Dk}+H.c.),$ (1)

where $a_{l}$ and $n_{l}=a_{l}^{{\dagger}}a_{l}$ are annihilation and number
operators of atoms at lattice site $l$ ($l=1$, $2$), respectively.
$\varepsilon_{l}$ denotes bare energy level in the well $l$. In the
oscillating reference, a cold atom in the well $l$ would feel a driving
potential $\Omega_{l}(t)=-r_{l}\cdot F(t)$ Eckardt2017 , in which $F(t)$ is
the inertial force written as $F(t)=F_{\Omega}\cos(\Omega
t)(\cos(\theta)e_{x}+\sin(\theta)e_{y})$. Shaking frequency $\Omega$ of the
potential is identical to frequency of the external vibration. Here,
$F_{\Omega}$ is amplitude of the oscillating force and $\theta$ represents the
angle between the direction of mechanical wave propagation and the one
dimensional optical lattice. The second term on the right side of Hamiltonian
indicates inter tunneling between the two wells with a tunneling strength $J$.
In the third term, the source ($\alpha=S$) and drain ($\alpha=D$) are
described by free quantum gases of single atoms with annihilation operator
$a_{\alpha k}$ and number operator $n_{\alpha k}=a_{\alpha
k}^{{\dagger}}a_{\alpha k}$. Here, $\mu_{\alpha k}$ represents energy of a
single atom with momentum $k$. The last term in the Hamiltonian represents
coupling energy between the shaken system and atomic reservoirs (the source
and drain) with the coupling amplitude $g$.

According to standard derivation of quantum master equation under Born-
Markovian approximation in quantum optics Scully ; Lai , equation of motion of
the shaking open system could be deduced based on Hamiltonian (1),

$\displaystyle\frac{\partial}{\partial t}\rho$ $\displaystyle=$
$\displaystyle-i[\varepsilon_{1}n_{1}+\varepsilon_{2}n_{2}-(Je^{i\phi(t)}a_{1}^{{\dagger}}a_{2}+H.c.),\rho]+\mathcal{L}_{S}\rho+\mathcal{L}_{D}\rho,$
(2)

where $\rho$ is the system density matrix and $\phi(t)$ indicates the time
dependent Peierls phase $\phi(t)=\frac{K}{\hbar\omega}\cos(\theta)\sin(\Omega
t)$. The Peierls phase is proportional to the shaking strength $K=F_{\Omega}d$
in which $d$ denotes distance between the two wells as shown in Fig. 1. The
Lindblad super operators depict atom transport between the system and atomic
electrodes, they have following typical expressions,

$\displaystyle\mathcal{L}_{S}\rho$ $\displaystyle=$
$\displaystyle\frac{\Gamma}{2}(1-f_{S}(\varepsilon_{1}))(2a_{1}\rho
a_{1}^{{\dagger}}-\\{a_{1}^{{\dagger}}a_{1},\rho\\})+\frac{\Gamma}{2}f_{S}(\varepsilon_{1})(2a_{1}^{{\dagger}}\rho
a_{1}-\\{a_{1}a_{1}^{{\dagger}},\rho\\})$ (3)

and

$\displaystyle\mathcal{L}_{D}\rho$ $\displaystyle=$
$\displaystyle\frac{\Gamma}{2}(1-f_{D}(\varepsilon_{2}))(2a_{2}\rho
a_{2}^{{\dagger}}-\\{a_{2}^{{\dagger}}a_{2},\rho\\})+\frac{\Gamma}{2}f_{D}(\varepsilon_{2})(2a_{2}^{{\dagger}}\rho
a_{2}-\\{a_{2}a_{2}^{{\dagger}},\rho\\}),$ (4)

where the brace $\\{,\\}$ represents anti commutation. The two coupling rates
are set to be equal, $\Gamma=2\pi D(\varepsilon_{1})|g|^{2}=2\pi
D(\varepsilon_{2})|g|^{2}$, in which the function $D$ denotes density of
energy states in the source and the drain. Chemical potentials of the source
$\mu_{S}$ and the drain $\mu_{D}$ are appeared in the corresponding Fermi-
Dirac distribution functions
$f_{S}(\varepsilon_{1})=\frac{1}{e^{(\varepsilon_{1}-\mu_{S})/k_{B}}+1}$ and
$f_{D}(\varepsilon_{2})=\frac{1}{e^{(\varepsilon_{2}-\mu_{D})/k_{B}T}+1}$.
Here, $k_{B}$ is the Boltzmann constant and $T$ is temperature in the two
atomic reservoirs. Eq. (2), as a differential equation of the density matrix
$\rho$ with time variable coefficient, could be written in the form
$\frac{\partial}{\partial t}\rho=M(t)\rho$ and it has a solution
$\rho=e^{\int_{0}^{t}M(t^{\prime})dt^{\prime}}\rho(0)$ for a given initial
state $\rho(0)$. Hilbert space of the double-well system is generated from the
basic states $|00\rangle$, $|01\rangle$, $|10\rangle$ and $|11\rangle$. The
given parameters throughout this article are $J/2\pi=200$ Hz Mancini ; Livi ,
$\Gamma=0.1J$, $k_{B}T=0.1J$ ($T=0.96$ nK),
$\varepsilon_{1}=\varepsilon_{2}=5J$, $\mu_{S}=10J$ and $\mu_{D}=0$.
Additionally, $\theta=0$ except the last figure.

Figure 2: (Color on line) (a) Average current as a function of shaking
amplitude $K$. (b) Average current as a function of shaking frequency
$\omega$. (c) Changes of diagonal density matrix elements corresponding to the
average current in (a).(d) Changes of diagonal density matrix elements
corresponding to the average current in (b). (e)-(h) Current behavior versus
real time under different values of shaking amplitude chosen from (a). (i)-(l)
The current spectrums corresponding to the current fluctuations in (e)-(h),
respectively.

Time dependent current is direct outcome of the atomic vibration transistor.
The current could be derived from the continuity equation
$I_{S}-I_{D}=\frac{\partial}{\partial
t}Tr_{S}[\rho(t)\sum_{l=1}^{2}a_{l}^{{\dagger}}a_{l}]$ Davies , where $I_{S}$
and $I_{D}$ are currents detected in the source and drain, respectively.
$Tr_{S}$ indicates trace over the system states. Using this formula, we
achieve the right side current $I(t)=-\Gamma
f_{D}(\varepsilon_{2})(\rho_{00,00}+\rho_{10,10})+\Gamma(1-f_{D}(\varepsilon_{2}))(\rho_{01,01}+\rho_{11,11})$,
where $I(t)=I_{D}(t)$ for simplicity. This current expression could be
equivalently derived from many-body Schrödinger equation approach Gurvitz .
For the atomic vibration sensor, Fourier transformation of the current $I(t)$

$\displaystyle I(\omega)$ $\displaystyle=$
$\displaystyle\int_{0}^{\tau}I(t)e^{-i\omega t}dt$ (5)

is significant. Here, time $\tau$ takes infinitely large value,
$\tau\rightarrow\infty$, as the current $I(t)$ is generally not a periodic
function. It is taken be $\tau=1$ s in all the present numerical treatments.
Modulus of the Fourier transformation
$|I(\omega)|=\sqrt{Re[I(\omega)]^{2}+Im[I(\omega)]^{2}}$ is taken in our
results. Another important quantity for the sensor is the timely averaged
current,

$\displaystyle\langle I\rangle=\frac{1}{\tau}\int_{0}^{\tau}I(t)dt,$ (6)

which can be seen as zero order term of the Fourier transformation with
respect to $\omega=0$ in Eq.(5).

Firstly, average currents calculated using Eq.(6) are plotted in Figs. 2 (a)
and (b). They show there are particular points where current $\langle
I\rangle$ is close to be zero. This effect appears when the ratio of vibration
strength and frequency $K/\Omega$ takes certain values, such as
$K_{1}/\Omega_{1}=2.405$, $K_{2}/\Omega_{2}=5.520$, $K_{3}/\Omega_{3}=8.654$,
$K_{4}/\Omega_{4}=11.791$, $K_{5}/\Omega_{5}=14.931$,
$K_{6}/\Omega_{6}=18.071$, $K_{7}/\Omega_{7}=21.213$,
$K_{7}/\Omega_{7}=24.352$ and so on. One can check these ratios that they
approximately form an arithmetic sequence with a common difference of $\pi$.
They have a general formula as,

$\displaystyle\frac{K_{n}}{\Omega_{n}}\approx\frac{K_{2}}{\Omega_{2}}+(n-2)\pi,$
(7)

where $\frac{K_{2}}{\Omega_{2}}=5.520$ and $n$ is any positive integer number.
The first point, $K_{1}/\Omega_{1}\approx 2.4$, for obtaining Mott insulator
phase has been reported in previous studies on driven optical lattices
Zenesini ; Eckardt ; Lignier . Therefore, the empirical formula (7) is a
general equation of the factor $K_{n}/\Omega_{n}$ for occurrence of Mott
insulator. In our atomic vibration sensor, it can supply information of
vibration frequency and strength. In Eq.(7), there are three variables,
$K_{n}$, $\Omega_{n}$ and $n$. One can set $K_{n}$ and $n$ by choosing system
parameters to design a vibration sensor for verification of a particular
frequency $\Omega_{n}$. Alternatively, one can test a particular $K_{n}$ when
$n$ and $\Omega_{n}$ are known.

To understand the Mott insulator mentioned above, diagonal elements of density
matrix has been shown in Figs. 2 (c) and (d) under the identical parameters as
in Figs. 2 (a) and (b). Both of these two figures illustrate the current
destructive occurs at the point $K_{n}/\Omega_{n}$ when atom population is
localization in one of the two optical wells. It is originated from the
coherent destruction of tunneling in shaking optical lattices due to
destructive interference in coherent tunnelings Kierig ; Lignier . As a
result, atom transition from one site to the other site would be blocked. It
is similar to coherent population trapping in quantum optics where atoms are
coherently trapped in one of its internal electronic states Whitley ; Arimondo
. The later effect is also explained by quantum interference Scully .

Figure 3: (Color on line) (a)-(e) Current as function of real time at
different shaking frequency $\Omega$. (f)-(j) Current spectrums corresponding
to the current fluctuations in (a)-(e), respectively.

In fact, the vibration sensor can directly detect vibrational frequency. To
show this process we choose four typical points in Fig. 2 (a), corresponding
to four different shaking strengths $K/2\pi=350$ Hz, $400$ Hz, $420$ Hz and
$433$ Hz, but the same shaking frequency $\Omega/2\pi=50$ Hz. Time evolution
of atomic currents $I(t)$ are shown in Figs. 2 (e)-(h) with respect to these
four shaking amplitudes. Consequently, frequency spectrum of these time
dependent currents are obtained through the Fourier transformation given in
Eq.(5). Spectrums of currents in Figs. 2 (i)-(l) reveal frequency of shaking
potential is encoded in atomic currents of the transistor. In other words,
frequency of the vibration could be directly extracted from Fourier
transformations of atomic currents. For example, at the point $K/2\pi=350$ Hz
in Fig. 2 (a), corresponding spectrum of current plotted in Fig. 2 (i)
supplies information of the frequency as $\omega=2\Omega$ and $4\Omega$.
Fortunately, in the area represented by the points $K/2\pi=400$ Hz, $420$ Hz
and $433$ Hz, the exact frequency $\omega=\Omega$ could be read as illustrated
in Figs. 2 (j)-(l).

Current destruction effect is significant for direct read of the vibrational
frequency. The three points $K/2\pi=400$ Hz, $420$ Hz and $433$ Hz in Fig. 2
(a) are located in the area of current destruction which is originated from
the time dependent Peierls phase $\phi(t)$. The time dependent Peierls phase
$\phi(t)$ leads to polarization of density matrix, as shown in Figs. 2 (c) and
(d), through the evolution process
$\rho(t)=exp[i\int_{0}^{\tau}M(t)dt]\rho(0)$. Therefore, the current
destructive is originated from the Peierls phase. Furthermore, the Peierls
phase $\phi(t)$ directly comes from the mechanical vibration induced inertial
potential $\phi(t)=\int_{0}^{t}\Omega(t)dt^{\prime}$. It indicates information
of the mechanical vibration is transferred to atomic current through the time
dependent Peierls phase.

Figure 4: (Color on line) Frequency spectrum of current with respect to
shaking frequency $\Omega$ under the given shaking amplitude $K=45J$.

The present sensor has large detection range of frequency. Fig. 3 shows
frequency monitoring from the value $0.1$ Hz to $1$ kHz. In fact, frequency
lower than $0.1$ Hz and higher than $1$ kHz could be detected as soon as the
vibration frequency $\Omega$ and vibration strength $K$ satisfy Eq.(7). With
contemporary cold atom experiments, the sensor may encounter some challenges
facing ultra low or ultra high vibrational frequency. For much low frequency,
for example, $\Omega=0.1$ Hz as shown in Figs. 3 (a) and (f), one needs
several tens of seconds to monitor atomic current. For much higher frequency,
for example, $\Omega=1000$ Hz as shown in Figs. 3 (e) and (j), one should
require large shaking amplitude around $K=2.405\Omega$ to read clear
information of the frequency.

Frequency spectrum of current with respect to continuous change of vibrational
frequency $\Omega$ with a definite shaking strength $K=45J$ is shown in Fig.
4. Peaks of the spectrums are appeared at the positions $\omega=n\Omega$ ($n$
is positive integer number). It is clear that frequency $\Omega$ of the
mechanical vibration could be found directly from Fourier transformation
$I(\omega)$ of the current. In this figure, one observes many (black) gaps
where the expected vibrational frequency could not be observed. Gaps in the
spectrum for a given shaking strength $K$ could be supplemented by tuning $K$
through adjusting the lattice constant $d$, since $K=F_{\Omega}d$.

There is a simple method to judge direction of a mechanical wave. This method
takes the advantage of Peierls phase $\phi(t)$ which is proportional to
$K\cos(\theta)$. Here, $\theta$ is angle between direction of the atomic
vibration sensor and the mechanical wave to be measured as illustrated in Fig.
1. Therefore, changing direction of the vibrational sensor is equivalently
changing the shaking strength. For a vibration with a given frequency $\Omega$
originated from a monochromatic mechanical wave, there are even number of
current destruction (Mott insulator) would occur by tuning $\theta$ from
$-\pi/2$ to $\pi/2$. This results have been shown in Fig. 5 for different
frequency $\Omega$. For example, if the frequency of vibration is
$\Omega=0.35$ kHz, current destruction should be observed four times. The even
number positions of the current destruction are symmetrically arranged on two
side of the line in the direction $\theta=0$. Then, one can point out
direction of the mechanical wave propagation is parallel to the line on angle
$\theta=0$. Generally, the direction could point to any angle, depending on
the choice of coordinate system.

Figure 5: Average current as a function of shaking frequency $\Omega$ and
mechanical wave direction $\theta$ with respect to the given shaking amplitude
$K=12.5J$.

In experiments, the oscillating mirror could works as a transducer which
transfer an external mechanical wave into shaking of optical potential in a
cavity Ivanov ; Zenesini . It can be comprehended from periodically modulated
distance between two mirrors of a cavity. The modulated distance between the
two mirrors would change frequency of laser field which generate standing wave
in the cavity. Furthermore, frequency change of laser beams lead to shaking of
optical lattices Struck ; Reitter . For progresses in open optical systems,
some simply structured open optical potentials with non-equilibrium atomic
gases have been experimentally implemented recent years Caliga17 ; Krinner ;
Amico . Corresponding theoretical works on atomic transport in open optical
lattices are studied for future applications of atomtronic transistors and
devices Seaman ; Caliga ; Wilsmann .

As conclusions, a vibration sensor based on quantum interference in cold atom
transport has been proved here. Peierls phase induced quantum interference is
the main reason of the coherent destruction of tunneling and Mott insulator.
Vibration frequency and Vibration strength satisfy a general formula which
leads to Mott insulator in the atomic vibration transistor. In the parameter
space, current around the point of Mott insulator is destructed and
fluctuates. Fourier transformations of the fluctuating currents can be used to
read accurate information of vibrations. Quantum evolution of the atomic
vibration transistor is described by time variable coefficient quantum master
equation. It is reliable and general equation with only Born-Markovian
approximation.

## References

  * (1) Q. Zhou, C. Wang, Y. Chen, S. Chen, and J. Lin, Sensors 16, 657 (2016).
  * (2) M. Kamata, M. Obara, R. R. Gattass, L. R. Cerami, and E. Mazur, Appl. Phys. Lett. 87, 051106 (2005).
  * (3) R. Micheletto and Y. Kawakami, Appl. Phys. Lett. 90, 244108 (2007).
  * (4) P. Lu, Y. Xu, F. Baset, X. Bao, and R. Bhardwaj, Appl. Phys. Lett. 103, 211113 (2013).
  * (5) E. H. Feng and R. E. Jones. Phys. Rev. B 83, 195412 (2011).
  * (6) M. C. Strus and A. Raman, Phys. Rev. B 80, 224105 (2009).
  * (7) S. C. Masmanidis, R. B. Karabalin, I. De Vlaminck, G. Borghs, M. R. Freeman, and M. L. Roukes, Science 317, 780 (2007).
  * (8) Y. Liang, H. Sun, L. Cheng, L. Jin, and B. Guan, Nat. Commun. 12, 4139 (2021).
  * (9) Y. Lu, H. Tang, S. Fung, Q. Wang, J. M. Tsai, M. Daneman, B. E. Boser, and D. A. Horsley, Appl. Phys. Lett. 106, 263503 (2015).
  * (10) Y. Lu, M. Zhang, H. Zhang, Y. Jiang, H. Zhang, and W. Pang, Phys. Rev. Applied 11, 044091 (2019).
  * (11) J. Hihath, C. R. Arroyo, G. Rubio-Bollinger, and N. Tao, Nano Letter, 8, 1673 (2008).
  * (12) C. Ritter, R. B. Muniz, S. S. Makler, and A. Latgé, Phys. Rev. B, 82, 113407 (2010).
  * (13) Y. Tian, P. Navarro, and M. Orrit, Phys. Rev. Lett., 113, 135505 (2014).
  * (14) W. Chen, P. Roelli, H. Hu, S. Verlekar, S. P. Amirtharaj, A. I. Barreda, T. J. Kippenberg, M. Kovylina, E. Verhagen, A. Martíhez, and C. Galland, Science 374, 1264 (2021).
  * (15) X. Bian, Z. Chen, J. K. Sowa, C. Evangeli, B. Limburg, J. L. Swett, J. Baugh, G. A. D. Briggs, H. L. Anderson, J. A. Mol, and J. O. Thomas, Phys. Rev. Lett. 129, 207702 (2022).
  * (16) Y. Liu, W. Ni, L. Yang, S. Huang, H. Liu, Y. Sun, R. Xia, Y. Yao, L. Yan, Y. Luo, Z. Xu, G. Xu, Q. Sun, X. Tang, and P. Shum, Appl. Phys. Lett. 123, 211105 (2023).
  * (17) J. Struck, C. Ölschläger, M. Weinberg, P. Hauke, J. Simonet, A. Eckardt, M. Lewenstein, K. Sengstock, and P. Windpassinger, Phys. Rev. Lett. 108, 225304 (2012).
  * (18) P. Hauke, O. Tieleman, A. Celi, C. Ölschläger, J. Simonet, J. Struck, M. Weinberg, P. Windpassinger, K. Sengstock, M. Lewenstein, and A. Eckardt, Phys. Rev. Lett. 109, 145301 (2012).
  * (19) H. M. Price, T. Ozawa, and N. Goldman, Phys. Rev. A 95, 023607 (2017).
  * (20) A. Eckardt, C. Weiss, and M. Holthaus, Phys. Rev. Lett. 95, 260404 (2005).
  * (21) A. Zenesini, H. Lignier, D. Ciampini, O. Morsch, and E. Arimondo, Phys. Rev. Lett. 102, 100403 (2009).
  * (22) M. Di Liberto, T. Comparin, T. Kock, M. Ölschläger, A. Hemmerich, and C. Morais Smith, Nat. Commun. 5, 5735 (2014).
  * (23) A. S. Sørensen, E. Demler, and M. D. Lukin, Phys. Rev. Lett. 94, 086803 (2005).
  * (24) S. Miao, Z. Zhang, Y. Zhao, Z. Zhao, H. Wang, and J. Hu, Phys. Rev. B 106, 054310 (2022).
  * (25) T. Kitagawa, E. Berg, M. Rudner, and E. Demler, Phys. Rev. B 82, 235114 (2010).
  * (26) M. S. Rudner, N. H. Lindner, E. Berg, and M. Levin, Phys. Rev. X 3, 031005 (2013).
  * (27) W. Zheng and H. Zhai, Phys. Rev. A 89, 061603(R) (2014).
  * (28) K. Wintersperger, C. Braun, F. Nur Ünal, A. Eckardt, M. D. Liberto, N. Goldman, I. Bloch, and M. Aidelsburger, Nat. Phys. 16, 1058 (2020).
  * (29) S. Cheng, H. Yin, Z. Lu, C. He, P. Wang, and G. Xianlong, Phys. Rev. A 101, 043620 (2020).
  * (30) J.-Y. Zhang, C.-R. Yi, L. Zhang, R.-H. Jiao, K.-Y. Shi, H. Yuan, W. Zhang, X.-J. Liu, S. Chen, and J.-W. Pan, Phys. Rev. Lett. 130, 043201 (2023).
  * (31) H. Lignier, C. Sias, D. Ciampini, Y. Singh, A. Zenesini, O. Morsch, and E. Arimondo, Phys. Rev. Lett. 99, 220403 (2007).
  * (32) E. Kierig, U. Schnorrberger, A. Schietinger, J. Tomkovic, and M. K. Oberthaler, Phys. Rev. Lett. 100, 190405 (2008).
  * (33) A. Eckardt, Rev. Mod. Phys. 89, 011004 (2017).
  * (34) G. Sun and A. Eckardt, Phys. Rev. Research 2, 013241 (2020).
  * (35) J. Ye and P. Zoller, Phys. Rev. Lett. 132, 190001 (2024).
  * (36) V. V. Ivanov, A. Alberti, M. Schioppo, G. Ferrari, M. Artoni, M. L. Chiofalo, and G. M. Tino, Phys. Rev. Lett. 100, 043602 (2008).
  * (37) M. O. Scully and M. S. Zubairy, Quantum Optics, Cambridge University Press (1997).
  * (38) W. Lai, Y. Cao and Z. Ma, J. Phys.: Condens. Matter, 24, 175301 (2012).
  * (39) M. Mancini, G. Pagano, G. Cappellini, L. Livi, M. Rider, J. Catani, C. Sias, P. Zoller, M. Inguscio, M. Dalmonte, L. Fallani, Science 349, 1510 (2015).
  * (40) L. F. Livi, G. Cappellini, M. Diem, L. Franchi, C. Clivati, M. Frittelli, F. Levi, D. Calonico, J. Catani, M. Inguscio, and L. Fallani, Phys. Rev. Lett. 117, 220401 (2016).
  * (41) J. H. Davies, S. Hershfield, P. Hyldgaard, and J. W. Wilkins, Phys. Rev. B, 47, 4603 (1993).
  * (42) S. A. Gurvitz and Ya. S. Prager, Phys. Rev. B, 53, 15932 (1996).
  * (43) R. M. Whitley and C. R. Stroud, Phys. Rev. A 14, 1498 (1976).
  * (44) E. Arimondo, Prog. Opt. 35, 257 (1996).
  * (45) M. Reitter, J. Näger, K. Wintersperger, C. Sträter, I. Bloch, A. Eckardt, and U. Schneider, Phys. Rev. Lett. 119, 200402 (2017).
  * (46) S. C. Caliga, C. J. E. Straatsma, and D. Z. Anderson, New J. Phys. 19, 013036 (2017).
  * (47) S. Krinner, T. Esslinger, and J.-P. Brantut, J. Phys.: Condens. Matter 29, 343003 (2017).
  * (48) L. Amico, D. Anderson, M. Boshier, J.-P. Brantut, L.-C. Kwek, A. Minguzzi, and W. von Klitzing, Rev. Mod. Phys. 94, 041001 (2022).
  * (49) B. T. Seaman, M. Krämer, D. Z. Anderson, and M. J. Holland, Entropy 75, 023615 (2007).
  * (50) S. C. Caliga, C. J. E. Straatsma, A. A. Zozulya, and D. Z. Anderson, New J. Phys. 18, 015012 (2016).
  * (51) K. W. Wilsmann, L. H. Ymai, A. P. Tonel, J. Links, and A. Foerster, Commun. Phy. 1, 91 (2018).


# Zero-Shot Transfer Learning for Event Extraction

Lifu Huang  Heng Ji
Rensselaer Polytechnic Institute
{huangl7<EMAIL_ADDRESS>Kyunghyun Cho
New York University
<EMAIL_ADDRESS>&Clare R. Voss
US Army Research Lab
<EMAIL_ADDRESS>
###### Abstract

Most previous event extraction studies have relied heavily on features derived
from annotated event mentions, thus cannot be applied to new event types
without annotation effort. In this work, we take a fresh look at event
extraction and model it as a grounding problem. We design a transferable
neural architecture, mapping event mentions and types jointly into a shared
semantic space using structural and compositional neural networks, where the
type of each event mention can be determined by the closest of all candidate
types . By leveraging (1) available manual annotations for a small set of
existing event types and (2) existing event ontologies, our framework applies
to new event types without requiring additional annotation. Experiments on
both existing event types (e.g., ACE, ERE) and new event types (e.g.,
FrameNet) demonstrate the effectiveness of our approach. Without any manual
annotations for 23 new event types, our zero-shot framework achieved
performance comparable to a state-of-the-art supervised model which is trained
from the annotations of 500 event mentions.

##  1 Introduction

The goal of event extraction is to extract event triggers and arguments from
unstructured data. An example is shown in Figure 1. Major obstacles to making
progress on event extraction have been the poor portability of traditional
supervised methods and the limited coverage of available event annotations.
Handling new event types means to start from scratch without being able to re-
use annotations for old event types. The main reason is that these approaches
modeled event extraction as a classification problem, encoding features only
by measuring the similarity between rich features encoded for test event
mentions and annotated event mentions. In these models, an event type (e.g.,
Transport-Person) or an argument role (e.g., Destination) is simply treated as
an atomic symbol (i.e., a surface lexical form). Therefore it’s not feasible
to repeat the high-cost annotation process for each of the 3,000+ event types.

Figure 1: Event Mention Example: dispatching is the trigger of a Transport-
Person event with four arguments.

In fact, many rich event ontologies have been recently developed, including
FrameNet Baker and Sato (2003), VerbNet Kipper et al. (2008), Propbank Palmer
et al. (2005) and OntoNotes Pradhan et al. (2007), where each event type is
associated with a set of predefined argument roles. We observe that both event
mentions and types can be represented with structures, where event mention
structure is constructed from trigger and candidate arguments, and event type
structure consists of event type and predefined roles. Consider two example
sentences:

  1. E1.

The Government of China has ruled Tibet since 1951 after dispatching troops to
the Himalayan region in 1950.

  2. E2.

Iranian state television stated that the conflict between the Iranian police
and the drug smugglers took place near the town of mirjaveh.

E1, as can be seen in Figure 1, includes a Transport_Person event mention
triggered by dispatching and E2 includes an Attack event mention triggered by
conflict. For each event mention, we apply Abstract Meaning Representation
(AMR) Banarescu et al. (2013) to identify candidate arguments and construct
event mention structures. Meanwhile, the two event types can also be
represented with structures from ERE (Entity Relation Event) Song et al.
(2015), as shown in Figure 2. We can see that, besides the lexical semantics
that relates a trigger to its type, their structures also tend to be similar:
a Transport_Person event typically involves a Person instead of an Artifact as
the patient, while an Attack event involves a Person or Location as an
Attacker. This observation is similar to the theory that “the semantics of an
event structure can be generalized and mapped to event mention structures in a
systematic and predictable way” Pustejovsky (1991). Inspired by this theory,
we take a fresh look at the event extraction task and model it as a
“grounding” problem, by mapping each mention to its semantically closest event
type in the ontology.

Figure 2: Examples of Event Mention and Type Structures from ERE.

One possible implementation of this idea is Zero-Shot Learning (ZSL), which
has been successfully exploited in visual object classification Frome et al.
(2013); Norouzi et al. (2013); Socher et al. (2013a). The main idea of
applying ZSL for vision tasks is to represent both images and type labels in a
multi-dimensional vector space separately, and then learn a regression model
to map from image semantic space to type label semantic space based on
annotated images for seen labels. This regression model can be further used to
predict the unseen labels of any given image.

In this paper, we apply ZSL to event extraction. Given an event ontology,
where each event type is defined with a rich structure (e.g., argument roles),
we call the event types with annotated event mentions as seen types, while
those without annotations as unseen. Our goal is to effectively transfer the
knowledge of events from seen types to unseen types, so we can extract event
mentions of any types defined in the ontology. We design a transferable neural
architecture, which jointly learns and maps the structural representations of
both event mentions and types into a shared semantic space by minimizing the
distance between each event mention and its corresponding type. For event
mentions with unseen types, their structures will be projected into the same
semantic space using the same framework and assigned types with top-ranked
similarity values.

There are two appealing advantages of this new view, which also manifest our
contributions:

  1. $\bullet$

This mapping/ranking function is “universal” and independent of event types,
thus we can transfer resources from existing types to new types without any
additional annotation effort;

  2. $\bullet$

Many existing event ontologies cover a wider range of event types, which allow
us to extend the scope of event extraction from several dozen types to
thousands of types.

##  2 Approach

###  2.1 Overview

Figure 3: Architecture Overview

Event extraction aims to extract both triggers and arguments. Figure 3
illustrates the overall architecture of our approach for trigger typing, while
argument typing follows the same pipeline.

Given a sentence $s$, we start by identifying candidate triggers and arguments
based on AMR parsing Wang et al. (2015b). An example is shown in Figure 1. For
each trigger $t$, e.g., dispatch-01, we build a structure $S_{t}$ using AMR as
shown in Figure 3. Each structure is composed of a set of tuples, e.g,
$\langle$dispatch-01, :ARG0, China$\rangle$. We use a matrix to represent each
AMR relation, composing its semantics with two concepts for each tuple, and
feed all tuple representations into CNN to generate event mention structure
representation $V_{S_{t}}$.

Given a target event ontology, for each type $y$, e.g., Transport_Person, we
construct a type structure $S_{y}$ by incorporating its predefined roles, and
use a tensor to denote the implicit relation between any types and arguments.
We compose the semantics of type and argument role with the tensor for each
tuple, e.g., $\langle$Transport_Person, Destination$\rangle$. We generate the
event type structure representation $V_{S_{y}}$ using the same CNN. By
minimizing the semantic distance between dispatch-01 and Transport_Person
using $V_{S_{t}}$ and $V_{S_{y}}$, we jointly map the representations of event
mention and event types into a shared semantic space, where each mention is
closest to its annotated type.

After training, the compositional functions and CNNs can be further used to
project any new event mention (e.g., donate-01) into the semantic space and
find its closest event type (e.g., Donation).

###  2.2 Candidate Trigger and Argument Identification

Similar to Huang et al. (2016), we identify candidate triggers and arguments
based on AMR Parsing Wang et al. (2015b) and apply the same word sense
disambiguation (WSD) tool Zhong and Ng (2010) to disambiguate word senses and
link each sense to OntoNotes, as shown in Figure 1.

Given a sentence, we consider all noun and verb concepts that can be mapped to
OntoNotes senses by WSD as candidate event triggers. In addition, the concepts
that can be matched with verbs or nominal lexical units in FrameNet are also
considered as candidate triggers. For each candidate trigger, its candidate
arguments are specified by a subset of AMR relations, as shown in Table 1.

Categories |  Relations
---|---
Core roles |  ARG0, ARG1, ARG2, ARG3, ARG4
Non-core roles |  mod, location, instrument, poss, manner, topic, medium, prep-X
Temporal |  year, duration, decade, weekday, time
Spatial |  destination, path, location
Table 1: Event-Related AMR Relations.

###  2.3 Structure Construction and Composition

As Figure 3 shows, for each candidate trigger $t$, we construct its event
mention structure $S_{t}$ based on its candidate arguments and AMR parsing.
For each type $y$ of the target event ontology, we construct a structure
$S_{y}$ by incorporating its pre-defined roles and take the type as the root.

Each $S_{t}$ or $S_{y}$ is composed of a collection of tuples. For each event
mention structure, a tuple consists of two AMR concepts and a AMR relation,
while for each event type structure, a tuple consists of a type name and an
argument role name. Here we propose two approaches to incorporate the
semantics of relations into the two words of each tuple.

Event Mention Structure  For each tuple $u=\langle w_{1},\lambda,w_{2}\rangle$
in an event mention structure, we use a matrix to represent each AMR relation,
and compose the semantics of the AMR relation $\lambda$ to the two concepts
$w_{1}$ and $w_{2}$ as:

$V_{u}=[V_{w_{1}}^{{}^{\prime}};V_{w_{2}}^{{}^{\prime}}]=f([V_{w_{1}};V_{w_{2}}]\cdot
M_{\lambda})\vspace{-0.1cm}$

where $V_{w_{1}}$, $V_{w_{2}}$ $\in\mathbb{R}^{d}$ are the vector
representations of words $w_{1}$ and $w_{2}$. $d$ is the dimension size of
each word vector. $[\ ;\ ]$ denotes the concatenation of two vectors.
$M_{\lambda}\in\mathbb{R}^{2d\times 2d}$ is the matrix representation for AMR
relation $\lambda$. $V_{u}$ is the composition representation of tuple $u$,
which consists of two updated vector representations
$V_{w_{1}}^{{}^{\prime}}$, $V_{w_{2}}^{{}^{\prime}}$ for $w_{1}$ and $w_{2}$
by incorporating the semantics of $\lambda$.

Event Type Structure  For each tuple $u^{{}^{\prime}}=\langle y,r\rangle$ in
an event type structure, where $y$ denotes the event type and $r$ denotes an
argument role, following Socher et al. (2013b), we assume an implicit and
“universal” relation between any pairs of type and argument, and use a single
and powerful tensor to represent the implicit relation:

$V_{u^{{}^{\prime}}}=[V_{y}^{{}^{\prime}};V_{r}^{{}^{\prime}}]=f([V_{y};V_{r}]^{T}\cdot
U^{[1:2d]}\cdot[V_{y};V_{r}])$

where $V_{y}$ and $V_{r}$ are vector representations for $y$ and $r$.
$U^{[1:2d]}\in\mathbb{R}^{2d\times 2d\times 2d}$ is a 3-order tensor.
$V_{u}^{{}^{\prime}}$ is the composition representation of tuple
$u^{{}^{\prime}}$, which consists of two updated vector representations
$V_{y}^{{}^{\prime}}$, $V_{r}^{{}^{\prime}}$ for $y$ and $r$ by incorporating
the semantics of their implicit relation $U^{[1:2d]}$.

###  2.4 Joint Event Mention and Type Label Embedding

CNN is good at capturing sentence level information in various Natural
Language Processing tasks. In this work, we use it to generate structure-level
representations. For each event mention structure
$S_{t}=(u_{1},u_{2},...,u_{h})$ and each event type structure
$S_{y}=(u^{{}^{\prime}}_{1},u^{{}^{\prime}}_{2},...,u^{{}^{\prime}}_{p})$,
which contains $h$ and $p$ tuples respectively, we apply a weight-sharing CNN
to each input structure to jointly learn event mention and type structural
representations, which will be later used to learn the ranking function for
zero-shot event extraction.

Input layer  is a sequence of tuples, where the order of tuples is from top to
bottom in the structure. Each tuple is represented by a $d\times 2$
dimensional vector, thus each mention structure and each type structure are
represented as a feature map of dimensionality $d\times 2h^{*}$ and $d\times
2p^{*}$ respectively, where $h^{*}$ and $p^{*}$ are the maximal number of
tuples for event mention and type structures. We use zero-padding to the right
to make the volume of all input structures consistent.

Convolution layer Take $S_{t}$ with $h^{*}$ tuples:
$u_{1},u_{2},...,u_{h^{*}}$ as an example. The input matrix of $S_{t}$ is a
feature map of dimensionality $d\times 2h^{*}$. We make $c_{i}$ as the
concatenated embeddings of $n$ continuous columns from the feature map, where
$n$ is the filter width and $0<i<2h^{*}+n$. A convolution operation involves a
filter $W\in\mathbb{R}^{nd}$, which is applied to each sliding window $c_{i}$:

$c^{{}^{\prime}}_{i}=\tanh(W\cdot c_{i}+b)\vspace{-0.1cm}$

where $c^{{}^{\prime}}_{i}$ is the new feature representation, and
$b\in\mathbb{R}^{d}$ is a biased vector. We set filter width as 2 and stride
as 2 to make the convolution function operate on each tuple with two input
columns.

#### Max-Pooling:

All tuple representations $c^{{}^{\prime}}_{i}$ are used to generate the
representation of the input sequence by max-pooling.

#### Learning:

For each event mention $t$, we name the correct type as positive and all the
other types in the target event ontology as negative. To train the composition
functions and CNN, we first consider the following hinge ranking loss:

$L_{1}(t,y)=\sum_{j\in Y,\ j\neq y}\max\\{0,m-C_{t,y}+C_{t,j}\\}$
$C_{t,y}=\cos([V_{t};V_{S_{t}}],[V_{y};V_{S_{y}}])$

where $y$ is the positive event type for $t$. $Y$ is the type set of the event
ontology. $[V_{t};V_{S_{t}}]$ denotes the concatenation of representations of
$t$ and $S_{t}$. $j$ is a negative event type for $t$ from $Y$. $m$ is a
margin. $C_{t,y}$ denotes the cosine similarity between $t$ and $y$.

The hinge loss is commonly used in zero-shot visual object classification
task. However, it tends to overfit the seen types in our experiments. While
clever data augmentation can help alleviate overfitting, we propose two
strategies: (1) we add “negative” event mentions into the training process.
Here a “negative” event mention means that the mention has no positive event
type among all seen types, namely it belongs to Other. (2) we design a new
loss function as follows:

$L^{d}_{1}(t,y)=\begin{cases}\max\limits_{j\in Y,j\neq
y}\max\\{0,m-C_{t,y}+C_{t,j}\\},\quad\quad y\neq Other\\\ \max\limits_{j\in
Y^{{}^{\prime}},j\neq
y^{{}^{\prime}}}\max\\{0,m-C_{t,y^{{}^{\prime}}}+C_{t,j}\\},\quad
y=Other\end{cases}$

where $Y$ is the type set of the event ontology. $Y^{{}^{\prime}}$ is the seen
type set. $y$ is the annotated type. $y^{{}^{\prime}}$ is the type which ranks
the highest among all event types for event mention $t$, while $t$ belongs to
Other.

By minimizing $L^{d}_{1}$, we can learn the optimized model which can compose
structure representations and map both event mention and types into a shared
semantic space, where the positive type ranks the highest for each mention.

###  2.5 Joint Event Argument and Role Embedding

For each mention, we map each candidate argument to a specific role based on
the semantic similarity of the argument path. Take E1 as an example. China is
matched to Agent based on the semantic similarity between
dispatch-01$\rightarrow$ :ARG0$\rightarrow$ China and Transport-
Person$\rightarrow$Agent.

Given a trigger $t$ and a candidate argument $a$, we first extract a path
$S_{a}=(u_{1},u_{2},...,u_{p})$, which connects $t$ and $a$ and consists of
$p$ tuples. Each predefined role $r$ is also represented as a structure by
incorporating the trigger type, $S_{r}=\langle y,r\rangle$. We apply the same
framework to take the sequence of tuples contained in $S_{a}$ and $S_{r}$ into
a weight-sharing CNN to rank all possible roles for $a$.

$L^{d}_{2}(a,r)=\begin{cases}\max\limits_{j\in R_{y},j\neq
r}\max\\{0,m-C_{a,r}+C_{a,j}\\}\quad\quad r\neq Other\\\ \max\limits_{j\in
R_{Y^{{}^{\prime}}},j\neq
r^{{}^{\prime}}}\max\\{0,m-C_{a,r^{{}^{\prime}}}+C_{a,j}\\}\ \
r|y=Other\end{cases}$

where $R_{y}$ and $R_{Y^{{}^{\prime}}}$ are the set of argument roles which
are predefined for trigger type $y$ and all seen types $Y^{{}^{\prime}}$. $r$
is the annotated role and $r^{{}^{\prime}}$ is the argument role which ranks
the highest for $a$ when $a$ or $y$ is annotated as Other.

In our experiments, we sample various size of “negative” training data for
trigger and argument labeling respectively. In Section 3.2 we describe how the
negative training instances are generated. We adopt a pipelined framework and
train the model for trigger labeling and argument labeling separately.

###  2.6 Zero-Shot Classification

During test, given a new event mention $t^{{}^{\prime}}$, we compute its
mention structure representation for $S_{t^{{}^{\prime}}}$ and all event type
structure representations for $S_{Y}=\\{S_{y_{1}},S_{y_{2}},...,S_{y_{n}}\\}$
using the same parameters trained from seen types. We rank all event types
based on their similarity scores with mention $t^{{}^{\prime}}$. The top
ranked prediction for $t^{{}^{\prime}}$ from the event type set, denoted as
$\widehat{y}(t^{{}^{\prime}},1)$, is given by:

$\widehat{y}(t^{{}^{\prime}},1)=\arg\max_{y\in
Y}{\cos([V_{t^{{}^{\prime}}};V_{S_{t^{{}^{\prime}}}}],[V_{y};V_{S_{y}}])}\vspace{-0.2cm}$

Moreover, $\widehat{y}(t^{{}^{\prime}},k)$ denotes the $k^{th}$ most probable
event type predicted for $t^{{}^{\prime}}$. We will investigate the event
extraction performance based on the top-$k$ predicted event types.

After determining the type $y^{{}^{\prime}}$ for mention $t^{{}^{\prime}}$,
for each candidate argument, we adopt the same ranking function to find the
most appropriate role from the role set defined for $y^{{}^{\prime}}$.

##  3 Experiments

###  3.1 Hyper-Parameters

We use an August 11, 2014 English Wikipedia dump to learn trigger sense and
argument embeddings based on the Continuous Skip-gram model Mikolov et al.
(2013). Table 2 shows the hyper-parameters we used to train models.

Parameter Name |  Value
---|---
Word Sense Embedding Size |  200
Initial Learning Rate |  0.1
# of Filters in Convolution Layer |  500
Maximal # of Tuples for Mention Structure |  10
Maximal # of Tuples for Argument Path |  5
Maximal # of Tuples for Event Type Structure |  5
Maximal # of Tuples for Argument Role Path |  1
Table 2: Hyper-parameters.

###  3.2 ACE Event Classification

We first use ACE event schema as our target event ontology and assume the
boundaries of triggers and arguments are given. Of the 33 ACE event types, we
select the top-$N$ most popular event types from ACE05 data as “seen” types,
and use 90% event annotations of these for training and 10% for development.
$N$ is set as 1, 3, 5, 10 respectively. We test the zero-shot classification
performances on the annotations for the remaining 23 unseen types. Table 3
shows the types that we selected for training in each experiment setting.

The negative event mentions and arguments that belong to Other are sampled
from the output of the system developed by Huang et al. (2016) based on ACE05
training sentences, which groups all candidate triggers and arguments into
clusters based on semantic representations and assigns a type/role name to
each cluster. We sample the negative event mentions from the clusters (e.g.,
Build, Threaten) which cannot be mapped to ACE event types. We sample the
negative arguments from the arguments associated with these negative event
mentions. Table 4 shows the statistics of the training, development and
testing data sets.

Setting |  N |  Seen Types for Training/Dev
---|---|---
A |  1 |  Attack
B |  3 |  Attack, Transport, Die
C |  5 |  Attack, Transport, Die, Meet, Arrest-Jail
D |  10 |  Attack, Transport, Die, Meet, Sentence, Arrest-Jail, Transfer-Money, Elect, Transfer-Ownership, End-Position
Table 3: Seen Types in Each Experiment Setting. Setting Index |  Training |  Development |  Test
---|---|---|---
|  # of Types/Roles |  # of Events |  # of Arguments |  # of Events |  # of Arguments |  # of Types/Roles |  # of Events |  # of Arguments
A |  1/5 |  953/900 |  894/1,097 |  105/105 |  86/130 |  23/59 |  753 |  879
B |  3/14 |  1,803/1,500 |  2,035/1,791 |  200/200 |  191/237 |  |  |
C |  5/18 |  2,033/1,300 |  2,281/1,503 |  225/225 |  233/241 |  |  |
D |  10/37 |  2537/700 |  2,816/879 |  281/281 |  322/365 |  |  |
Table 4: Statistics for Positive/Negative Instances in Training, Dev, and Test Sets for Each Experiment. Setting |  Method |  Hit@k Trigger Classification (%) |  Hit@k Argument Classification (%)
---|---|---|---
|  1 |  3 |  5 |  1 |  3 |  5
|  WSD-Embedding |  1.73 |  13.01 |  22.84 |  2.39 |  2.84 |  2.84
A |  Our Approach |  3.98 |  23.77 |  32.54 |  1.25 |  3.41 |  3.64
B |  7.04 |  12.48 |  36.79 |  3.53 |  6.03 |  6.26
C |  20.05 |  34.66 |  46.48 |  9.56 |  14.68 |  15.70
D |  33.47 |  51.40 |  68.26 |  14.68 |  26.51 |  27.65
Table 5: Hit@K Performance on Trigger and Argument Classification.

To show the effectiveness of structural similarity in our approach, we design
a baseline, WSD-Embedding, which directly grounds event mentions and arguments
to their candidate types and roles using our pre-trained word sense
embeddings. Table 5 shows that the structural similarity is much more
effective than lexical similarity for both trigger and argument
classification. Also, as the number of seen types in training increases, the
transfer model’s performance improves.

We further evaluate the performance of our transfer approach on similar and
distinct unseen types. The 33 sub-types defined in ACE fall within 8 coarse-
grained main types, such as Life, Justice. Each subtype belongs to one main
type. Subtypes that belong to the same main type tend to have similar
structures. For example, Trial-Hearing and Charge-Indict have the same set of
argument roles. For training our transfer model, we select 4 subtypes of
Justice: Arrest-Jail, Convict, Charge-Indict, Execute. For testing, we select
another 3 subtypes of Justice: Sentence, Appeal, Release-Parole. Additionally,
we also select one subtype from each of the other seven main types for
comparison. Table 6 shows that, when testing on a new unseen type, the more
similar it is to the seen types, the better performance is achieved.

Type |  Subtype |  Hit@k Trigger Classification
---|---|---
1 |  3 |  5
Justice |  Sentence |  68.29 |  68.29 |  69.51
Justice |  Appeal |  67.50 |  97.50 |  97.50
Justice |  Release-Parole |  73.91 |  73.91 |  73.91
Conflict |  Attack |  26.47 |  44.52 |  46.69
Transaction |  Transfer-Money |  48.36 |  68.85 |  79.51
Business |  Start-Org |  0 |  33.33 |  66.67
Movement |  Transport |  2.60 |  3.71 |  7.81
Personnel |  End-Position |  9.09 |  50.41 |  53.72
Contact |  Phone-Write |  60.78 |  88.24 |  90.20
Life |  Injure |  87.64 |  91.01 |  91.01
Table 6: Performance on Various Types Using Justice Subtypes for Training

###  3.3 ACE Event Identification & Classification

Considering that ACE05 corpus includes the richest annotations for event
extraction to date, to assess our transferable neural architecture on a large
number of unseen types when trained on limited annotations of seen types, we
construct a new event ontology which combines 33 ACE event types and argument
roles, and 1,161 frames from FrameNet except for the most generic frames such
as Entity, Locale. Some ACE event types easily align to frames, e.g., Die is
aligned with Death. Some frames are instead more accurately treated as
inheritors of ACE types, such as Suicide-Attack, which inherits from Attack.
We manually mapped the selected frames to ACE types.

We compare our approach against the following supervised methods:

  1. $\bullet$

LSTM: A long short-term memory neural network Hochreiter and Schmidhuber
(1997) based on distributed semantic features, similar to Feng et al. (2016).

  2. $\bullet$

Joint: A structured perceptron model based on symbolic semantic features Li et
al. (2013).

For our approach, we follow the experiment setting D in Section 3.2, but
target at the 1191 event types in our new event ontology. For evaluation, we
sample 150 sentences from the remaining ACE05 data, which contain 129
annotated event mentions for the 23 testing types. For both LSTM and Joint
approaches, we use the entire ACE05 annotated data for 33 ACE event types for
training except for the held-out 150 evaluation sentences.

We identify the candidate triggers and arguments based on the approach in
Section 2.2, and map each candidate trigger and argument to the target event
ontology. We evaluate on the event mentions which are classified into the 23
testing ACE types. Table 7 shows the performances.

Setting |  Method |  Trigger Identification |  Trigger Identification + Classification |  Arg Identification |  Arg Identification + Classification
---|---|---|---|---|---
|  |  P |  R |  F |  P |  R |  F |  P |  R |  F |  P |  R |  F
D |  LSTM |  94.7 |  41.8 |  58.0 |  89.4 |  39.5 |  54.8 |  47.8 |  22.6 |  30.6 |  28.9 |  13.7 |  18.6
D |  Joint |  55.8 |  67.4 |  61.1 |  50.6 |  61.2 |  55.4 |  36.4 |  28.1 |  31.7 |  33.3 |  25.7 |  29.0
D |  Transfer |  85.7 |  41.2 |  55.6 |  75.5 |  36.3 |  49.1 |  28.2 |  27.3 |  27.8 |  16.1 |  15.6 |  15.8
Table 7: Performance of Trigger and Argument Extraction on ACE Types. (%)

To further demonstrate the zero-shot learning ability of our framework and the
significance on saving human annotation effort, we use the supervised LSTM
approach for comparison, because it achieved state-of-the-art performance on
ACE event extraction Feng et al. (2016). The training data of LSTM contains
3,464 sentences with 905 annotated event mentions for the 23 testing event
types. We divide these event annotations into 10-fold and successively add
another 10% into the training data of LSTM. Figure 4 shows the learning curve.
Without any annotated mentions of the 23 test event types in its training set,
our transfer learning approach achieves performance comparable to that of the
LSTM, which is trained on 3,000 sentences with 500 annotated event mentions.

Figure 4: Comparison between Our Approach with LSTM on 23 Testing Event Types.

In analyzing the triggers which are annotated with ACE types but misclassified
into incorrect types or frames, we observe that most errors occur among the
types that are defined under the same scenario. For example, in the following
sentence “Abby was a true water birth ( 3kg - normal ) and with Fiona I was
dragged out of the pool after the head crowned”, birth should be a Being-Born
event while our approach misclassified it as Giving-Birth because both Being-
Born and Giving-Birth are defined for Birth-Scenario and have very similar
predefined roles. For argument classification, our approach heavily relies on
the semantics of argument path and argument concepts, while many argument
roles such as Entity, Organization, are not informative enough to be matched
with argument concepts.

###  3.4 Event Extraction on New Types

In Section 3.3, as we use 1,194 event types as the target ontology, we further
evaluate the performance of our approach on non-ACE types. From the testing
results of Section 3.3, we randomly sample 200 event mentions assigned with
non-ACE types and ask a linguistic expert to manually assess them. For each
mention, the annotator can see its trigger, arguments, the source sentence,
the frame and roles assigned by our approach, as well as the definition and
examples of the frame from
FrameNet111https://framenet.icsi.berkeley.edu/fndrupal/. The annotator marks
true or false by judging whether the type and argument roles are correct.
Table 8 shows the performance.

Performance |  Accuracy (%)
---|---
Trigger Identification + Classification |  40.9
Arg Identification + Classification |  17.41
Table 8: Overall Performance on All Event Types

Our approach can discover a lot of new events that are not annotated in ACE.
For example, in the sentence “15 dead as suicide bomber blasts student bus,
Israel hits back in Gaza”, blasts is correctly identified as an Explosion
event. However, many triggers are mapped to the correct scenario but assigned
with incorrect types. For example, in the sentence “But Anwar’s lawyers said
they were filing a fresh request for bail pending a further appeal.”, filing
is identified as a trigger and mapped to the correct Bail related scenario but
misclassified as a Bail-Decision event. We find that, to determine the type of
an event mention, besides the consistent semantics between event mentions and
type structures, the trigger sense should also be consistent with the
definition of the event type.

###  3.5 Impact of AMR

In our work, we use AMR parsing output to construct event structures. To
assess the impact of AMR parser Wang et al. (2015a) on event extraction, we
choose a subset of ERE corpus which has perfect AMR annotations222This subset
contains 304 documents with 1,022 annotated event mentions of 40 types.. We
select the top-6 most popular event types (Arrest-Jail, Execute, Die, Meet,
Sentence, Charge-Indict) with 548 manual annotations as seen types. We sample
500 negative event mentions from distinct types of clusters generated from the
system Huang et al. (2016) based on ERE training sentences. We combine the
annotated events for seen types and the negative event mentions, and use 90%
for training and 10% for development. For evaluation, we select 200 sentences
from the remaining ERE subset, which contains 128 Attack event mentions and 40
Convict event mentions. Table 9 shows the event extraction performances based
on perfect AMR and system AMR respectively.

Using the same data sets, we further evaluate the performance of our approach
using different semantic parsing outputs. We compare AMR with Semantic Role
Labeling (SRL) output Palmer et al. (2010) by keeping only the core roles
(e.g., :ARG0, :ARG1) from AMR annotations. As Table 9 shows, compared with SRL
output, the fine-grained AMR semantic relations such as :location, :instrument
appear to be more informative to infer the argument roles.

Method |  Trigger $F_{1}$ |  Arg $F_{1}$
---|---|---
P |  R |  $F_{1}$ |  P |  R |  $F_{1}$
Perfect AMR |  79.1 |  47.1 |  59.1 |  25.4 |  21.4 |  23.2
Perfect AMR Core Roles (SRL) |  77.1 |  47.0 |  58.4 |  19.7 |  16.9 |  18.2
System AMR |  85.7 |  32.0 |  46.7 |  22.6 |  15.8 |  18.6
Table 9: Impact of AMR Parser and Semantic Information on Trigger and Argument
Identification and Classification (%).

##  4 Related Work

Most of previous event extraction methods were based on supervised learning
using symbolic features Ji and Grishman (2008); Miwa et al. (2009); Liao and
Grishman (2010); Liu et al. (2010); Hong et al. (2011); McClosky et al.
(2011); Riedel and McCallum (2011); Chen and Ng (2012); Li et al. (2013); Liu
et al. (2016) or distributional features Chen et al. (2015); Nguyen and
Grishman (2015); Feng et al. (2016); Nguyen et al. (2016) from a large amount
of training data, regarding event types and argument roles as symbols. Such
work can achieve high quality for given types, but cannot be applied to new
types without annotation. In contrast, we provide a new angle to vision event
extraction and model it as a “grounding” task by taking advantage of rich
semantics of event types.

Some other IE paradigms such as Open IE Etzioni et al. (2005); Banko et al.
(2007, 2008); Etzioni et al. (2011); Ritter et al. (2012), Pre-emptive IE
Shinyama and Sekine (2006), On-demand IE Sekine (2006), Liberal IE Huang et
al. (2016, 2017), and semantic frame based event discovery Kim et al. (2013)
can discover many events without pre-defined event schema. The event types and
argument roles are inferred by a cluster of similar events. These paradigms
heavily rely on information redundancy, so cannot work when the input consists
of only a few sentences. Our work can discover events from any size of given
corpus and can also be complementary with these paradigms because it can
ground each event cluster to a rich predefined event ontology.

Zero-Shot learning has been widely applied in visual object classification
Frome et al. (2013); Norouzi et al. (2013); Socher et al. (2013a), fine-
grained name tagging Ma et al. (2016); Qu et al. (2016) and relation
extraction Verga et al. (2016). Different from these tasks, the seen types in
event extraction are limited. The most popular event schemas, such as ACE,
only defined 33 event types while most visual object training sets contain
more than 1,000 types. Thus, the methods proposed for zero-shot visual object
classification cannot be directly applied to event extraction because of
overfitting. Thus, we design a new loss function by creating “negative”
training instances to avoid overfitting.

##  5 Conclusions and Future Work

In this work, we take a fresh look at the event extraction task and model it
as a grounding problem. We propose a transferable neural architecture, which
leverages existing human constructed event schemas and manual annotations for
a small set of seen types, and transfers the knowledge from the existing types
to the extraction of unseen types, to improve the scalability of event
extraction as well as save human effort. Without any annotation, our approach
can achieve comparable performance with state-of-the-art supervised models
trained from a large amount of labeled data. In the future, we will extend our
framework by incorporating event definitions and argument descriptions to
improve the event extraction performance.

## References

  * Baker and Sato (2003) C. F. Baker and H. Sato. 2003.  The framenet data and software.  In Proc. ACL2003.
  * Banarescu et al. (2013) L. Banarescu, C. Bonial, S. Cai, M. Georgescu, K. Griffitt, U. Hermjakob, K. Knight, P. Koehn, M. Palmer, and N. Schneider. 2013.  Abstract meaning representation for sembanking.  In Proc. ACL2013 Workshop on Linguistic Annotation and Interoperability with Discourse.
  * Banko et al. (2007) M. Banko, M. Cafarella, S. Soderland, M. Broadhead, and O. Etzioni. 2007.  Open information extraction for the web.  In Proc. IJCAI2007.
  * Banko et al. (2008) M. Banko, O. Etzioni, and T. Center. 2008.  The tradeoffs between open and traditional relation extraction.  In Proc. ACL-HLT2008.
  * Chen and Ng (2012) C. Chen and V. Ng. 2012.  Joint modeling for chinese event extraction with rich linguistic features.  In Proc. COLING2012.
  * Chen et al. (2015) Y. Chen, L. Xu, K. Liu, D. Zeng, and J. Zhao. 2015.  Event extraction via dynamic multi-pooling convolutional neural networks.  In Proc. ACL2015.
  * Etzioni et al. (2005) O. Etzioni, M. Cafarella, D. Downey, A. Popescu, T. Shaked, S. Soderland, D. Weld, and A. Yates. 2005.  Unsupervised named-entity extraction from the web: An experimental study.  Artificial Intelligence .
  * Etzioni et al. (2011) O. Etzioni, A. Fader, J. Christensen, S. Soderland, and M. Mausam. 2011.  Open information extraction: The second generation.  In Proc. IJCAI2011.
  * Feng et al. (2016) X. Feng, L. Huang, D. Tang, B. Qin, H. Ji, and T. Liu. 2016.  A language-independent neural network for event detection.  In Proc. ACL2016.
  * Frome et al. (2013) A. Frome, G. Corrado, J. Shlens, S. Bengio, J. Dean, and T. Mikolov. 2013.  Devise: A deep visual-semantic embedding model.  In Proc. NIPS2013.
  * Hochreiter and Schmidhuber (1997) S. Hochreiter and J. Schmidhuber. 1997.  Long short-term memory.  Neural computation .
  * Hong et al. (2011) Y. Hong, J. Zhang, B. Ma, J. Yao, G. Zhou, and Q. Zhu. 2011.  Using cross-entity inference to improve event extraction.  In Proc. ACL2011.
  * Huang et al. (2016) L. Huang, T. Cassidy, X. Feng, H. Ji, C. Voss, J. Han, and A. Sil. 2016.  Liberal event extraction and event schema induction.  In Proc. ACL2016.
  * Huang et al. (2017) L. Huang, J. May, X. Pan, H. Ji, X. Ren, J. Han, L. Zhao, and J. Hendler. 2017.  Liberal entity extraction: Rapid construction of fine-grained entity typing systems.  Big Data .
  * Ji and Grishman (2008) H. Ji and R. Grishman. 2008.  Refining event extraction through cross-document inference.  In Proc. ACL2008.
  * Kim et al. (2013) H. Kim, X. Ren, Y. Sun, C. Wang, and J. Han. 2013.  Semantic frame-based document representation for comparable corpora.  In Proc. ICDM2013.
  * Kipper et al. (2008) K. Kipper, A. Korhonen, N. Ryant, and M. Palmer. 2008.  A large-scale classification of english verbs.  Language Resources and Evaluation Journal .
  * Li et al. (2013) Q. Li, H. Ji, and L. Huang. 2013.  Joint event extraction via structured prediction with global features.  In Proc. ACL2013.
  * Liao and Grishman (2010) S. Liao and R. Grishman. 2010.  Using document level cross-event inference to improve event extraction.  In Proc. ACL2010.
  * Liu et al. (2010) B. Liu, L. Qian, H. Wang, and G. Zhou. 2010.  Dependency-driven feature-based learning for extracting protein-protein interactions from biomedical text.  In Proc. COLING2010.
  * Liu et al. (2016) S. Liu, Y. Chen, S. He, K. Liu, and J. Zhao. 2016.  Leveraging framenet to improve automatic event detection.  In Proc. ACL2016.
  * Ma et al. (2016) Y. Ma, E. Cambria, and S. Gao. 2016.  Label embedding for zero-shot fine-grained named entity typing.  In Proc. COLING2016.
  * McClosky et al. (2011) D. McClosky, M. Surdeanu, and C. D. Manning. 2011.  Event extraction as dependency parsing.  In Proc. ACL2011.
  * Mikolov et al. (2013) T. Mikolov, K. Chen, G. Corrado, and J. Dean. 2013.  Efficient estimation of word representations in vector space.  CoRR abs/1301.3781.
  * Miwa et al. (2009) M. Miwa, R. Sætre, Y. Miyao, and J. Tsujii. 2009.  A rich feature vector for protein-protein interaction extraction from multiple corpora.  In Proc. EMNLP2009.
  * Nguyen et al. (2016) T. Nguyen, K. Cho, and R. Grishman. 2016.  Joint event extraction via recurrent neural networks.  In Proc. NAACL-HLT2016.
  * Nguyen and Grishman (2015) T. Nguyen and R. Grishman. 2015.  Event detection and domain adaptation with convolutional neural networks.  In Proc. ACL2015.
  * Norouzi et al. (2013) M. Norouzi, T. Mikolov, S. Bengio, Y. Singer, J. Shlens, A. Frome, G. Corrado, and J. Dean. 2013.  Zero-shot learning by convex combination of semantic embeddings.  arXiv preprint arXiv:1312.5650 .
  * Palmer et al. (2005) M. Palmer, D. Gildea, and P. Kingsbury. 2005.  The proposition bank: An annotated corpus of semantic roles.  Computational Linguistics .
  * Palmer et al. (2010) M. Palmer, D. Gildea, and N. Xue. 2010.  Semantic role labeling.  Synthesis Lectures on Human Language Technologies .
  * Pradhan et al. (2007) S. Pradhan, E. Hovy, M. Marcus, M. Palmer, L. Ramshaw, and R. Weischedel. 2007.  Ontonotes: A unified relational semantic representation.  International Journal of Semantic Computing .
  * Pustejovsky (1991) J. Pustejovsky. 1991.  The syntax of event structure.  Cognition .
  * Qu et al. (2016) L. Qu, G. Ferraro, L. Zhou, W. Hou, and T. Baldwin. 2016.  Named entity recognition for novel types by transfer learning.  In Proc. ACL2016.
  * Riedel and McCallum (2011) S. Riedel and A. McCallum. 2011.  Fast and robust joint models for biomedical event extraction.  In Proc. EMNLP2011.
  * Ritter et al. (2012) A. Ritter, O. Etzioni, and S. Clark. 2012.  Open domain event extraction from twitter.  In Proc. SIGKDD2012.
  * Sekine (2006) S. Sekine. 2006.  On-demand information extraction.  In Proc. COLING-ACL2006.
  * Shinyama and Sekine (2006) Y. Shinyama and S. Sekine. 2006.  Preemptive information extraction using unrestricted relation discovery.  In Proc. HLT-NAACL2006.
  * Socher et al. (2013a) R. Socher, M. Ganjoo, C. Manning, and A. Ng. 2013a.  Zero-shot learning through cross-modal transfer.  In Proc. NIPS2013.
  * Socher et al. (2013b) R. Socher, A. Perelygin, J. Wu, J. Chuang, C. Manning, A. Ng, and C. Potts. 2013b.  Recursive deep models for semantic compositionality over a sentiment treebank.  In Proc. EMNLP2013.
  * Song et al. (2015) Z. Song, A. Bies, S. Strassel, T. Riese, J. Mott, J. Ellis, J. Wright, S. Kulick, N. Ryant, and X. Ma. 2015.  From light to rich ere: annotation of entities, relations, and events.  In In Proc. NAACL-HLT2015.
  * Verga et al. (2016) P. Verga, D. Belanger, E. Strubell, B. Roth, and A. McCallum. 2016.  Multilingual relation extraction using compositional universal schema.  In Proc. NAACL2016.
  * Wang et al. (2015a) C. Wang, N. Xue, and S. Pradhan. 2015a.  Boosting transition-based amr parsing with refined actions and auxiliary analyzers.  In Proc. ACL2015.
  * Wang et al. (2015b) Chuan Wang, Nianwen Xue, Sameer Pradhan, and Sameer Pradhan. 2015b.  A transition-based algorithm for amr parsing.  In HLT-NAACL.
  * Zhong and Ng (2010) Z. Zhong and H. T. Ng. 2010.  It makes sense: A wide-coverage word sense disambiguation system for free text.  In Proc. ACL2010.