# Electronic and optical properties of ternary kagome Rb2Ni3S4

Gang Bahadur Acharya1,2, and Madhav Prasad Ghimire1,3 1 Central Department of
Physics, Tribhuvan University, Kirtipur, 44613 Kathmandu, Nepal  2 Condensed
Matter Physics Research Center, Butwal-11, 32907 Butwal, Nepal  3 Leibniz
Institute for Solid State and Materials Research—IFW Dresden, Helmholtzstr.
20, 01069 Dresden, Germany<EMAIL_ADDRESS>
###### Abstract

The application of semiconductors with optical properties has grown
significantly in the development of semiconductor photovoltaics. In this
study, the electronic and optical properties of ternary transition metal
sulfide Rb2Ni3S4 is studied by means of density functional theory calculation
within the framework of generalized gradient approximation. From the
structural perspective, Ni atoms is found to form a kagome-like lattice in a
two-dimensional plane of Rb2Ni3S4. From the electronic structure calculations,
the material is found to be a semiconductor with an indirect band gap of
$\sim$0.67 eV. The orbital contributions to the density of states and band
structure around the Fermi level are from the Ni-$3d$ and S-$3p$ states.
Significant hybridization was observed between the S-$3p_{x}$ and S-$3p_{y}$
with the Ni-3$d_{xz}$ and Ni-3$d_{yz}$ orbitals. Interestingly, flat band was
noticed below the Fermi level demonstrating one significant feature of kagome
lattice. From the optical calculations, Rb2Ni3S4 is found to exhibit optical
activity in both the visible and ultraviolet regions of the incident photon
energies. This optical response suggests that the material as a potential
candidate for opto-electronic device application, given its ability to
interact with light across a broad range of wavelengths. This work is expected
to motivate experimental group for transport measurements and may provide a
new foundation for the development of opto-electronic applications.

Keywords: Density functional theory, Kagome Lattice, Electronic structure,
Semiconductor, Optoelectronic applications

##  1 Introduction

The family of ternary transition metal chalcogenides represented by the
molecular formula A2M3X4 (where A includes K, Rb, Cs; M includes Ni, Pd, Pt;
and X includes S, Se) demonstrates diverse crystal symmetries [1]. These
compounds possess unique properties, featuring a quasi-two dimensional (2D)
layered structure, where the transition metal elements create kagome nets
together with chalcogenide atoms. Kagome compounds are widely recognized as
excellent platforms for exploring novel topological properties. The specific
arrangement of atoms in a kagome pattern leads to remarkable transport
characteristics, including negative magneto-resistance, anomalous Hall
conductivity, and Fermi arc surface states [2, 3, 4, 5, 6, 7, 8, 9].

Recently, a kagome CsV3Sb5 was reported to exhibit efficient absorption of
ultraviolet radiation and has a strong reflectivity for visible light [10] by
means of density functional theory (DFT) calculations. In the family of
ternary chalcogenides Rb2Pd3Se4, Li et al. predicted the superconductivity by
applying pressure wherein Lifshitz transition was observed [11]. Elder et al.
performed the magnetic and transport measurements in K2Ni3S4 suggesting the
possibility of co-existence of Ni1+ and Ni2+ in the crystal [12]. Likewise, we
recently studied the electronic and optical properties of Cs2Ni3S4 in addition
to the effect of vacancy defects in the crystal. Our study suggest this
material to be a suitable candidate for optoelectronic device [13].

Few literatures were found for Rb2Ni3S4 which suggest this material to be
interesting. For instance, Bronger et al. first synthesized and carried out
the magnetic susceptibility measurements above 90 K to explore magnetism in
Rb2Ni3S4. There did not find any evidence of magnetism [14]. Fukamachi et al.
reported the observation of weak ferromagnetism and semi-conducting
characteristics in Rb2Ni3S4. However, their findings, based on both nuclear
magnetic resonance and magnetic measurements, led them to propose that
Rb2Ni3S4 is a band insulator. Consequently, they suggested that the intrinsic
property of this system to be non-magnetic [15]. Later in 2002, Hondou et al.
performed the DFT calculations and this material to be a band gap
semiconductor, featuring flat bands below the Fermi level (E${}_{\mbox{F}}$)
[16]. Additional investigations using photoemission spectroscopy have unveiled
the moderate electron correlation and substantial hybridization between Ni-3d
and S-3p orbitals, in agreement with the results from the band structure
calculations [17]. Because of the presence of a planar sulfur coordination
around Ni2+, results in a low spin state with Ni in its 3d8 configuration.
This is suggested to be a non-magnetic semiconductor.

Despite a number of studies, there are no reports on optical properties of
Rb2Ni3S4. With an aim to fill this gap and to show the interesting behaviour
of this material, we explore the detailed electronic structure and optical
properties of Rb2Ni3S4. Our calculations suggest this material to be a non-
magnetic semiconductor with an indirect band gap of $\sim$0.67 eV. The optical
property shows that optical activity is prominent within the visible and lower
ultraviolet spectrum.

##  2 Computational and structural details

Figure 1: (a) Crystal structure of Rb2Ni3S4. (b) Three dimensional Brillouin
zone. Red dots are indicates the high symmetry points in momentum space. (c)
Ni-ions constitute a kagome lattice in ab plane.

Rb2Ni3S4 crystallizes in an orthorhombic crystal structure with space group
Fmmm (no. 69) and belongs to the point group D2h. The experimental lattice
parameters used in this study are $a$ = 5.862 Å, $b$= 9.937 Å and $c$ = 13.758
Å [1]. The crystal system consists of four in-equivalent atoms labeled as Rb,
Ni(I), Ni(II), and S, respectively. In Rb2Ni3S4, Rb1+ ions are surrounded by
eight equivalent S2- ions in an 8-coordinate geometry. The Rb–S bond distances
exhibit a range from 3.38 Å to 3.45 Å. All Ni2+ ions are coordinated by four
equivalent S2- ions, adopting a square co-planar arrangement [see Fig. 1(a)].
At the first and second Ni2+ site, the Ni–S bond length measured are 2.22 Å.
Furthermore, Rb2Ni3S4 exhibits symmorphic crystal symmetry, and due to the
presence of a center of inversion, they are considered centrosymmetric
crystal. Table 1 presents the fully relaxed atomic coordinates for Rb2Ni3S4.

In order to investigate the electronic and optical characteristics of selected
compound, we employed the DFT approach using the full potential local orbital
(FPLO) code [18], version 18.00. The standard generalized gradient
approximation (GGA) with the Perdew, Burke, and Ernzerhof (PBE-96) [19]
parameterization was taken into the exchange and correlation energy. Both,
scalar-relativistic (without spin-orbit coupling) and full-relativistic (with
spin-orbit coupling) modes were considered for the self-consistent
calculations. However, since the inclusion of spin-orbit coupling did not
significantly impact the electronic properties, we focus our discussion only
on the outcomes from the scalar-relativistic calculations. For the optical
property calculations, we used the FOPTICS module of the FPLO. In the
electronic structure calculations, a k-mesh subdivision of $12\times 12\times
12$ was used to cover the full Brillouin zone. As for the optical property
calculations, a finer k-mesh subdivision of $32\times 32\times 32$ was
employed. To ensure the accuracy of the energy, the convergence criteria for
self-consistency as set to 10-8 Hartree. During the structural optimization, a
force convergence of 10-3 eV/Å was applied.

Table 1: Total number of in-equivalent atoms and their Wyckoff positions in Rb2Ni3S4. Atom | Point location | x | y | z
---|---|---|---|---
Rb | 8i | 0 | 0 | 0.3425
Ni(I) | 8e | -1/4 | -1/4 | 0
Ni(II) | 4a | 0 | 0 | 0
S | 16m | 0 | 0.1677 | 0.1061

##  3 Results and discussion

###  3.1 Electronic properties

Figure 2: Total, partial and local DOS of Rb2Ni3S4. (a) Total DOS. (b) partial
DOS. (c) Projected local DOS for Ni(I)-3d, (d) Ni(II)-3d and (e) S-3p. The
vertical solid line at zero energy denotes E${}_{\mbox{F}}$.

To understand the electronic properties, we performed the density of states
(DOS) and band structure calculation. Based on the total and partial DOS
analysis, as shown in Fig. 2, it is evident that 3d orbital of Ni and 3p
orbital of S atoms contribute significantly to the DOS, while Rb has minor
contributions (see Fig. 2(b)). This difference is due to the presence of only
one electron in the outermost 5s orbital of Rb atom. To enhance clarity, we
display the local DOS projections individually, focusing on Ni(I)-3d as
depicted in Figure 2(c), and of Ni(II)-3d as shown in Figure 2(d). The
analysis of the local DOS (LDOS) projection (see Fig. 2(c-e)) reveals that the
insulating behavior originates from the splitting of energy bands due to the
crystal field effect. It is well-established that each nickel (Ni) ion is
surrounded by four S ions, forming a tetrahedral square planar coordination
arrangement (see Fig. 1(a)). In this scenario, the S2- ions act as strong
ligand for the central Ni2+ metal, which leads to crystal field splitting.
Consequently, specific d-orbitals ($d_{xy}$, $d_{yz}$, $d_{xz}$), known as the
t2g state, possess higher energy levels compared to $d_{x^{2}-y^{2}}$ and
$d_{z}^{2}$ orbitals, referred to as the eg state. Furthermore, below the
E${}_{\mbox{F}}$, a pronounced hybridization occurs between the S-3px and
S-3py orbitals with the Ni-3$d_{xy}$, $d_{x^{2}-y^{2}}$, and $d_{z}^{2}$
orbitals in the energy range spanning from 0 to -0.035 eV. Conversely, above
E${}_{\mbox{F}}$, S-3px and S-3py orbitals exhibit significant hybridization
with the Ni-3$d_{xz}$ and Ni-3$d_{yz}$ orbitals within the energy range from
0.7 to 1.2 eV (see Fig. 2(c-e)).

Additionally, the kagome lattice of Ni2+ leads to the formation of flat bands
below E${}_{\mbox{F}}$, which is mainly contributed by 3dxy orbital of Ni
atom. Theoretical studies have suggested that such flat band systems in a
kagome lattice can give rise to various exotic many-body phenomena [20, 16,
21, 22, 23]. Based on the analysis of the electronic band structure, Rb2Ni3S4
exhibit semiconducting behavior with an energy band gap of $\sim$0.67 eV.
These findings are consistent with the previously reported DFT value of 0.66
eV [24] and the experimental value of 0.8 eV deduced from a transport study
[16]. The nature of the band gap is indirect. This is attributed to the
occurrence of the VBM at the high symmetry point X and the CBM at the T-Z line
in the momentum space (see Fig. 3).

Figure 3: Electronic band structure of Rb2Ni3S4. The horizontal solid line at
zero energy denotes E${}_{\mbox{F}}$.

###  3.2 Optical properties

Figure 4: Optical properties of Rb2Ni3S4. (a) real part of the dielectric
function $\epsilon_{1}$$(\omega)$. (b) Imaginary part of the dielectric
function $\epsilon_{2}$$(\omega)$. (c) Energy loss spectrum L$(\omega)$. (d)
Optical conductivity $\sigma$$(\omega)$ [in Ohm-1cm-1].

The linear response of the system to electromagnetic radiation, which is
associated to the interaction of photons with electrons, can be defined using
the complex dielectric function, $\epsilon(\omega)$.

$\epsilon(\omega)=\epsilon_{1}(\omega)+i\epsilon_{2}(\omega)$ (3.1)

where, $\epsilon_{1}$ and $\epsilon_{2}$ are the real and imaginary part of
the dielectric function, respectively [25, 26].

In accordance with the selection criteria, the momentum matrix elements
between the occupied and unoccupied state are used to calculate the imaginary
component of the dielectric function, which is given by the following equation
[27, 28],

$\epsilon_{2}(\omega)=\frac{Ve^{2}}{2\pi hm^{2}\omega^{2}}\int
d^{3}k\sum|<kg|p|kg^{\prime}>|^{2}$

$\times f(kg)[1-f(kg^{\prime})]\delta(E_{kg}-E_{kg^{\prime}}-\hbar\omega)$
(3.2)

where p is the momentum operator. The Fermi distribution function is indicated
by f(kg), and Ekg is the eigenvalue associated with eigenfunction $|kg>$. The
real part is given by Kramers-Kronig relationship [29],

$\epsilon_{1}(\omega)=1+\frac{2}{\pi}\int\frac{\epsilon_{2}(\omega^{\prime})\omega^{\prime}}{(\omega^{\prime})^{2}-\omega^{2}}d\omega^{\prime}$
(3.3)

The optical conductivity and electron loss function is given by following the
equation,

$\sigma(\omega)=\frac{-i\omega(\epsilon_{2}(\omega)-1)}{4\pi}$ (3.4)

$L(\omega)=\frac{\epsilon_{2}}{\epsilon_{1}^{2}(\omega)+\epsilon_{2}^{2}(\omega)}$
(3.5)

As already observed from electronic structure calculations, Rb2Ni3S4 exhibits
a semiconducting nature with a band gap of $\sim$0.67 eV. Thus when exposed to
incident photons, transitions can be possible at arbitrarily high photon
energy. In such transitions, photons excite electrons from occupied states in
the valence band region to unoccupied states in the conduction band region
known as interband transition [30]. To understand these phenomena, to study
the underlying optical properties, we consider the polarization components
along the x, y, and z directions due to an orthorhombic crystal structure. To
understand the electronic polarizability of the material under an applied
electric field, we analyze the real part of the dielectric function
$\epsilon_{1}(\omega)$ [31], as depicted in Fig. 4(a). $\epsilon_{1}(\omega)$
along x, y and z direction denoted by $\epsilon_{1xx}$, $\epsilon_{1yy}$, and
$\epsilon_{1zz}$, respectively. The real part of the dielectric function at
zero frequency, (which is connected to the material’s band gap is known as
static dielectric constants $\epsilon_{1}$(0)) are found to be $\sim$7.5 along
both the x and y directions, while it is $\sim$6.2 along the z direction.
Correspondingly, the refractive index was determined using the established
Penn Relation n(0) = $\sqrt{\epsilon_{1}(0)}$ [32] gives the static refractive
index as 2.74 along x and y direction while 2.49 along z direction. The
obtained value shows that the material has optical anisotropy. In the energy
range 0 to 1.3 eV for the x, y direction, and 0 to 1 eV for the z direction,
we observe a gradual increase in $\epsilon_{1}(\omega)$, which indicates that
the material interaction effectively with photons. Notably, at 1.63 eV, 1.60
eV, and 1.16 eV for $\epsilon_{1xx}$, $\epsilon_{1yy}$, and $\epsilon_{1zz}$,
respectively, sharp peaks are observed. These peaks correspond to the inter-
band transition, indicating the photon-induced electron transitions from
occupied valence bands to the unoccupied conduction bands. This means that the
material can absorb light particularly well at that specific energy. Within
the photon energy range exceeding 1.16 eV, the real part of the dielectric
function $\epsilon_{1}(\omega)$ exhibits a remarkable characteristic: it
adopts a negative value. This phenomenon is indicative of incident light being
reflected by the medium. Consequently, the material demonstrates metallic
attributes in this specific energy range [33, 34]. Furthermore, we also
observe sharp peaks at approximately 2.01 eV, 1.96 eV, and 3.93 eV for x, y
and z direction, respectively. $\epsilon_{2}(\omega)$, the imaginary part of
the dielectric function, reveals details about optical absorption in crystal.
The absorption begins at 0.84 eV for the imaginary part of the dielectric
function, which act as the threshold energy associated with the occurrence of
an optical band gap (see Fig. 4(b)). Beyond the threshold point, curve
increases rapidly. Notably, sharp peaks are seen along the x, y, and z
directions, respectively, at 2.02 eV, 1.97 eV, and 1.22 eV. These absorption
peaks may clearly be associated with transitions from the Ni(3d) valence band
to the S(3p) conduction band based on the predicted PDOS (see Fig. 2(b)).
Additionally, within the energy range of 2.83 eV to 6.05 eV, various
significant peaks are observed. These peaks correspond to inter-band
transitions from the valence band to the conduction band [35, 36]. However,
beyond the energy of 6.05 eV, the peaks begin to diminish, eventually
disappearing at higher energy ranges. Another important optical property
calculated in this work is the electron energy loss function L($\omega$). Fig
4(c) shows the variation of L($\omega$) with photon energy. The L($\omega$)
provides the information about the energy loss of a fast-moving electron
traversing in the material, and is reciprocal of the imaginary part of the
complex dielectric function [37, 38]. The visible area of the loss spectrum
exhibits a slight energy loss, which rises as photon energy increases. The
identified peaks in the L($\omega$) spectra provide information about the
material’s plasma frequency. The nature of the material, i.e., whether it
exhibits metallic properties or dielectric behavior, is known to be described
by the plasma frequency. It is well known that materials exhibit metallic
[$\epsilon_{1}(\omega)$ $\textless$ 0] behavior below the plasma resonance
frequency and dielectric [$\epsilon_{1}(\omega)$$\textgreater$ 0] behavior
above this frequency [34, 39, 40, 41]. The maximal peaks for Lxx, Lyy, and Lzz
are clearly visible in the energy loss spectra at approximately 6.44 eV, 6.41
eV, and 4.91 eV, respectively. The resonant energy loss is seen at 6.44 eV
along x direction with a maximum value of 0.8. After these peaks, the spectrum
starts to decline. In Fig. 4(d), show the optical conductivity which depend on
photon energy. This indicating that the Rb2Ni3S4 exhibits optical activity in
the range from 1.25 eV to 8.60 eV. The $\sigma(\omega)$ edge starts 0.84 eV,
which corresponds to the calculated optical band gap of the material, and
numerous peaks are observed in the spectrum. Maximum optical conductivity,
$\sigma_{xx}$, $\sigma_{yy}$ and $\sigma_{zz}$ was found to be 6.5$\times$103
Ohm-1cm-1, 16$\times$103 Ohm-1cm-1 and 5.9$\times$103 Ohm-1cm-1, respectively
at photon energy of 2.02 eV to 4.05 eV. These findings are consistent with the
imaginary part of the dielectric function, results obtained in Fig. 4(b).

##  4 Conclusions

We have investigated the electronic structure and optical properties of
Rb2Ni3S4 by means of density functional theory approach. The studied material
is found to be nonmagnetic semiconductor with an indirect band gap of
$\sim$0.67 eV. The Ni atoms form a kagome lattice in a two-dimensional plane
resulting in a flat band located below the Fermi energy. From the optical
properties calculations, the dielectric function including its real and
imaginary parts, the loss function, and the optical conductivity has been
obtained. We found that the material is optically active in the visible and
lower ultraviolet region. We predicti that Rb2Ni3S4 could be a promising
candidate for optoelectronic device fabrication.

## Acknowledgments

M.P.G. was supported by a grant from UNESCO-TWAS and the Swedish International
Development Cooperation Agency (SIDA). The views expressed herein do not
necessarily represent those of UNESCO-TWAS, SIDA or its Board of Governors.
M.P.G. acknowledges IFW-Dresden for providing the large-scale compute nodes to
Tribhuvan University for scientific computations. G.B.A. thanks Nepal Academy
of Science and Technology (NAST) for the PhD fellowship. The author thanks
Manuel Richter, IFW-Dresden for careful reading of the manuscripts and
suggestions. M.P.G. and G.B.A. thanks Ulrike Nitzsche for the technical
assistance.

## References

  * [1] Bronger W, Rennau R and Schmitz D 1991 Z. fur Anorg. Allg. Chem. 597 27–32
  * [2] Kang M, Ye L, Fang S, You J S, Levitan A, Han M, Facio J I, Jozwiak C, Bostwick A, Rotenberg E et al. 2020 Nat. Mater. 19 163–169
  * [3] Chen D, Le C, Fu C, Lin H, Schnelle W, Sun Y and Felser C 2021 Phys. Rev. B 103 144410
  * [4] Liu E, Sun Y, Kumar N, Muechler L, Sun A, Jiao L, Yang S Y, Liu D, Liang A, Xu Q et al. 2018 Nat. Phys. 14 1125–1131
  * [5] Ghimire M P, Facio J I, You J S, Ye L, Checkelsky J G, Fang S, Kaxiras E, Richter M and Van Den Brink J 2019 Phys. Rev. Res. 1 032044
  * [6] Yang H, Sun Y, Zhang Y, Shi W J, Parkin S S and Yan B 2017 New J. Phys. 19 015008
  * [7] Kübler J and Felser C 2018 Europhys. Lett. 120 47002
  * [8] Liu J and Balents L 2017 Phys. Rev. Lett. 119 087202
  * [9] Ren Z, Li H, Sharma S, Bhattarai D, Zhao H, Rachmilowitz B, Bahrami F, Tafti F, Fang S, Ghimire M P et al. 2022 npj Quantum Mater. 7 109
  * [10] Naher M, Ali M A, Hossain M, Uddin M and Naqib S 2023 arXiv preprint arXiv:2304.07669
  * [11] Li Q, Wu Y, Fan X, Zhang Y J, Zhu X, Zhu Z, Li Y and Wen H H 2022 Phys. Rev. B 106 214501
  * [12] Elder S H, Jobic S, Brec R, Gelabert M and DiSalvo F J 1996 J. Alloys Compd. 235 135–142
  * [13] Acharya G B, Belbase B P and Ghimire M P 2023 Electronic Structure URL :DOI 10.1088/2516-1075/ad0951
  * [14] Bronger W, Eyck J, Rüdorff W and Stöussel A 1970 Z. fur Anorg. Allg. Chem. 375 1–7
  * [15] Fukamachi T, Kobayashi Y, Nakamura A, Harashina H and Sato M 1999 J. Phys. Soc. Japan 68 3668–3672
  * [16] Hondou K, Fujiwara Y, Kato T, Iio K, Saiki A, Usuda M and Hamada N 2002 J. Alloys Compd. 333 274–281
  * [17] Nawai S, Okazaki K, Mizokawa T, Fujimori A, Hondou K, Fujiwara Y, Iio K, Usuda M and Hamada N 2004 Phys. Rev. B 69 045103
  * [18] Koepernik K and Eschrig H 1999 Phys. Rev. B 59 1743
  * [19] Perdew J P, Burke K and Ernzerhof M 1996 Phys. Rev. Lett. 77 3865
  * [20] Hondou K and Chikazawa S 2007 J. Magn. Magn. Mater. 310 1815–1817
  * [21] Kang M, Fang S, Ye L, Po H C, Denlinger J, Jozwiak C, Bostwick A, Rotenberg E, Kaxiras E, Checkelsky J G et al. 2020 Nat. Commun. 11 1–9
  * [22] Wang W S, Li Z Z, Xiang Y Y and Wang Q H 2013 Phys. Rev. B 87 115135
  * [23] Yu S L and Li J X 2012 Phys. Rev. B 85 144402
  * [24] Nawai S, Okazaki K, Mizokawa T, Fujimori A, Hondou K, Fujiwara Y, Iio K, Usuda M and Hamada N 2004 Phys. Rev. B 69(4) 045103
  * [25] He H, Orlando R, Blanco M A, Pandey R, Amzallag E, Baraille I and Rérat M 2006 Phys. Rev. B 74 195123
  * [26] Ghosh A and Thangavel R 2017 Ind. J. Phys. 91 1339–1344
  * [27] Ambrosch-Draxl C and Sofo J O 2006 Comput. Phys. Commun. 175 1–14
  * [28] Hoat D 2019 J. Mol. Graph. 90 153–160
  * [29] Toll J S 1956 Phys. Rev. 104 1760
  * [30] Stepanjuga A, Ray R, Richter M, Carrocci S, Hampel S, Galle L, Grafe H J and Valldor M 2022 New J. Chem. 46 4900–4910
  * [31] Hoat D, Silva J R and Blas A M 2019 J. Solid State Chem. 270 85–91
  * [32] Penn D R 1962 Phys. Rev. 128 2093
  * [33] Shore A 2019 RSC Adv. 9 12813–12813
  * [34] Kumar M, Raj A, Kumar A, Sharma S, Bherwani H, Gupta A and Anshul A 2021 Optik 242 166764
  * [35] Kumar A and Ahluwalia P 2012 Mater. Chem. Phys. 135 755–761
  * [36] Belbase B P, Jaishi D R, Adhikari R P and Ghimire M P 2023 Solid State Commun. 361 115085
  * [37] Wang C and Klein B 1981 Phys. Rev. B 24 3417
  * [38] Zhang X, Chen Z, Zhang S, Liu R, Zong H, Jing Q, Li G, Ma M and Wang W 2007 J. Condens. Matter Phys. 19 425231
  * [39] Hu J M, Huang S P, Xie Z, Hu H and Cheng W D 2007 J. Condens. Matter Phys. 19 496215
  * [40] Kumar P, Soni A, Bhamu K and Sahariya J 2017 Mater. Res. Bull. 86 131–138
  * [41] Sun J, Zhou X F, Fan Y X, Chen J, Wang H T, Guo X, He J and Tian Y 2006 Phys. Rev. B 73 045108


# Instructions for *ACL Proceedings

Kung-Hsiang Huang$\spadesuit$ Kathleen McKeown$\clubsuit$
Preslav Nakov$\diamondsuit$ Yejin Choi$\heartsuit\vardiamondsuit$ Heng
Ji$\spadesuit$
$\spadesuit$ University of Illinois Urbana-Champaign $\clubsuit$ Columbia
University
$\diamondsuit$ Hamad Bin Khalifa University $\heartsuit$ University of
Washington $\vardiamondsuit$Allen Institute for AI
{khhuang3<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

# Faking Fake News for Real Fake News Detection:
Propaganda-loaded Training Data Generation

Kung-Hsiang Huang$\spadesuit$ Kathleen McKeown$\clubsuit$
Preslav Nakov$\diamondsuit$ Yejin Choi$\heartsuit\vardiamondsuit$ Heng
Ji$\spadesuit$
$\spadesuit$ University of Illinois Urbana-Champaign $\clubsuit$ Columbia
University
$\diamondsuit$ Hamad Bin Khalifa University $\heartsuit$ University of
Washington $\vardiamondsuit$Allen Institute for AI
{khhuang3<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

While there has been a lot of research and many recent advances in neural fake
news detection, defending against human-written disinformation remains
underexplored. Upon analyzing current approaches for fake news generation and
human-crafted articles, we found that there is a gap between them, which can
explain the poor performance on detecting human-written fake news for
detectors trained on automatically generated data. To address this issue, we
propose a novel framework for generating articles closer to human-written
ones. Specifically, we perform self-critical sequence training with natural
language inference to ensure the validity of the generated articles. We then
explicitly incorporate propaganda techniques into the generated articles to
mimic how humans craft fake news. Eventually, we create a fake news detection
training dataset, PropaNews, which includes 2,256 examples. Our experimental
results show that detectors trained on PropaNews are 7.3% to 12.0% more
accurate for detecting human-written disinformation than for counterparts
trained on data generated by state-of-the-art approaches.111The dataset and
the code will be made publicly available.

##  1 Introduction

The dissemination of falsified information can cause chaos, hatred, and trust
issues among humans, and can eventually hinder the development of society
Dewatana and Adillah (2021); Wasserman and Madrid-Morales (2019). In
particular, human-written disinformation222There are many types of fake news.
We focus on text-only disinformation in this paper., which is often used to
manipulate certain populations, had catastrophic impact on multiple events,
such as the 2016 US Presidential Election Grinberg et al. (2019), Brexit
Bastos and Mercea (2019), the COVID-19 pandemic van Der Linden et al. (2020),
and the recent Russia’s assault on Ukraine. Hence, we are in urgent need of a
defending mechanism against human-written disinformation.333WARNING: This
paper contains disinformation that may be sensitive or offensive in nature.

To construct such a mechanism, we need a substantial amount of training data
to train the detectors. A naïve solution is to collect human-written news
articles that contain inaccurate information by crawling untrustworthy news
media and to recruit human annotators to label the veracity of each article.
Unfortunately, news articles published by suspicious sources do not
necessarily contain false information, which means that the annotators are
required to perform fact-checking for every claim in each untrustworthy
article. Additionally, these news outlets are often removed shortly after
having been posted. Hence, such a solution is neither scalable nor reliable. A
more appealing approach is to generate training data automatically that avoid
these issues while enjoying the benefit of able to produc update-to-date
disinformation.

Our goal here is to further advance the field of disinformation detection by
generating a dataset that is closer to human-written disinformation, so that
automatic detectors trained on it are more robust at detecting disinformation
in human-written articles. We started by collecting human-written
disinformative articles from untrustworthy sites444These new sources are rated
low on factuality of reporting by mediabiasfactcheck.com, and we analyzed
around 40 of them that contain falsified information. Throughout our analysis,
we found two characteristics of these human-written disinformation. First,
about 33% of the articles use propaganda techniques to convince the audience
that the fake information is authentic. These techniques usually involve the
use of emotion-triggering terms, appealing to authorities or logical fallacies
Da San Martino et al. (2019) for increasing the credibility of the article.
For instance, there is strong evidence showing that information is more
persuasive and appears to be more true when it comes from sources perceived to
be credible Nadarevic et al. (2020), especially non-media sources such as
authoritative individuals and domain experts Nadarevic et al. (2020); Walter
and Tukachinsky (2020). Second, more than 55% of the articles that we analyzed
contain inaccurate information mixed with correct information. Essentially,
all claims, except for one or two, in these disinformation articles were
factual. Thus, the fact that the majority of the claims in these articles are
real makes these few false claims in these articles even more believable.Prior
work has made great progress in generating fake news. For instance, Zellers et
al. (2019) pre-trained a sequence-to-sequence (seq2seq) model, while Fung et
al. (2021) presented a finer-grained generator that conditions on perturbed
knowledge elements. A major issue with these approaches is that the vast
majority of the contents in the articles they generate is inaccurate, which is
in stark contrast with our observations about human-written disinformation,
where accurate information accounts for a much greater proportion.

AJDABIYAH , Libya | Thu Apr 7 , 2011 6:34 pm EDT AJDABIYAH , Libya -LRB- Reuters -RRB- - Rebels fighting to overthrow Muammar Gaddafi said five of their fighters were killed when NATO planes mistakenly bombed a rebel tank column near the contested port of Brega in eastern Libya . In Washington , the head of U.S. Africa command told a Senate hearing the United States should not provide arms to the rebels without a better idea of who they were . Asked if there was an emerging stalemate in the seven-week-old conflict , General Carter Ham replied : “ I would agree with that at present , on the ground . ” In rebel-held eastern Libya , wounded rebels being brought to a hospital Ajdabiyah said their trucks and tanks were hit on Thursday by a NATO air strike outside Brega. NATO said it was investigating an attack by its aircraft on a tank column in the area along the Mediterranean coast on Thursday , saying the situation was “ unclear and fluid . ” Rebels said at least five of their fighters were killed when NATO planes mistakenly bombed a rebel tank column near the contested port. “A number of vehicles were hit by a NATO strike ”, officers from UN concluded. The fighting for Brega , the only active front , has dragged on for a week and has entered a daily pattern of advances back and forth with neither side making major gains .
---

Table 1: An example of our generated fake news. Our approach first identifies
a salient sentence in a given authentic news article (the strikethrough
texts). Then, we generate a plausible but disinformative sentence that is
coherent to the context ( texts in orange). Finally, we generate propaganda to
make it resemble human-written fake news ( texts in blue)

.

Moreover, the articles generated using these methods do not explicitly use
propaganda techniques, which are common in human-written disinformation Da San
Martino et al. (2019).

Given an authentic news article, we replace a salient sentence with a
plausible but fake information using a sequence-to-sequence (seq2seq) language
model. One issue is that if the generated sentence can be entailed by the
replaced sentence, the generated sentence is actually accurate, which is not
desirable. To penalize such an entailment, we fine tune the seq2seq model with
a self-critical sequence training objective Rennie et al. (2017) that uses a
natural inference (NLI) model in addition to maximum likelihood estimation.
Additionally, as a post-processing step, we further use the NLI model to
filter out generated sentences that can be inferred from the replaced
sentences. Then, we add propaganda techniques into the generated
disinformation for mimicking how humans craft disinformation. In particular,
we adopt two commonly used propaganda techniques: appeal to authority and
loaded language Da San Martino et al. (2019). The entire generation process is
performed automatically to generate silver-standard training data for the
detector. An example is shown in Table 1. For comparison, we recruited
crowdsourcing workers to correct grammatical errors and to validate that some
of the generated texts are indeed fake to construct a gold-standard training
dataset. Compared to the aforementioned annotation process, this data
generation approach is much more viable as the annotators only need to fact-
check the generated texts.

To evaluate the detector’s performance on human-written disinformation, we
also collected 200 human-written articles (HumanNews), with a balanced portion
of real and fake articles. We compare our generation method with other state-
of-the-art fake generation approaches, such as Zellers et al. (2019) and Shu
et al. (2021). The results show that detectors are significantly better at
detecting human-written disinformation when trained on our generated dataset.
Our ablation studies further confirm the effectiveness of incorporating
propaganda into the generated articles in identifying human-crafted
disinformation.

Our contributions can be summarized as follows:

  * •

We propose an effective method to automatically generate more realistic
disinformation compared to prior work.

  * •

We incorporate self-critical sequence training with an NLI model into the
learning objectives of the generator to discourage generating sentences that
can be entailed from the original context.

  * •

We develop the first automatic methods for generating specific propaganda
techniques such that the generated articles are closer to human-written
disinformation.

  * •

We demonstrate that detectors trained on our generated data, compared to
generated articles using other methods, are better in detecting human-written
disinformation.

  * •

We release two disinformation detection datasets. (1) PropaNews: 2.2K articles
generated using our approach and validated by humans. (2) HumanNews: 200
human-written articles for evaluation in a real-world setting.

##  2 Training Data Generation

Our process of generating training data for propaganda-loaded disinformation
consists of two main steps. First, we perform disinformation generation by
replacing a salient sentence in an authentic news article with a plausible but
inaccurate sentence (Section 2.1). Second, we incorporate propaganda
techniques, including appeal to authority and loaded language, into the
generated sentence (Section 2.2). Below, we describe each of these steps in
detail.

Figure 1: Illustration of our self-critical sequence training. Given a
corrupted input article $\tilde{x}$, BART generates two sequences with Nucleus
sampling and greedy decoding, respectively. The reward for each sequence is
computed as the negative entailment probability $-P_{ent}$ as output from the
NLI model.

###  2.1 Disinformation Generation

Our disinformation generation algorithm aims at two sub-goals: (_i_) replacing
a salient sentence in the given article with a sequence of generated coherent
texts that look plausible, and (_ii_) ensuring that the generated information
cannot be entailed by the original masked-out sentence; otherwise, the
generated texts will not be disinformative. To achieve the first sub-goal, we
first identify salient sentences using an extractive summarization model, and
we then perform mask infilling with BART Lewis et al. (2020), a pre-trained
seq2seq architecture that has demonstrated strong performance in multiple text
generation tasks. The second sub-goal is accomplished using self-critical
sequence training Rennie et al. (2017); Bosselut et al. (2018) with a Natural
Language Inference (NLI) component, which is used as a reward function for
generation that meets this sub-goal.

#### Salient Sentence Identification

A salient sentence is critical for the overall semantics of the article. When
a salient sentence is manipulated or replaced, the news complex event
described in the article may be drastically changed. Yet, there is no salient
sentence identification dataset publicly available. Motivated by the fact that
sentences included in an extractive summary are often of higher importance, we
take the scores computed by an extractive summarization model Liu and Lapata
(2019), which predicts how likely each sentence is to belong to the summary,
to estimate the saliency of each sentence. Empirically, we found that this
approach provides a reasonably good estimation for salient sentence
identification. For each news outlet, we replace one sentence that has the
highest extractive summarization score with our generated disinformation.

#### Mask Infilling with BART

To perform infilling, we take an approach that is similar to that of Donahue
et al. (2020), but we use BART Lewis et al. (2020), a pre-trained language
model with an encoder–decoder architecture. During training time, we randomly
mask out a sentence $y^{*}$ from a given article $x$. The bidirectional
encoder first produces contextualized representations
$\bm{h_{e}}=\mathrm{Encoder}(\tilde{x})$ given the article with a masked-out
sentence $\tilde{x}=x-y^{*}$. Then, the auto-regressive decoder learns a
maximum likelihood estimation that aims to maximize the probability of
generating the next token $y^{*}_{t}$ at time step $t$ given all tokens in
previous time steps $\\{y^{*}_{0},...,y^{*}_{t-1}\\}$ and the encoder hidden
states $\bm{h_{e}}$ by minimizing the negative log probability of generating
$y^{*}_{t}$ as follows:

$\displaystyle\mathcal{L}_{m}=-\sum^{T}_{t=1}\log
P(y^{*}_{t}|y^{*}_{0},...,y^{*}_{t-1},\bm{h_{e}})$ (1)

During inference time, rather than random masking, $\tilde{x}$ is formed by
masking out the sentence with the highest score computed by the extractive
summarization model given the original document $x$, as discussed in the
previous paragraph.

#### Self-critical Sequence Training

BART optimized via maximum likelihood estimation alone is capable of
generating coherent texts. However, although the generated texts $y^{\prime}$
may be very different from the originally masked out sentence $y^{*}$, there
is no guarantee that $y^{\prime}$ contains incorrect information. If the
generated texts $y^{\prime}$ can be entailed by the masked out sentence
$y^{*}$, then $y^{\prime}$ is actually not disinformative. An example is shown
in Figure 2. Here, except for the lack of details, the generated sentence
$y^{\prime}$ delivers the same message as the masked out sentence $y^{*}$. To
reduce the probability that $y^{\prime}$ can be entailed by $y^{*}$, we
leverage self-critical sequence training Rennie et al. (2017); Bosselut et al.
(2018) that rewards the model for generating sequences that cannot be entailed
by the masked-out sentences.

Figure 2: An example showing the NLI model predicts an entailment from the
masked out sentence $y^{*}$ to the generated sentence $y^{\prime}$.

Self-critical sequence training (SCST) is a form of the REINFORCE algorithm
Williams (1992) that allows for direct optimization on non-differentiable
functions. Utilizing a baseline output $y^{\prime\prime}$ of the model itself
to normalize the rewards, SCST avoids the challenge of directly estimating the
reward signal or estimating normalization Rennie et al. (2017). Since our goal
is to avoid entailment from $y^{*}$ to $y^{\prime}$, we define the reward as
the negative entailment probability produced by a RoBERTa-based Liu et al.
(2019) NLI model fine-tuned on Multi-NLI Williams et al. (2018)555We use the
fine-tuned NLI model from https://huggingface.co/roberta-large-mnli. Its
accuracy is 90.2% on the dev set of MNLI, which is on par with state-of-the-
art methods.,

$\displaystyle r(y^{\prime})=-P_{nli}(y^{*},y^{\prime}),$ (2)

where $r(y^{\prime})$ denotes the reward of the sequence sampled from the
current policy $y^{\prime}$, and $P_{nli}(y^{*},y^{\prime})$ is the
probability that $y^{*}$ entails $y^{\prime}$. To generate $y^{\prime}$, we
use Nucleus Sampling Holtzman et al. (2020) with $p=0.96$ since such sampling
method has demonstrated advantages over top-k sampling in open-ended
generation Holtzman et al. (2020); Zellers et al. (2019). Specifically, at
each time step, we sample each token from the most probable set of words whose
cumulative probability comprises at least $p$ of the total vocabulary
distribution. As for generating the baseline output $y^{\prime\prime}$, we use
greedy decoding by taking argmax at every timestep. Upon generating
$y^{\prime}$ and $y^{\prime\prime}$, we obtain the entailment probabilities of
both sequences from the NLI model, and then compute the self-critical sequence
training loss:

$\displaystyle\mathcal{L}_{s}=-(r(y^{\prime})-r(y^{\prime\prime}))\sum^{T}_{t=1}\log
P(y^{\prime}_{t}|y^{\prime}_{0},..,y^{\prime}_{t-1},\bm{h_{e}}).$ (3)

$r(y^{\prime\prime})$ here can be viewed as a baseline reward, and
$r(y^{\prime})-r(y^{\prime\prime})$ can be regarded as a normalized reward.
This loss function encourages BART to generate $y^{\prime}$ when
$r(y^{\prime})>r(y^{\prime\prime})$, whereas suppresses the probability of
decoding $y^{\prime}$ when $r(y^{\prime})<r(y^{\prime\prime})$. An overview of
SCST is shown in Figure 1.

The final objective function that BART learns to minimize is a weighted sum of
Equation 1 and Equation 3,

$\displaystyle\mathcal{L}_{final}=\alpha\mathcal{L}_{m}+\beta\mathcal{L}_{s},$
(4)

where $\alpha$ and $\beta$ are the weights for each loss666Empirically, we set
$\alpha=1$ and $\beta=0.01$..

#### Post-processing

To further ensure the quality of the disinformation generated, we reuse the
NLI model discussed in the previous paragraph to filter out invalid outputs
$y^{\prime}$ that can be intailed from the masked-out sentence $y^{*}$, as
demonstrated in Figure 2. We observe that the incorporation of the SCST loss
(Equation 3) into the training objective successfully reduces the invalid rate
from 7.8% to 3.2%.

Technique |  Generated Disinformation and Propaganda
---|---
Appeal to Authority |  Cairo’s Tahrir Square was the scene of clashes between protesters and police on Wednesday. “ At least three people were killed and more than 600 were injured in the clashes, ” said Egypt’s President.
Loaded Language |  Cairo’s Tahrir Square was the scene of deadly clashes between protesters and police on Wednesday.
Table 2: Examples of the generated propaganda using two techniques, as shown
by texts in blue. Given the same disinformation generated (i.e. the context),
the first row demonstrates how loaded language is introduced by inserting an
emotion-triggering term, while the second row shows how the argument is
strengthened with reference to a statement made by authorities.

###  2.2 Propaganda Generation

After generating inaccurate information, we then incorporate propaganda into
each generated article. In this work, we focus on generating two types of
commonly used propaganda techniques, loaded language and appeal to authority
Da San Martino et al. (2019, 2020).

#### Appeal to Authority

Appeal to authority is a propaganda technique that attempts to strengthen or
invalidate an argument by referring to a statement made by authorities or
experts Da San Martino et al. (2019). These statements can be incorrect in two
ways: (1) the statement itself is inaccurate, or (2) the authority never made
such a statement. In this work, we focus on the latter case. For each article,
we collect authority candidates $Z$ from two sources, domain experts collected
from Wikidata and entities gathered from the input article. With the Wikidata
Query Service777https://query.wikidata.org/, we collect experts from various
domains, including economists, biologists, immunologists, and decision makers.
Particularly, we specify the occupation (P108) of each expert and filter out
entities that were born before 1940 to ensure recency. To consider only the
impactful entities, we rank all the candidates based on the number of
corresponding outcoming statements (i.e. connected concepts in Wikidata),
inspired by PageRank Page et al. (1999), and select the top 100 entities for
each occupation into the candidate $Z$. Additionally, authorities that are
more relevant to the local context are also considered. We include the person
named entities extracted by a name
tagger888https://stanfordnlp.github.io/stanza/ner.html into the list of
authority candidates based on the our findings that more than 73% of the news
articles contain authorities. More details about how authority candidates $Z$
are collected can be found in Appendix B.

Once we collect a list of authority candidates $Z$, we can generate fake
arguments made by each $z_{i}\in Z$ with the BART model that has already been
fine-tuned in Section 2.1. In specific, a “<mask>” token is inserted right
after the filled-in sentence $y^{\prime}$ in the input article to BART so that
it knows where to perform infilling. To inform BART that it should generate a
statement made by one of the authorities, we prefix the decoder with a
template such as [$z_{i}$ confirmed that “”], where $z_{i}\in Z$ is the name
of the authority. The prefix ends with an opening quotation mark to indicate
that it should be followed by a statement made by authority $z_{i}$.
Furthermore, to increase the diversity of the generated statements, we devise
a variety of templates, as detailed in Appendix B. Finally, the best sequence
$s^{*}$ is selected with the lowest perplexity
$s^{*}=\operatorname*{argmin}_{s_{i}}\textrm{Perplexity}(s_{i})$ where $s_{i}$
denotes the generated sequence using $z_{i}$ as the authority.

#### Loaded Language

Loaded language is another propaganda technique that utilizes emotion-
triggering terms or phrases to influence audiences’ opinions Da San Martino et
al. (2019); Dimitrov et al. (2021). Often times, loaded language involves the
use of sensational adverbs or adjectives to exaggerate a statement. Based on
this observation, we utilize the propaganda dataset released by Da San Martino
et al. (2019) where propaganda techniques are annotated at the fragment level
(i.e. span level). The dataset contains 2,547 loaded language instances. Yet,
not every instance contains adjectives or adverbs that are emotion-triggering.
To create valid training data for loaded language generation, we first use
SpaCy999https://spacy.io/ to perform part of speech tagging and dependency
parsing, and then keep the examples where there exists an adverb pointing to a
verb or an adjective pointing to a noun through dependency parsing edges. This
results in 1,017 samples of valid loaded language instances. Some generated
examples for appeal to authority and loaded language are shown in Table 2.

Upon collecting the training data for generating loaded language, we fine-tune
another BART Lewis et al. (2020) on this dataset. A naive method would be
taking the article with emotion-triggering adverbs or adjectives removed as
input to BART and using the original article as the decoding target. However,
we found that around 25% of the time BART does not exactly reproduce the
unmasked texts due to hallucination. This observation is consistent with
Donahue et al. (2020)’s findings. To this end, we propose a two-step
generation approach. First, we train BART to insert a “<mask>” token into the
target sentence in the input document that is marked with special tokens.
Then, BART learns to infill the “<mask>” with the similar approach as
discussed in section 2.1 but without the SCST objective. Empirically, we found
such approach successfully reduces the chance of failure in generating the
exact unmasked contexts to around 2%.

###  2.3 Intermediate Pre-training

Directly fine-tuning BART on these two datasets does not yield satisfactory
results as the size of Timeline17 Tran et al. (2013) and the propaganda
dataset Da San Martino et al. (2019) are relatively small. To enhance the
domain adaptation capabilities on small datasets, we perform intermediate pre-
training (IPT) on the source news articles in the CNN/DM dataset Hermann et
al. (2015), a large summarization corpus containing more than 280K news
articles from CNN and Daily Mail. The overall IPT objectives for
disinformation generation and propaganda generation are mostly the same as
described in Section 2.1 and Section 2.2, but with some minor changes due to
different goals in the IPT phase. When performing IPT for disinformation
generation, we remove $\mathcal{L}_{s}$ from the final loss function (Equation
4) as the goal for IPT is only to learn to generate coherent sentences. On the
other hand, the objective for loaded language IPT is to enable BART to
identify where to insert which adjectives or adverbs. Hence, we create
training samples by gathering all appearances of adjectives pointing to a noun
or adverbs pointing to a verb via dependency parsing graphs.

##  3 The PropaNews Dataset

Using the generation method discussed in Section 2, we produce a set of
realistic-looking disinformative articles for our proposed PropaNews dataset.
To create a gold-standard training set for comparison, we hire human workers
to validate that the generated articles are indeed disinformative. The details
of this dataset are discussed in the following sections.

###  3.1 Data Source

When selecting the source of data to construct our dataset, we consider the
following two criteria. First, the news articles must have high
trustworthiness. This ensures that, except for our manipulated sentences, the
rest of the articles are genuine. Second, the news events described in the
articles must be important. If the disinformation presented in the articles is
not important, such as gossip news, then these disinformative articles are not
good subjects for studying disinformation detection. Motivated by these two
criteria, we repurpose the Timeline17 dataset Tran et al. (2013) as our source
of data. Timeline17 contains 17 timelines, each of which corresponds to a news
event. Each timeline is associated with a series of news articles that span
across a wide time span, implying the high importance and impact of these news
events. Additionally, the news articles are from trustworthy media, such as
The New York Times and The Guardian. In total, there are 4,535 news articles
in Timeline17.

###  3.2 Crowdsourcing

We use Amazon’s Mechanical Turk (AMT) to verify the quality and correctness of
the generated disinformation. In total, there are around 400 unique
crowdsourcing workers contributing to approximately 2,000 Human Intelligence
Tasks (HITs). For each HIT, annotators are tasked to look for supporting
evidence from trustworthy news media to determine whether the generated
sentences are indeed inaccurate. Only those labeled as inaccurate will be
included in PropaNews, while the accurate counterparts are discarded. Appendix
C provides the details of the annotation interface.

To measure the inter-annotator agreement (IAA), we use the Worker Agreement
With Aggregate (WAWA) score, following Ning et al. (2020) and Sheng et al.
(2021)101010We did not use other IAA metrics, such as Cohen’s Kappa Cohen
(1960), as we expect vast majority of our generated disinformation to be
inaccurate. WAWA provides a better approximation for inter-annotator agreement
in our scenario.. WAWA compares each annotator’s answer with the aggregated
answer obtained via majority votes and micro-averages the results across all
samples. The resulting WAWA precision, recall, and $\mathrm{F}_{1}$ are
80.01%, 78.94%, and 79.47%, respetively, which indicates a moderate to high
agreement.

##  4 Disinformation Detection

The disinformation detection task challenges detectors to determine whether a
given input article contains inaccurate information or not. We experiment with
four competitive detectors, including HDSF Karimi and Tang (2019), Grover
Zellers et al. (2019), Bert Devlin et al. (2019), RoBERTa Liu et al. (2019).
HDSF leverages the hierarchical structures of discourse-level features, such
as dependency trees, to predict the veracity of a news article. Grover is a
unidirectional seq2seq model pre-trained on news-domain documents. It utilizes
the encoder components of its generator counterpart for detection. The
predictions are made by feeding the final hidden states of the encoder to a
multi-layer perceptron. Similarly, Bert and RoBERTa take in the entire article
as input and feed the representations of the first token to a classification
head to determine the veracity of each article. In addition, InfoSurgeon is a
more fine-grained detector that models the inconsistency at the level of
knowledge elements across image and text modalities with a graph neural
network-based architecture. Since we focus on text-only data, we set the input
image features to zero vectors. All models are optimized using cross entropy.
For fair comparison, we set the maximal sequence length to 512 and use the
Large variants for all models. Details of the implementation can be found in
Appendix D.

##  5 Experiments

In our experiments, we aim to (1) analyze the performance of different models
on the PropaNews dataset, (2) examine the effect of various training data
sets, and (3) investigate how much silver-standard data is equivalent to gold-
standard data.

###  5.1 Data

#### PropaNews

The PropaNews dataset consists of 2,256 distinct articles, with a balanced
portion of fake and real documents. Within the fake articles, 30% of them use
appeal to authority, another 30% include loaded language, and the remaining
40% simply contains inaccurate information. We split the data into 1,256: 500:
500 for training, validation, and testing.

#### Human-written news articles

To evaluate the performance of our models on defending against human-written
disinformation, we first collected around 60 disinformative articles debunked
on politifact.com that were published between 2015 to 2020. Then, we further
expanded the set of fake articles to 100 by fact checking news outlets
published on untrustworthy news media11footnotemark: 1. The fact checking
procedure was done by a Computer Science majored graduate student. For real
news articles, 100 articles from Los Angeles Times are curated. Eventually,
this dataset (HumanNews) contains 200 human-written news articles.

#### Other training data

We compare the performance of detectors when trained on the following datasets
to study the effectiveness of our dataset in defending against human-written
disinformation. To understand the impact of human validation, we form the PN-
silver dataset by resampling our generated articles but disregarding the
annotator validation discussed in Section 3.2. In addition, we compare with
two other approaches, Grover-gen Zellers et al. (2019), FactGen Shu et al.
(2021) and FakeEvent, where fake articles in PN-silver are replaced with
documents generated by according methods. Grover-gen generates headlines with
condition on the original body texts, followed by body text generation
conditioning on the generated headlines. FactGen enhances the factual
consistency of the generated article with a fact retriever that fetches
supporting information from external corpora. FakeEvent generates each
sentence sequentially, with each generation step conditioning on the
manipulated knowledge elements of the current sentence and previously
generated sentences, following Fung et al. (2021). To ensure fair comparison,
all generators take in the same set of authentic articles as inputs.

Detector Training Data Grover-large RoBERTa-large Without human validation
(silver) Grover-gen 59.5 ($\pm$3.0) 54.9 ($\pm$2.6) FakeEvent 44.7 ($\pm$1.8)
42.4 ($\pm$2.5) FactGen 45.8 ($\pm$7.7) 43.9 ($\pm$3.2) PN-silver 62.1
($\pm$4.6) 61.8 ($\pm$1.3) With human validation (gold) PropaNews 66.8
($\pm$2.0) 66.9 ($\pm$2.1) w/o AA 64.2 ($\pm$2.1) 64.8 ($\pm$2.9) w/o LL 65.6
($\pm$2.9) 65.2 ($\pm$1.8) w/o AA & LL 63.8 ($\pm$4.8) 63.8 ($\pm$4.4)

Table 3: Accuracy (in %) on the HumanNews dataset for different models when
trained on various data sets. The bottom rows show different variants of
PropaNews, with each component ablated. AA denotes appeal to authority,
whereas LL refers to loaded language. Article and Analysis
---
Article: … “It is true that the Democratic Party should have put more
resources into that election,” Sanders said on CNN’s “State of the Union” of
the Thompson campaign. "But it is also true that he ran 20 points better than
the Democratic candidate for president did in Kansas …
Analysis: Appealing to authority is common in human-written fake news.
Article: … Fraudulent White House Tells Businesses to Ignore Court Order on
Vaccine Mandates Quick note : Tech giants are shutting us down …
Analysis: The use of loaded language often indicates disinformation.
Table 4: Examples from HumanNews where RoBERTa successfully predicts the
veracity when trained on PN-silver, but classifies incorrectly when trained on
Grover-gen.

###  5.2 Results and Discussion

#### Human-written disinformation detection

To study the effectiveness of detecting human-written disinformation, we train
Grover and RoBERTa on different training datasets and evaluate them on the
HumanNews dataset, as shown in Table 3. We see that both models perform the
best when trained on PropaNews, compared to training on other datasets.
Consider ablating human validation, detectors trained on PN-silver still
outperform their counterparts trained on other datasets. This shows that our
generative method produces articles that are more similar to human-written
disinformation.To further verify such a finding, we measure the similarity
between generated articles in different training data and the disinformative
articles in the HumanNews dataset using the Mauve metric Pillutla et al.
(2021). Mauve computes the similarity between two text distributions by
summing the areas under a divergence curve, and it has been shown to produce
better approximations than other metrics such as JS divergence Martins et al.
(2020). We found that the Mauve score with HumanNews for PropaNews and Grover-
gen are 33.1% and 27.1%, respectively, suggesting that the generated documents
in PropaNews are closer to human-written disinformation. These results confirm
that the advantage of our generated articles in defending against human-
written disinformation is resulted from the closer gap between them.

In Table 4, we show two disinformative articles from HumanNews where RoBERTa
is able to classify them as inaccurate when trained on PN-silver but fails to
do so when trained on Grover-gen. Both articles contain propaganda, which are
incorporated into PN-silver but not into Grover-gen. This demonstrates that
detectors are better at detecting human-written disinformation that has such
properties.

#### Is propaganda generation helpful for disinformation detection?

We have demonstrated that detectors trained on PropaNews perform better in
identifying human-written disinformation compared to training on other
datasets. We further conduct an ablation study to analyze the contributions of
each propaganda technique. As shown in the bottom of Table 3, both appeal to
authority and loaded language prove beneficial to enhancing models’
capabilities in detecting human-written disinformation. Furthermore, comparing
PropaNews w/o AA& LL with other disinformation generation approaches, we find
that both models trained on our generated data, even without the incorporation
of propaganda techniques, still outperform their counterparts trained on other
datasets. This illustrates that our generated articles, with disinformation
mixed within real information, are more similar to human-written
disinformation than articles generated using other approaches.

#### How important is human validation?

The creation of gold-standard PropaNews involves human validation to ensure
data quality. Although such a procedure is far more efficient than manual
annotation of human-written disinformation, our ultimate goal is to completely
relieve human efforts for even higher scalability. Therefore, we seek to
understand the importance of such human validation process by comparing models
trained on our generated data with (gold) or without (silver) human
validation. In Table 5, we have shown the importance of human validation with
the advantage of PropaNews over PN-silver. We further expand the experiment by
training RoBERTa-large on different scale of silver data, ranging from 1 time
to 10 times the size of their gold counterpart (i.e. PropaNews). The 1 time
silver data is equivalent to the PN-silver dataset. Note that since the
Timeline17 dataset only contains around 4K samples, we additionally crawl news
articles from New York Times as inputs to our generator for the “5 times” to
“10 times” experiments. The results are shown in Figure 3. We discover that
when the size of silver data is increased to 5 times or above of the gold data
size, the performance of the two models are comparable. The finding is likely
due to the fact that even without human validation, the vast majority of the
generated articles are indeed disinformative ($\sim$ 82%). This also reflects
that when silver-standard data is abundant, human validation is not
necessarily needed. However, our human validated gold-standard data can serve
as a valuable resource for future research on pinpointing the disinformative
passage in a given article as such directions may require higher-quality data.

Figure 3: Performance of RoBERTa on the HumanNews dataset when trained on
different size of data without human validation (silver), from 1 to 10 times
of the data that involves human validation (gold). Each data point represents
the average accuracy of four runs.

#### How good is the generation quality?

To evaluate the quality of our generation approach, we utilized AMT workers to
rate the plausibility of 100 generated articles from PropaNews and determine
the degree by which their answer to this question is influenced by the
generated propaganda. Each article is rated by 3 workers. As a comparison, we
also ask AMT workers to rate the plausibility of 100 generated articles from
Grover-gen. Details of the survey are discussed in Appendix E. The average
plausibility scores for PropaNews and Grover-gen are 2.25 and 2.15 (out of 3),
indicating that our generation approach has slight advantage over Grover-gen
in terms of plausibility. Furthermore, among the articles in PropaNews that
are rated highly plausible, 29.2% of the workers think that the generated
propaganda highly affects their response (i.e. rated 3 out of 3) that the
generated article is plausible. This demonstrates the effectiveness of our
propaganda techniques in convincing the audience of the plausibility of
generated articles.

#### Benchmarking detectors

The performance of various detectors on the PropaNews dataset is shown in
Table 5. We have the following observations. First, we find that RoBERTa and
Grover demonstrate advantages over Bert. This could be explained by the fact
that RoBERTa and Grover are pre-trained on news domain corpora, whereas Bert
has no access to such domains during pre-training. In addition, we found that
HDSF performs much worse than the other three models. This reflects that
large-scale pre-training of language models brings more benefit to detection
performance than explicit modeling of discourse-level features.

Detector | Dev Acc. (%) | Test Acc. (%)
---|---|---
HDSF | 52.4 ($\pm$0.6) | 50.6 ($\pm$2.4)
Bert | 57.7 ($\pm$1.0) | 58.0 ($\pm$1.2)
Grover | 60.3 ($\pm$5.8) | 63.3 ($\pm$5.0)
RoBERTa |  70.5 ($\pm$0.3) |  69.8 ($\pm$1.1)
Table 5: Evaluation of various detectors on the PropaNews development and test
set. We report the mean and standard deviation of four runs.

###  5.3 Remaining Challenges

To better understand the remaining disinformative articles that the detectors
failed to identify, we conduct an analysis by comparing the RoBERTa
predictions on the HumanNews dataset and the labels. Three major modeling
capabilities required for successful detection are identified, as listed
below:

#### Static knowledge enrichment

About 30% of the misclassification is resulted from the lack of static
knowledge that can be found in public databases, such as law dictionaries. For
example, in this article111111https://bit.ly/3h5Qz00, Alexandria Ocasio Cortez
falsely states that the U.S. Immigration Customs Enforcement (ICE) is required
to fill 34,000 beds every day. According to the Appropriations Act of
2016121212https://www.congress.gov/114/bills/hr2029/BILLS-114hr2029enr.pdf,
however, ICE is only required to detain 34,000 available beds. Therefore, to
detect such a misinformation, the detector needs to be enriched with static
knowledge bases.

#### Dynamic knowledge acquisition

Around 48% of the misclassified human-written disinformation are caused by the
inability to acquire dynamic knowledge from new news sources. For instance,
COVID-related articles are usually published after 2020, while RoBERTA was
pre-trained on news articles released before 2019. It is very challenging for
RoBERTA to detect disinformation of such topics unless the detector is
equipped with the capabilities of acquiring dynamic knowledge from news
articles. Particularly, RoBERTa achieves an accuracy of 69.0% on detecting
fake articles published before 2019, but its accuracy drops to 51.9% when
testing on articles published after 2019.

#### Multi-document reasoning

The rest of the incorrect detection is caused by the lack of multi-document
reasoning ability. For instance, a news article131313https://bit.ly/3t0OV5v
wrongly associates Hillary Clinton with a flawed immigration policy of the
former government, and strengthens such a statement by referring to a Senate
report and relevant news articles. However, the cited report does not mention
Clinton, and the other news articles contain disinformation. To correctly
detect this piece of disinformation, detectors should be capable of reasoning
across multiple documents.

##  6 Related Work

#### Fake News Generation and Detection

Due to the rise of neural models and the potential threats of machine-
generated fake news, prior study mostly focuses on how to automatically
generate fake news with neural networks to defend against them. Zellers et al.
(2019) pre-train a generator with the same architecture as GPT-2 Radford et
al. (2019) on a large-scale news corpus and demonstrate that this generator is
effective in detecting neural fake news. More recently, Fung et al. (2021)
improve the controllability of the generated fake news by conditioning the
generator on knowledge elements, such as entities, relations and events,
extracted from the original news article. Shu et al. (2021) enhances the
factual consistency of the generated article by introducing a fact retriever
that fetches relevant information from external corpora. Mosallanezhad et al.
(2021) utilizes adversarial reinforcement learning to generate topic-
preserving articles. These studies have developed methods for generating fake
news that is hard to distinguish from real news to humans. Nevertheless, due
to the overwhelming amount of inaccurate information introduced and the lack
of propaganda techniques in the generated texts, these approaches are sub-
optimal for detecting human-written fake news, as shown in Section 5.2. In
contrast, our work generates fake news by incorporating propaganda techniques
and preserving the majority of the correct information. Hence, our approach is
more suitable for studying defense against human-written fake news.
Additionally, since our released dataset is annotated with the exact offset of
the disinformative passages, this work opens up future research opportunities
on interpretable detection of fake news.

#### Propaganda Generation and Detection

There is little previous study on propaganda generation. Zellers et al. (2019)
is the only relevant work that we know of, which studies propaganda generation
for communicating targeted disinformation. In contrast, our work focuses on
generating specific propaganda techniques to make the generated articles
closer to human-written fake news. To the best of our knowledge, we are the
first to study the incorporation of specific propaganda techniques into
generated articles. Prior work on propaganda detection mostly focuses on
document-level detection. Early works collect propaganda datasets using
distant supervision Rashkin et al. (2017); Barrón-Cedeno et al. (2019) by
assigning the same propaganda label to each news outlet under the same source
based on the news-media-level label of corresponding news source listed in
trustworthy sites, such as US News & World Report. However, classifiers
trained on such datasets may only learn to recognize the bias of each news
source instead of propaganda Da San Martino et al. (2020). Our dataset avoids
such issues by explicitly incorporating propaganda into each generated
article. Along another line of work, Da San Martino et al. (2019) present a
fragment-level propaganda detection dataset, where specific propaganda
techniques are labeled onto spans of text instead of each document. Recent
approaches for detecting these propaganda techniques rely on pre-trained
transformers Morishita et al. (2020); Chernyavskiy et al. (2020); Feng et al.
(2021); Tian et al. (2021). By contrast, we focus on detecting disinformative
articles with propaganda signals.

##  7 Conclusions and Future Work

We have proposed a novel method for generating disinformation that are closer
to human-written fake news. Evaluation on the human-written fake news dataset
HumanNews demonstrate the effectiveness of our generated data PropaNews in
enabling better detection performance on human-written fake news. We hope that
the datasets presented in this work, PropaNews and HumanNews, can serve as
enabling resources for human-written fake news detection and encourage future
research along this direction. For future work, we plan to extend our approach
to other languages and explore the cross-lingual detection settings. In
addition, we are also interested in studying the generation of other
propaganda techniques that involves logical fallacy, such as straw man and red
herring, to enable more robust detection of human-written disinformation.

##  8 Ethical Statement and Broader Impact

Our objective for developing a generative approach that produces more
realistic news articles is to advance the field of disinformation detection
and to bring awareness that the current approaches for generating training
data for fake news detection are sub-optimal for defending against human-
written fake news.

We acknowledge the dual-use concerns for such a generation framework, and
therefore decided to release the codebase for only the detectors used in the
experiments but not the generator.

## Acknowledgement

This research is based upon work supported by U.S. DARPA SemaFor Program No.
HR001120C0123. The views and conclusions contained herein are those of the
authors and should not be interpreted as necessarily representing the official
policies, either expressed or implied, of DARPA, or the U.S. Government. The
U.S. Government is authorized to reproduce and distribute reprints for
governmental purposes notwithstanding any copyright annotation therein.

## References

  * Barrón-Cedeno et al. (2019) Alberto Barrón-Cedeno, Israa Jaradat, Giovanni Da San Martino, and Preslav Nakov. 2019.  Proppy: Organizing the news based on their propagandistic content.  _Information Processing & Management_, 56(5):1849–1864.
  * Bastos and Mercea (2019) Marco T Bastos and Dan Mercea. 2019.  The brexit botnet and user-generated hyperpartisan news.  _Social science computer review_ , 37(1):38–54.
  * Bosselut et al. (2018) Antoine Bosselut, Asli Celikyilmaz, Xiaodong He, Jianfeng Gao, Po-Sen Huang, and Yejin Choi. 2018.  Discourse-aware neural rewards for coherent text generation.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 173–184, New Orleans, Louisiana. Association for Computational Linguistics.
  * Chernyavskiy et al. (2020) Anton Chernyavskiy, Dmitry Ilvovsky, and Preslav Nakov. 2020.  Aschern at SemEval-2020 task 11: It takes three to tango: RoBERTa, CRF, and transfer learning.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 1462–1468, Barcelona (online). International Committee for Computational Linguistics.
  * Cohen (1960) Jacob Cohen. 1960.  A coefficient of agreement for nominal scales.  _Educational and psychological measurement_ , 20(1):37–46.
  * Da San Martino et al. (2020) Giovanni Da San Martino, Stefano Cresci, Alberto Barrón-Cedeño, Seunghak Yu, Roberto Di Pietro, and Preslav Nakov. 2020.  A survey on computational propaganda detection.  _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence_.
  * Da San Martino et al. (2019) Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019.  Fine-grained analysis of propaganda in news article.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5636–5646, Hong Kong, China. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dewatana and Adillah (2021) Hernawan Dewatana and Siti Ummu Adillah. 2021.  The effectiveness of criminal eradication on hoax information and fake news.  _Law Development Journal_ , 3(3):513–520.
  * Dimitrov et al. (2021) Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021.  Detecting propaganda techniques in memes.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 6603–6617, Online. Association for Computational Linguistics.
  * Donahue et al. (2020) Chris Donahue, Mina Lee, and Percy Liang. 2020.  Enabling language models to fill in the blanks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2492–2501, Online. Association for Computational Linguistics.
  * Feng et al. (2021) Zhida Feng, Jiji Tang, Jiaxiang Liu, Weichong Yin, Shikun Feng, Yu Sun, and Li Chen. 2021.  Alpha at SemEval-2021 task 6: Transformer based propaganda classification.  In _Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)_ , pages 99–104, Online. Association for Computational Linguistics.
  * Fung et al. (2021) Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, and Avi Sil. 2021.  InfoSurgeon: Cross-media fine-grained information consistency checking for fake news detection.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1683–1698, Online. Association for Computational Linguistics.
  * Grinberg et al. (2019) Nir Grinberg, Kenneth Joseph, Lisa Friedland, Briony Swire-Thompson, and David Lazer. 2019.  Fake news on twitter during the 2016 us presidential election.  _Science_ , 363(6425):374–378.
  * Hermann et al. (2015) Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015.  Teaching machines to read and comprehend.  In _NIPS_.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  In _International Conference on Learning Representations_.
  * Karimi and Tang (2019) Hamid Karimi and Jiliang Tang. 2019.  Learning hierarchical discourse-level structure for fake news detection.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 3432–3442, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Liu and Lapata (2019) Yang Liu and Mirella Lapata. 2019.  Text summarization with pretrained encoders.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3730–3740, Hong Kong, China. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
  * Martins et al. (2020) Pedro Henrique Martins, Zita Marinho, and André F. T. Martins. 2020.  Sparse text generation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4252–4273, Online. Association for Computational Linguistics.
  * Morishita et al. (2020) Terufumi Morishita, Gaku Morio, Hiroaki Ozaki, and Toshinori Miyoshi. 2020.  Hitachi at SemEval-2020 task 3: Exploring the representation spaces of transformers for human sense word similarity.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 286–291, Barcelona (online). International Committee for Computational Linguistics.
  * Mosallanezhad et al. (2021) Ahmadreza Mosallanezhad, Kai Shu, and Huan Liu. 2021.  Generating topic-preserving synthetic news.  In _2021 IEEE International Conference on Big Data (Big Data)_ , pages 490–499.
  * Nadarevic et al. (2020) Lena Nadarevic, Rolf Reber, Anne Josephine Helmecke, and Dilara Köse. 2020.  Perceived truth of statements and simulated social media postings: an experimental investigation of source credibility, repeated exposure, and presentation format.  _Cognitive Research: Principles and Implications_ , 5(56).
  * Ning et al. (2020) Qiang Ning, Hao Wu, Rujun Han, Nanyun Peng, Matt Gardner, and Dan Roth. 2020.  TORQUE: A reading comprehension dataset of temporal ordering questions.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1158–1172, Online. Association for Computational Linguistics.
  * Page et al. (1999) Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999.  The pagerank citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford InfoLab.  Previous number = SIDL-WP-1999-0120.
  * Pillutla et al. (2021) Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021.  Mauve: Measuring the gap between neural text and human text using divergence frontiers.  In _NeurIPS_.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Rashkin et al. (2017) Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017\.  Truth of varying shades: Analyzing language in fake news and political fact-checking.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2931–2937, Copenhagen, Denmark. Association for Computational Linguistics.
  * Rennie et al. (2017) Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017.  Self-critical sequence training for image captioning.  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Salehi et al. (2015) Niloufar Salehi, Lilly C. Irani, Michael S. Bernstein, Ali Alkhatib, Eva Ogbe, Kristy Milland, and Clickhappier. 2015.  We are dynamo: Overcoming stalling and friction in collective action for crowd workers.  In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_ , CHI ’15, page 1621–1630, New York, NY, USA. Association for Computing Machinery.
  * Sheng et al. (2021) Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2021.  “nice try, kiddo”: Investigating ad hominems in dialogue responses.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 750–767, Online. Association for Computational Linguistics.
  * Shu et al. (2021) Kai Shu, Yichuan Li, Kaize Ding, and Huan Liu. 2021.  Fact-enhanced synthetic news generation.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 35(15):13825–13833.
  * Tian et al. (2021) Junfeng Tian, Min Gui, Chenliang Li, Ming Yan, and Wenming Xiao. 2021.  MinD at SemEval-2021 task 6: Propaganda detection using transfer learning and multimodal fusion.  In _Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)_ , pages 1082–1087, Online. Association for Computational Linguistics.
  * Tran et al. (2013) Giang Binh Tran, Tuan Tran, Nam Khanh Tran, Mohammad Alrifai, and Nattiya Kanhabua. 2013.  Leveraging learning to rank in an optimization framework for timeline summarization.
  * van Der Linden et al. (2020) Sander van Der Linden, Jon Roozenbeek, and Josh Compton. 2020.  Inoculating against fake news about covid-19.  _Frontiers in psychology_ , 11:2928.
  * Walter and Tukachinsky (2020) Nathan Walter and Riva Tukachinsky. 2020.  A meta-analytic examination of the continued influence of misinformation in the face of correction: How powerful is it, why does it happen, and how to stop it?  _Communication Research_ , 47:155–177.
  * Wasserman and Madrid-Morales (2019) Herman Wasserman and Dani Madrid-Morales. 2019.  An exploratory study of “fake news” and media trust in kenya, nigeria and south africa.  _African Journalism Studies_ , 40(1):107–123.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Williams (1992) Ronald J Williams. 1992.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  _Machine learning_ , 8(3):229–256.
  * Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019.  Defending against neural fake news.  In _Advances in Neural Information Processing Systems 32_.

Step |  Generated Sequence
---|---
1 |  Panmure Gordon analyst Peter Hitchens confirmed that “ the US government is likely to agree to reduce its estimate of the size of the spill, which would cut BP fines ”.
2 |  “ The US government is likely to agree to reduce its estimate of the size of the spill, which would cut BP fines, ” Panmure Gordon analyst Peter Hitchens confirmed.
3 |  “ The US government is likely to agree to reduce its estimate of the size of the spill, which would cut BP fines, ” Panmure Gordon analyst Peter Hitchens said.
4 |  “ The US government is likely to agree to reduce its estimate of the size of the spill, which would cut BP fines, ” Panmure Gordon analyst Peter Hitchens said in a conference.
Table 6: An illustration of how appeal to authority is performed. In step 1,
we generate a statement using BART with the prefix “Panmure Gordon analyst
Peter Hitchens confirmed that “ ”. In step 2, we move the subject and verb to
the back of the sentence to diversify the sentence structure. In step 3, we
swap the verb with another verb from the verb set $V$. In step 4, we append a
preposition in to the sequence in step 3 and use the resulting sequence as
prefix to BART’s decoder to generate the rest of the context. For step 1 and
step 4, we mark the prefix sequence to the decoder in yellow, and the
generated sequence in blue. To increase the diversity of the generated
sequences, step 2 to step 4 are each performed 50% of the time.

##  Appendix A Qualitative Examples of Generated Articles

Method |  Generated Article
---|---
Grover-gen |  Libya to restart three oilfields on Sept 1 : minister BENGHAZI , Libya , Aug 28 ( Reuters ) - Libya will restart two eastern oil fields in mid-September and resume crude shipment from Tobruk by the end of the month , SNCM deputy oil minister Habib Hummed said on Sunday . He said the west of Sirte was being restored and a third field in the eastern province of Brega was about to restart operations . “ Production at the Brega field has been increased by two to four percent and will resume its production on the end of the month , ” he said at a political conference . “ The second phase of production at the eastern field of Sirte came to full production by the end of the month and restarting the production from Brega field will begin in September , ” he said . He said production at the west of Sirte and east of Tobruk had already reached 95,000 barrels per day ( bpd ) on their main export terminals . Libya , awash with millions of dollars , most of it believed to have been siphoned off through crude theft , is pumping around 290,000 bpd in total . The first fuel terminals in Libya have reopened and are loaded with diesel and petrol and tankers from Europe are ploughing in cargo to get through a key smuggling route from Derna to the Mediterranean . Libya was sharply divided last year when armed tribesmen struck crude export terminals along its border , denying it the lucrative gas exports it was using to service southern EU countries . Libya ’s southern oil terminal was hit in September by militant attacks . ( Writing by Ulf Laessing ; Editing by David Holmes )
FactGen |  Shares Augmented and stored pipelines replaced by light pipes .Disputed concerns over biggest damage to Libya since 2003 . Despite concerns oil could be leaking . ’Russia says production is safe and it is ready to resume production . ’Greece could use more sulfur oil by 2020 . ’Energy ministers say it could be used to power two-thirds of the world economy . ’Saudi had been the biggest oil producer in the world before meltdown . ”French to continue to contain crisis . ”Russia on Libyan crisis : ‘ we are facing a serious and continuing public health crisis that is needed ’ ”But not in a vacuum of oil ’ ”World affairs . ”Greece : ‘ we need to be doing more to combat damage to the energy system . ’ ” leftasing all oil from Libya crisis . ”
Ours (PN-silver) |  Tue Aug 23 , 2011 4:05 pm EDT * Limited Libyan oil production could start soon * Condition of oil export terminals still unclear * Clearing pipes clogged by waxy crude could delay output By David Sheppard and Sarah Young NEW YORK/LONDON , Aug 23 -LRB- Reuters -RRB- - Limited Libyan oil production ought to resume quite quickly as most fields appear to have survived the civil war unscathed , international oil services companies say , but many are still waiting for more stability in the country before sending employees back in . In interviews with some of the biggest oil services players in pre-civil war Libya , most echoed the view that major oil fields had largely avoided serious damage during the six months of fighting , though some voiced concerns about damage to export terminals and pipelines . OPS International Chairman Gavin De Salis told Reuters Insider television that Libyan crude oil , prized for its high yield of valuable light products such as gasoline and for its low sulfur content , was quite waxy , which could clog up pipelines if they had been left unused for some time . “ There might be a little bit of effort unplugging pipelines , which is two to three months ’ worth of effort before they can resume full production , ” De Salis said . “ But that will not affect all of the pipelines or all of the fields , so they can certainly start limited production quite quickly . ” Nilsson said contacts at Libya ’s rebel oil firm Arabian Gulf Oil Company -LRB- AGOCO -RRB- informed him there had been little damage to the oilfields in the east of the country during the six-month power struggle . “ We have n’t been able to work at the oilfields during the civil war as it has not been safe , but I think within a couple of weeks we could be back to almost normal , ” Nilsson said by telephone from his office in Stockholm . “ The oil income is essential to Libya and the new government so they will want to bring it back online as soon as possible . ” Nilsson said they had several Swedish , Indian and Sudanese employees who had stayed in the country during the civil war , but total staff numbers in the country were down from around 250-300 . Nilsson said there was still a lot of work to be done in the country . De Salis said that “ a lot of damage ” had been done to Libya ’s oil infrastructure , including the destruction of some of the country ’s main oil export terminals , but he said it was too early to estimate the full extent of the damage . DAMAGE Oil firm ’s who supported the rebel government during the civil war are expected to win the lion ’s share of contracts to help relaunch the Libyan oil industry , which before the war produced some 1.6 million barrels per day of crude …
Table 7: A qualitative comparison between generated articles from different
approaches. The texts marked in orange indicate disinformation, and the texts
in blue denote propaganda. We see that other approaches generate a large
amount of inaccurate information, which contrasts with a property of human-
written fake news mentioned in Section 1. We also note that the article
generated using FactGen appear to be low-quality. This is likely caused by the
fact that the checkpoints reported in the paper were not released and we train
FactGen from scratch by closely following the recipe described in Shu et al.
(2021). It is possible that some details of the training process of FactGen
were missing from the paper, and hence the low generation quality.

In Table 7, we show a comparison of generated articles given the same input
data across different generative methods. Our approach produces articles with
a small fraction of inaccurate information, which matches a property of human-
written fake news discussed in Section 1.

##  Appendix B Appeal to Authority Details

To recap, we first gather a list of authorities $Z$ for each article from
Wikidata and the corresponding context. The best appeal to authority sequence
$s^{*}$ is selected with the lowest perplexity
$s^{*}=\operatorname*{argmin}_{s_{i}}\textrm{PPL}(s_{i})$ where $s_{i}$
denotes the generated sequence using $z_{i}$ as the authority. However, this
process results in every sequence $s^{*}$ containing the the substring
“confirms that”, which makes it trivial for detectors to classify these
generated documents as fake by simply detecting such substrings. Therefore, we
devise an algorithm to diversify the templates so that these generated
articles are not easily detectable.

First, we define a set of verbs $V$ that can be swapped with “confirms”: $V=$.
Then, we diversify the generated structure of the generated sentence $s^{*}$
by reordering the subject, verb, and object. Next, we swap the verb with a
another verb from $V$. Finally, to diversify the context, we append a
preposition from the preposition set $PP=$ to the output of the previous step,
and then feed the sequence to BART to generate the context. An example of this
process is provided in Table 6.

##  Appendix C Human Validation Details

In this section, we describe the details of human validation where AMT workers
are tasked to validate whether the generated sentences contain inaccurate
information. To ensure the annotation quality, only workers who have an
acceptance rate greater than 95% and have more than 100 accepted HITs in the
past are allowed to work on our annotation task. Each HIT was designed such
that the annotators are rewarded $12-$15 per hour, which complies with the
ethical research standards outlined by AMT Salehi et al. (2015). In each HIT,
the annotators are presented an article with the generated part marked in
boldface. The questions and guidelines are illustrated as follows. (Note that
we only use the annotators’ response for Q1 to validate our generated data.
The annotations for the other questions will be used for future research.)

Q1: Is the generated text in boldface Accurate or Inaccurate? (If you cannot
find any supporting evidence, please select Inaccurate.) Note that: A
statement (in quotation marks) made by a person is only accurate if this
person actually made the exact same statement. If the statement in quotation
marks is just a paraphrase of what the person actually said, then the
statement is inaccurate.

  * -

Inaccurate: Any false information presented in the generated text makes it
inaccurate.

  * -

Accurate: All the information in the generated text must be accurate.

Q2: Enter the URL of the news article you found that supports your decision in
the previous response in the below box. Put down “from context" if the
evidence can be found in the context.

Q3: Does the generated text in boldface delivers the same sentiment as the
rest of the article?

  * -

False: The sentiment of the generated text is NOT the same as the rest of the
article.

  * -

True: The sentiment of the generated text is the same as the rest of the
article.

Q4: Is the discourse of the generated text in boldface consistent with the
rest of the article?

  * -

False: The discourse of the generated text is NOT consistent with the rest of
the article.

  * -

True: The discourse of the generated text is consistent with the rest of the
article.

Q5: If there is any grammatical error or inconsistent discourse, please
rewrite the correct the generated text and put it in the below box. Just put
down the corrected generated text in bold is enough. For example, “Harry is a
boy. He likes go to school.” Please put in “He likes to go to school.” in the
box below.

##  Appendix D Detector Implementation Details

For BERT adn RoBERTa experiments, we use AdamW Loshchilov and Hutter (2019) as
the optimizer with a batch size of 2 and gradient accumulation steps of 8. We
set the learning rate and weight decay to 5e-5 and 1e-5 for the the parameters
that have been pre-trained, and 1e-3 and 1e-3 for other parameters. For
experiments on the Grover detector, we follow the original detection setting.
Grover is trained using Adam Kingma and Ba (2015) with a learning rate of 2e-5
and a batch size of 64. Similarly, we follow the original recipe to train
HDSF, which is optimized with Adam with a learning rate of 1e-2.

All experiments are conducted on a Ubuntu 18.04 machine with NVIDIA Tesla
V100. We use PyTorch 1.10.0 and Transformers 4.3.0 for constructing all models
and loading pre-trained weights, except for Grover, which operates on
Tensorflow 1.13.1.

##  Appendix E Human Evaluation Details

In this section, we describe the survey we deliver to AMT workers for
evaluating the quality of the generated articles. Annotators are presented a
generated article and asked to answer a few questions regarding the quality of
it. Q2 is only applicable for evaluating generated articles from PropaNews, in
which we show the sentence that contains propaganda. The low, medium, and high
ratings in the response correspond to 1, 2, and 3 scores described in Section
5.2. The questions and guidelines are illustrated as follows:

Q1: How plausible do you think of the article above?

  * -

Low: It likely contains inaccurate information.

  * -

Medium: Not sure.

  * -

High: It unlikely contain inaccurate information.

Q2: How much does this sentence in the article affects your decision on the
previous answer?

  * -

Low: This sentence does not affect my answer for the previous question.

  * -

Medium: This sentence somehow affect my answer for the previous question.

  * -

High: This sentence largely affects my answer for the previous question.