# Possible Unconventional Surface Superconductivity in the Half-Heusler YPtBi

Eylon Persky  Geballe Laboratory for Advanced Materials, Stanford University,
Stanford, CA 94305, USA  Stanford Institute for Materials and Energy Sciences,
SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, CA
94025, USA  Department of Applied Physics, Stanford University, Stanford, CA
94305, USA  Alan Fang  Geballe Laboratory for Advanced Materials, Stanford
University, Stanford, CA 94305, USA  Stanford Institute for Materials and
Energy Sciences, SLAC National Accelerator Laboratory, 2575 Sand Hill Road,
Menlo Park, CA 94025, USA  Xinyang Zhang  Geballe Laboratory for Advanced
Materials, Stanford University, Stanford, CA 94305, USA  Department of Applied
Physics, Stanford University, Stanford, CA 94305, USA  Stanford Institute for
Materials and Energy Sciences, SLAC National Accelerator Laboratory, 2575 Sand
Hill Road, Menlo Park, CA 94025, USA  Carolina Adamo  Geballe Laboratory for
Advanced Materials, Stanford University, Stanford, CA 94305, USA  Stanford
Institute for Materials and Energy Sciences, SLAC National Accelerator
Laboratory, 2575 Sand Hill Road, Menlo Park, CA 94025, USA  Eli Levenson-Falk
Geballe Laboratory for Advanced Materials, Stanford University, Stanford, CA
94305, USA  Department of Applied Physics, Stanford University, Stanford, CA
94305, USA  Chandra Shekhar  Max Planck Institute for Chemical Physics of
Solids, 01187 Dresden, Germany.  Claudia Felser  Max Planck Institute for
Chemical Physics of Solids, 01187 Dresden, Germany.  Binghai Yan  Department
of Condensed Matter Physics, Weizmann Institute of Science, Rehovot, Israel
Aharon Kapitulnik  Geballe Laboratory for Advanced Materials, Stanford
University, Stanford, CA 94305, USA  Stanford Institute for Materials and
Energy Sciences, SLAC National Accelerator Laboratory, 2575 Sand Hill Road,
Menlo Park, CA 94025, USA  Department of Applied Physics, Stanford University,
Stanford, CA 94305, USA  Department of Physics, Stanford University, Stanford,
CA 94305, USA

###### Abstract

We report an extensive extensive study of the noncentrosymmetric half-Heusler
topological superconductor YPtBi, revealing unusual relation between bulk
superconductivity and the appearance of surface superconductivity at
temperatures up to 3 times the bulk transition temperature. Transport
measurements confirmed the low carrier density of the material and its bulk
superconducting transition, which was also observed in ac susceptibility
through mutual inductance (MI) measurements. However, a weak signature of
superconductivity in the MI measurements appeared much above the bulk
transition temperature, which was further observed in scanning tunneling
spectroscopy. Polar Kerr effect measurements suggest that while the bulk
superconductor may exhibit an unusual nodal superconducting state, only the
surface state breaks time reversal symmetry. Complementary tunneling
measurements on LuPtBi are used to establish the observations on YPtBi, while
density-functional theory (DFT) calculations may shed light on the origin of
this unusual surface state.

Introduction:

Topological superconductivity can be realized by inducing superconductivity at
the surface of a three dimensional topological insulator (3D TI) [1]. This can
be realized through interface engineering, or by using the surface states of a
bulk superconductor [4, 3, 2]. These mechanisms generally lead to time-
reversal invariant topological states [1], and their critical temperature is
determined by that of the bulk superconductor. An alternative approach, which
expands the limits of these techniques, is utilizing flat surface bands which
can support superconductivity without a proximity effect. Such superconducting
surface states have been observed in KZnBi [5] and t-PtBi2 [6]. However, the
topological properties of these states, particularly as related to the origin
and nature of their superconducting properties, are not fully understood.

Here, we explore the possible formation of a superconducting surface state at
the (111) surface of the half Heusler compound YPtBi through extensive
measurements that include transport, ac susceptibility measurements through
mutual inductance, scanning tunneling microscopy and spectroscopy and polar
Kerr effect. While robust bulk superconductivity appears below $T_{c}^{b}$, we
find that superconductivity at the surface persists to temperatures
$T_{c}^{s}\approx 3T_{c}^{b}$. Furthermore, our Kerr effect measurements
suggest that unlike the bulk, this superconducting surface state breaks time
reversal symmetry (TRS), thus revealing a delicate interplay between the
surface and bulk states in this paradigmatic topological system. We complement
the scanning tunneling spectroscopy measurements on YPtBi with measurements
LuPtBi to substantiate our arguments. To further support our experimental
findings, we performed ab-initio calculations on the Bi, Lu and Pt
terminations of the (111) direction, finding that a Bi-termination surface
exhibits a van-Hove singularity (VHS) near the Fermi energy, which could be a
source for enhanced susceptibility for a superconducting surface state. A
recent theoretical study [7] explored a similar surface band structure,
finding a possible domain for the onset of unconventional chiral topological
surface superconductivity ahead of the bulk transition, in agreement with our
results. Our results reveal a new platform for 2D chiral superconductors in
which the topologically-protected surface state condense into a TRS-breaking
superconducting state prior to the bulk.

In a half-Heusler compound, three metallic elements are arranged in
interpenetrating fcc lattices ($C1_{b}$ structure) [8]. The choice of
elements, with thousands of possible combinations, enables control over a wide
variety of properties in the resulting compounds: structures can be metallic,
semiconducting, magnetic or non-magnetic, superconducting, and they can have
topological surface states [9, 10]. Of these materials, YPtBi and LuPtBi have
the largest inverted gaps [11], leading to topologically protected surface
states [12]. Furthermore, YPtBi and LuPtBi are bulk superconductors, with
critical temperatures of 0.8 K and 1 K respectively [13, 14]. The bulk
superconductivity should be a combination of odd and even parity states, due
to the lack of inversion symmetry [15]. While much work was done to explore
the bulk superconductivity in these materials[13, 14, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28], studies of surface states and their propensity to
unconventional superconductivity have been limited [7].

Methods:

Single crystals of LuPtBi and YPtBi were grown by self-flux solution growth
method, with Bi used as the flux medium [29, 30, 31]. The composition and
structure were verified using dispersive X-ray analysis and Laue X-ray
diffraction, respectively. We used X-ray Photoelectron Spectroscopy (XPS) to
characterize the elemental surface composition of the samples. With a typical
measurement depth of about a nanometer, pristine crystals showed traces of
carbon, oxygen, and bismuth on the surface. The carbon traces were eliminated
after several seconds of Ar+ ion milling (equivalent to a few nm), and all
three elements: Lu, Pt and Bi seem to appear with the correct 1:1:1 ratio,
together with traces of oxygen. The surface composition did not change upon
subsequent milling of several more nanometers of the surface, confirming that
no other elements exist in the samples.

Resistivity and Hall effect on both types of samples, confirming the low
carrier densities, $n\sim 1\times 10^{19}$/cm3, and relatively high
mobilities, typically $\mu\sim 2500$ cm2/V-s at helium temperature. Sharp bulk
superconductivity was verified with both, resistive transition and two-coils
mutual inductance (MI) measurements.

Scanning Tunneling Microscopy (STM) and spectroscopy (STS) measurements were
performed in a modified UNISOKU Co., USM1300S3HeTM system with Ultra High
Vacuum (UHV), 350 mK base temperature, and magnetic field capabilities. Our
studies focused on samples that exhibited a clear (111) surface such as shown
in the insets of Fig. 1. The data shown in this work were obtained on native
as-grown surfaces without any ex-situ preparation. We performed in-situ
surface cleaning by removing the top few surface layers with the STM tip
itself. The details of this procedure are described in the Supplemental
Material [31], and are based on a similar procedure previously used to expose
superconducting regions in vacuum-cleaved YPtBi crystals [32].

High-resolution measurements of Polar Kerr Effect (PKE) were performed using a
continuous wave, Zero-Area-Loop Sagnac Interferometer (ZALSI) [33, 34], in a
variable temperature 3He system. This instrument was successfully used to
detect TRSB-breaking order parameters in unconventional superconductors and
subtle effects of magnetism in a variety of quantum materials (see
Supplemental Information [31]).

Results and Discussion:

Figure 1 shows representative base temperature ($\sim$350 mK) spectra of
pristine surface and tip-cleaned surface for YPtBi and LuPtBi samples. While
the tip-cleaned surface improved the spectra, particularly in reducing the
observed zero bias conductance to very low values, the pristine surface
spectra pointed to a similar large gap. At the same time the observed spectra
are rounded, even well below temperatures corresponding to the observed large
gap. Such a behavior could be because of nodal gap structure, high effective
temperature due to RF heating or due to an intermediate thin metallic layer
which depress the gap by proximity effect [35]. The fact that on occasion we
observe a flat gap as in Fig. 1b suggests that the rounding of the gap is not
due to nodes in the gap.

Figure 1: Base temperature tunneling spectra. Colored spectra as example
spectra after tip cleaning. a) YPtBi, b) LuPtBi. Both figures show a typical
spectrum on the pristine surface, already indicating a gap much larger than
the gap expected for the respective $T_{c}$s. Insets show typical crystals
used in this study, both exhibit a (111) surface.

The large gaps observed at base temperature indicate that there is a
superconducting system within either LuPtBi or YPtBi which involves a
transition temperature higher than the bulk. Indeed, temperature dependence of
tunneling spectra of both materials indicate gaps that persist much above the
bulk $T_{c}$. Focusing on YPtBi, we show in Fig. 2a a typical set of spectra
taken at different temperatures. Similar data on YPtBiwas reported in [32].
Since the location of the coherence peaks is not a good measure of the size of
the gap due to the rounded data at all temperatures, some assumptions are
required to estimate the gap as a function of temperature. In a first approach
we assume that “noise” interferes with the measurement and thus each spectrum
is considered as a standard BCS spectrum taken at an effective temperature
higher than the actual sample temperature $T$. Such an approach is similar to
assuming a an unknown broadening of the spectrum [36]. The fit to each
spectrum yields the gap $\Delta$ and the effective temperature $T_{eff}$.
Alternatively we can estimate the gap from maximum slope of the spectrum
before it turns into the coherence peak (this is similar to using the maximum
derivative of the spectrum [37]). Both estimates give a maximum gap of 0.4 to
0.41 meV and the fit to the BCS gap function with $2\Delta/k_{B}T_{c}=3.52$
yields a $T_{c}\approx 2.85$ K.

Figure 2: Temperature dependence of tunneling spectra for YPtBi. a) a typical
set of data with fits assuming noise-induced higher effective temperature (see
text). b) Extracted gap as a function of temperature using two approaches: i)
maximum slope in the gap ( ) using actual measured temperature and ii)
effective temperature higher than the actual temperature ( ). Solid lines are
BCS gap functions, assuming $2\Delta/k_{B}T_{c}=3.52$. For LuPtBi (see [31])
only the maximum slope approach ( , ) ) is used to determine the gap.

To further investigate the onset of superconductivity, we measured the ac
susceptibility of both materials using a mutual inductance technique [38].
Both materials showed a large increase of the diamagnetic response at their
respective bulk transition temperatures (Figure 3), in agreement with previous
reports [13, 14]. However, a careful measurement of YPtBi revealed a small
increase, $\sim$0.5% of the change due to the bulk transition, which onsets at
$\sim$3 K (Fig. 3a). This may indicate that a small volume fraction of the
sample turns superconducting at a higher temperature.

Figure 3: Mutual inductance determination of bulk superconducting transition
for (a) LuPtBi, and (b) YPtBi. Inset in (a) shows the onset of a small
diamagnetic signal in YPtBi, at $\sim$3 K.

The observation of a similar transition temperature for the vanishing of the
superconducting gap in tunneling and a weak onset of inductive signal in
mutual inductance points to an intrinsic origin of superconductivity that
onsets at a temperature much above the bulk $T_{c}^{b}$. We envision two
possible origins for the above observations. First, given our inability to
make clean, atomically flat cleaves that are known to expose the (111) plane,
the signals may alternatively be attributed to another compound, created
during the surface cleaning, a combination of the HH elements, possibly
combined with contaminants, particularly oxygen. Superconducting compounds
that may be formed this way are listed in Table 1. While the $T_{c}$ of both
amorphous and polycrystalline Bi are suitable, such compounds on the surface
would generate similar transitions for both LuPtBi and YPtBi, inconsistent
with the huge difference between the two compounds observed in STM
measurements. We further note that LuPtBi and YPtBi are very similar
chemically and structurally (lattice constants $a=6.58$Å and $a=6.65$Å,
respectively [11]). Thus, any strained inclusion of a different compound on
the surface would be expected to yield a similar gap. Moreover, the similarity
in the behavior of LuBi and YBi may rule this possibility out as well,
particularly since they require pressure to exhibit superconductivity.

Compound | Bi | a-Bi | p-Bi | BiPt | Bi2Pt | YBi | LuBi
---|---|---|---|---|---|---|---
P (GPa) | amb. | amb. | amb. | amb. | 5-6 | $\gtrsim 2.6$ | $\gtrsim 3$
$T_{c}(K)$ | $\ll 0.1^{1}$ | 5-62 | 6.13 | 1.34 |  $\sim$25 |  $\sim$56 | 4-7.57
Table 1: Possible superconducting compounds derived from YPtBi and LuPtBi.
While “amb.” represents ambient pressure, some compounds only become
superconducting at a finite pressure. Source references: 1[39], 2[40], 3[41,
42], 4[43], 5[44], 6[45], 7[46].

The alternative explanation for the observed higher $T_{c}$ in the two
compounds is the creation of a chiral surface state. Particularly, since both
LuPtBi and YPtBi are known to be topological with a large band inversion [11],
it was suggested that superconductivity associated with this material is
topological and may break TRS [22, 23]. To test this possibility, particularly
for YPtBi, we performed high resolution polar Kerr effect measurements using a
zero-area-loop Sagnac interferometer (ZALSI) [33, 34]. By construction, the
ZALSI is inherently reciprocal and thus by its symmetry it can distinguish
between true TRSB Kerr effect and optical activity when reflected from a
chiral medium. In our implementation we use two orthogonally counter
propagating linearly polarized light beams, which are then converted into
right and left circular polarizations using a quarter-wave plate. An objective
lens focuses the light onto a small interaction region on the sample, and the
reflected light beams are converted back to linear polarizations with
exchanged polarization directions. Owing to the reciprocity of the apparatus,
any non reciprocal phase shift $\phi_{nr}=2\theta_{K}$ will appear at the
detector only if TRS is broken through the interaction of the two circularly
polarized beams with the sample [33, 34] (see supplementary material [31]).

Figure 4: Polar Kerr effect measurements on a (111) face of a pristine YPtBi
crystal. (a) The sample was first cooled in a magnetic field to base
temperature ($\sim$300 mK). The field was then removed, and Kerr data was
taken while warming up the sample. (b) The resulting Kerr signals for various
low training fields. A finite Kerr signal appears much above the bulk
$T_{c}^{b}$ of the sample. The error bars represent a 95% confidence interval,
based on statistical averaging within each temperature bin.

We cooled down the sample to $\sim$300 mK from a high temperature in the
presence of a small external magnetic field, and then removed the field and
measured the Kerr signal while warming up the sample (Figure 4a). For small
training fields ($H\lesssim 50$ Oe) the initial signal (at 300 mK) following
the removal of the magnetic field was very small, consistent with zero (within
our resolution of $\sim 0.05\ \mu$rad). However, increasing the temperature
from base temperature, a finite Kerr signal of order $\pm 0.25\ \mu$rad
appeared above the bulk $T_{c}^{b}$. The signal was independent of the field
strength and changed sign upon changing the direction of the training field
(Figure 4b). These features indicate a TRS breaking state, and are generally
similar to those observed in chiral superconductors [47, 48].

With strong structural and electronic structure similarities to GdPtBi, we
estimate the optical penetration depth of YPtBi at $\sim 150$ nm [49]. Thus,
the Kerr signal can have both bulk and surface contributions. In particular
above the bulk $T_{c}^{b}$ the observed Kerr signal may be a result of TRS
breaking associated with the surface superconductivity - a favorable
possibility since Kerr effect persists very close to the surface $T_{c}^{s}$ ,
or it may be associated with vestigial order [50] originating from the bulk
superconducting state. However, the fact that Kerr effect is observed up to
three times the bulk $T_{c}^{b}$ makes the vestigial order possibility
unlikely.

While low field training (i.e. training in $H\lesssim 50$ Oe) yields similar
onset and size of Kerr signal, which follows the direction of the training
field, cooling the sample at higher magnetic field to base temperature (0.3
K), turning the field to zero and measuring while warming up the sample in
zero magnetic field yields a signal that is zero within our resolution of
$\sim 0.05\ \mu$rad. However, cooling the sample at higher magnetic field to
base temperature (0.3 K), but on the way turning the magnetic field to zero at
$T\approx 1.3$ K, then measuring the Kerr effect at zero field from base
temperature up yields results similar to the low field training experiments.
Fig. 5 summarizes these findings.

Figure 5: Polar Kerr effect after higher-field training. (a) An alternative
training protocol, where the magnetic field was removed at 1.3 K, before the
bulk superconducting transition. (b) The Kerr signals obtained after training
at low and high fields ($>$ 80 Oe) with the original training protocol, and
after high-field training with the new protocol. Note that $\theta_{K}$ is
suppressed or reversed when trained at a high field. However, the sample could
be trained even when the field was switched off before the bulk transition,
and the original sign of $\theta_{K}$ was recovered.

Such a peculiar effect may point to a delicate interplay between the vortex
state that appears in the bulk material upon cooling in a field much below the
bulk $T_{c}^{b}$ and the effect on the surface state. For example, cooling at
lower field and removing the field at low temperatures may result in low
vortex occupancy due to weak surface and geometrical barriers (note that
$H_{c1}\lesssim 3$ Oe, while the thermodynamic critical field is
$H_{c1}\lesssim 100$ Oe for the bulk superconducting state of these
materials). Above a field of order $\gtrsim 100$ Oe, enough vortices may
remain in the bulk, which upon crossing the bulk Tc will move out of the
sample, thus possibly “scrambling” the single domain nature of the TRS-
breaking state of the surface superconductivity. We note that the apparent
suppression of the Kerr effect at temperatures below $T_{c}^{b}$ suggests
points to a TRS-preserving bulk superconducting state. This is consistent with
the observations of nodes in the bulk and the analysis of the possible spin
and orbital pairing states [23]. Thus, the magnetization of the surface state
interacts with a conventional superconducting order parameter. This is similar
to magnetic superconductors, where the formation and motion of vortices can
change the ferromagnetic order parameter [51].

Taken together, the STM, mutual inductance and PKE results indicate that the
(111) surfaces of YPtBi and LuPtBi host a superconudcting state which breaks
TRS. To understand the origin of this state, we studied the surface band
structure of LuPtBi (see Supplementary Information [31]). In Bi-terminated
(111) surfaces, we found a van-Hove signularity close to the $M$-point, giving
rise to an enhanced density of states near the Fermi level. This could be the
driving force behind the enhanced surface $T_{c}^{s}$. The calculated band
structure of the Lu terminated surface, did not show van-Hove singularities.
Schwemmer et al. [7] have recently reported a detailed study of the surface
band structure and possible superconductivity of the Bi-terminated (111)
surface of LuPtBi. While the gross features of their surface band structure
are similar to our calculation, they also revealed flat-band regions near the
Fermi energy, and calculated propensity to a superconducting instability. In
particular, they argue that the transition temperature of the surface state is
independent of the bulk transition, and exhibit vestigial time-reversal
symmetry breaking in it fluctuation regime. Since inversion symmetry is
broken, simple singlet and/or triplet states were dismissed and pairing was
tested for pairs of electrons that are connected by the TR operator. The
result has been a pair-amplitude that transforms under the E irreducible
representation of the surface’s $C_{3v}$ point group (relevant to the (111)
crystal orientation) in the presence of strong Kane-Mele type and Rashba type
spin-orbit coupling [52]. Furthermore, they found that a chiral
superconducting state, which spontaneously breaks TRS, is energentically
favorable.

In conclusion, we presented above a comprehensive study of scanning tunneling
spectroscopy of the tip-cleaned (111) surface of the half-Heusler alloys YPtBi
and LuPtBi. While a bulk $T_{c}^{b}$ of $\sim$0.8 K is observed for both
systems, we find a much larger corresponding gap in our tunneling studies with
$T_{c}^{s}$’s of $\sim$2.3 K and $\sim$6.5 K for YPtBi and LuPtBi
respectively. Mutual inductance measurements of YPtBi further revealed a small
diamagnetic response that onsets around $T_{c}^{s}$. Polar Kerr effect on
YPtBi indicate that TRS is broken near or at $T_{c}^{s}$. Together, our data
indicate that chiral surface superconductivity may precede bulk condensation
in these materials, a hypothesis which is strongly supported by recent
calculations [7]. While additional work is required to determine the order
parameter of these states, our results open a new direction to engineering
topological superconducting states in half Heusler compounds. While our study
was conducted on single crystals, Kim et al. [53] recently demonstrated
epitaxially growth of YPtBi films, which can also be encapsulated by an
alumina film, preventing degradation of the surface. This system is therefore
highly promising for quantum device applications.

###### Acknowledgements.

We thank Ronny Thomale and Jörg Schmalian for illuminating discussions. This
work was supported by the U. S. Department of Energy (DOE) Office of Basic
Energy Science, Division of Materials Science and Engineering at Stanford
under contract No. DE-AC02-76SF00515. EP was partially supported by the Koret
Foundation.

## References

  * [1] L. Fu, and C. L. Kane, “Superconducting Proximity Effect and Majorana Fermions at the Surface of a Topological Insulator,” Phys. Rev. Lett. 100, 096407 (2008)
  * [2] A.K. Nayak, A. Steinbok, Y. Roet, J. Koo, G. Margalit, I. Feldman, A. Almoalem, A. Kanigel, G. Fiete, B. Yan, Y. Oreg, N. Avraham and H. Beidenopf, “Evidence of topological boundary modes with topological nodal-point superconductivity,” Nat. Phys. 17, 1413–1419 (2021).
  * [3] O.J. Clark, M.J. Neat, K. Okawa, L. Bawden, I. Marković, F. Mazzola, J. Feng, V. Sunko, J.M. Riley, W. Meevasana, J. Fujii, I. Vobornik, T.K. Kim, M. Hoesch, T. Sasagawa, P. Wahl, M.S. Bahramy, and P.D.C. King,“Fermiology and Superconductivity of Topological Surface States in PdTe2,” Phys. Rev. Lett. 120, 156401 (2018).
  * [4] P. Zhang, K. Yaji, T. Hashimoto, Y. Ota, T. Kondo, K. Okazani, Z. Wang, J. Wen, G.D. GU, H. Ding, and S. Shin, “Observation of topological superconductivity on the surface of an iron-based superconductor,” Science 360, 182-186 (2018).
  * [5] J. Song, S. Kim, Y. Kim, H. Fu, J. Koo, Z. Wang, G. Lee, J. Lee, S.H. Oh, J. Bang, T. Matsushita, N. Wada, H. Ikegami, J.D. Denlinger, Y.H. Lee, B. Yan, Y. Kim, and S.W. Kim, “Coexistence of Surface Superconducting and Three-Dimensional Topological Dirac States in Semimetal KZnBi,” Phys. Rev. X 11, 021065 (2021).
  * [6] Sebastian Schimmel, Yanina Fasano, Sven Hoffmann, Joaquín Puig, Grigory Shipunov, Danny Baumann, Saicharan Aswartham, Bernd Büchner, and Christian Hess, “High-$T_{C}$ surface superconductivity in topological Weyl semimetal t-PtBi2,” arXiv:2302.08968 [cond-mat.supr-con] (2023).
  * [7] Tilman Schwemmer, Domenico Di Sante, Jörg Schmalian, Ronny Thomale, “Chiral surface superconductivity in half-Heusler semimetals,” arXiv:2212.09786 [cond-mat.supr-con] (2022).
  * [8] Tanja Graf, Claudia Felser, and Stuart S.P. Parkin, “Simple rules for the understanding of Heusler compounds,” Prog. Solid. State Ch. 39, 1-50 (2011).
  * [9] Kaustuv Manna, Yan Sun, Lukas Muechler, Jürgen Kübler, and Claudia Felser, “Heusler, Weyl and Berry,” Nat Rev Mater 3, 244–256 (2018).
  * [10] Sheron Tavares, Kesong Yang, and Marc A. Meyers, “Heusler alloys: Past, properties, new alloys, and prospects,” Prog. Mater. Sci. 132, 101017 (2023).
  * [11] S. Chadov, X. L. Qi, J. Kübler, G. H. Fecher, C. Felser, and S. C. Zhang, “Tunable multifunctional topological insulators in ternary Heusler compounds,” Nat. Mater. 9, 541 (2010).
  * [12] Z. K. Liu, L. X. Yang, S.-C. Wu, C. Shekhar, J. Jiang, H. F. Yang, Y. Zhang, S.-K. Mo, Z. Hussain, B. Yan, C. Felser, and Y. L. Chen, “Observation of unusual topological surface states in half-Heusler compounds LnPtBi (Ln=Lu, Y),” Nat. Commun 7, 12924 (2016).
  * [13] N. P. Butch, P. Syers, K. Kirshenbaum, A. P. Hope, and J. Paglione, “Superconductivity in the topological semimetal YPtBi,” Phys. Rev. B 84, 220504 (2011).
  * [14] F. F. Tafti, Takenori Fujii, A. Juneau-Fecteau, S. René de Cotret, N. Doiron-Leyraud, Atsushi Asamitsu, and Louis Taillefer, “Superconductivity in the noncentrosymmetric half-Heusler compound LuPtBi: A candidate for topological superconductivity,” Phys. Rev. B 87, 184504 (2013).
  * [15] M Smidman, M. B. Salamon, H. Q. Yuan, and D. F. Agterberg, “Superconductivity and spin–orbit coupling in non-centrosymmetric materials: a review,” Rep. Prog. Phys. 80, 036501 (2017).
  * [16] T. V. Bay, T. Naka, Y. K. Huang, and A. de Visser, “Superconductivity in noncentrosymmetric YPtBi under pressure,” Phys. Rev. B 86, 064515 (2012).
  * [17] T.V. Bay, M. Jackson, C. Paulsen, C. Baines, A. Amato, T. Orvis, M.C. Aronson, Y.K. Huang, A. de Visser, “Low field magnetic response of the non-centrosymmetric superconductor YPtBi,” Solid State Comm. 183, 13 (2014).
  * [18] Markus Meinert, “Unconventional Superconductivity in YPtBi and Related Topological Semimetals,” Phys. Rev. Lett. 116, 137001 (2016).
  * [19] Orest Pavlosiuk, Dariusz Kaczorowski, and Piotr Wiśniewski, “Superconductivity and Shubnikov–de Haas oscillations in the noncentrosymmetric half-Heusler compound YPtBi,” Phys. Rev. B 94, 035130 (2016).
  * [20] C. Timm, A. P. Schnyder, D. F. Agterberg, and P. M. R. Brydon, “Inflated nodes and surface states in superconducting half-Heusler compounds,” Phys. Rev. B 96, 094526 (2017).
  * [21] Hao Yang, Jiabin Yu, Stuart S.P. Parkin, Claudia Felser, Chao Xing Liu, and Binghai Yan, “Prediction of Triple Point Fermions in Simple Half-Heusler Topological Insulators,” Phys. Rev. Lett. 119, 136401 (2017).
  * [22] Lucile Savary, Jonathan Ruhman, Jörn W. F. Venderbos, Liang Fu, and Patrick A. Lee, “Superconductivity in three-dimensional spin-orbit coupled semimetals,” Phys. Rev. B 96, 214514 (2017).
  * [23] Hyunsoo Kim, Kefeng Wang, Yasuyuki Nakajima, Rongwei Hu, Steven Ziemak, Paul Syers, Limin Wang, Halyna Hodovanets, Jonathan D. Denlinger, Philip M. R. Brydon, Daniel F. Agterberg, Makariy A. Tanatar, Ruslan Prozorov, Johnpierre Paglione, “Beyond triplet: Unconventional superconductivity in a spin-3/2 topological semimetal,” Sci. Adv. 4, eaao4513 (2018).
  * [24] Jörn W.F. Venderbos, Lucile Savary, Jonathan Ruhman, Patrick A. Lee, and Liang Fu, “Pairing States of Spin-3/2 Fermions: Symmetry-Enforced Topological Gap Functions,” Phys. Rev. X 8, 011029 (2018).
  * [25] Qing-Ze Wang, Jiabin Yu, and Chao-Xing Liu, “Unconventional superconductivity and surface pairing symmetry in half-Heusler compounds,” Phys. Rev. B 97, 224507 (2018).
  * [26] Bitan Roy, Sayed Ali Akbar Ghorashi, Matthew S. Foster, and Andriy H. Nevidomskyy, “Topological superconductivity of spin-3/2 carriers in a three-dimensional doped Luttinger semimetal,” Phys. Rev. B 99, 054505 (2019).
  * [27] Hyunsoo Kim, Makariy A. Tanatar, Halyna Hodovanets, Kefeng Wang, Johnpierre Paglione, and Ruslan Prozorov, “Campbell penetration depth in low carrier density superconductor YPtBi,” Phys. Rev. B 104, 014510 (2021).
  * [28] Hyunsoo Kim, Junhyun Lee, Halyna Hodovanets, Kefeng Wang, Jay D. Sau, and Johnpierre Paglione, “Quantum oscillations of the $j=3/2$ Fermi surface in the topological semimetal YPtBi,” Phys. Rev. Research 4, 033169 (2022).
  * [29] P. C. Canfield, J. D. Thompson, W. P. Beyermann, A. Lacerda, M. F. Hundley, E. Peterson, Z. Fisk, and H. R. Ott, “Magnetism and heavy fermion‐like behavior in the RBiPt series,” J. Appl. Phys. 70, 5800 (1991).
  * [30] P. C. Canfield and Z. Fisk, “Growth of single crystals from metallic fluxes,” Philos. Mag. B 65, 1117 (1992).
  * [31] Supplemental Information - URL to be determined by publisher.
  * [32] Hongwoo Baek, Jeonghoon Ha, Duming Zhang, Bharath Natarajan, Jonathan P. Winterstein, Renu Sharma, Rongwei Hu, Kefeng Wang, Steven Ziemak, Johnpierre Paglione, Young Kuk, Nikolai B. Zhitenev, and Joseph A. Stroscio, “Creating nanostructured superconductors on demand by local current annealing,” Phys. Rev. B 92, 094510 (2015).
  * [33] Jing Xia, Peter T. Beyersdorf, Martin M. Fejer, Aharon Kapitulnik, “Modified Sagnac interferometer for high-sensitivity magneto-optic measurements at cryogenic temperatures,” Applied Physics Letters 89, 062508 (2006).
  * [34] Aharon Kapitulnik, Jing Xia, Elizabeth Schemm and Alexander Palevski, “Polar Kerr effect as probe for time-reversal symmetry breaking in unconventional superconductors,” New J. Phys. 11, 055060 (2009).
  * [35] N. Moussy, H. Courtois, and B. Pannetier, “Local spectroscopy of a proximity superconductor at very low temperature,” Europhys. Lett., 55, 861 (2001).
  * [36] R. C. Dynes, V. Narayanamurti, and J. P. Garno, “Direct Measurement of Quasiparticle-Lifetime Broadening in a Strong-Coupled Superconductor,” Phys. Rev. Lett. 41, 1509 (1978).
  * [37] Toshiyuki Kita, Daisuke Akasako, Hironaru Murakami and Ryozo Aoki, “Gap Value Determinations on the Tunneling Spectrum,” in Advances in Superconductivity VII, K. Yamafuji et al. (eds.), (Springer-Verlag Tokyo 1995.)
  * [38] Xinyang Zhang, Mark Zic, Dong Chen, Chandra Shekhar, Claudia Felser, Ian R. Fisher, and Aharon Kapitulnik, “Vortex phase diagram of kagome superconductor CsV3Sb5,” arXiv:2306.13297 [cond-mat.supr-con] (2023).
  * [39] O. Prakash, A. Kumar, A. Thamizhavel, S. Ramakrishnan, “Evidence for bulk superconductivity in pure bismuth single crystals at ambient pressure,” Science 355, 52 (2017).
  * [40] J. T. Chen, T. T. Chen, J. D. Leslie and H. J. T. Smith, “Strong-Coupling Superconductivity in Amorphous Bismuth,” Physics Letters A 25, 679 (1967).
  * [41] V. I. Petrosyan, V. N. Molin, O. I. Vasin, P. A. Skripkina, S. I. Stenin, and E. G. Batyev, “Superconductivity in polycrystalline bismuth films,” Zh. Eksp. Teor. Fiz. 66, 995-100 (1974).
  * [42] Mingliang Tian, Jinguo Wang, Nitesh Kumar, Tianheng Han, Yoji Kobayashi, Ying Liu, Thomas E. Mallouk, and Moses H. W. Chan, “Observation of Superconductivity in Granular Bi Nanowires Fabricated by Electrodeposition,” Nano Letters 6, 2773 (2006).
  * [43] N.N. Zhuravlev, A.A. Stepanova, N.I. Zyuzin, “On the Superconductivity of the Compound BiPt,” J. Exptl. Theoret. Phys. (U.S.S.R.) 37, 880-881 (1959).
  * [44] Jing Wang, Xuliang Chen , Yonghui Zhou , Chao An, Ying Zhou, Chuanchuan Gu , Mingliang Tian, and Zhaorong Yang, “Pressure-induced superconductivity in trigonal layered PtBi2 with triply degenerate point fermions,” Phys. Rev. B 103, 014507 (2021).
  * [45] C. Q. Xu, B. Li, M. R. van Delft, W. H. Jiao, W. Zhou, B. Qian, Nikolai D. Zhigadlo, Dong Qian, R. Sankar, N. E. Hussey, and Xiaofeng Xu, “Extreme magnetoresistance and pressure-induced superconductivity in the topological semimetal candidate YBi,” Phys. Rev. B 99, 024110 (2019).
  * [46] H. Gu, F. Tang, Y.-R. Ruan, J.-M. Zhang, R.-J. Tang, W. Zhao, R. Zhao, L. Zhang, Z.-D. Han, B. Qian, X.-F. Jiang, and Y. Fang, “Pressure effect on the topologically nontrivial electronic state and transport of lutecium monobismuthide,” Phys. Rev. Materials 4, 124204 (2020).
  * [47] E. R. Schemm, W. J. Gannon, C. M. Wishne, W. P. Halperin, and A. Kapitulnik, “Observation of broken time-reversal symmetry in the heavy-fermion superconductor UPt3,” Science 345, 190-193 (2014).
  * [48] I. M. Hayes, D. S. Wei, T. Metz, J. Zhang, Y. S. Eo, S. Ran, S. R. Saha, J. Collini, N. P. Butch, D. F. Agterberg, A. Kapitulnik, and J. Paglione, “Multicomponent superconducting order parameter in UTe2,” Science 373, 797-801 (2021).
  * [49] F. Hütt, A. Yaresko, M. B. Schilling, C. Shekhar, C. Felser, M. Dressel, and A. V. Pronin, “Linear-in-Frequency Optical Conductivity in GdPtBi due to Transitions near the Triple Points,” Phys. Rev. Lett. 121, 176601 (2018).
  * [50] Rafael M. Fernandes, Peter P. Orth, and Jörg Schmalian, “Intertwined Vestigial Order in Quantum Materials: Nematicity and Beyond,” Annual Review of Condensed Matter Physics 10, 133 (2019).
  * [51] C. Paulsen, D. J. Hykel, K. Hasselbach, and D. Aoki, “Observation of the Meissner-Ochsenfeld Effect and the Absence of the Meissner State in UCoGe,” Phys. Rev. Lett. 109, 237001 (2012).
  * [52] Manuel Laubach, Johannes Reuther, Ronny Thomale, and Stephan Rachel, “Rashba spin-orbit coupling in the Kane-Mele-Hubbard model,” Phys. Rev. B 90, 165136 (2014).
  * [53] Jiwoong Kim, Kajetan M. Fijalkowski, Johannes Kleinlein, Claus Schumacher, Anastasios Markou, Charles Gould, Steffen Schreyeck, Claudia Felser, and Laurens W. Molenkamp, “Molecular beam epitaxy of a half-Heusler topological superconductor candidate YPtBi,” Phys. Rev. Materials 7, 024802 (2023).
  * [54] G. Kresse and J. Hafner, “Ab initio molecular dynamics for liquid metals,” Phys. Rev. B 47, 558(R) (1993).

SUPPLEMENTAL INFORMATION

Possible Unconventional Surface Superconductivity in the Half-Heuslers LuPtBi
and YPtBi

Eylon A. Persky1,2,3, Alan Fang,1,2 Carolina Adamo,1,2 Eli Levenson-Falk,1,3
Qi I. Yang,1,4 Chandra Shekhar,5 Claudia Felser,5 Binghai Yan,6 and Aharon
Kapitulnik1,2,3,4

1Geballe Laboratory for Advanced Materials, Stanford University, Stanford,
California 94305, USA
2Stanford Institute for Materials and Energy Sciences, SLAC National
Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, California 94025, USA
3Department of Applied Physics, Stanford University, Stanford, California
94305, USA
4Department of Physics, Stanford University, Stanford, California 94305, USA
5Max Planck Institute for Chemical Physics of Solids, 01187 Dresden, Germany.
6Department of Condensed Matter Physics, Weizmann Institute of Science,
Rehovot, Israel

##  S1 Crystal growth and sample fabrication

Single crystals of YPtBi and LuPtBi were grown by self-flux solution growth
method, whereas Bi provides a flux medium. For example, the stoichiometry
quantities of freshly polished piece of elements Lu, Pt and Bi of purity
$>99.99\%$ in the 0.3:0.3:10 atomic ratio were put in tantalum crucible and
sealed in dry quartz ampoule under 3 mbar partial pressure of argon. The
mixture containing ampoule was heated at a rate of 100 K/h up to 1473 K and
followed by 12 hours soaking. For crystal growth, the temperature was slowly
reduced by 2 K/h to 873 K and removed extra Bi flux by decanting the ampoule
at 873 K. In this method, we get 2-3 mm regular triangle size of crystals, and
their preferred growth orientation is (111), which was confirmed by Laue
diffraction. The general method of crystals growth has been followed from
literatures [29, 30]. The composition and structure were checked by energy
dispersive X-ray analysis and Laue X-ray diffraction, respectively. The
lattice parameters of the cubic structure are 6.574, which are consistent with
previous reports [11, 14]. Energy dispersive x-ray spectrometry gives atomic
percentages 33.6: 33.0: 33.4 $\pm$ 3.0 $\%$ for Lu: Pt: Bi confirming the
stoichiometric ratio of the chemical compositions. Similar procedure and
results were obtained for YPtBi.

##  S2 Surface Properties and Morphology

Figure S1: XPS studies of a LuPtBi crystal before and after ion milling. We
also show an expanded region around 160 eV, showing the removal of bismuth-
oxide and strengthening the LuPtBi-bonded bismuth.

To understand the morphology and chemistry of the surface of the crystals we
used X-ray Photoelectron Spectroscopy (XPS) that allows for high sensitivity
elemental surface composition measurement and high resolution binding energy
chemical shift measurement of solid samples under high vacuum. With typical
measurement depth of about a nanometer, pristine crystals show traces of
carbon, oxygen, and bismuth on the surface. However, after a several seconds
of ion milling (equivalent to a few nm), all carbon traces are gone and all
three elements: Lu, Pt and Bi seem to appear with the correct 1:1:1 ratio,
together with traces of oxygen. This results does not changwee after
subsequent milling of several more nanometers of the surface, confirming that
no other elements exist in the samples.

##  S3 Surface cleaning procedure using STM tip

The ease by which the fresh surface oxidizes suggests that ex-situ milling and
then loading the sample into the STM may not solve the surface contamination
problem. To overcome this issue, we performed in-situ surface cleaning by
using the STM itself to remove the top few layers and expose deeper layers,
presumably reaching the true surface layer of the crystal. In a typical
conditions of this process, performed in-situ at helium temperature under UHV
conditions, the STM tip junction resistance was set by a bias of 20-50 mV and
current of 100 pA. Then voltage pulses of a few volts were applied until zero
bias conductance was minimized. Occasionally, we found regions of area of
$\sim 5\times 5$ nm2, which were flat and untilted to within a few angstroms,
yet had an indistinct but reproducible topography (no atomic corrugations
above 10’s of pm were seen, most likely due to tip bluntness). Since we
focused on the temperature range below $T_{c}^{b}$, the quality of spectra
were gauged by how close the DOS reached zero at zero bias, as expected for
gapped spectra. A similar method was previously used to expose superconducting
regions in vacuum-cleaved YPtBi crystals by Baek et al. [32], although no
particular effort was made to study the topological (111) direction of the
crystals. Notably our (111) starting surface, while showing “lumpy” topography
similar to [32], was overall much flatter, with only $\sim$0.3 nm height
variations over 50 nm regions (as opposed to .$\sim$1 nm in [32]).

##  S4 STM Measurements

As noted in the main text, we performed measurements on a modified UNISOKU
Co., USM1300S3HeTM system with Ultra High Vacuum (UHV), 350 mK base
temperature, and magnetic field capabilities.

Figure S2:  Examples of surface spectroscopy following tip-cleaning procedure
as a function of temperature for a) YPtBi and b) LuPtBi.

Figure S3:  Examples of surface spectroscopy following tip-cleaning procedure
performed at base temperature (365 mK) as a function of magnetic field for a)
YPtBi and b) LuPtBi.

##  S5 Kerr Effect Measurements

High-resolution measurements of Kerr rotation were performed using a fiber-
based zero-area loop Sagnac interferometer as first described in Ref. [33].
Following the main components shown in Fig. S4, two linearly polarized
counterpropagating beams comprising the interferometer are isolated along the
fast and slow axes of $\sim$10 m of polarization-maintaining fiber and are
modulated in phase with a frequency that matches the transit time of light
through the interferometer. The linearly polarized light traveling along each
of these paths passes through a quarter-waveplate and is converted into
circularly polarized light just above the sample. In the absence of magnetic
field, the apparatus is completely reciprocal by symmetry except for the
sample. Thus, upon reflection, one branch of the interferometer acquires a
phase shift of $+\theta_{K}$, while its orthogonally polarized counterpart
acquires an opposite phase shift of $-\theta_{K}$. The two phase shifts are
added at the detector and extracted from the ratio of the first and second
harmonics of the modulated signal.

Figure S4:  Schematic of the zero-area loop Sagnac interferometer used in this
study. On the right is the end-part showing one beam that enters the quarter
waveplate from the fast axis as right circularly polarized, and reflected back
into the slow axis as left circularly polarized after experiencing a Kerr
angle shift. A second beam propagates in the direction of the sample in the
slow axis and is reflected into the fast axis.

##  S6 Ab-initio calculations of the surface band structure of LuPtBi

Ab initio calculations were performed within the density-funcitonal theory
framework with the generalized gradient approximation, which is implemented
into the VASP package [54]. A 54-atomic-layer-thick slab model was employed to
simulate the LuPtBisurface, in which the top surface is terminated by Bi and
the bottom surface is by Lu. A vacuum layer with more than 50 Åthick is
employed in the $z$-direction to avoid inter-slab interactions. A grid of
12$\times$12 is used for the 2D Brillouin zone integration. Spin-orbit
coupling was included in all calculations.

In particular, the Bi-termination surface is shown in Fig. S5, exhibiting a
van-Hove singularity (VHS) at $E=-0.10$ eV near the M-point, about 5% offset
along the line from M to $\Gamma$. 6 such VHS points appear in the first
Brillouin zone.

Figure S5: The surface band structure and density of states (DOS) from ab-
initio calculations. (a) Band structure. The size of red circles represents
the projection to the surface Bi atom on the top surface. (b) DOS projected to
the surface Bi states. The Fermi energy is shifted to zero. A Van Hove
singularity (VHS) point exists in the surface band near the $M$ point (nearly
10% position from M to $\Gamma$), corresponding to DOS singularity at about
$-0.1$ eV. Two insets are shown to demonstrate the VHS point, in which the
left inset shows the up-opening parabolic dispersion along the $\Gamma-M$ line
while the right inset shows the down-opening parabolic dispersion along the
line perpendicular to the $\Gamma-M$ direction. The saddle (VHS) point is
highlighted by blue arrows.


# Rethinking Text Line Recognition Models

Daniel Hernandez Diaz
Google Research
<EMAIL_ADDRESS>Siyang Qin
Google Research
<EMAIL_ADDRESS>Reeve Ingle
Google Research
<EMAIL_ADDRESS>Yasuhisa Fujii
Google Research
<EMAIL_ADDRESS>Alessandro Bissacco
Google Research
<EMAIL_ADDRESS>

###### Abstract

In this paper, we study the problem of text line recognition. Unlike most
approaches targeting specific domains such as scene-text or handwritten
documents, we investigate the general problem of developing a universal
architecture that can extract text from any image, regardless of source or
input modality. We consider two decoder families (Connectionist Temporal
Classification and Transformer) and three encoder modules (Bidirectional
LSTMs, Self-Attention, and GRCLs), and conduct extensive experiments to
compare their accuracy and performance on widely used public datasets of scene
and handwritten text. We find that a combination that so far has received
little attention in the literature, namely a Self-Attention encoder coupled
with the CTC decoder, when compounded with an external language model and
trained on both public and internal data, outperforms all the others in
accuracy and computational complexity. Unlike the more common Transformer-
based models, this architecture can handle inputs of arbitrary length, a
requirement for universal line recognition. Using an internal dataset
collected from multiple sources, we also expose the limitations of current
public datasets in evaluating the accuracy of line recognizers, as the
relatively narrow image width and sequence length distributions do not allow
to observe the quality degradation of the Transformer approach when applied to
the transcription of long lines.

##  1 Introduction

Optical character recognition (OCR) is a crucial component of a wide range of
practical applications, such as visual search, document digitization,
autonomous vehicles, augmented reality (e.g., visual translation), and can
increase the environmental awareness of visually impaired people [38]. In the
last decade, increasing sophistication in OCR models has come hand-in-hand
with a remarkable increase in accuracy for various domains [17, 3, 8, 33].

Traditional approaches usually separate the text extraction task into two
subproblems: _text detection_ and _text recognition_. Text detection
algorithms try to detect text instances (words or lines) in the input images,
while text recognition models try to decode the textual content from cropped
and rectified text patches. For scene (sparse) texts, the majority of the text
spotting methods operate at the _word-level_ , driven by the fact that most of
the public datasets in the domain only provide word-level labeling. On the
other hand, handwriting and dense text recognition systems typically adopt
_lines_ as the processing unit [3, 8, 24].

Figure 1: Example text line images in our Internal dataset, which contains
handwriting, scene-text, and document text images of various lengths.

Despite the popularity of word-level systems for scene text recognition,
operating at world-level could be suboptimal for a general purpose system
supporting many scripts. First, for handwritten text, it is not always
straightforward to separate words since detected boxes for neighboring words
could partially overlap. Second, for dense printed documents, it is difficult
to reliably detect a large amount of small and dense words. Third, it is
challenging for detectors to separate words for scripts that do not separate
words by spaces such as Chinese, Japanese, and Korean. Last but not least,
word-level OCR models are more likely to miss punctuation and diacritic marks.

In this work, we consider a universal _line-based_ OCR pipeline that can
extract dense printed or handwritten text as well as sparse scene-text with
the goal of determining the optimal _text-line recognition_ (TLR) model in
this scenario. In such a pipeline, a separate the text detection model is
responsible for detecting and rectifying all lines of text in the image. As
this step is outside the scope of this paper, we exclude from the analysis
here public datasets with irregular scene text where detection and
rectification are the main focus, such as [11]. In a line-based OCR system,
_the recognizer needs to handle arbitrarily long text-lines effectively and
efficiently_. This challenge has been overlooked by previous state-of-the-art
word-based models. Public datasets (mostly word level) have relatively narrow
image width and sequence length distributions (see Figure 5). At the same
time, the design, performance and speed of TLR models depends on these
variables. To understand these issues, we study TLR models trained on an
internal dataset containing examples within a wide range of lengths and with a
much larger symbol coverage (675 classes containing most of the symbols in
Latin script and special characters, compared to alphanumeric characters used
in majority of the public datasets). We found that techniques such as _image
chunking_ substantially alleviate the challenges posed by examples of wildly
different sizes.

We explore a number of architectures for TLR and compare their accuracy and
performance on the internal dataset. From this study emerge concrete
recommendations for the design of universal, line-level, TLR systems. _We find
that the model that uses a Self-Attention encoder[51] coupled with the
Connectionist Temporal Classification (CTC) decoder [16], compounded with an
explicit language model, outperforms all other models having both maximal
text-line recognition accuracy and minimal complexity_. To the best of our
knowledge, this model architecture has not been previously studied in the
literature. It also fits within the overarching trend in machine learning of
disposing of recurrent networks in favor of attention modules. More generally,
encoder modules based on the self-attention mechanism emerge winners in our
encoder comparison, regardless of decoder type. Our performance on widely used
public datasets (including printed, handwritten and scene-text) are reported.
We also report results of training this model on public datasets only, and
show that even when using such a restricted train set accuracy is comparable
to the widely used Transformer-based approach, and not far from significantly
more complex state-of-the-art models.

To summarize, our contributions are four-fold:

  * •

We fairly compare 9 different model design choices on an internal dataset and
on a _universal_ TLR task, pairing each of three encoders (Self-Attention,
GRCL, and BiLSTM) to each of three decoders (CTC, CTC with LM and the
Transformer decoder).

  * •

We identify as the optimal model architecture a previously unstudied
combination of Self-Attention encoding with CTC decoding. We found that this
model yielded the top accuracy with low memory requirements and very good
latency, making it an easy choice for low-resource environments.

  * •

We propose the use of image chunking to ensure that the model works
efficiently and effectively on arbitrary long input images without shrinking.

  * •

We find that models based on the Transformer decoder have significant
regression in quality if the image length is outside the length distribution
of the training data.

##  2 Related Work

Handwriting and scene-text are usually treated separately in TLR, and most
work in the literature focuses on one or the other. Nevertheless, models for
these domains share many features in common. Generally speaking, they are
designed around the use of a CNN for extracting a sequence of visual features,
that is subsequently passed to an encoder/decoder network that outputs the
labels.

For the case of Scene-Text Recognition (STR), an in-depth survey was recently
carried out by Chen et al. [10]. Close in spirit to this paper is Baek et al.
[2] in the domain of scene-text recognition. That work emphasized the
inconsistencies among the public datasets typically used for training and
evaluation in the literature, and was the first attempt to provide a uniform
framework for the comparison of recognizer models. In their analysis, they
include a transformation module, that preprocesses a potentially irregular
input image before it is fed to the vision backbone. Compared to that work,
this paper is a more focused effort. Assuming that images are rectified and
fixing the vision backbone allows us to perform a more straightforward
encoder/decoder comparison, also scanning over different encoder
configurations.

CTC models for STR are well studied [46, 15, 49]. The seminal work by Shi et
al. [46] proposed the standard, end-to-end backbone + encoder/decoder
architecture for TLR tasks. Our Self-Attention / CTC model replaces the RNN in
that work by a Self-Attention module. Sequence-to-Sequence (Seq2Seq) models
with attention have become more prevalent for STR in recent years [59, 54, 45,
4, 30, 54], in part due to the advent of the Transformer architecture [51],
that has allowed the field to loosen its ties to the recurrent connection.
Sheng et al. [45] first applied the Transformer to STR [4]. Lee et al. [30]
devised a novel 2D dynamic positional encoding method with impressive results.
Other models with an initial rectification step [47, 2, 32, 48, 58] have been
most successful in irregular STR. In particular, the drop in accuracy for
longer lines was noticed in [48]. Wang et al [55] combine a fully
convolutional (FCN) encoder with an RNN decoder based on Gated Recurrent Units
(GRU).

Modern methods for handwriting recognition (HWR) have been reviewed by Memon
et al. [36]. Michael et al. [37] compare different attention mechanisms for
Seq2Seq methods in the HWR domain. RNN encoders with CTC decoders have been
customary for some time, with Long-Short Term Memory (LSTM) being the typical
RNN cell of choice [6, 18, 39]. Multidimensional LSTM encoders have also been
tried, e.g. [18, 52, 41]. Seq2Seq approaches to handwriting recognition
including attention-based decoding have been catching up [37, 28, 40, 12, 5].
The difficulties with scaling handwriting recognition were tackled in [24].

Figure 2: This figure shows the image chunking process. Input images are split
into overlapping chunks with bidirectional padding before being fed to the
backbone. The valid portions of the produced sequence features are
concatenated before being fed to the decoder.

##  3 Anatomy of a Text-Line Recognition Model

Most state-of-the-art TLR algorithms consists of three major components: a
convolutional backbone to extract visual features; a sequential encoder which
aggregates features from part or entire sequence; and finally a decoder which
produces the final transcription given the encoder output. In this work, we
study different combination of encoder and decoder with a fixed backbone and
propose an optimal model architecture.

###  3.1 Backbone

Similar to other vision tasks, a convolutional backbone network is used to
extract relevant visual features from text-line images. Any of the many
mainstream vision modules in the literature can be reused in principle as a
TLR backbone; ResNet [20], Inception [50], and MobileNet-like [22, 44]
networks are standard choices. Generally speaking, increasing the complexity
of the backbone translates into moderate performance gains. In this work, we
use a fixed backbone across all of the experiments while focusing on the
question of finding the optimal encoder/decoder combination.

Figure 3: Backbone used in our experiments. We first reduce the resolution of
input grayscale image by 4x with a space-to-depth operation, then we apply
$11$ Fused Inverted Bottleneck [56] layers with expansion rate $8$ and $64$
output channels, and use a residual convolution block to project the output
into a tensor of height 1.

Our backbone is an isometric architecture [43] using Fused Inverted Bottleneck
layers as building block, a variant of the Inverted Bottleneck layer [44] that
replaces separable with full convolutions for higher model capacity and
inference efficiency on modern accelerators [56]. Isometric architectures
maintain constant internal resolution throughout all layers, allowing a low
activation memory footprint and making it easier to customize the model to
specialized hardware and achieve the highest utilization. Figure 3 illustrates
the network in detail. It consists of a space-to-depth layer with block size
4, followed by 11 Fused Inverted Bottleneck layers with $3\times 3$ kernels
and $8\times$ expansion rate with $64$ output channels. A final fully
convolutional residual block is applied to reduce the tensor height to 1, to
be fed as input to the encoder network [24].

###  3.2 Encoder

The backbone has a finite receptive field, which limits its ability to encode
wide range context information. This makes it difficult to decode long
sequences under challenging scenarios (e.g., handwritten text). Standard
sequential encoders can be used to capture long-distance context features,
making it a crucial component of the TLR model. In this work, we compare the
following encoders:

BiLSTM: Bidirectional RNNs based on the LSTM cell [21] are natural candidates
to capture long-range correlations in the output feature sequence of the
backbone. In our comparison, we stack 1 to 3 LSTM layers, with each cell
having 512 hidden units.

Gated Recurrent Convolution Layer (GRCL): GRCL was introduced in [53] and has
demonstrated its effectiveness on OCR. Our GRCL encoder consists of 3 sets of
repeated GRCL blocks. The convolutions in each of the 3 sets use [384, 256,
128] output filters and kernel widths (1-D convolution along width dimension)
[3, 5, 7] respectively (Fig. 4). The basic layer in the repeated blocks of
each set contains a convolutional gating mechanism, which uses a convolution
with sigmoid activation to control information propagation from a convolution
with ReLU activation. We compare 6 different variations where the number of
GRCL blocks within each set varied from 1 to 6. Batch Renormalization [25] was
used for each of the blocks and a dropout rate of 0.1 was applied in all the
convolutional layers.

Figure 4: The GRCL encoder architecture.

Self-Attention: The Self-Attention encoder, proposed in Vaswani et al. [51]
has been widely used in numerous NLP and vision tasks. As an image-to-sequence
task, text-line recognition is no exception. The Self-Attention encoder can
effectively output features that summarize an entire sequence without making
use of recurrent connections. The output of the backbone, with height
dimension being removed ($X\in\mathbb{R}^{n\times d}$), is fed to the encoder.
The encoded feature $Y$ is computed as:

$\begin{gathered}Q=XW_{q}\,,\quad K=XW_{k}\,,\quad V=XW_{v}\,,\\\
Y=\textrm{softmax}\left(\frac{QK^{T}}{\sqrt{d}}\right)V\,,\end{gathered}$ (1)

where $W_{q}$, $W_{k}$ and $W_{v}$ are $d\times d$ learned parameters which
project the input sequence $X$ into queries, keys and values respectively. The
encoded feature $Y$ is a convex combination of the computed values $V$, the
similarity matrix is computed by the dot-product of queries and keys.

Figure 5: Height-normalized (40 px) distribution of text line images widths
for the different datasets used in this work.

We use multi-head attention with 4 separate heads. The hidden size is set to
256. To prevent over-fitting, we apply dropout after each sub-layer with drop
ratio 0.1. Sinusoidal relative positional encoding is added to make the
encoder position-aware. In our experiments, we compare the accuracy and
complexity of different model variations by stacking $k$ encoder layers with
$k\in\\{4,8,12,16,20\\}$.

###  3.3 Decoder

Decoders take the encoded sequence features and try to decode its text
content. CTC [16] and attention decoders are among the popular choices for
text-line recognition. In this work, we compare the CTC decoder (with and
without language model) and the Transformer decoder [51].

CTC: The CTC decoder was originally used in speech recognition, and
researchers have had huge success adopting it to OCR tasks. The input encoded
features can be viewed as vertical feature frames along the width dimension.
The input is fed to a dense layer to obtain per-frame probabilities over
possible labels. At inference time, we use a simple greedy CTC decoding
strategy which removes all repeated symbols and all blanks from the per-frame
predictions. Beam search decoding can be optionally used to consider multiple
valid paths which are mapped to the same final output sequence. Our empirical
study shows that beam search is not needed when CTC is paired with a powerful
encoder such as the self-attention encoder.

CTC w/ LM: Incorporating an explicit language model on top of the logits from
the optical model can significantly improve accuracy. We follow the approach
described in [14] to form a log-linear model, combining the cost of CTC
logits, the cost of a character-based $N$-gram language model, the cost of a
character unigram prior, and transition costs defined for new characters,
blank labels, and repeated characters. The weights of the feature functions
were optimized using minimum error rate training [35].

Transformer Decoder: The Transformer decoder has become the decoder of choice
for sequence prediction tasks such as machine translation. In our experiments,
we incorporate a standard Transformer decoder with a stack of 8 repeated
layers. In addition to self-attention and feed-forward layers, the decoder
adds a third sub-layer which applies single-head attention over the encoded
features (cross-attention to compute a context vector). The decoder’s hidden
size is set to 256. Greedy decoding is applied at inference time. Per-step
cross-entropy loss with label smoothing is used during training.

###  3.4 Chunking

Due to the dot-product attention in the self-attention layer, the model
complexity and memory footprint of the encoder grows quadratically as a
function of the image width. This can cause issues for inputs of large width.
Shrinking wide images can avoid such issues but it will inevitably affect
recognition accuracy, especially for narrow or closely-spaced characters.

We propose a simple yet effective chunking strategy to ensure that the model
works well on arbitrarily wide input images without shrinking (see Figure 2),
similar to [9]. We resize the input image to 40 pixels height, preserving the
aspect ratio. Then the text-line is split into overlapping chunks with
bidirectional padding to reduce possible boundary effects (note the last chunk
has extra padding to ensure a uniform shape for batching purpose). We feed
overlapping chunks into the backbone and self-attention encoder to produce
sequence features for each chunk. Finally, we merge the valid regions back
into a full sequence, removing the padding areas.

This approach splits long sequences into $k$ shorter chunks, effectively
reducing model complexity and memory usage by a factor of $k$ for self-
attention layers. This strategy is used both at training and inference time to
keep a consistent behavior. Via a controlled experiment, we find no regression
by applying chunking to the inputs. The aspect ratio of each chunk is set to 8
(i.e., 320 pixel width for 40 pixel height) to provide a reasonable amount of
context information.

##  4 Experimental Setup

Table 1: Evaluation results on public handwriting and scene-text datasets of our best models and selected works. The “Rect.” column indicates whether the model includes a rectification module. “S-Attn”, “Attn”, and “Tfmr Dec.” stand for Self-Attention, Attention and Transformer Decoder respectively. “MJ”, “ST” and “SA” stand for MJSynth[26, 27], SynthText [19] and SynthAdd [31] respectively. | Rect. | Encoder / Decoder | Train Dataset |  IAM $\downarrow$ |  RIMES $\downarrow$ |  IIIT5K $\uparrow$ |  SVT $\uparrow$ |  IC13 $\uparrow$ |  IC15 $\uparrow$
---|---|---|---|---|---|---|---|---|---
Bluche and Messina [6] | No | GCRNN/CTC | IAM (50k lexicon) | 3.2 | 1.9 | - | - | - | -
Michael et al. [37] | No | LSTM/LSTM w/Attn | IAM | 4.87 | - | - | - | - | -
Kang et al. [28] | No | Transformer | IAM | 4.67 | - | - | - | - | -
Bleeker and de Rijke [4] | No | Transformer | MJ + ST | - | - | 94.7 | 89.0 | 93.4 | 75.7
Lu et al. [34] | No |  Global Context Attn / Tfmr Dec. | MJ + ST + SA | - | - | 95.0 | 90.6 | 95.3 | 79.4
Qiao et al. [42] | No | LSTM / LSTM + Attn | MJ + ST | - | - | 94.4 | 90.1 | 93.3 | 77.1
Sheng et al. [45] | Yes | S-Attn / Attn | MJ + ST | - | - | 93.4 | 89.5 | 91.8 | 76.1
Shi et al. [48] | Yes | LSTM / LSTM + Attn | MJ + ST | - | - | 91.93 | 93.49 | 89.75 | -
Wang et al. [55] | No | FCN / GRU | MJ + ST | 6.4 | 2.7 | 94.3 | 89.2 | 93.9 | 74.5
SCATTER [32] | Yes | CNN / BiLSTM | MJ + ST + SA | - | - | 93.9 | 92.7 | 94.7 | 82.8
Yu et al. [57] | No |  Attn / Semantic Attn. | MJ + ST | - | - | 94.8 | 91.5 | 95.5 | 82.7
Ours | No | S-Attn / CTC | MJ + ST + Public | - | - | 92.06 | 87.94 | 92.15 | 72.36
No | S-Attn / CTC + LM | MJ + ST + Public | - | - | 93.63 | 91.50 | 93.61 | 74.77
No | Transformer | MJ + ST + Public | - | - | 93.93 | 92.27 | 93.88 | 77.08
No | S-Attn / CTC | Internal | 4.62 | 10.80 | 96.26 | 91.96 | 94.43 | 78.43
No | S-Attn / CTC + LM | Internal | 3.15 | 7.79 | 96.83 | 94.59 | 95.98 | 80.36
No | Transformer | Internal | 3.99 | 9.71 | 96.54 | 92.59 | 94.34 | 79.68
No | S-Attn / CTC | Internal + Public | 3.53 | 2.48 | 95.66 | 91.34 | 93.70 | 78.38
No | S-Attn / CTC + LM | Internal + Public | 2.75 | 1.99 | 96.43 | 93.66 | 95.25 | 79.92
No | Transformer | Internal + Public | 2.96 | 2.01 | 96.44 | 92.50 | 93.97 | 80.45

###  4.1 Datasets

To better train and evaluate TLR models for a line-based OCR engine with broad
language support, we built an Internal dataset which contains both single
words and long text-lines (see Figure 5). This dataset has multiple image
sources, including synthetic text, scene-text, handwritten, documents, and
photos reflecting some common phone camera use cases (translate, receipts,
notes and memos). Scene-text training consists of the Uber-Text [60] and the
OpenImages dataset [29] self-annotated with Google Cloud Vision TEXT_DETECTION
API [23]. Other images are from public sources - Flickr, Web pages - or
collected through vendors specialized in crowdsourcing, which were given
detailed instructions on how to collect the images to mimic photos taken by
phone application users. The images contain text from many Latin languages,
with 675 symbols including punctuation and special characters, and were
annotated by human raters with a transcription/verification iterative process
seeded from the OCR predictions.

We also evaluate our models on six commonly used public benchmarks in the
scene-text and handwriting domains, namely, IAM and RIMES for handwritten
text, and IIIT5K, SVT, ICDAR13 and ICDAR 15 for scene-text (detailed
descriptions of these datasets can be found in the Supplementary Material).
Despite their wide usage, public datasets share two features that make them
somewhat unrealistic, namely, _relatively narrow image-length distribution_
and _lack of symbol coverage_. The height-normalized image width distribution
of the test set of the public datasets is shown in the left panels in Figure
5. The IAM dataset lacks short lines while in the other word-level datasets
long lines are missing.

###  4.2 Implementation Details

Models were trained on three different training data configurations : _Synth +
Public_ , _Internal_ , and _Internal + Public_. Most of the analysis presented
below corresponds to training on the _Internal_ configuration, where only the
internal dataset was used for training. To establish a qualitative comparison
with other works, we also trained on the _Internal + Public_ configuration,
where data from _Internal_ was combined with the training data from the public
datasets (_Public_). These two were then mixed in a 9:4 ratio.

The _Synth + Public_ runs are scene-text, public-data-only runs, that combine
a merge of MJSynth[26, 27] and Synthtext[19] with the training data from the
public datasets. The latter was further augmented 10 times by applying random
distortions (affine, blur, noise, etc). During training, _synth_ and _train_
were mixed in a 10:3 ratio.

All of our models were trained and tested using grayscale images. Image
chunking (Section 3.4) is applied to _CTC based models_ at both training and
inference time to efficiently handle arbitrarily long inputs. We normalize the
image height to 40 pixels while keeping the aspect ratio. For models which use
the _Transformer decoder_ , to prevent accuracy regression on long lines
(Section 5.3), we feed fixed size images (resized and padded to $1024\times
40$) to the models. Since the image width is bounded, chunking strategy is not
used for _Transformer decoder_ based models.

The character-based $N$-gram language model for the CTC w/ LM decoder was
trained from Web texts with stupid-backoff [7] and $N$ was set to 9. The
weights of the feature functions were tuned on the development portion of the
internal dataset.

All the models are implemented in TensorFlow [1], trained using 32 TPU-v3
cores with a batch size of 1024. The SGD optimizer is used with momentum set
to 0.9. We use a linear learning rate warm-up followed by stepped learning
rate decay. Training iterations, initial learning rate, weight decay factor
and global gradient clipping ratio are optimized for each model using grid
search.

###  4.3 Evaluation Protocol

The handwriting datasets IAM and RIMES are evaluated using the case-sensitive
Character Error Rate (CER), defined as the Levenshtein distance at the
character level between the prediction and the ground truth, normalized by the
ground truth length. For the scene-text public datasets, we use case-
insensitive word prediction accuracy (WPA) to evaluate the recognition results
as is customary. This is defined as $1-W$ where $W$ is the Word Error Rate,
defined through the Levenshtein distance, analogously to CER, but at the word
level.

Evaluation on the internal test set, that includes images from the printed,
handwritten and scene-text domains, is evaluated using both WPA and CER. No
lexicon is used in any of our experiments.

##  5 Result Analysis

###  5.1 Encoder / Decoder Comparisons

Evaluation results on the _internal_ test set are shown in Fig. 6 for CER and
WPA, respectively. Within each figure, line color is used to indicate the
encoder type: Self-Attention (pink), GRCL (green), or BiLSTM (blue).
Alternatively, decoder type is indicated by the symbol: CTC (circle), CTC with
LM (square), or Transformer Decoder (triangle).

Figure 6: CERs and WPAs for the different encoder/decoder combinations
evaluated on the internal test set. _Self-Attention:_ 5 models are plotted per
decoder corresponding to 4, 8, 12, 16, and 20 Self-Attention layers. _GRCL:_ 6
models are plotted corresponding to the number of GRCL blocks per set, from 1
to 6. _BiLSTM:_ Three models are plotted with increasing depth 1, 2, and 3.
Figure 7: CER vs Latency for the non-recurrent models in Fig. 6. CTC decoding
is 5-10 times faster. The LM adds a small overhead (1 ms). For all models,
latency was computed on a NVIDIA V100 GPU card.

The Self-Attention encoder (pink lines) is the most performant encoder
irrespective of the decoder. With similar number of parameters, the CER for
models with the Self-Attention encoder was $\sim$ 2.5 - 10% lower (relative)
than for their GRCL and BiLSTM counterparts, translating into WPA gains of
$\sim$ 1% absolute. Among all the models considered, the models with best CER
and WPA for all three decoders were Self-Attention models.

CTC decoders compounded with an explicit LM achieve the best CER and WPA for
the Self-Attention and GRCL encoders. The CER gains afforded by the LM applied
to each of the CTC models considered, were relatively modest ($<$5% relative
for fixed number of parameters for Self-Attention and GRCL). There is a bigger
improvement from the LM for the BiLSTM encoder, but in that case, the
Transformer Decoder outperformed CTC, an indication of the importance of the
Attention mechanism in RNN architectures.

Figure 8: CER per 100 px bucket of height-normalized image width showing the
impact of resizing during inference in the Transformer architecture trained
with images up to 256px wide (left) and up to 512px wide (right). The Self-
Attention / CTC + LM result is with the same training data is also shown for
comparison. Figure 9: Failure cases for the Self-Attention / Transformer
Decoder model, trained with images of maximum width 256 and evaluated with no
resizing. Below each image, the TLR results. The vertical blue line denotes
256 pixel mark (height-normalized to 40 pixels)

The clear winner overall is the model that combines a Self-Attention encoder,
the CTC decoder with LM, using 12 Self-Attention layers or more. Our absolute
best run, with 16 Self-Attention layers, produces a 4% relative CER
improvement, with 8 million less parameters, over the second best model, a
Self-Attention / Transformer Decoder with 20 encoder layers.

Different from other encoders, the BiLSTM encoder improved only slightly with
more layers. This encoder is also more prone to overfitting and overall
performs worse than the other two.

Without the explicit language model, CTC and the Transformer are overall
competitive as decoders for the Self-Attention Module, but there are trade-
offs. The CTC model has a better CER at the same number of parameters with
similar WPA. However, the biggest Transformer model performs somewhat better
in both metrics than the best CTC. Latency tells a different story. In Fig. 7,
latencies for our non-recurrent models are represented for an input size of
$40\times 320$. CTC is 5 to 10 times faster than the Transformer decoder while
increasing the capacity of the encoder or adding a LM contributes only a small
overhead

###  5.2 Evaluation on Public Datasets

In Table 1 we show the performance on the test sets of the public datasets of
our best models trained on _Synth + Public_ , along with some of the best
results reported in the literature. The table shows that in this training
configuration, these models are competitive with SOTA, especially if one
considers only models that do not include a rectification module. The Self-
Attention/CTC model performs slightly worse than Self-Attention/Transformer in
this configuration. Despite its small relative size, the inclusion of the
_Public_ dataset is key to achieve this performance; training on synthetic
data only yields significantly worse results.

We also show the evaluation results for the _Internal_ and _Internal + Public_
training runs, on the test sets of the public datasets for both decoders. We
stress that by presenting these results we do not intend to establish a
comparison with SOTA, since the training data in this case differs from the
public datasets typically used. No benefit was found from including the
training portion of the scene-text public datasets in this training
configuration. In fact, the runs that performed best on scene-text datasets
corresponded to models trained with only _Internal_ data. Conversely, adding
the train sets of the handwritten public data have a substantial impact. This
is most manifest for RIMES, in which the official training and test splits
contain many examples with repeated labels.

###  5.3 Resizing

In order to train the Transformer decoder models on TPUs, images are first
resized to a fixed height and then either padded or anisotropically shrunk to
a fixed maximum width $M$. Reducing $M$ could in principle lead to faster
training times, while potentially impacting performance, particularly for
longer lines.

The performance changes with respect to $M$ are shown in Fig. 8. In the figure
we plot CER as a function of validation image width, grouped into buckets of
100 pixels. We considered two inference configurations differing on whether
the validation images were resized to width $M$ or not. The Self-Attention/CTC
+ LM result, when trained on the same data, is shown for comparison.

It is observed in Fig. 8 that the CERs drastically increase after $M$ without
resizing in both max widths. Fig. 9 shows typical failure cases without
resizing. The recognizer appears to essentially stop decoding at the maximum
width, often outputting the same characters observed around $M$ repeatedly.
Resizing the images to the maximum width at inference time alleviates the
issue. However, there is a limitation with the approach. In our experiments,
the accuracy started to noticeably degrade once the width of the image becomes
longer than roughly $2\times M$. In contrast, the accuracy of the CTC decoder
is insensitive to the image width, which is another advantage of the CTC
decoder.

##  6 Conclusions

In this work we investigated the performance of representative encoder/decoder
architectures as universal text-line recognizers. In the decoder comparison,
we found that CTC, compounded with a language model, yielded the overall
superior performance. In the absence of a LM, CTC and the Transformer are
competitive, with CTC dominating in some cases (GRCL) and the Transformer in
others (BiLSTM). On the other hand, in the encoder comparison, Self-Attention
was the overall winner and both decoders were similarly accurate without LM.
Interestingly, the unstudied Self-Attention/CTC + LM model, is our best. [13]
showed that attention-based decoders can still benefit from an external
language model. Investigations on the effectiveness of external language
models with transformer decoders will be future work.

We also considered issues derived from the presence of long images in the
distribution of examples. There are at least two new aspects that need to be
considered, efficiency and performance. Long images affect the efficiency of
models with the Self-Attention encoder due to quadratic scaling with image
length. We showed that this problem can be solved for CTC models without
performance loss by chunking the images. Training on images of fixed maximum
width affects the performance on longer images of models that make use of the
Transformer decoder. This problem can be alleviated, although not entirely
eliminated, by resizing the images to the train width.

## References

  * [1] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al.  Tensorflow: A system for large-scale machine learning.  In 12th $\\{$USENIX$\\}$ symposium on operating systems design and implementation ($\\{$OSDI$\\}$ 16), pages 265–283, 2016.
  * [2] Jeonghun Baek, Geewook Kim, Junyeop Lee, Sungrae Park, Dongyoon Han, Sangdoo Yun, Seong Joon Oh, and Hwalsuk Lee.  What is wrong with scene text recognition model comparisons?  2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4714–4722, 2019.
  * [3] A. Bissacco, M. Cummins, Y. Netzer, and H. Neven.  Photoocr: Reading text in uncontrolled conditions.  In 2013 IEEE International Conference on Computer Vision, pages 785–792, 2013.
  * [4] Maurits Bleeker and Maarten de Rijke.  Bidirectional scene text recognition with a single decoder.  arXiv preprint arXiv:1912.03656, 2019.
  * [5] Theodore Bluche.  Joint line segmentation and transcription for end-to-end handwritten paragraph recognition.  In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29, pages 838–846. Curran Associates, Inc., 2016.
  * [6] T. Bluche and R. Messina.  Gated convolutional recurrent neural networks for multilingual handwriting recognition.  In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), volume 01, pages 646–651, 2017.
  * [7] T. Brants, A. C. Popat, P. Xu, F. J. Och, and J. Dean.  Large language models in machine translation.  In 2007 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 858–867, 2007.
  * [8] T. M. Breuel, A. Ul-Hasan, M. A. Al-Azawi, and F. Shafait.  High-performance ocr for printed english and fraktur using lstm networks.  In 2013 12th International Conference on Document Analysis and Recognition, pages 683–687, 2013.
  * [9] Kai Chen, Zhi-Jie Yan, , and Qiang Huo.  A context-sensitive-chunk bptt approach to training deep lstm/blstm recurrent neural networks for offline handwriting recognition.  In 2015 International Conference on Document Analysis and Recognition (ICDAR), pages 411–415, 2015.
  * [10] Xiaoxue Chen, Lianwen Jin, Yuanzhi Zhu, Canjie Luo, and T. Wang.  Text recognition in the wild: A survey.  ArXiv, abs/2005.03492, 2020.
  * [11] Chee Kheng Chng and Chee Seng Chan.  Total-text: A comprehensive dataset for scene text detection and recognition.  CoRR, abs/1710.10400, 2017.
  * [12] Arindam Chowdhury and Lovekesh Vig.  An Efficient End-to-End Neural Model for Handwritten Text Recognition.  arXiv e-prints, page arXiv:1807.07965, July 2018.
  * [13] Fuze Cong, Wenping Hu, Qiang Huo, and Li Guo.  A comparative study of attention-based encoder-decoder approaches to natural scene text recognition.  In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 916–921, 2019.
  * [14] Y. Fujii, K. Driesen, J. Baccash, A. Hurst, and A. C. Popat.  Sequence-to-label script identification for multilingual ocr.  In 2017 International Conference on Document Analysis and Recognition (ICDAR), pages 161–168, 2017.
  * [15] Yunze Gao, Yingying Chen, Jinqiao Wang, Ming Tang, and Hanqing Lu.  Reading scene text with fully convolutional sequence modeling.  Neurocomputing, 339:161 – 170, 2019.
  * [16] Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber.  Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks.  In Proceedings of the 23rd International Conference on Machine Learning, ICML ’06, page 369–376, New York, NY, USA, 2006. Association for Computing Machinery.
  * [17] A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber.  A novel connectionist system for unconstrained handwriting recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5):855–868, 2009.
  * [18] Alex Graves and Jürgen Schmidhuber.  Offline handwriting recognition with multidimensional recurrent neural networks.  In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21, pages 545–552. Curran Associates, Inc., 2009.
  * [19] Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman.  Synthetic data for text localisation in natural images.  In IEEE Conference on Computer Vision and Pattern Recognition, 2016\.
  * [20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  * [21] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  9(8):1735–1780, Nov. 1997.
  * [22] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  CoRR, abs/1704.04861, 2017.
  * [23] OCR in Google Cloud Vision API.  http://cloud.google.com/vision/docs/ocr.
  * [24] R. R. Ingle, Y. Fujii, T. Deselaers, J. Baccash, and A. C. Popat.  A scalable handwritten text recognition system.  In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 17–24, 2019.
  * [25] Sergey Ioffe.  Batch renormalization: Towards reducing minibatch dependence in batch-normalized models.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30, pages 1945–1953. Curran Associates, Inc., 2017\.
  * [26] Max Jaderberg, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.  Synthetic data and artificial neural networks for natural scene text recognition.  In Workshop on Deep Learning, NIPS, 2014.
  * [27] Max Jaderberg, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.  Reading text in the wild with convolutional neural networks.  International Journal of Computer Vision, 116(1):1–20, jan 2016\.
  * [28] Lei Kang, Pau Riba, Marçal Rusiñol, Alicia Fornés, and Mauricio Villegas.  Pay attention to what you read: Non-recurrent handwritten text-line recognition, 2020.
  * [29] Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, and et al.  The open images dataset v4.  International Journal of Computer Vision, 128(7):1956–1981, Mar 2020.
  * [30] Junyeop Lee, Sungrae Park, Jeonghun Baek, Seong Joon Oh, Seonghyeon Kim, and Hwalsuk Lee.  On recognizing texts of arbitrary shapes with 2d self-attention.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 546–547, 2020.
  * [31] Hui Li, Peng Wang, Chunhua Shen, and Guyu Zhang.  Show, attend and read: A simple and strong baseline for irregular text recognition.  Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):8610–8617, Jul. 2019.
  * [32] Ron Litman, Oron Anschel, Shahar Tsiper, Roee Litman, Shai Mazor, and R. Manmatha.  Scatter: Selective context attentional scene text recognizer.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [33] Shangbang Long, Xin He, and Cong Yao.  Scene text detection and recognition: The deep learning era.  International Journal of Computer Vision, pages 1–24, 2020.
  * [34] Ning Lu, Wenwen Yu, Xianbiao Qi, Yihao Chen, Ping Gong, and Rong Xiao.  MASTER: multi-aspect non-local network for scene text recognition.  CoRR, abs/1910.02562, 2019.
  * [35] W. Macherey, F. J. Och, I. Thayer, and J. Uszkoreit.  Lattice-based minimum error rate training for statistical machine translation.  pages 725–734, 2008.
  * [36] J. Memon, M. Sami, R. A. Khan, and M. Uddin.  Handwritten optical character recognition (ocr): A comprehensive systematic literature review (slr).  IEEE Access, 8:142642–142668, 2020.
  * [37] J. Michael, R. Labahn, T. Grüning, and J. Zöllner.  Evaluating sequence-to-sequence models for handwritten text recognition.  In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1286–1293, 2019.
  * [38] Leo Neat, Ren Peng, Siyang Qin, and Roberto Manduchi.  Scene text access: A comparison of mobile ocr modalities for blind users.  In Proceedings of the 24th International Conference on Intelligent User Interfaces, pages 197–207, 2019.
  * [39] V. Pham, T. Bluche, C. Kermorvant, and J. Louradour.  Dropout improves recurrent neural networks for handwriting recognition.  In 2014 14th International Conference on Frontiers in Handwriting Recognition, pages 285–290, 2014.
  * [40] J. Poulos and Rafael Valle.  Attention networks for image-to-text.  ArXiv, abs/1712.04046, 2017.
  * [41] J. Puigcerver.  Are multidimensional recurrent layers really necessary for handwritten text recognition?  In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), volume 01, pages 67–72, 2017.
  * [42] Zhi Qiao, Xugong Qin, Yu Zhou, Fei Yang, and Weiping Wang.  Gaussian constrained attention network for scene text recognition, 2020\.
  * [43] Mark Sandler, Jonathan Baccash, Andrey Zhmoginov, and Andrew Howard.  Non-discriminative data or weak model? on the relative importance of data and model resolution, 2019.
  * [44] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4510–4520, 2018.
  * [45] Fenfen Sheng, Zhineng Chen, and Bo Xu.  NRTR: A no-recurrence sequence-to-sequence model for scene text recognition.  CoRR, abs/1806.00926, 2018.
  * [46] B. Shi, X. Bai, and C. Yao.  An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition.  IEEE Transactions on Pattern Analysis & Machine Intelligence, 39(11):2298–2304, nov 2017.
  * [47] Baoguang Shi, Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai.  Robust scene text recognition with automatic rectification.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
  * [48] B. Shi, M. Yang, X. Wang, P. Lyu, C. Yao, and X. Bai.  Aster: An attentional scene text recognizer with flexible rectification.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2035–2048, 2019.
  * [49] Bolan Su and Shijian Lu.  Accurate scene text recognition based on recurrent neural network.  In Daniel Cremers, Ian Reid, Hideo Saito, and Ming-Hsuan Yang, editors, Computer Vision – ACCV 2014, Cham, 2015. Springer International Publishing.
  * [50] C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich.  Going deeper with convolutions.  In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1–9, 2015.
  * [51] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc., 2017.
  * [52] P. Voigtlaender, P. Doetsch, and H. Ney.  Handwriting recognition with large multidimensional long short-term memory recurrent neural networks.  In 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pages 228–233, 2016.
  * [53] Jianfeng Wang and Xiaolin Hu.  Gated recurrent convolution neural network for ocr.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 335–344. Curran Associates, Inc., 2017.
  * [54] S. Wang, Y. Wang, X. Qin, Q. Zhao, and Z. Tang.  Scene text recognition via gated cascade attention.  In 2019 IEEE International Conference on Multimedia and Expo (ICME), pages 1018–1023, 2019.
  * [55] Tianwei Wang, Yuanzhi Zhu, Lianwen Jin, Canjie Luo, Xiaoxue Chen, Yaqiang Wu, Qianying Wang, and Mingxiang Cai.  Decoupled attention network for text recognition.  In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 12216–12224. AAAI Press, 2020.
  * [56] Yunyang Xiong, Hanxiao Liu, Suyog Gupta, Berkin Akin, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Vikas Singh, and Bo Chen.  Mobiledets: Searching for object detection architectures for mobile accelerators, 2020.
  * [57] Deli Yu, Xuan Li, Chengquan Zhang, Tao Liu, Junyu Han, Jingtuo Liu, and Errui Ding.  Towards accurate scene text recognition with semantic reasoning networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12113–12122, 2020.
  * [58] F. Zhan and S. Lu.  Esir: End-to-end scene text recognition via iterative image rectification.  In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2054–2063, 2019.
  * [59] Jiaxin Zhang, Canjie Luo, Lianwen Jin, Tianwei Wang, Ziyan Li, and Weiying Zhou.  Sahan: Scale-aware hierarchical attention network for scene text recognition.  Pattern Recognition Letters, 136:205 – 211, 2020.
  * [60] Ying Zhang, Lionel Gueguen, Ilya Zharkov, Peter Zhang, Keith Seifert, and Ben Kadlec.  Uber-text: A large-scale dataset for optical character recognition from street-level imagery.  In SUNw: Scene Understanding Workshop - CVPR 2017, Hawaii, U.S.A., 2017.